A powerful, easily deployable network traffic analysis tool suite
There are several ways to customize Malcolm’s runtime behavior via local changes to configuration files. Many commonly-tweaked settings are discussed in the project README (see Environment Variable Files and Customizing event severity scoring for some examples).
Some configuration changes can be put in place by modifying local copies of configuration files and then using a Docker bind mount to overlay the modified file onto the running Malcolm container. This is already done for many files and directories used to persist Malcolm configuration and data. For example, the default list of bind mounted files and directories for each Malcolm service is as follows:
$ grep -P "^( - ./| [\w-]+:)" docker-compose-standalone.yml
opensearch:
- ./nginx/ca-trust:/var/local/ca-trust:ro
- ./.opensearch.primary.curlrc:/var/local/curlrc/.opensearch.primary.curlrc:ro
- ./.opensearch.secondary.curlrc:/var/local/curlrc/.opensearch.secondary.curlrc:ro
- ./opensearch:/usr/share/opensearch/data:delegated
- ./opensearch-backup:/opt/opensearch/backup:delegated
- ./opensearch/opensearch.keystore:/usr/share/opensearch/config/persist/opensearch.keystore:rw
dashboards-helper:
- ./nginx/ca-trust:/var/local/ca-trust:ro
- ./.opensearch.primary.curlrc:/var/local/curlrc/.opensearch.primary.curlrc:ro
- ./.opensearch.secondary.curlrc:/var/local/curlrc/.opensearch.secondary.curlrc:ro
dashboards:
- ./nginx/ca-trust:/var/local/ca-trust:ro
- ./.opensearch.primary.curlrc:/var/local/curlrc/.opensearch.primary.curlrc:ro
logstash:
- ./nginx/ca-trust:/var/local/ca-trust:ro
- ./.opensearch.primary.curlrc:/var/local/curlrc/.opensearch.primary.curlrc:ro
- ./.opensearch.secondary.curlrc:/var/local/curlrc/.opensearch.secondary.curlrc:ro
- ./logstash/maps/malcolm_severity.yaml:/etc/malcolm_severity.yaml:ro
- ./logstash/certs/ca.crt:/certs/ca.crt:ro
- ./logstash/certs/server.crt:/certs/server.crt:ro
- ./logstash/certs/server.key:/certs/server.key:ro
filebeat:
- ./nginx/ca-trust:/var/local/ca-trust:ro
- ./.opensearch.primary.curlrc:/var/local/curlrc/.opensearch.primary.curlrc:ro
- ./zeek-logs:/zeek
- ./suricata-logs:/suricata
- ./filebeat/certs/ca.crt:/certs/ca.crt:ro
- ./filebeat/certs/client.crt:/certs/client.crt:ro
- ./filebeat/certs/client.key:/certs/client.key:ro
arkime:
- ./nginx/ca-trust:/var/local/ca-trust:ro
- ./.opensearch.primary.curlrc:/var/local/curlrc/.opensearch.primary.curlrc:ro
- ./pcap:/data/pcap
- ./arkime-logs:/opt/arkime/logs
- ./arkime-raw:/opt/arkime/raw
zeek:
- ./nginx/ca-trust:/var/local/ca-trust:ro
- ./pcap:/pcap
- ./zeek-logs/upload:/zeek/upload
- ./zeek-logs/extract_files:/zeek/extract_files
- ./zeek/intel:/opt/zeek/share/zeek/site/intel
zeek-live:
- ./nginx/ca-trust:/var/local/ca-trust:ro
- ./zeek-logs/live:/zeek/live
- ./zeek-logs/extract_files:/zeek/extract_files
- ./zeek/intel:/opt/zeek/share/zeek/site/intel
suricata:
- ./nginx/ca-trust:/var/local/ca-trust:ro
- ./suricata-logs:/var/log/suricata
- ./pcap:/data/pcap
- ./suricata/rules:/opt/suricata/rules:ro
suricata-live:
- ./nginx/ca-trust:/var/local/ca-trust:ro
- ./suricata-logs:/var/log/suricata
- ./suricata/rules:/opt/suricata/rules:ro
file-monitor:
- ./nginx/ca-trust:/var/local/ca-trust:ro
- ./zeek-logs/extract_files:/zeek/extract_files
- ./zeek-logs/current:/zeek/logs
- ./yara/rules:/yara-rules/custom:ro
pcap-capture:
- ./nginx/ca-trust:/var/local/ca-trust:ro
- ./pcap/upload:/pcap
pcap-monitor:
- ./nginx/ca-trust:/var/local/ca-trust:ro
- ./.opensearch.primary.curlrc:/var/local/curlrc/.opensearch.primary.curlrc:ro
- ./zeek-logs:/zeek
- ./pcap:/pcap
upload:
- ./nginx/ca-trust:/var/local/ca-trust:ro
- ./pcap/upload:/var/www/upload/server/php/chroot/files
htadmin:
- ./nginx/ca-trust:/var/local/ca-trust:ro
- ./htadmin/config.ini:/var/www/htadmin/config/config.ini:rw
- ./htadmin/metadata:/var/www/htadmin/config/metadata:rw
- ./nginx/htpasswd:/var/www/htadmin/auth/htpasswd:rw
freq:
- ./nginx/ca-trust:/var/local/ca-trust:ro
netbox:
- ./nginx/ca-trust:/var/local/ca-trust:ro
- ./netbox/config:/etc/netbox/config:ro
- ./netbox/media:/opt/netbox/netbox/media:rw
- ./net-map.json:/usr/local/share/net-map.json:ro
netbox-postgres:
- ./nginx/ca-trust:/var/local/ca-trust:ro
- ./netbox/postgres:/var/lib/postgresql/data:rw
netbox-redis:
- ./nginx/ca-trust:/var/local/ca-trust:ro
- ./netbox/redis:/data
netbox-redis-cache:
- ./nginx/ca-trust:/var/local/ca-trust:ro
api:
- ./nginx/ca-trust:/var/local/ca-trust:ro
- ./.opensearch.primary.curlrc:/var/local/curlrc/.opensearch.primary.curlrc:ro
nginx-proxy:
- ./nginx/ca-trust:/var/local/ca-trust:ro
- ./nginx/nginx_ldap.conf:/etc/nginx/nginx_ldap.conf:ro
- ./nginx/htpasswd:/etc/nginx/auth/htpasswd:ro
- ./nginx/certs:/etc/nginx/certs:ro
- ./nginx/certs/dhparam.pem:/etc/nginx/dhparam/dhparam.pem:ro
So, for example, if a user wanted to make a change to the nginx-proxy
container’s nginx.conf
file, they could add the following line to the volumes:
section of the nginx-proxy
service in the docker-compose.yml
file:
- ./nginx/nginx.conf:/etc/nginx/nginx.conf:ro
The change would take effect after stopping and starting Malcolm.
See the documentation on Docker bind mount for more information on this technique.
Another method for modifying local copies of Malcolm’s services’ containers is to build custom containers with the modifications baked-in.
For example, imagine a user wanted to create a Malcolm container that includes a new dashboard for OpenSearch Dashboards and a new enrichment filter .conf
file for Logstash. After placing these files under ./dashboards/dashboards
and ./logstash/pipelines/enrichment
, respectively, in the Malcolm working copy, run ./build.sh dashboards-helper logstash
to build just those containers. After the build completes, run docker images
to see the fresh images for ghcr.io/idaholab/malcolm/dashboards-helper
and ghcr.io/idaholab/malcolm/logstash-oss
. Users may need to review the contents of the Dockerfiles to determine the correct service and filesystem location within that service’s Docker image depending on the nature of the task.
Alternately, forks of Malcolm on GitHub contain workflow files that contain instructions for GitHub to build the docker images and sensor and Malcolm installer ISOs. The resulting images are named according to the pattern ghcr.io/owner/malcolm/image:branch
(e.g., if the GitHub user romeogdetlevjr
has forked Malcolm, the arkime
container built for the main
would be named ghcr.io/romeogdetlevjr/malcolm/arkime:main
). To run a local instance of Malcolm using these images instead of the official ones, users would need to edit their docker-compose.yml
file(s) and replace the image:
tags according to this new pattern, or use the bash helper script ./shared/bin/github_image_helper.sh
to pull and re-tag the images.