A dedicated network sensor appliance is the recommended method for capturing and analyzing live network traffic when performance and throughput is of utmost importance. Hedgehog Linux is a custom Debian-based operating system built to:
Please see the Hedgehog Linux README for more information.
zeek-live containers can monitor one or more local network interfaces, specified by the
PCAP_IFACE environment variable in
docker-compose.yml. These containers are started with additional privileges (
SYS_ADMIN) to allow opening network interfaces in promiscuous mode for capture.
The instances of Zeek and Suricata (in the
zeek-live containers when the
ZEEK_LIVE_CAPTURE environment variables in
docker-compose.yml are set to
true, respectively) analyze traffic on-the-fly and generate log files containing network session metadata. These log files are in turn scanned by Filebeat and forwarded to Logstash for enrichment and indexing into the OpenSearch document store.
In contrast, the
pcap-capture container buffers traffic to PCAP files and periodically rotates these files for processing (by Arkime’s
capture utlity in the
arkime container) according to the thresholds defined by the
PCAP_ROTATE_MINUTES environment variables in
docker-compose.yml. If for some reason (e.g., a low resources environment) you also want Zeek and Suricata to process these intermediate PCAP files rather than monitoring the network interfaces directly, you can set
ZEEK_LIVE_CAPTURE to false.
These various options for monitoring traffic on local network interfaces can also be configured by running
Note that currently Microsoft Windows and Apple macOS platforms run Docker inside of a virtualized environment. Live traffic capture and analysis on those platforms would require additional configuration of virtual interfaces and port forwarding in Docker which is outside of the scope of this document.
Malcolm’s Logstash instance can also be configured to accept logs from a remote forwarder by running
./scripts/install.py --configure and answering “yes” to “
Expose Logstash port to external hosts?.” Enabling encrypted transport of these logs files is discussed in Configure authentication and the description of the
BEATS_SSL environment variable in the
Configuring Filebeat to forward Zeek logs to Malcolm might look something like this example
filebeat.inputs: - type: log paths: - /var/zeek/*.log fields_under_root: true compression_level: 0 exclude_lines: ['^\s*#'] scan_frequency: 10s clean_inactive: 180m ignore_older: 120m close_inactive: 90m close_renamed: true close_removed: true close_eof: false clean_renamed: true clean_removed: true output.logstash: hosts: ["192.0.2.123:5044"] ssl.enabled: true ssl.certificate_authorities: ["/foo/bar/ca.crt"] ssl.certificate: "/foo/bar/client.crt" ssl.key: "/foo/bar/client.key" ssl.supported_protocols: "TLSv1.2" ssl.verification_mode: "none"