Automatic file extraction and scanning

Malcolm can leverage Zeek’s knowledge of network protocols to automatically detect file transfers and extract those files from PCAPs as Zeek processes them. This behavior can be enabled globally by modifying the ZEEK_EXTRACTOR_MODE environment variable in docker-compose.yml, or on a per-upload basis for PCAP files uploaded via the browser-based upload form when Analyze with Zeek is selected.

To specify which files should be extracted, the following values are acceptable in ZEEK_EXTRACTOR_MODE:

• none: no file extraction
• interesting: extraction of files with mime types of common attack vectors
• mapped: extraction of files with recognized mime types
• known: extraction of files for which any mime type can be determined
• all: extract all files

Extracted files can be examined through any of the following methods:

• submitting file hashes to VirusTotal; to enable this method, specify the VTOT_API2_KEY environment variable in docker-compose.yml
• scanning files with ClamAV; to enable this method, set the EXTRACTED_FILE_ENABLE_CLAMAV environment variable in docker-compose.yml to true
• scanning files with Yara; to enable this method, set the EXTRACTED_FILE_ENABLE_YARA environment variable in docker-compose.yml to true
• scanning PE (portable executable) files with Capa; to enable this method, set the EXTRACTED_FILE_ENABLE_CAPA environment variable in docker-compose.yml to true

Files which are flagged via any of these methods will be logged as Zeek signatures.log entries, and can be viewed in the Signatures dashboard in OpenSearch Dashboards.

The EXTRACTED_FILE_PRESERVATION environment variable in docker-compose.yml determines the behavior for preservation of Zeek-extracted files:

• quarantined: preserve only flagged files in ./zeek-logs/extract_files/quarantine
• all: preserve flagged files in ./zeek-logs/extract_files/quarantine and all other extracted files in ./zeek-logs/extract_files/preserved
• none: preserve no extracted files

The EXTRACTED_FILE_HTTP_SERVER_… environment variables in docker-compose.yml configure access to the Zeek-extracted files path through the means of a simple HTTPS directory server. Beware that Zeek-extracted files may contain malware. As such, the files may be optionally encrypted upon download.