A powerful, easily deployable network traffic analysis tool suite
As several of the sections in this document will reference adding new data source fields, we’ll cover that here at the beginning.
Although OpenSearch is a NoSQL database and as-such is “unstructured” and “schemaless,” in order to add a new data source field you’ll need to define that field in a few places in order for it to show up and be usable throughout Malcolm. Minimally, you’ll probably want to do it in these three files:
arkime/etc/config.ini
- follow existing examples in the [custom-fields]
and [custom-views]
sections in order for Arkime to be aware of your new fieldsarkime/wise/source.zeeklogs.js
- add new fields to the allFields
array for Malcolm to create Arkime value actions for your fieldsdashboards/templates/composable/component/__(name)__.json
- add new fields to a new composable index template file in this directory and add its name (prefixed with custom_
) to the composed_of
section of dashboards/templates/malcolm_template.json
in order for it to be included as part of the arkime_sessions3-*
index template used by Arkime and OpenSearch Dashboards in MalcolmWhen possible, I recommend you to use (or at least take inspiration from) the Elastic Common Schema (ECS) Reference when deciding how to define new field names.
If your new log fields are coming from a Zeek script where they are defined in a record
as &log
fields, the script scripts/zeek_script_to_malcolm_boilerplate.py
may help you by autogenerating the parts of the files mentioned above.