Logo

A powerful, easily deployable network traffic analysis tool suite

Quick Start

Documentation

Components

Supported Protocols

Configuring

Arkime

Dashboards

Hedgehog Linux

Contribution Guide

Deploying Malcolm in Other Third-Party Environments

Generating a Malcolm Amazon Machine Image (AMI) for Use on Amazon Web Services (AWS)

This section outlines the process of using packer’s Amazon AMI Builder to create an EBS-backed Malcolm AMI for either the x86-64 or arm64 CPU architecture. This section assumes you have good working knowledge of Amazon Web Services (AWS).

Prerequisites

Procedure

The files referenced in this section can be found in scripts/third-party-environments/aws/ami.

  1. Copy packer_vars.json.example to packer_vars.json
     $ cp ./packer_vars.json.example ./packer_vars.json
    
  2. Edit packer_vars.json
    • set aws_access_key, aws_secret_key, vpc_region, instance_arch, and other variables as needed
  3. Validate the packer configuration
     $ packer validate packer_build.json
     The configuration is valid.
    
  4. Launch packer to build the AMI
     $ packer build -var-file=packer_vars.json packer_build.json
    
     amazon-ebs: output will be in this color.
    
     ==> amazon-ebs: Prevalidating any provided VPC information
     ==> amazon-ebs: Prevalidating AMI Name: malcolm-v24.06.0-arm64-2024-05-30T13-57-31Z
         amazon-ebs: Found Image ID: ami-xxxxxxxxxxxxxxxxx
    
     ...
    
     ==> amazon-ebs: Waiting for AMI to become ready...
     ==> amazon-ebs: Skipping Enable AMI deprecation...
     ==> amazon-ebs: Adding tags to AMI (ami-xxxxxxxxxxxxxxxxx)...
     ==> amazon-ebs: Tagging snapshot: snap-xxxxxxxxxxxxxxxxx
     ==> amazon-ebs: Creating AMI tags
         amazon-ebs: Adding tag: "Malcolm": "idaholab/Malcolm/v24.06.0"
         amazon-ebs: Adding tag: "source_ami_name": "amzn2-ami-kernel-5.10-hvm-2.0.20240521.0-arm64-gp2"
     ==> amazon-ebs: Creating snapshot tags
     ==> amazon-ebs: Terminating the source AWS instance...
     ==> amazon-ebs: Cleaning up any extra volumes...
     ==> amazon-ebs: No volumes to clean up, skipping
     ==> amazon-ebs: Deleting temporary keypair...
     Build 'amazon-ebs' finished after 23 minutes 58 seconds.
    
     ==> Wait completed after 23 minutes 58 seconds
    
     ==> Builds finished. The artifacts of successful builds are:
     --> amazon-ebs: AMIs were created:
     us-east-1: ami-xxxxxxxxxxxxxxxxx
    
  5. Use aws (or the Amazon EC2 console) to verify that the new AMI exists
     $ aws ec2 describe-images --owners self --filters "Name=root-device-type,Values=ebs" --filters "Name=name,Values=malcolm-*"
    
     {
         "Images": [
             {
                 "Architecture": "arm64",
                 "CreationDate": "2024-05-30T14:02:21.000Z",
                 "ImageId": "ami-xxxxxxxxxxxxxxxxx",
                 "ImageLocation": "xxxxxxxxxxxx/malcolm-v24.06.0-arm64-2024-05-30T13-57-31Z",
                 "ImageType": "machine",
                 "Public": false,
                 "OwnerId": "xxxxxxxxxxxx",
                 "PlatformDetails": "Linux/UNIX",
                 "UsageOperation": "RunInstances",
                 "State": "available",
                 "BlockDeviceMappings": [
                     {
                         "DeviceName": "/dev/xvda",
                         "Ebs": {
                             "DeleteOnTermination": true,
                             "SnapshotId": "snap-xxxxxxxxxxxxxxxxx",
                             "VolumeSize": 20,
                             "VolumeType": "gp2",
                             "Encrypted": false
                         }
                     }
                 ],
                 "EnaSupport": true,
                 "Hypervisor": "xen",
                 "Name": "malcolm-v24.06.0-arm64-2024-05-30T13-57-31Z",
                 "RootDeviceName": "/dev/xvda",
                 "RootDeviceType": "ebs",
                 "SriovNetSupport": "simple",
                 "Tags": [
                     {
                         "Key": "Malcolm",
                         "Value": "idaholab/Malcolm/v24.06.0"
                     },
                     {
                         "Key": "source_ami_name",
                         "Value": "amzn2-ami-kernel-5.10-hvm-2.0.20240521.0-arm64-gp2"
                     }
                 ],
                 "VirtualizationType": "hvm",
                 "BootMode": "uefi",
                 "SourceInstanceId": "i-xxxxxxxxxxxxxxxxx",
                 "DeregistrationProtection": "disabled"
             }
         ]
     }
    
  6. Launch an instance from the new AMI
    • for x86-64 instances c4.4xlarge, t2.2xlarge, and t3a.2xlarge seem to be good instance types for Malcolm
    • for arm64 instances, m6gd.2xlarge, m6g.2xlarge, m7g.2xlarge, and t4g.2xlarge seem to be good instance types for Malcolm
    • see recommended system requirements for Malcolm
  7. SSH into the instance
  8. Run ~/Malcolm/scripts/configure to configure Malcolm
  9. Run ~/Malcolm/scripts/auth_setup to set up authentication for Malcolm
  10. Run ~/Malcolm/scripts/start to start Malcolm

Attribution

Amazon Web Services, AWS, the Powered by AWS logo, and Amazon Machine Image (AMI) are trademarks of Amazon.com, Inc. or its affiliates. The information about providers and services contained in this document is for instructional purposes and does not constitute endorsement or recommendation.