This article describes how to connect your existing AWS CloudTrail to the Expel Workbench.

If you're...

use...

Notes

connecting an existing CloudTrail

this article

You must already have a trail configured to send CloudTrail events to a S3 bucket.

You must already have an IAM (Identity and Access Management) Role for Workbench to authenticate with. To set this up, follow Step 6 of the AWS CloudTrail - New CloudTrail setup for Workbench article.

setting up new AWS CloudTrails

the onboarding wizard

You can also use the manual connection procedure.

connecting an existing CloudTrail that includes Control Tower

the Control Tower connection steps

Amazon S3 notifications can only be configured for 1 Simple Queue Service (SQS) queue per bucket. This poses a problem if you want to hook up Workbench to your CloudTrail bucket if you're already sending notifications to an SQS queue for internal reasons.

In this article, we show how you can route S3 notifications through SNS (simple notification service) to a different SQS queue that Workbench can poll from.

Connecting your device to Workbench allows Workbench to ingest the logs. AWS logs include a great deal of information that can take hours to manually review. And not all AWS alerts need attention.

Expel collects data through direct API integrations with the AWS platform. Expel supports authentication with an IAM Role (recommended) or IAM User with a set of read-only permissions. To collect data, Expel communicates directly with AWS APIs (like AWS GuardDuty and Inspector) and pulls in CloudTrail data from S3.

Expel processes all product alerts with a library of Expel created rules focused on the MITRE attack framework. This makes it possible for a product alert that wouldn't be reviewed to be elevated to an Expel alert.

Information to record

DField

Description

S3 bucket ARN (Amazon Resource Name)

The unique identifier for the S3 bucket where CloudTrail events are being sent.

SQS queue ARN

The unique identifier for the SQS queue that Workbench uses to consume S3 notifications.

SQS queue URL

The URL for the SQS queue that Workbench uses to consume S3 notifications.

SNS topic ARN

The unique identifier for the SNS topic that routes S3 notifications to the SQS queue.

Step 1: Create a new SNS topic

Caution

Make sure to create the SNS topic in the same region as the S3 bucket CloudTrail events are being sent to!

  1. Navigate to Services > Simple Notification Service > Topics and click Create Topic.

  2. On the next screen, select Standard as the Type and create a Topic Name.

  3. Under Access Policy, select Advanced.

  4. In the JSON editor, paste the below policy substituting the YOUR_TOPIC_ARN and YOUR_S3_ARN fields with your values. This policy allows S3 to publish notifications to the topic for your CloudTrail bucket.

    {
    "Version": "2008-10-17",
    "Id": "expel-topic-policy-ID",
    "Statement": [
      {
       "Sid": "expel-statement-ID",
       "Effect": "Allow",
       "Principal": {
        "AWS":"*"
        },
        "Action": [
         "SNS:Publish"
        ],
        "Resource": "YOUR_TOPIC_ARN",
        "Condition": {
           "ArnLike": { 
               "aws:SourceArn": "YOUR_S3_ARN"
        }
       }
      }
     ]
    }
  5. Click Create Topic.

Step 2: Create an SQS queue for S3 notifications

In this step, we create a new SQS queue for S3 notifications. Workbench polls notifications from this queue to know when new CloudTrail data is added.

Caution

Make sure you create the SQS queue in the same region as the SNS topic and S3 bucket!

  1. Navigate to Services > Simple Queue Service > Queues and click Create queue.

  2. On the next screen, select Standard Queue and name the new queue.

    • Visibility timeout: 30 Seconds.

    • Message retention period: 7 days.

    • Delivery delay: 0 Seconds.

    • Maximum message size: 256 KB.

    • Receive message wait time: 0 Seconds.

  3. Under Access Policy, select Advanced.

  4. In the JSON editor, paste the below policy substituting the YOUR_SQS_QUEUE_ARN and YOUR_SNS_TOPIC_ARN fields with your values.

    {
    "Version": "2012-10-17",
    "Id": "YOUR_QUEUE_ARN/SQSDefaultPolicy",
    "Statement": [
    {
    "Sid": "Sid1572965666162",
    "Effect": "Allow",
    "Principal": {
    "AWS": "*"
    },
    "Action": "SQS:SendMessage",
    "Resource": "YOUR_SQS_QUEUE_ARN",
    "Condition": {
    "StringEquals": {
    "aws:SourceArn": "YOUR_SNS_TOPIC_ARN"
    }
    }
    }
    ]
    }

Step 3: Subscribe SQS queue to SNS topic

Now that we’ve created an SNS topic and SQS queue, we need to configure SNS to send events to the SQS queue.

  1. Navigate to Services > Simple Notification Service > Subscriptions and click Create subscription.

  2. On the next screen, type the required fields to configure the subscription.

    Field

    Value

    Topic ARN

    Your SNS Topic ARN

    Protocol

    Select Amazon SQS

    Endpoint

    Your SNS Queue ARN

    Enable raw message delivery

    Important

    Selecting enable raw message delivery makes sure SNS doesn’t add extra metadata headers to the message when it sends to SQS. Make sure you select this!

  3. Click Create subscription to finish this step.

Step 4: Enable S3 event notifications to SNS topic

In this step, we configure the CloudTrail S3 bucket to send SNS notifications when CloudTrail adds logs to the bucket.

  1. Navigate to Services > S3 > Your S3 CloudTrail Bucket.

  2. Open Properties for your S3 bucket and navigate to Event notifications. Click Create event notification.

  3. On the next screen:

    • Create a name for your notification rule.

    • Select All object create events.

    • Send to SNS topic.

    • Select your SNS topic we created earlier.

Step 5: Grant ExpelIAM Role access to SQS queue

At this point we configured S3 notifications → SNS topic → SQS queue. The final step involves granting the existing ExpelIAM Role access to poll events from the SQS queue and the S3 bucket.

  1. Navigate to Services > IAM > Expel Role.

  2. Create and add a new inline policy to the Expel Role. Grant the policy these permissions:

    • SQS Permissions

      • DeleteMessage

      • DeleteMessageBatch

      • ReceiveMessage

    • S3 Permissions

      • GetObject

    • Expel Permissions

      {
         “Version”: “2012-10-17”,
         “Statement”: [
         {
                 “Action”: [
                 “sqs:DeleteMessage”,
                 “sqs:DeleteMessageBatch”,
                 “sqs:ReceiveMessage”
                 ],
                 “Effect”: “Allow”,
                 “Resource”: “YOUR_SQS_QUEUE_ARN”
          },
          {
                 “Action”: [
                 “s3:GetObject”
          ],
                 “Effect”: “Allow”,
                 “Resource”: “YOUR_S3_BUCKET_ARN/*”
          }
        ]
      }               
  3. Congratulations! You configured S3 notifications to an SQS queue through SNS. Contact your engagement manager for details on how to finish onboarding.

Tip

This article was accurate at the time of writing, but changes happen. If you find the instructions are outdated, leave a description in the comment field below and let us know!