This manual setup guide is for those who have an existing trail that includes Control Tower available for use in their CloudTrail service. This article assumes you are using the default service control policies. If you encounter problems during setup, contact support for help.
Prerequisites
-
You must already have an AWS Control Tower with Landing Zone.
-
You must have Administrator privileges in the AWS Control Tower management and Log Archive accounts.
- You must download the attachment found at the end of this article, which includes the code blocks you need to complete this process.
Quick Start
Step 1: Set Up ExpelIAM Role and Policy
In this step we create a permissions policy to assign to the IAM Role.
For AWS Control Tower, the primary Expel role is created within the organization’s Log Archive account where the CloudTrail S3 Log Bucket exists.
The role and policy must be replicated across all the other accounts in the organization to allow Expel to perform investigative actions within your AWS environment. The policy replicated to the other non-Log Archive sub-accounts can be modified to exclude the policy elements providing access permissions to the S3 resources if necessary.
-
Create IAM Role and Policy in management account.
-
Navigate to the CloudFormation > Stacks service portal.
-
Click Create Stack (with new resources) and select Upload a template file as the source.
-
Upload a JSON file containing the stack template. You can find that code in the attachment for this article in the Step 1 area.
-
Provide a relevant name to the stack, such as ExpelIAMStack.
-
In the WorkbenchExternalID parameter field, type in your Workbench GUID, then click Next.
Note
You can find this GUID by logging into Workbench and then navigating to https://workbench.expel.io/settings/organizations. The redirected URL contains the value to be used for this parameter.
-
You can leave all defaults for configuring the stack options and click Next.
-
Verify the stack details, then check the acknowledgement and click Submit.
-
The new IAM role and attached policy should exist for Expel within the management account on completion of the stack.
-
-
Create IAM Role and Policy in all sub accounts.
-
Navigate to the CloudFormation > Stacksets service portal.
-
Click Create StackSet and select Upload a template file as the source.
-
Follow the same process to fill out the stackset details as was done in deploying the stack above (Step 1.1).
-
Step 2: Update CloudTrail CMK Policy If Applicable
Note
You can proceed directly to Step 3 if your CloudTrail is not configured to encrypt the S3 logs using CMK. Otherwise, we need to update the key policy to provide Workbench with the kms:Decrypt permission so that we can properly get objects from the S3 bucket containing the CloudTrail logs.
You can determine if this step applies to you by navigating to Services > CloudTrail > Trails and then selecting the trail created by Control Tower in your management account. In the General details section you see a value under Log file SSE-KMS encryption and an associated AWS KMS key populated as well.
-
Navigate to the CMK key used to encrypt the CloudTrail logs.
-
You can find the specific key arn by looking for AWS KMS Key in the General details section of the CloudTrail.
-
-
Select the Key policy tab and then click Edit to change it.
-
Add the decrypt permission to the existing list of policy statements. You can find that code in the attachment for this article in the Step 2 area.
Note
Make sure to update the principal value with the correct ARN path to the Expel Role created in the Log Archive account as a result of Step 1.2 via the StackSet.
-
Click Save changes.
Step 3: Confirm S3 Log Bucket ACL Settings
This step (through Step 8) must be performed from within your AWS Control Tower Log Archive account. See the AWS documentation for any clarification.
Amazon S3 access control lists allow you to manage access to buckets and any objects contained within. We want to confirm that the S3 bucket created in your log archive account allows the bucket owner (Log Archive) and Expel by extension through the deployed role read permissions for the objects within.
-
Navigate to Services > S3 > Buckets > [Your S3 Log Bucket] (usually follows the naming scheme “aws-controltower-logs-”.
-
Select the Permissions tab, then look at the Object Ownership section to confirm the current ACL setting.
-
If you keep ACLs enabled on the bucket then confirm the current Object Ownership is set to Bucket owner preferred. AWS recommends that you disable ACLs on S3 for a majority of modern use-cases which then delegate ownership to the bucket owner account (source).
Step 4: Set Up SNS Topic for S3 Notifications
Option 1: Using CloudFormation (Recommended)
-
Navigate to Services > CloudFormation > Stacks and click Create stack (with new resources).
-
In the Specify template section select Upload a template file. Then upload a JSON file containing the stack template. Then click Next. You can find that code in the attachment for this article in the Step 4, Option 1 area.
-
Fill in a name for the stack.
-
Fill in the following required parameters.
-
S3LogBucketARN: the ARN associated with the S3 Bucket in your Log Archive account that retains the CloudTrail logs.
-
CloudTrailKeyARN: the ARN for the customer-managed kms key configured on your CloudTrail (in your management account).
-
-
Proceed through the rest of the wizard keeping the defaults, then click Submit to initiate the stack.
-
After the stack creation completes, make note of all values returned in the Outputs tab of the stack console. You need these returned values to complete setup in Workbench.
-
RoleARN: the IAM Role created in the log archive account needed to complete setup in Workbench.
-
SqsURL: the SQS url path needed to complete setup of in Workbench.
-
- Proceed directly to Step 8: Enable S3 event notifications to SNS topic.
Option 2: Manual Setup
Make sure to create the SNS topic in the same region as the S3 bucket CloudTrail events are being sent to!
-
Navigate to Services > Simple Notification Service > Topics and click Create Topic.
-
On the next screen, select Standard as the Type and create a Topic Name.
-
Under Access Policy, select Advanced.
In the JSON editor, paste the below policy substituting the YOUR_TOPIC_ARN and YOUR_S3_ARN fields with your values. This policy allows S3 to publish notifications to the topic for your CloudTrail bucket. You can find that code in the attachment for this article in the Step 4, Option 2 area.
-
Click Create Topic.
Step 5: Set Up SQS Queue for SNS Notification
In this step, we create a new SQS queue for S3 notifications. Workbench polls notifications from this queue to know when new CloudTrail data is added.
Make sure you create the SQS queue in the same region as the SNS topic and S3 bucket.
-
Navigate to Services > Simple Queue Service > Queues and click Create queue.
-
On the next screen, select Standard Queue and name the new queue.
-
Visibility timeout: 30 Seconds.
-
Message retention period: 7 days.
-
Delivery delay: 0 Seconds.
-
Maximum message size: 256 KB.
-
Receive message wait time: 0 Seconds.
-
-
Under Access Policy, select Advanced.
-
In the JSON editor, paste the policy substituting the YOUR_SQS_QUEUE_ARN and YOUR_SNS_TOPIC_ARN fields with your values. You can find that code in the attachment for this article in the Step 5 area.
Step 6: Subscribe SQS to SNS Topic
Now that we created an SNS topic and SQS queue, we need to configure SNS to send events to the SQS queue.
-
Navigate to Services > Simple Notification Service > Subscriptions and click Create subscription.
-
On the next screen, configure the required fields to complete the subscription.
Field
Value
Topic ARN
Your SNS Topic ARN
Protocol
Select Amazon SQS
Endpoint
Your SNS Queue ARN
Enable raw message delivery
Selecting enable raw message delivery makes sure SNS doesn’t add extra metadata headers to the message when it sends to SQS. Make sure you select this! -
Click Create subscription to finish this step.
Step 7: Set Up KMS Encryption for SNS and SQS
This step must be performed from within your AWS Control Tower Log Archive account.
-
Navigate to Services > Key Management Service (KMS) > Customer managed keys and click Create Key.
-
Retain the default configuration values as shown below, then click Next.
-
Fill in values for the alias and description for the key, then click Next.
-
Select any additional key administrators for the new key, then click Next until you get to the Review page.
-
Look for the Key policy generated for this key in the Review page and add the policies to the statement list to give S3 & SNS the right permissions they need to decrypt with this key. You can find that code in the attachment for this article in the Step 7 area.
-
Click Finish to complete the creation of the new key.
-
Enable Encryption on the SNS topic you created in Step 3 using the new key.
-
Navigate to Services > Simple Notification Service > Topics > YourTopic.
-
Click Edit, then click Encryption on.
-
In the Customer master key (CMK) selection, click the KMS key you created above.
-
Click Save changes.
-
-
Enable Encryption on the SQS topic you created in Step 4 using the new key.
-
Navigate to Services > Simple Queue Service > Queues > YourQueue.
-
Click Edit, then toggle the Encryption option on.
-
Set the Server-side encryption option to Enabled.
-
Set the Encryption key type to SSE-KMS.
-
In the Customer master key selection, click the KMS key you created above.
-
Click Save.
-
Step 8: Enable S3 Event Notifications to SNS Topic
In this step, we configure the CloudTrail S3 bucket to send SNS notifications when CloudTrail adds logs to the bucket.
-
Navigate to Services > S3 > Your S3 CloudTrail Bucket.
-
Open Properties for your S3 bucket and navigate to Event notifications. Click Create event notification.
-
On the next screen:
-
Create a name for your notification rule.
-
Select All object create events from the Event types section.
-
Select SNS topic from the Destination section.
-
Select your SNS topic created in Step 3.
-
Click Save changes.
-
Step 9: Grant ExpelIAM Role Necessary Access
If you onboarded the Expel resources with CloudFormation using Step 4, Option 1, proceed directly to Step 10.
At this point we configured S3 notifications → SNS topic → SQS queue. The final step involves granting the existing ExpelIAM Role the necessary access to poll events from the SQS queue and the S3 bucket.
-
Navigate to Services > IAM > Roles.
-
Create and add a new inline policy to the Expel Role that was propagated to the Log Archive account as part of Step 1.
Note
Name it ExpelAssumeRole if you used the provided StackSet template. The policy grants the permissions. You can find that code in the attachment for this article in the Step 9 area.
Step 10: Complete Workbench Setup
Congratulations! You configured S3 notifications to an SQS queue through SNS. Go to Step 8 of the AWS CloudTrail Manual Setup - New Trail page to add the integration as a security device in Workbench. Make sure to select Manual Connection for the connection type.
You require the following details to complete this step:
-
Role ARN: Expel Role ARN created in Step 1 specific to the Log Archive account.
-
SQS URL: the full URL path of the SQS created in Step 4.
-
Organization Management Account: the account ID of your AWS organization’s management account.
Reference
Permissions description and explanation
Expel requires these permissions when connecting to AWS CloudTrail. These permissions are set to Read, except as shown below.
This permission... |
does this... |
Notes |
---|---|---|
ec2:DescribeRegions |
Dynamically list all enabled regions for a customer account |
|
ec2:DescribeInstances |
Cloud investigative actions and inventory |
Allows SOC analysts to triage issues and get additional content for the environment. |
ec2:DescribeSecurityGroups |
||
iam:List* |
||
iam:Get* |
||
lambda:GetFunction |
||
rds:DescribeDBInstances |
||
rds:ListTagsForResource |
||
eks:DescribeCluster |
||
eks:ListClusters |
||
ecs:ListContainerInstances |
||
ecs:DescribeContainerInstances |
||
ecs:DescribeClusters |
||
ecs:ListClusters |
||
s3:GetBucketNotification |
||
s3:GetEncryptionConfiguration |
||
cloudtrail:GetTrail |
||
cloudtrail:DescribeTrails |
||
cloudtrail:GetTrailStatus |
||
cloudtrail:ListTrails |
||
organizations:ListAccounts |
Enumerate accounts in an org |
|
sqs:DeleteMessage |
Remove Processed messages from the queue |
Requires Write permission. |
sqs:ReceiveMessage |
Read messages |
|
s3:GetObject |
Download cloudtrail log files |
|
ec2:DescribeVolumes |
Cloud investigative actions and inventory |
Allows SOC analysts to triage issues and get additional content for the environment. |
ecs:DescribeTaskDefinition |
Inventory |
|
ecs:ListTaskDefinitions |
||
config:ListDiscoveredResources |
||
config:GetDiscoveredResourceCounts |
||
lambda:ListFunctions |
Cloud investigative actions and inventory |
Allows SOC analysts to triage issues and get additional content for the environment. |
lightsail:GetInstances |
||
lightsail:GetRegions |
||
s3:ListAllMyBuckets |
||
kms:Decrypt |
Decrypt S3 buckets |
|
organizations:DescribeOrganization |
Organization information |
Retrieves information about the organization the user's account belongs to. |