1. Expel Help Center
  2. Connect Your Technology
  3. A-C Integrations
  4. Amazon/AWS
  5. AWS CloudTrail

AWS CloudTrail Manual Setup - Existing Trail

This manual setup guide is for those who have an existing trail available for use in their CloudTrail service. If you would like to use the Wizard instead, see Setup Options.

This guide takes you through required manual configurations to allow Expel to access your S3 bucket securely via an API. It also shows you how to add CloudTrail as a security device in Workbench, which completes the integration and enables you to monitor all activity.

Scope and Limitations

When choosing to set up this integration, remember the following:

  • For those using AWS Organizations, this guide assumes your SNS Topic, SQS Queue and KMS Key encrypting SNS are stored in the same account as your S3 bucket. It also assumes you have encrypted both your CloudTrail and S3 bucket with KMS (Key Management Service). Please see the Reference section for more information.

  • The AWS setup described here is Expel’s recommended best practice.

  • There are a number of ways to customize your AWS configurations based on your own environment or protocols. These steps are intended to show the necessary settings that must be configured in order for your Expel integration to work, with basic instructions for how to do so; they do not cover most of the optional settings or explain every possible option in AWS.

Prerequisites

  1. You must already have an existing trail that is configured to send CloudTrail events to an S3 bucket.
  2. Verify you have the necessary permissions to create and modify IAM policies and roles for your AWS account.
  3. Make sure you are in the management console and that you have your AWS Account ID if you are using AWS Organizations, or are in the proper AWS account if you manage more than one account.
  4. If your organization centralizes CloudTrail logs from multiple subsidiaries into a single S3 bucket, make sure you know the AWS Organizational Unit (OU) ID for the subsidiary.
    • This is an optional prerequisite that enables us to filter logs so that only the relevant data is sent to each subsidiary's Workbench instance, preventing data overlap.
  5. Check the AWS region in the top menu bar (i.e. us-east-2, us-west-1, etc.) and make sure you are in the home region you want to be in; you must use the same home region throughout the entire AWS configuration process, and you must later specify this region correctly in Workbench.
  6. Make sure you can log into Workbench successfully, and are able to see your organization settings.
    • Note: This is a good time to copy and save your Workbench GUID, which is a unique alphanumeric value assigned by Expel to your organization and found in the My Organization page; see this Reference topic for detailed instructions.
  7. Create a new file or other space to keep track of all of the names, ARNs, and other values from the AWS configuration as you go (a list of all needed values is available in the Quick Start, and additional help is available in this Reference section).

Quick Links

Setup includes the following steps (select any step for detailed instructions):

  1. Download the CloudFormation Templates
  2. Make Note of Your Needed Values
  3. Create a New KMS key
  4. Create a New SNS topic
  5. Edit the SNS SQS KMS Key Policy
  6. Create the SQS Queue
  7. Create the SNS Subscription
  8. Enable S3 Event Notifications
  9. Set Up CloudFormation
  10. Add AWS CloudTrail as a Security Device in Workbench
  11. Troubleshooting
  12. Reference

Step 1: Download the CloudFormation Templates

Scroll to the bottom of this page to download the CloudFormationTemplates.zip file, which contains two different templates. You will need one or both CloudFormation templates to complete this guide (depending on how you use AWS).

About the Templates

Template Application Actions
CloudFormationTemplate_Accounts.json
  • For those with a single AWS account and/or those who need to update the management account
  • Used when deploying a stack
  • Creates Expel's IAM role (ExpelAssumeRole) in your management account
  • Creates the required IAM policy (ExpelAccessPolicy); you will define the SQS queue, S3 bucket, and AWS KMS key ARN
CloudFormationTemplate_Orgs.json
  • For those using AWS Organizations
  • Used when deploying a stackset
  • Creates Expel's IAM role (ExpelAssumeRole) in your children accounts; you will create this role in your management account separately
  • Creates the required IAM policy (ExpelAccessPolicy) for your children accounts; you will grant additional access separately

Step 2: Make Note of Your Needed Values

During the AWS configuration process, you will need to copy out and save a number of values as they are added by you (e.g. the AWS KMS Alias) or generated by AWS (e.g. the SQS Queue ARN).

Values you should have already:

  • Workbench Unique ID (if you do not have this value, get it now)
  • AWS Account ID (AWS organizations only; if you do not have this value, get it now)
  • AWS OU ID (only if you use a single S3 bucket for multiple subsidiaries; if you do not have this value, get it now)
  • AWS Region
  • S3 bucket ARN
  • S3 Bucket KMS Key ARN (KMS ARN of the key set as default encryption of your S3 bucket)
  • CloudTrail KMS Key ARN (KMS ARN of the key Cloudtrail uses to encrypt logs that are sent to S3 bucket)

Values you will acquire as you follow this guide:

  • SNS SQS KMS Key ARN
  • SNS Topic ARN
  • SQS Queue ARN
  • SQS URL
  • S3 bucket account IAM Role ARN
  • AWS OU Role ARN (only if you use a single S3 bucket for multiple subsidiaries and you choose to create a separate IAM Role with the additional permissions)

Knowing each of these values is necessary to successfully complete all AWS configuration steps and to also add AWS as a security device in Workbench.

Note:

For examples of what some of these values look like or instructions on where to find them if you forget to save one of them, refer to the Reference section.

Step 3: Create a New KMS Key

First you will create a KMS key to use to encrypt our SNS Topic and SQS queue. Create this key in the same account as the S3 bucket account.

  1. Use the Search bar to quickly navigate to Key Management Service, or find it in the Services menu.
  2. Select Create key.
  3. Leave the Configure key page as is and select Next.
  4. Enter an alias or display name for the key.
  5. Select Next.
  6. Leave the Define key administrative permissions page as is and select Next.
  7. Leave the Define key usage permissions page as is and select Next.
  8. Review your configuration if desired.
  9. Select Finish.
  10. In the key list, select the key you just made. Copy the ARN and save it to a safe place as your SNS SQS KMS Key ARN.

Step 4: Create a New SNS Topic

If you have already configured Amazon Simple Notification Service (SNS) as an event notifier for your existing CloudTrail bucket that includes the event type “All object create events”, skip to Step 5. Otherwise, you will create an SNS topic and edit its access policy to give the S3 bucket permission to push events to the topic. These communications will eventually be sent from the SNS topic to an SQS queue (configured in a later section).

  1. Use the Search bar to quickly navigate to Simple Notification Service (SNS), or find it in the Services menu.
  2. Select Topics.
  3. Select Create topic.
  4. In the Details section, select Standard as the Type.
  5. Enter a topic name in the Name field.
  6. Expand the Encryption section.
    • Toggle Encryption to on.
    • Use the dropdown menu to select the SNS SQS KMS key you created in Step 3.
  7. Expand the Access policy section.
    • Leave the method as Basic.
    • Look at the JSON preview and scroll down to the Resource key.
    • Copy the Resource value and save it to a safe place as your SNS Topic ARN (you will need it to configure the new access policy). The format should look something like this: arn:aws:sns:us-east-1:123456789012:YourTopicName
    • Next, select Advanced to define a new access policy.

    • Highlight and Delete the existing policy and paste the below policy instead. Make sure to use your SNS Topic ARN as the Resource value and your S3 bucket ARN as the aws:SourceARN value.
       

      {
  "Version": "2008-10-17",
  "Id": "expel-topic-policy-ID",
  "Statement": [
    {
      "Sid": "expel-statement-ID",
      "Effect": "Allow",
      "Principal": {
        "Service": "s3.amazonaws.com"
      },
      "Action": [
        "SNS:Publish"
      ],
      "Resource": "YOUR_SNS_TOPIC_ARN",
      "Condition": {
        "ArnLike": {
          "aws:SourceArn": "YOUR_S3_BUCKET_ARN"
        }
      }
    }
  ]
}
  8. Important: Did you use your SNS Topic ARN and S3 bucket ARN in the new code? Check to be sure before continuing.
  9. Leave the remaining sections as is and select Create topic. 

Step 5: Edit the SNS SQS KMS Key Policy

Next, you need to edit the SNS SQS KMS key policy to include proper permissions for our SNS topic and S3 bucket.

  1. Use the Search bar to quickly navigate to Key Management Service, or find it in the Services menu.
  2. Select the key you created in Step 3.
  3. Under Key policy, select Switch to policy view.
  4. Select Edit.
  5. Append the following JSON snippet into the existing policy. Make sure to use your S3 bucket ARN and SNS Topic ARN where specified.
 ,
        {
            "Sid": "Allow cloudtrail bucket to encrypt/decrypt",
            "Effect": "Allow",
            "Principal": {
                "Service": "s3.amazonaws.com"
            },
            "Action": [
                "kms:GenerateDataKey",
                "kms:Decrypt"
            ],
            "Resource": "*",
            "Condition": {
                "ArnLike": {
                    "aws:SourceArn": "YOUR_S3_BUCKET_ARN"
                }
            }
        },
        {
            "Sid": "Allow SNS to encrypt/decrypt",
            "Effect": "Allow",
            "Principal": {
                "Service": "sns.amazonaws.com"
            },
            "Action": [
                "kms:GenerateDataKey",
                "kms:Decrypt"
            ],
            "Resource": "*",
            "Condition": {
                "ArnLike": {
                    "aws:SourceArn": "YOUR_SNS_TOPIC_ARN"
                }
            }
        }

If you need help appending your policy, go to the Troubleshooting section.

  1. Important: Did you use your S3 bucket ARN and SNS Topic ARN in the code? Check to be sure before continuing.
  2. Select Save changes.

Step 6: Create the SQS Queue

Next, you'll create a dedicated SQS queue to receive messages from the SNS topic, and edit the access policy to allow the SNS topic to send messages to it. Expel will use this queue to poll for notifications of new CloudTrail data, and then will update Workbench accordingly.

  1. Use the Search bar to quickly navigate to Simple Queue Service (SQS), or find it in the Services menu.
  2. Select Queues.
  3. Select Create queue.
  4. In the Details section, select Standard as the type.
  5. Enter a name in the Name field.
  6. In the Configuration section:
    • Change the Message Retention period to 7 days.
    • Leave all other defaults.
  7. In the Encryption section:
    • Leave Server-side encryption as Enabled.
    • Choose AWS Key Management Service key (SSE-KMS) as the encryption key type.
    • Choose the SNS SQS KMS key you created in Step 3.
    • Leave the data key reuse period as is.

  8. In the Access policy section:

    • Select Advanced.
    • First, go to the Resource value and copy the ARN, then save it to a safe place as your SQS Queue ARN (you will need it for your new access policy).
    • Highlight and delete the existing policy and paste the below policy instead. Make sure to use your SQS Queue ARN as the Resource value and your SNS Topic ARN as the aws:SourceARN value. 
    {
  "Version": "2012-10-17",
  "Id": "SQSDefaultPolicy",
  "Statement": [
    {
      "Sid": "Sid1572965666162",
      "Effect": "Allow",
      "Principal": {
        "Service": "sns.amazonaws.com"
      },
      "Action": "SQS:SendMessage",
      "Resource": "YOUR_SQS_QUEUE_ARN",
      "Condition": {
        "ArnLike": {
          "aws:SourceArn": "YOUR_SNS_TOPIC_ARN"
        }
      }
    }
  ]
}
  9. Important: Did you use your SQS Queue ARN and SNS Topic ARN in the new code? Check to be sure before continuing.
  10. Leave the remaining sections as is and select Create Queue.
  11. Before leaving the confirmation screen, copy the URL and save it to a safe place as your SQS URL (you will need it in a later section).

Step 7: Create the SNS Subscription

Now that you have an SNS topic and SQS queue, and have also edited both of their access policies to allow for communication, you need to create a new subscription within the SNS topic so that the topic can send events to the queue.

  1. Use the Search bar to quickly navigate to Simple Notification Service (SNS), or find it in the Services menu.
  2. Select Subscriptions.
  3. Select Create subscription.
  4. In the Details section, complete the fields as follows:
    • Topic ARN - Select the Topic ARN you obtained in Step 4.
    • Protocol - Amazon SQS.
    • Endpoint - Select your SQS Queue ARN.
    • Select Enable raw message delivery. Selecting this option ensures SNS doesn’t add metadata to the notifications it sends to SQS, so be sure this is enabled.
  5. Leave the remaining sections as is and select Create subscription.

Step 8: Enable S3 Event Notifications

If you have already configured Amazon Simple Notification Service (SNS) as an event notifier for your existing CloudTrail bucket that includes the event type “All object create events”, skip to Step 9. Now that you have fully configured the communications between the SNS topic and SQS queue, you can create an event notification. This notification will tell the S3 bucket to send certain events (you will specify which ones) to the SNS topic whenever CloudTrail adds event logs to the bucket.

  1. Use the Search bar to quickly navigate to S3, or find it in the Services menu.
  2. Select your S3 bucket.
  3. Select Properties.
  4. Go to the Event Notifications section and select Create event notification. Then:
    • Enter a name.
    • In the Event types section, use the checkbox to enable All object create events. Leave the rest of the boxes unchecked.
    • In the Destination section, select SNS topic and choose your SNS topic from the dropdown.
  5. Select Save changes.

Step 9: Set Up CloudFormation

Expel will authenticate its credentials using an IAM Role and will be granted permissions to your AWS account(s) based on a corresponding IAM policy. You can complete this step most efficiently by using CloudFormation instead of doing a manual configuration via IAM.

Find Your CloudFormation Instructions:

Important note about objects stored in different accounts: The instructions assume that your SNS Topic, SQS Queue and KMS Key encrypting SNS/SQS are stored in the same account as your S3 bucket, with your CloudTrail in the management account. Resources existing in additional accounts will require additional configuration not provided in this guide. Please see the Reference section for additional information.

AWS Single Account Instructions

Creating a stack with our CloudFormation template is the recommended best practice for single accounts because it is a faster configuration process that is less likely to fail due to human error. However, you may use IAM to create your policy and role manually if you wish, then go to Step 12 to continue the process.

Note:

Before you begin, make sure you have your SQS Queue ARN, S3 Bucket ARN, AWS KMS Key ARN, and Workbench GUID. Make sure you have also downloaded and unzipped the CloudFormationTemplates.zip file at the bottom of this page. If you need additional help finding any of the above values, see the Reference for detailed instructions.

  1. Use the Search bar to quickly navigate to CloudFormation, or find it in the Services menu.
  2. Go to Stacks.
  3. Select Create stack. If you see a dropdown menu, choose the "With new resources" option.
  4. For your stack:
    • Leave the Prerequisite as Template is ready.
    • For the template source, choose Upload a template file.
    • Choose the CloudFormationTemplate_Accounts.json template file.
  5. Select Next.
  6. Configure your stack details.
    • Enter a name for your stack.
    • Enter your AWS KMS Key ARN, S3 Bucket ARN, SQS Queue ARN, and Workbench GUID.
  7. Select Next.
  8. Leave all defaults for stack options as is.
  9. Select Next.
  10. Review your configuration if desired, and select Submit.
  11. Now, you must grab your IAM Role ARN (you will need it for the next section). Use the Search bar to quickly navigate to IAM, or find it in the Services menu.
  12. Go to Roles.
  13. Look for ExpelAssumeRole and select it.
  14. In the Summary section, copy the ARN and save it to a safe place as your IAM Role ARN.

You can now skip to Step 10: Add AWS CloudTrail as a Security Device in Workbench.

AWS Organizations Instructions, Objects in Same Account

If you use AWS Organizations and your trail and S3 bucket are in the same account, you will first create a stackset in the management account (this is where the primary Expel role resides) using our CloudFormation template. Then you will create an additional stack, with a different CloudFormation template, that is just for the management organization.

Note:

Before you begin, make sure you have downloaded and unzipped the CloudFormationTemplates.zip file at the bottom of this page. Make sure you also have your SQS Queue ARN, S3 Bucket ARN, AWS KMS Key ARN, and Workbench GUID. If you need additional help finding these values, see the Reference for instructions.

Step 1: Create a Stackset in the Management Account

This stackset allows AWS to automatically replicate and deploy the template's IAM role and policy via stacks to all of the other accounts in your organization. The permissions in the template enable Expel to perform all necessary Investigative Actions, and also prevent you from having to update or redo the configuration if a new account is added to your organization. 

  1. Use the Search bar to quickly navigate to CloudFormation, or find it in the Services menu.
  2. Go to StackSets.
  3. Select Create StackSet. If you see a dropdown menu, choose the "With new resources" option.
  4. For the template:
    • Leave Permissions as Service-managed permissions.
    • Leave the Prerequisite as Template is ready.
    • For the template source, choose Upload a template file.
    • Choose the CloudFormationTemplate_Orgs.json template file.
  5. Select Next.
  6. Configure your stackset details.
    • Enter a name for your stackset.
    • Enter your Workbench GUID.
  7. Select Next.
  8. Leave Tags and Execution configuration as is.
  9. Select Next.
  10. For your deployment options:
    • Scroll down to Specify regions and choose your AWS region.
    • Leave all other defaults on this page as is.
  11. Select Next.
  12. Review your configuration if desired, and select Submit.

Step 2: Create an Additional Stack

This stack will enable additional permissions related to the SQS Queue ARN, S3 Bucket ARN, and KMS Key ARN.

  1. Use the Search bar to quickly navigate to CloudFormation, or find it in the Services menu.
  2. Go to Stacks.
  3. Select Create stack. If you see a dropdown menu, choose the "With new resources" option.
  4. For your stack:
    • Leave the Prerequisite as Template is ready.
    • For the template source, choose Upload a template file.
    • Choose the CloudFormationTemplate_Accounts.json template file.
  5. Select Next.
  6. Configure your stack details.
    • Enter a name for your stack.
    • Enter your AWS KMS Key ARN, S3 Bucket ARN, SQS Queue ARN, and Workbench GUID.
  7. Select Next.
  8. Leave all defaults for stack options as is.
  9. Select Next.
  10. Review your configuration if desired, and select Submit.
  11. Now, you must grab your IAM Role ARN (you will need it for the next section). Use the Search bar to quickly navigate to IAM, or find it in the Services menu.
  12. Go to Roles.
  13. Look for ExpelAssumeRole and select it.
  14. In the Summary section, copy the ARN and save it to a safe place as your IAM Role ARN.

Important

If your organization centralizes CloudTrail logs from multiple subsidiaries into a single S3 bucket, you will need to either edit this IAM Role to add two additional permissions for Expel, or create a separate IAM role with those permissions.

  • The two additional permissions are: organizations:ListAccountsForParent and organizations:ListOrganizationalUnitsForParent.
  • If you choose to create a separate IAM Role with these permissions, make sure to save that IAM Role ARN as your AWS OU Role ARN. You will need it when you set up the security device in Workbench. 

You can now skip to Step 10: Add AWS CloudTrail as a Security Device in Workbench.

AWS Organizations Instructions, Objects in Different Accounts

If you use AWS Organizations and your trail and S3 bucket are not in the same account, you must use CloudFormation templates to deploy a generic IAM policy via a stack in your management account and a generic IAM policy via a stackset to all of your children accounts, set additional policy permissions in the child account to enable communication with the S3 bucket, and update the KMS Key Policy in the management account.

Note: 

Make sure you have your SQS Queue ARN (from Step 6), S3 Bucket ARN (pre-existing), AWS KMS Key ARN your trail uses to encrypt S3 bucket logs (pre-existing), Workbench GUID, and the CloudFormationTemplates (the zip file at the bottom of this page). If you need additional help finding any of the above values, see the Reference for detailed instructions.

Step 1: Create a Stack in the Management Account

The permissions in the CloudFormation templates contain generic IAM policies that enable Expel to perform all necessary Investigative Actions. In this step, you will create the generic IAM policies in the management account.

  1. Log into your management account. Please also ensure you are still in the same region as in previous steps.
  2. Use the Search bar to quickly navigate to CloudFormation, or find it in the Services menu.
  3. Go to Stacks.
  4. Select Create stack. If you see a dropdown menu, choose the "With new resources" option.
  5. For your stack:
    • Leave the Prerequisite as Choose an existing template.
    • For the template source, choose Upload a template file.
    • Choose the CloudFormationTemplate_Accounts.json template file.
  6. Select Next.
  7. Configure your stack details.
    • Enter a name for your stack.
    • Enter the KMS Key ARN your trail uses to encrypt S3 bucket logs, S3 Bucket ARN, SQS Queue ARN, and Workbench GUID.
  8. Select Next.
  9. Leave all defaults for stack options as is.
  10. Select Next.
  11. Review your configuration if desired, and select Submit. 

Step 2: Create a Stackset in the Management Account

You will use the other CloudFormation template to create the generic IAM policies in all children accounts, including in the S3 bucket account.

  1. Use the Search bar to quickly navigate to CloudFormation, or find it in the Services menu.
  2. Go to StackSets.
  3. Select Create StackSet. If you see a dropdown menu, choose the "With new resources" option.
  4. For the template:
    • Leave Permissions as Service-managed permissions.
    • Leave the Prerequisite as Template is ready.
    • For the template source, choose Upload a template file.
    • Choose the CloudFormationTemplate_Orgs.json template file.
  5. Select Next.
  6. Configure your stackset details.
    • Enter a name for your stackset.
    • Enter your Workbench GUID.
  7. Select Next.
  8. Leave Tags and Execution configuration as is.
  9. Select Next.
  10. For your deployment options:
    • Scroll down to Specify regions and choose your AWS region.
    • Leave all other defaults on this page as is.
  11. Select Next.
  12. Review your configuration if desired, and select Submit.

Step 3: Edit the S3 Bucket Role in the Child Account

You will now add a new inline policy to a role in the S3 bucket account, which will allow permissions to other resources.

  1. Log into the child account containing the S3 bucket.
  2. Use the Search bar to quickly navigate to IAM, or find it in the Services menu.
  3. Select Roles.
  4. In the Roles list, search for or select ExpelAssumeRole.
  5. In the Summary section, copy the ARN and save it to a safe place as your S3 bucket account IAM Role ARN. You will need this value later when you add the CloudTrail in Workbench.
  6. In the Permissions policies section, select Add permissions > Create inline policy.
  7. In the JSON editor section, select JSON.

  8. Highlight and Delete the existing policy, and paste the below policy instead. Make sure to enter your own values for the SQS ARN (Step 6), S3 bucket ARN, KMS ARN, SNS SQS KMS ARN (Step 3), and KMS ARN.

     

    {
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "sqs:DeleteMessage",
        "sqs:ReceiveMessage"
      ],
      "Resource": "YOUR_SQS_QUEUE_ARN"
    },
    {
      "Effect": "Allow",
      "Action": [
        "s3:GetObject"
      ],
      "Resource": "YOUR_S3_BUCKET_ARN/*"
    },
    {
      "Effect": "Allow",
      "Action": [
        "kms:Decrypt"
      ],
      "Resource": "YOUR_SNS_SQS_KMS_ARN"
    },
    {
      "Effect": "Allow",
      "Action": [
        "kms:Decrypt"
      ],
      "Resource": "YOUR_CLOUDTRAIL_KMS_KEY_ARN"
    }
  ]
}

     

  9. Important: Did you use your own resource values in the policy? Check to be sure before continuing.
  10. Select Next.
  11. Name your policy.
  12. Select Create policy.

Step 4: Add or Edit an IAM Role

If your organization centralizes CloudTrail logs from multiple subsidiaries into a single S3 bucket, you will need to either edit the associated IAM Role to add two additional permissions for Expel, or create a separate IAM role with the following two permissions:

  • organizations:ListAccountsForParent
  • organizations:ListOrganizationalUnitsForParent

If you choose to create a separate IAM Role with these permissions, make sure to save that IAM Role ARN as your AWS OU Role ARN. You will need it when you set up the security device in Workbench.

If this situation does not apply to you, continue to the next step.

Step 5: Update the KMS Key Policy

You must now update the KMS Key Policy of the key that your trail uses to encrypt S3 bucket logs, so that the S3 bucket's Expel IAM Role has permission to decrypt:

  1. Switch to your management account.

  2. Use the Search bar to quickly navigate to Key Management Service, or find it in the Services menu.

  3. Select the KMS key encrypting your CloudTrail.

  4. In the Key policy section, select Edit.

  5. Append the following JSON snippet to the Key policy, being sure to insert the account ID of the child account your S3 bucket resides in where specified:

    ,
       {
           "Sid": "Allow Expel to use this KMS key to decrypt logs",
           "Effect": "Allow",
           "Principal": {
               "AWS": "arn:aws:iam::YOUR_S3_BUCKET_ACCOUNT_ID:role/ExpelAssumeRole"
           },
           "Action": "kms:Decrypt",
           "Resource": "*"
       }
    If you need help appending JSON snippets, see the Troubleshooting for an example.
  6. Important: Did you use the account ID for the account your S3 bucket resides in in the new code? Check to be sure before continuing.
  7. Select Save changes.

You can now go to Step 10.

Step 10: Add AWS CloudTrail as a Security Device in Workbench

Now, you can add a security device in Workbench to complete the integration. Before you begin, make sure you have your IAM Role ARN and SQS URL. If you use AWS Organizations, you will also need your AWS Account ID as well as the subsidiary's AWS Organizational Unit (OU) ID (the OU ID is only necessary if you use a single S3 bucket for multiple subsidiaries). If you need additional help finding any of these values, see the Reference for detailed instructions.

  1. Log in to Workbench.
  2. In the side menu, navigate to Organization Settings > Security Devices.
  3. Select Add Security Device.
  4. Select AWS Cloudtrail (or search for it if you do not see it listed). Then select the bullets as follows:
    • “Are you using AWS organizations?” - AWS Organizations should leave it as Yes; single accounts should select No.
    • “Do you have an existing CloudTrail that you want Expel to reuse?” - leave it as Yes.
    • “How would you like to connect?” - select Manual connection.
    • Select Save.
  5. In the next screen, complete the fields as follows:
    • Name - enter a name that might help you more easily identify this integration, such as “CompanyName CloudTrail”; this name will display in Workbench under the Name column, and is a text string that you can filter on.
    • Location - enter the location of your integration, for example “cloud” or “AWS cloud” or “on prem;” this is also a text string that you can filter on, so we recommend being consistent with location naming across your Expel integrations.
    • Role ARN - enter the IAM Role ARN you saved from the last section. Format: arn:aws:iam::123456789012:role/RoleName
    • Role session name - enter a text string to label your IAM role session. Example: orgname-expel-trail-session
    • AWS region - choose the AWS region where you created your configuration in AWS.
    • SQS URL - enter the SQS URL you copied and saved in a previous step.
    • Organization management account - if you use AWS Organizations, input the AWS Account ID of the management account; single account users can leave this field blank.
    • AWS OU ID - if you use AWS Organizations to centralize CloudTrail logs from multiple subsidiaries into a single S3 bucket, enter the AWS OU ID for the relevant subsidiary (format: ou-abc1-defghi2j); otherwise, leave this field blank. This field tells our system to filter the logs based on that ID, ensuring that each subsidiary's Workbench instance only ingests and analyzes the logs relevant to its own accounts, which helps prevent any overlap of data.
    • AWS OU Role ARN - if you entered an AWS OU ID, you will also need to enter the IAM Role ARN that contains the appropriate permissions. If you chose to add these permissions to your existing IAM Role, enter the IAM Role ARN again here. If you chose to create a new IAM Role with those permissions, enter your AWS OU Role ARN. Otherwise, leave this field blank.
    • Select Save.
  6. Your device should be created successfully within a few seconds. A few reminders:
    • After your connection is healthy, it will take some time for your device to begin polling and receiving data.
    • To check on the status, select the downward arrow for your device in the first column and choose View details. You can then scroll to the Connection section to see if your device is fully connected.
    • Polling will happen first; data will be received after that. You must refresh the page to see updates.
    • If your device does not begin polling within 15 minutes, and does not begin receiving data within 30 minutes, contact our support team for help.

Troubleshooting

If you are encountering an error or your Trail isn't connecting to Workbench, try checking for some common issues:

  • You left one of your ARN values out of a JSON object in a prior step, put an ARN in the wrong location, or formatted an ARN incorrectly.
  • The JSON objects were copied, appended, or formatted incorrectly in a prior step.
  • You have forgotten to enable something, or have configured something incorrectly, in one of the prior steps.
    • Examples: Not choosing your AWS KMS key from the dropdown menu when setting up default encryption in Step 4, or not using the same region throughout your AWS configuration.

Appending JSON Code

Make sure to put this JSON snippet in the correct place, using a comma, and make sure the final bracket is still there. Lines 13-47 of your JSON code should look like this (the appended part is in orange):

{
    "Version": "2012-10-17",
    "Id": "key-consolepolicy-3",
    "Statement": [
        {
            "Sid": "Enable IAM User Permissions",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::241008450902:root"
            },
            "Action": "kms:*",
            "Resource": "*"
        },
       {
           "Sid": "Allow cloudtrail bucket to encrypt/decrypt",
           "Effect": "Allow",
           "Principal": {
               "Service": "s3.amazonaws.com"
           },
           "Action": [
               "kms:GenerateDataKey",
               "kms:Decrypt"
           ],
           "Resource": "*",
           "Condition": {
               "ArnLike": {
                   "aws:SourceArn": "YOUR_S3_BUCKET_ARN"
               }
           }
       },
       {
           "Sid": "Allow SNS to encrypt/decrypt",
           "Effect": "Allow",
           "Principal": {
               "Service": "sns.amazonaws.com"
           },
           "Action": [
               "kms:GenerateDataKey",
               "kms:Decrypt"
           ],
           "Resource": "*",
           "Condition": {
               "ArnLike": {
                   "aws:SourceArn": "YOUR_SNS_TOPIC_ARN"
               }
           }
       }
    ]
}

Reference

Cross-Account Configuration Resources

The diagram below illustrates the relationship and location of AWS resources for a possible environment covered in this guide: an existing CloudTrail configuration involving AWS Organizations where the trail is in the management account and the S3 bucket, SNS topic, and SQS queue are all together in a child account. KMS encryption can be applied as illustrated and IAM roles are deployed to the accounts using CloudFormation Stacks and Stacksets.

AWS Cloudtrail design schematic - Data flow & IAM (1).png

For more information on configuring a cross-account environment, refer to these AWS developer guide pages:

ARNs and URLs

This chart gives examples of ARNs and the SQS URL so you can check on the general formatting, and also tells you where to find any of the values if you’ve forgotten to copy and save them during the configuration process. 

 

Value Where to Find Example
S3 Bucket ARN S3 > Buckets > bucket name > Properties arn:aws:s3:::MyS3BucketARN
SNS Topic ARN Simple Notification Service > Topics > topic name > Details section arn:aws:sns:us-east-1:123456789012:MyTopicName
AWS KMS Key ARN Key Management Service > key alias > General configuration section arn:aws:kms:us-west-2:123456789012:key/123a4567-890b-1234-c5d6-7ef89012g345
SQS Queue ARN Simple Queue Service > queue name > Details section arn:aws:sqs:us-east-1:123456789012:MySQSQueueName
SQS URL Simple Queue Service > queue name > Details section https://sqs.us-west-1.amazonaws.com/123456789012/MySQSQueueName
IAM Role ARN IAM > Access Management > Roles > role name > Summary section arn:aws:iam::123456789012:role/RoleName

(most users will look for the ExpelAssumeRole)

AWS Organization-Specific Values

This chart helps you locate the additional values you need if you are using AWS Organizations. The OU ID is only necessary if you use a single S3 bucket for multiple subsidiaries.

Value Where to Find Example
AWS Account ID AWS Organizations Console, upper right of screen 1234-5678-9012
AWS OU ID AWS Organizations Console

ou-abc1-defghi2j

(format is ou-rootid-ouid and this entire value should be entered into the Workbench security device)

Workbench GUID

The Workbench GUID is a unique alphanumeric value assigned by Expel to your organization.

Format

a123b456-7c89-0def-g1hi-2j3k45l6mn7o

Where to Find

  1. Log in to Workbench.
  2. Go to Organization Settings > My Organization.
  3. On the organization's page, look for the Organization GUID and select the Copy button to copy the GUID.

Note

If you have multiple organizations, you must first select the organization name that will be associated with your CloudTrail integration to access the page with the Copy button. Or, you can stay on the page and highlight then copy the GUID shown for that organization in the GUID column.

 

Finished downloading the template files? Return to Step 1.

Related articles