This article describes how to troubleshoot AWS CloudTrail errors.

CloudTrail Log File Read Is Forbidden

Error name: AWS_FILE_READ_FORBIDDEN

What’s wrong: The Expel role in your AWS environment doesn't have permission to read the CloudTrail log file. Make sure the get object permission is properly applied.

Steps to fix:

  1. Log in to your AWS instance.

  2. Navigate to your IAM service.

  3. View your Expel role. This is usually named ExpelAssumeRole.

    Read about the Expel role

  4. Examine the policy applied to the role. This is usually named expel-aws-CloudTrail-CloudTrail-manager-policy.

    Read about policies

  5. Check that the S3 get object permission is set to Allow and has the correct S3 bucket resource Amazon Resource Name (ARN) applied.

    Read about applying permissions to policies (AWS)

    Example JSON (formatting can vary):

    {
                "Action": "s3:GetObject",
                "Effect": "Allow",
                "Resource": "insert S3 Key arn here"
           	},
  6. Wait a few minutes for the system to refresh, then check for the error again.

    If you still see the error, submit a Workbench support request. Be sure to document all the steps you tried and list the error specifically.

CloudTrail Log File Decrypt Forbidden

Error name: AWS_FILE_DECRYPT_FORBIDDEN

What’s wrong: The Expel role within your AWS environment doesn't have permission to decrypt the CloudTrail log file. Apply the correct permission to the Expel role.

Steps to fix:

  1. Log in to your AWS instance.

  2. Navigate to your IAM service.

  3. View your Expel role. This is usually named ExpelAssumeRole.

    Read about the Expel role

  4. Examine the policy applied to the role. This is usually named expel-aws-CloudTrail-CloudTrail-manager-policy.

    Read about policies

  5. View the permissions policy.

    Make sure the KMS:Decrypt permission is set to Allow and the resource contains the KMS Key ARN for your policy. See the following example for details:

    {
                "Action": "kms:Decrypt",
                "Effect": "Allow",
                "Resource": "insert KMS Key ARN here"
           	},
    

    Read about the KMS Key ARN

  6. Make sure the permission exists and it's applied to the correct KMS Key ARN resource.

  7. Save changes.

  8. Wait a few minutes for the system to refresh, then check for the error again.

    If you still see the error, submit a Workbench support request. Be sure to document all the steps you tried and list the error specifically.

CloudTrail Log File Is Not Valid JSON

Error name: AWS_INVALID_JSON

What’s wrong: The CloudTrail log file is not in JSON format. Verify the format of your CloudTrail log records.

Steps to fix:

  1. Log in to your AWS instance.

  2. Navigate to your S3 service.

  3. Open the bucket containing the CloudTrail logs.

    This is the bucket you selected when you set up CloudTrail. If you don't know it, navigate to your CloudTrail service on the CloudTrail service dashboard. Under general details, view the Trail log location. This shows the S3 bucket containing your CloudTrail logs.

  4. Find the relevant logs within the bucket and make sure they are in JSON format.

    Read about working with CloudTrail log files

  5. If you are unable to determine the source of your issue, submit a Workbench support request. Be sure to document all the steps you tried and list the error specifically.

CloudTrail Log File Is Non Tar.gz Format

Error name: AWS_NON_TAR_FILE

What’s wrong: The CloudTrail log file is in the incorrect format. Confirm the S3 bucket contains compressed JSON files with the .gz file extension.

Steps to fix:

  1. Log in to your AWS instance.

  2. Navigate to your S3 service.

  3. Open the bucket containing the CloudTrail logs.

    This is the bucket you selected when you set up CloudTrail. If you don't know it, navigate to your CloudTrail service on the CloudTrail service dashboard. Under general details, view the Trail log location. This shows the S3 bucket containing your CloudTrail logs.

  4. Find the relevant logs within the bucket and make sure they are in the proper format.

    If they are in the wrong format, view AWS documentation on how to get your S3 in tar.gz format.

    Read about working with CloudTrail log files

  5. If you are unable to determine the source of your issue, submit a Workbench support request. Be sure to document all the steps you tried and list the error specifically.

CloudTrail Log File Is Empty

Error name: AWS_NO_RECORDS_IN_S3_FILE

What’s wrong: No records were found in the S3 file. Confirm the CloudTrail is writing logs to the correct S3 log bucket. This error is caused by a configuration of CloudTrail that is not compatible with how Expel reads logs.

Steps to fix:

  1. Log in to your AWS instance.

  2. Navigate to your S3 service.

  3. Open the bucket containing the CloudTrail logs.

    This is the bucket you selected when you set up CloudTrail. If you don't know it, navigate to your CloudTrail service on the CloudTrail service dashboard. Under general details, view the Trail log location. This shows the S3 bucket containing your CloudTrail logs.

  4. Look in the folders and make sure they are not empty.

    If they are empty, check your CloudTrail configuration.

    Read about working with CloudTrail log files

  5. If you are unable to determine the source of your issue, submit a Workbench support request. Be sure to document all the steps you tried and list the error specifically.

CloudTrail Log File Not Found

Error name: AWS_LOG_FILE_NOT_FOUND

What’s wrong: No log file was found in the S3 bucket. Confirm the CloudTrail is writing logs to the correct S3 log bucket.

Steps to fix:

  1. Log in to your AWS instance.

  2. Navigate to your S3 service.

  3. Open the bucket containing the CloudTrail logs.

    This is the bucket you selected when you set up CloudTrail. If you don't know it, navigate to your CloudTrail service on the CloudTrail service dashboard. Under general details, view the Trail log location. This shows the S3 bucket containing your CloudTrail logs.

  4. Look in the folders and make sure they are not empty.

    If they are empty, do the following:

    • Make sure the notification service is connected to the correct S3 log bucket.

    • Check your CloudTrail configuration.

      Read about working with CloudTrail log files

    • Check if the log file was deleted.

      Expel can't delete your log files, so a deletion would have happened on the AWS account owner's side. Check with your AWS console and team to determine why this happened.

  5. If you are unable to determine the source of your issue, submit a Workbench support request. Be sure to document all the steps you tried and list the error specifically.

Botocore.exceptions.ClientError: An Error Occurred (AccessDenied) When Calling the AssumeRole Operation: User: arn:aws:iam::012205512454:user/ExpelCloudService Is Not Authorized to Perform: sts:AssumeRole on Resource: arn:aws:iam::*****:role/ExpelCloudTrailAccessRole

Error name: STS:AssumeRole

What’s wrong: Expel doesn't have the permission to use AssumeRole.

Steps to fix:

  1. Log in to your AWS instance.

  2. Navigate to your IAM service.

  3. View your Expel role. This is usually named ExpelAssumeRole.

    Read about the Expel role

  4. Examine the policy applied to the role. This is usually named expel-aws-CloudTrail-CloudTrail-manager-policy.

    Read about policies

  5. Verify the trust policy exists, in case it was omitted during configuration.

    The trust policy should look as in the following example:

    {
        "Version": "2012-10-17",
        "Statement": [
            {
                "Effect": "Allow",
                "Principal": {
                    "AWS": "arn:aws:iam::111111111:user/ExpelCloudService"
                },
                "Action": "sts:AssumeRole",
                "Condition": {
                    "StringEquals": {
                        "sts:ExternalId": "INSERT ORGANIZATION GUID"
                    }
                }
            }
        ]
    }
    
    • If the trust policy doesn’t exist, create the policy by copying the data from the above example.

    • If the trust policy exists, make sure it has the correct organization GUID as the ExternalId.

      You can find the Expel organization GUID in Workbench, on the Organization settings page: the 36-character GUID (numbers, letters, and hyphens) is a suffix in the URL address.

      Copy the entire GUID, including hyphens, and use it as the ExternalId.

  6. Wait a few minutes for the system to refresh, then check for the error again.

    If you still see the error, submit a Workbench support request. Be sure to document all the steps you tried and list the error specifically.