7 AWS CloudTrail Best Practices with Console and CLI Examples

In AWS, whether you perform an action from Console, use AWS CLI, use AWS SDK, or when a AWS service does an action on your behalf, all of those API activities are logged in AWS CloudTrail.

This tutorials explains the following 7 essential AWS Cloudtrail best practices with examples on how to do it from both Console and using AWS CloudTrail CLI command.

1. Enable CloudTrail in All Regions
2. Encrypt CloudTrail Logs using KMS
3. Set Key Policy for Encrypted CloudTrail Logs
4. Enable CloudTrail Log file Validation
5. Send CloudTrail Logs to Cloudwatch
6. For Multi-Account: Send CloudTrail logs to Centralized S3 bucket
7. For Multi-Account: Enable CloudTrail at Organization Level

1. Enable CloudTrail in All Regions

When you create a CloudTrail you have the option of creating it for one region, or for all the regions in your AWS account.

Even when you are putting your workloads in only one region, as a best practice you should still enable cloudtrail in ALL AWS regions. This way, when an activity happens in any other region, other than your primary working region, you can track them and take action immediately.

From Console, set the “Apply trail to all regions” option to “Yes” as shown below.

From CLI, when you are creating a cloudtrail, use the –is-multi-region-trail option as shown below:

aws cloudtrail create-trail --name thegeekstuff \
  --s3-bucket-name tgs-logs \
  --is-multi-region-trail

To manage your S3 bucket, refer to this: 28 Essential AWS S3 CLI Command Examples to Manage Buckets and Objects

The following is the output of the above command. In the following output, notice how it says IsMultiRegionTrail as true.

{
    "IncludeGlobalServiceEvents": true,
    "Name": "thegeekstuff",
    "TrailARN": "arn:aws:cloudtrail:us-east-1:111111111111:trail/thegeekstuff",
    "LogFileValidationEnabled": false,
    "IsMultiRegionTrail": true,
    "S3BucketName": "tgs-logs"
}

2. Encrypt CloudTrail Logs using KMS

By default, the cloudtrail logs that are delivered are encrypted using Amazon S3-managed encryption keys (SSE-S3). SSE stands for server side encryption.

However you can change this to encrypt log files with AWS Key Management Service (SSE-KMS).

From Console, when creating a cloudtrail:

• Encrypt log files with SSE-KMS: Set this option to “Yes”. When you set this option, you’ll get next two options.
• Create a new KMS key: Set this to “Yes” to create a new KMS key. This will automatically create appropriate KMS key policy for cloudtrail to allow access.
• KMS key: Give the name of the key alias that should be given to the new KMS key that cloudtrail creates.

From CLI, use the –kms-key-id to specify the SSE-KMS key-id as shown below.

aws cloudtrail create-trail --name thegeekstuff \
  --s3-bucket-name tgs-logs \
  --is-multi-region-trail \
  --enable-log-file-validation \
  --kms-key-id alias/thegeekstuff/key

When you specify the kms-key-id, the following are possible formats that you can specify for this option:

• alias/YourKeyAliasName
• arn:aws:kms:us-east-2:111111111111:alias/YourKeyAliasName
• arn:aws:kms:us-east-2:111111111111:key/11111111-2222-1111-2222-111111111111
• 11111111-2222-1111-2222-111111111111

When specifying alias for ksm-key-id, if you don’t use the prefix “alias”, then you’ll get the following error message:

An error occurred (InvalidKmsKeyIdException) when calling the CreateTrail operation: KMS key ID thegeekstuff/key is not a valid format.

Also, if appropriate S3 bucket policy is not set, then you’ll get the following error message.

An error occurred (InsufficientEncryptionPolicyException) when calling the CreateTrail operation: Insufficient permissions to access S3 bucket tgs-logs or KMS key arn:aws:kms:us-east-1:111111111111:alias/thegeekstuff/key.

3. Set Key Policy for Encrypted CloudTrail Logs

If you are using SSE-KMS to encrypt your cloudtrial logs, make sure your KMS key policy has the following three SIDs.

KMS Key Policy SID 1: Allow CloudTrail to encrypt logs

{
  "Sid": "Allow CloudTrail to encrypt logs",
  "Effect": "Allow",
  "Principal": {
    "Service": "cloudtrail.amazonaws.com"
  },
  "Action": "kms:GenerateDataKey*",
  "Resource": "*",
  "Condition": {
    "StringLike": {
      "kms:EncryptionContext:aws:cloudtrail:arn": [
        "arn:aws:cloudtrail:*:111111111111:trail/*"
      ]
    }
  }
}

KMS Key Policy SID 2: Enable CloudTrail log decrypt permissions

{
  "Sid": "Enable CloudTrail log decrypt permissions",
  "Effect": "Allow",
  "Principal": {
    "AWS": "arn:aws:iam::111111111111:user/ramesh"
  },
  "Action": "kms:Decrypt",
  "Resource": "*",
  "Condition": {
    "Null": {
      "kms:EncryptionContext:aws:cloudtrail:arn": "false"
    }
  }
}

Note: In the above, change the username accordingly. If you prefer to use a role, then in the Principal section, instead of user, use the following:
arn:aws:iam::11111111111:role/MyCloudTrailReadRole

KMS Key Policy SID 3: CloudTrail to describe CMK properties

{
  "Sid": "Allow CloudTrail access",
  "Effect": "Allow",
  "Principal": {
    "Service": "cloudtrail.amazonaws.com"
  },
  "Action": "kms:DescribeKey",
  "Resource": "*"
}

4. Enable CloudTrail Log file Validation

Apart from delivering the cloudtrail events to your S3 bucket, you can also instruct cloudtrail to create a digest file for your log files and deliver them to the same S3 bucket.

You can then use the digest file to validate your cloudtrail log file integrity. i.e You can make sure the cloudtrail log files are not tampered with after it was delivered to your s3 bucket. The log file validation is done using SHA-256 for hashing and SHA-256 with RSA for digital signing.

From Console, while creating the cloudtrail, under Storage location section, set the “Enable log file validation” to “Yes” as shown below.

From CLI, when you create cloudtrail, use the enable-log-file-validation option as shown below.

aws cloudtrail create-trail --name thegeekstuff \
  --s3-bucket-name tgs-logs \
  --is-multi-region-trail \
  --enable-log-file-validation

The following is the output of the above command. In the following output, notice how it says LogFileValidationEnabled as true.

{
    "IncludeGlobalServiceEvents": true,
    "Name": "thegeekstuff",
    "TrailARN": "arn:aws:cloudtrail:us-east-1:111111111111:trail/thegeekstuff",
    "LogFileValidationEnabled": true,
    "IsMultiRegionTrail": true,
    "S3BucketName": "tgs-logs"
}

5. Send CloudTrail Logs to Cloudwatch

You can also send your cloudtrail events to cloudwatch logs for monitoring.

From Console, select an existing cloudtrail, under “Cloudwatch Logs” section, click on “Configure”. This will ask you to enter the name of the log group. If the given log group already exists, it will use that. If not, it’ll create a new log group.

From CLI, use the cloud-watch-logs-log-group-arn and cloud-watch-logs-role-arn option as shown below to enable cloudwatch logs for cloudtrail.

aws cloudtrail create-trail --name thegeekstuff \
  --s3-bucket-name tgs-logs \
  --is-multi-region-trail \
  --enable-log-file-validation \
  --cloud-watch-logs-log-group-arn arn:aws:logs:us-east-1:111111111111:log-group:/thegeekstuff/cloudtrail:* \
  --cloud-watch-logs-role-arn arn:aws:iam::111111111111:role/CloudTrail_CloudWatchLogs_Role

If you specify only the log-group-arn without logs-role-arn, you’ll get the following error message.

An error occurred (InvalidCloudWatchLogsRoleArnException) when calling the CreateTrail operation: You must specify a role ARN as well as a log group.

Also, make sure the given role has appropriate permission for cloudtrail to access cloudwatch logs. If not, you’ll get the following error message.

An error occurred (InvalidCloudWatchLogsLogGroupArnException) when calling the CreateTrail operation: Access denied. Check the permissions for your role.

6. For Multi-Account: Send CloudTrail logs to Centralized S3 bucket

When you have multiple AWS accounts, sending cloudtrail logs to a S3 bucket in the individual accounts might create some operational challenges. You need a way to centrally manage and monitor cloudtrail logs from all your accounts.

For this, you can enable cloudtrail in all your accounts, but send the cloudtrail logs to a centralized S3 bucket in one account. You can create this centralized S3 bucket in a dedicated logging account where all other accounts will send their cloudtrail logs.

From Console, when creating cloudtrail, under “Storage location” settings, set “Create a new S3 bucket” to “No”. For “S3 bucket”, specify the name of the centralized S3 bucket that is in a different account (i.e from your logging account where that centralized S3 bucket exist)

From CLI, while creating cloudtrail, just like how we did earlier, specify the name of the S3 bucket here.

aws cloudtrail create-trail --name thegeekstuff \
  --s3-bucket-name thegeekstuff-cloudtrail-logs \
  --is-multi-region-trail \
  --enable-log-file-validation

For this to work properly, in the centralized S3 bucket, make sure to set the following bucket policy.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "BucketAclForCloudTrail",
      "Effect": "Allow",
      "Principal": {
        "Service": "cloudtrail.amazonaws.com"
      },
      "Action": "s3:GetBucketAcl",
      "Resource": "arn:aws:s3:::thegeekstuff-cloudtrail-logs"
    },
    {
      "Sid": "WritePermissionForCloudTrail",
      "Effect": "Allow",
      "Principal": {
        "Service": "cloudtrail.amazonaws.com"
      },
      "Action": "s3:PutObject",
      "Resource": [
        "arn:aws:s3:::thegeekstuff-cloudtrail-logs/AWSLogs/111111111111/*",
        "arn:aws:s3:::thegeekstuff-cloudtrail-logs/AWSLogs/222222222222/*"
      ],
      "Condition": {
        "StringEquals": {
          "s3:x-amz-acl": "bucket-owner-full-control"
        }
      }
    }
  ]
}

The above indicates that cloudtrail from two accounts (111111111111 and 222222222222) can send their cloudtrail logs to this centralized S3 bucket.

If you want to add another account (3333333333333), modify the above policy and add new account to the Resource section in the WritePermissionForCloudTrail SID as shown below.

"Resource": [
  "arn:aws:s3:::thegeekstuff-cloudtrail-logs/AWSLogs/111111111111/*",
  "arn:aws:s3:::thegeekstuff-cloudtrail-logs/AWSLogs/222222222222/*",
  "arn:aws:s3:::thegeekstuff-cloudtrail-logs/AWSLogs/3333333333333/*"
],

7. For Multi-Account: Enable CloudTrail at Organization Level

If you are creating a cloudtrail on your master account where you have AWS organizations enabled, you also have the option to enable cloudtrail at your organization level.

When you select this option, this will log cloudtrail events for all your AWS accounts that is under that organization.

From Console, on your master account where AWS organization is enabled, while creating a cloudtrail, set the “Apply trail to my organization” option to “Yes” as shown below.

From CLI, while creating the cloudtrail, specify the is-organization-trail option as shown below.

aws cloudtrail create-trail --name thegeekstuff \
  --s3-bucket-name tgs-logs \
  --is-multi-region-trail \
  --is-organization-trail \
  --enable-log-file-validation

You may get the following error message for the above command if you are using a older version of aws-cli. For example, aws-cli/1.16.6 gives the following error message:

Unknown options: --is-organization-trail

$ aws --version
aws-cli/1.16.6 Python/2.7.10 Darwin/16.7.0 botocore/1.11.6

To avoid that, upgrade your aws cli to the latest version. For example, aws-cli/1.16.139 and above will not give the above error message.

$ aws --version
aws-cli/1.16.139 Python/2.7.10 Darwin/16.7.0 botocore/1.12.129