# Validating CloudTrail log file integrity To determine whether a log file was modified, deleted, or unchanged after CloudTrail delivered it, you can use CloudTrail log file integrity validation. This feature is built using industry standard algorithms: SHA-256 for hashing and SHA-256 with RSA for digital signing. This makes it computationally infeasible to modify, delete or forge CloudTrail log files without detection. You can use the AWS CLI to validate the files in the location where CloudTrail delivered them. ## Why use it? Validated log files are invaluable in security and forensic investigations. For example, a validated log file enables you to assert positively that the log file itself has not changed, or that particular user credentials performed specific API activity. The CloudTrail log file integrity validation process also lets you know if a log file has been deleted or changed, or assert positively that no log files were delivered to your account during a given period of time. ## How it works When you enable log file integrity validation, CloudTrail creates a hash for every log file that it delivers. Every hour, CloudTrail also creates and delivers a file that references the log files for the last hour and contains a hash of each. This file is called a digest file. CloudTrail signs each digest file using the private key of a public and private key pair. After delivery, you can use the public key to validate the digest file. CloudTrail uses different key pairs for each AWS Region. The digest files are delivered to the same Amazon S3 bucket associated with your trail as your CloudTrail log files. If your log files are delivered from all Regions or from multiple accounts into a single Amazon S3 bucket, CloudTrail will deliver the digest files from those Regions and accounts into the same bucket. The digest files are put into a folder separate from the log files. This separation of digest files and log files enables you to enforce granular security policies and permits existing log processing solutions to continue to operate without modification. Each digest file also contains the digital signature of the previous digest file if one exists. The signature for the current digest file is in the metadata properties of the digest file Amazon S3 object. For more information about digest file contents, see [CloudTrail digest file structure](cloudtrail-log-file-validation-digest-file-structure.md). ### Storing log and digest files You can store the CloudTrail log files and digest files in Amazon S3 or Amazon Glacier securely, durably and inexpensively for an indefinite period of time. To enhance the security of the digest files stored in Amazon S3, you can use [Amazon S3 MFA Delete](https://docs.aws.amazon.com/AmazonS3/latest/userguide/UsingMFADelete.html). ### Enabling validation and validating files To enable log file integrity validation, you can use the AWS Management Console, the AWS CLI, or CloudTrail API. Enabling log file integrity validation allows CloudTrail to deliver digest log files to your Amazon S3 bucket, but does not validate the integrity of the files. For more information, see [Enabling log file integrity validation for CloudTrail](cloudtrail-log-file-validation-enabling.md). To validate the integrity of CloudTrail log files, you can use the AWS CLI or create your own solution. The AWS CLI will validate files in the location where CloudTrail delivered them. If you want to validate logs that you have moved to a different location, either in Amazon S3 or elsewhere, you can create your own validation tools. For information on validating logs by using the AWS CLI, see [Validating CloudTrail log file integrity with the AWS CLI](cloudtrail-log-file-validation-cli.md). For information on developing custom implementations of CloudTrail log file validation, see [Custom implementations of CloudTrail log file integrity validation](cloudtrail-log-file-custom-validation.md). # Enabling log file integrity validation for CloudTrail You can enable log file integrity validation by using the AWS Management Console, AWS Command Line Interface (AWS CLI), or CloudTrail API. CloudTrail starts delivering digest files in about an hour. ## AWS Management Console To enable log file integrity validation with the CloudTrail console, choose **Yes** for the **Enable log file validation** option when you create or update a trail. By default, this feature is enabled for new trails. For more information, see [Creating and updating a trail with the console](cloudtrail-create-and-update-a-trail-by-using-the-console.md). ## AWS CLI To enable log file integrity validation with the AWS CLI, use the `--enable-log-file-validation` option with the [create-trail](https://docs.aws.amazon.com/cli/latest/reference/cloudtrail/create-trail.html) or [update-trail](https://docs.aws.amazon.com/cli/latest/reference/cloudtrail/update-trail.html) commands. To disable log file integrity validation, use the `--no-enable-log-file-validation` option. **Example** The following `update-trail` command enables log file validation and starts delivering digest files to the Amazon S3 bucket for the specified trail. ``` aws cloudtrail update-trail --name your-trail-name --enable-log-file-validation ``` ## CloudTrail API To enable log file integrity validation with the CloudTrail API, set the `EnableLogFileValidation` request parameter to `true` when calling `CreateTrail` or `UpdateTrail`. For more information, see [CreateTrail](https://docs.aws.amazon.com/awscloudtrail/latest/APIReference/API_CreateTrail.html) and [UpdateTrail](https://docs.aws.amazon.com/awscloudtrail/latest/APIReference/API_UpdateTrail.html) in the [AWS CloudTrail API Reference](https://docs.aws.amazon.com/awscloudtrail/latest/APIReference/). # Validating CloudTrail log file integrity with the AWS CLI To validate logs with the AWS Command Line Interface, use the CloudTrail `validate-logs` command. The command uses the digest files delivered to your Amazon S3 bucket to perform the validation. For information about digest files, see [CloudTrail digest file structure](cloudtrail-log-file-validation-digest-file-structure.md). The AWS CLI allows you to detect the following types of changes: + Modification or deletion of CloudTrail log files + Modification or deletion of CloudTrail digest files + Modification or deletion of both of the above **Note** The AWS CLI validates only log files that are referenced by digest files. For more information, see [Checking whether a particular file was delivered by CloudTrail](#cloudtrail-log-file-validation-cli-validate-logs-check-file). ## Prerequisites To validate log file integrity with the AWS CLI, the following conditions must be met: + You must have online connectivity to AWS. + You must have read access to the Amazon S3 bucket that contains the digest and log files. + The digest and log files must not have been moved from the original Amazon S3 location where CloudTrail delivered them. + The role executing the command must have permissions to call `ListObjects`, `GetObject`, and `GetBucketLocation` for each S3 bucket referenced by the trail. **Note** Log files that have been downloaded to local disk cannot be validated with the AWS CLI. For guidance on creating your own tools for validation, see [Custom implementations of CloudTrail log file integrity validation](cloudtrail-log-file-custom-validation.md). ## validate-logs ### Syntax The following is the syntax for `validate-logs`. Optional parameters are shown in brackets. `aws cloudtrail validate-logs --trail-arn --start-time [--end-time ] [--s3-bucket ] [--s3-prefix ] [--account-id ] [--verbose]` **Note** The `validate-logs` command is Region specific. You must specify the `--region` global option to validate logs for a specific AWS Region. ### Options The following are the command-line options for `validate-logs`. The `--trail-arn` and `--start-time` options are required. The `--account-id` option is additionally required for organizational trails. `--start-time` Specifies that log files delivered on or after the specified UTC timestamp value will be validated. Example: `2015-01-08T05:21:42Z`. `--end-time` Optionally specifies that log files delivered on or before the specified UTC timestamp value will be validated. The default value is the current UTC time (`Date.now()`). Example: `2015-01-08T12:31:41Z`. For the time range specified, the `validate-logs` command checks only the log files that are referenced in their corresponding digest files. No other log files in the Amazon S3 bucket are checked. For more information, see [Checking whether a particular file was delivered by CloudTrail](#cloudtrail-log-file-validation-cli-validate-logs-check-file). `--s3-bucket` Optionally specifies the Amazon S3 bucket where the digest files are stored. If a bucket name is not specified, the AWS CLI will retrieve it by calling `DescribeTrails()`. `--s3-prefix` Optionally specifies the Amazon S3 prefix where the digest files are stored. If not specified, the AWS CLI will retrieve it by calling `DescribeTrails()`. You should use this option only if your current prefix is different from the prefix that was in use during the time range that you specify. `--account-id` Optionally specifies the account for validating logs. This parameter is required for organization trails for validating logs for the specific account inside an organization. `--trail-arn` Specifies the Amazon Resource Name (ARN) of the trail to be validated. The format of a trail ARN follows. ``` arn:aws:cloudtrail:us-east-2:111111111111:trail/MyTrailName ``` To obtain the trail ARN for a trail, you can use the `describe-trails` command before running `validate-logs`. You may want to specify the bucket name and prefix in addition to the trail ARN if log files have been delivered to more than one bucket in the time range that you specified, and you want to restrict the validation to the log files in only one of the buckets. `--verbose` Optionally outputs validation information for every log or digest file in the specified time range. The output indicates whether the file remains unchanged or has been modified or deleted. In non-verbose mode (the default), information is returned only for those cases in which there was a validation failure. ### Example The following example validates log files from the specified start time to the present, using the Amazon S3 bucket configured for the current trail and specifying verbose output. ``` aws cloudtrail validate-logs --start-time 2015-08-27T00:00:00Z --end-time 2015-08-28T00:00:00Z --trail-arn arn:aws:cloudtrail:us-east-2:111111111111:trail/my-trail-name --verbose ``` ### How `validate-logs` works The `validate-logs` command starts by validating the most recent digest file in the specified time range. First, it verifies that the digest file has been downloaded from the location to which it claims to belong. In other words, if the CLI downloads digest file `df1` from the S3 location `p1`, validate-logs will verify that `p1 == df1.digestS3Bucket + '/' + df1.digestS3Object`. If the signature of the digest file is valid, it checks the hash value of each of the logs referenced in the digest file. The command then goes back in time, validating the previous digest files and their referenced log files in succession. It continues until the specified value for `start-time` is reached, or until the digest chain ends. If a digest file is missing or not valid, the time range that cannot be validated is indicated in the output. The `validate-logs` command first operates on the standard digest chain. After completing standard digest validation, it validates backfill digest files, if present. Backfill digests form a separate validation chain and are processed independently from standard digests. ## Validation results Validation results begin with a summary header in the following format: ``` Validating log files for trail trail_ARN between time_stamp and time_stamp ``` Each line of the main output contains the validation results for a single digest or log file in the following format. Lines prefixed with `(backfill)` indicate backfill digest files, which form a separate validation chain from the standard digest files. ``` ``` The following table describes the possible validation messages for log and digest files. **** | File Type | Validation Message | Description | | --- | --- | --- | | Digest file | valid | The digest file signature is valid. The log files it references can be checked. This message is included only in verbose mode. | | Digest file | INVALID: has been moved from its original location | The S3 bucket or S3 object from which the digest file was retrieved do not match the S3 bucket or S3 object locations that are recorded in the digest file itself. | | Digest file | INVALID: invalid format | The format of the digest file is invalid. The log files corresponding to the time range that the digest file represents cannot be validated. | | Digest file | INVALID: not found | The digest file was not found. The log files corresponding to the time range that the digest file represents cannot be validated. | | Digest file | INVALID: public key not found for fingerprint fingerprint | The public key corresponding to the fingerprint recorded in the digest file was not found. The digest file cannot be validated. | | Digest file | INVALID: signature verification failed | The digest file signature is not valid. Because the digest file is not valid, the log files it references cannot be validated, and no assertions can be made about the API activity in them. | | Digest file | INVALID: Unable to load PKCS \$11 key with fingerprint fingerprint | Because the DER encoded public key in PKCS \$11 format having the specified fingerprint could not be loaded, the digest file cannot be validated. | | Log file | valid | The log file has been validated and has not been modified since the time of delivery. This message is included only in verbose mode. | | Log file | INVALID: hash value doesn't match | The hash for the log file does not match. The log file has been modified after delivery by CloudTrail. | | Log file | INVALID: invalid format | The format of the log file is invalid. The log file cannot be validated. | | Log file | INVALID: not found | The log file was not found and cannot be validated. | The output includes summary information about the results returned. ## Example outputs ### Verbose The following example `validate-logs` command uses the `--verbose` flag and produces the sample output that follows. `[...]` indicates the sample output has been abbreviated. ``` aws cloudtrail validate-logs --trail-arn arn:aws:cloudtrail:us-east-2:111111111111:trail/example-trail-name --start-time 2015-08-31T22:00:00Z --end-time 2015-09-01T19:17:29Z --verbose ``` ``` Validating log files for trail arn:aws:cloudtrail:us-east-2:111111111111:trail/example-trail-name between 2015-08-31T22:00:00Z and 2015-09-01T19:17:29Z Digest file s3://amzn-s3-demo-bucket/AWSLogs/111111111111/CloudTrail-Digest/us-east-2/2015/09/01/111111111111_CloudTrail-Digest_us-east-2_example-trail-name_us-east-2_20150901T201728Z.json.gz valid Log file s3://amzn-s3-demo-bucket/AWSLogs/111111111111/CloudTrail/us-east-2/2015/09/01/111111111111_CloudTrail_us-east-2_20150901T1925Z_WZZw1RymnjCRjxXc.json.gz valid Log file s3://amzn-s3-demo-bucket/AWSLogs/111111111111/CloudTrail/us-east-2/2015/09/01/111111111111_CloudTrail_us-east-2_20150901T1915Z_POuvV87nu6pfAV2W.json.gz valid Log file s3://amzn-s3-demo-bucket/AWSLogs/111111111111/CloudTrail/us-east-2/2015/09/01/111111111111_CloudTrail_us-east-2_20150901T1930Z_l2QgXhAKVm1QXiIA.json.gz valid Log file s3://amzn-s3-demo-bucket/AWSLogs/111111111111/CloudTrail/us-east-2/2015/09/01/111111111111_CloudTrail_us-east-2_20150901T1920Z_eQJteBBrfpBCqOqw.json.gz valid Log file s3://amzn-s3-demo-bucket/AWSLogs/111111111111/CloudTrail/us-east-2/2015/09/01/111111111111_CloudTrail_us-east-2_20150901T1950Z_9g5A6qlR2B5KaRdq.json.gz valid Log file s3://amzn-s3-demo-bucket/AWSLogs/111111111111/CloudTrail/us-east-2/2015/09/01/111111111111_CloudTrail_us-east-2_20150901T1920Z_i4DNCC12BuXd6Ru7.json.gz valid Log file s3://amzn-s3-demo-bucket/AWSLogs/111111111111/CloudTrail/us-east-2/2015/09/01/111111111111_CloudTrail_us-east-2_20150901T1915Z_Sg5caf2RH6Jdx0EJ.json.gz valid Digest file s3://amzn-s3-demo-bucket/AWSLogs/111111111111/CloudTrail-Digest/us-east-2/2015/09/01/111111111111_CloudTrail-Digest_us-east-2_example-trail-name_us-east-2_20150901T191728Z.json.gz valid Log file s3://amzn-s3-demo-bucket/AWSLogs/111111111111/CloudTrail/us-east-2/2015/09/01/111111111111_CloudTrail_us-east-2_20150901T1910Z_YYSFiuFQk4nrtnEW.json.gz valid [...] Log file s3://amzn-s3-demo-bucket/AWSLogs/144218288521/CloudTrail/us-east-2/2015/09/01/144218288521_CloudTrail_us-east-2_20150901T1055Z_0Sfy6m9f6iBzmoPF.json.gz valid Log file s3://amzn-s3-demo-bucket/AWSLogs/144218288521/CloudTrail/us-east-2/2015/09/01/144218288521_CloudTrail_us-east-2_20150901T1040Z_lLa3QzVLpOed7igR.json.gz valid Digest file s3://amzn-s3-demo-bucket/AWSLogs/144218288521/CloudTrail-Digest/us-east-2/2015/09/01/144218288521_CloudTrail-Digest_us-east-2_example-trail-name_us-east-2_20150901T101728Z.json.gz INVALID: signature verification failed Digest file s3://amzn-s3-demo-bucket/AWSLogs/144218288521/CloudTrail-Digest/us-east-2/2015/09/01/144218288521_CloudTrail-Digest_us-east-2_example-trail-name_us-east-2_20150901T091728Z.json.gz valid Log file s3://amzn-s3-demo-bucket/AWSLogs/144218288521/CloudTrail/us-east-2/2015/09/01/144218288521_CloudTrail_us-east-2_20150901T0830Z_eaFvO3dwHo4NCqqc.json.gz valid Digest file s3://amzn-s3-demo-bucket/AWSLogs/144218288521/CloudTrail-Digest/us-east-2/2015/09/01/144218288521_CloudTrail-Digest_us-east-2_example-trail-name_us-east-2_20150901T081728Z.json.gz valid Digest file s3://amzn-s3-demo-bucket/AWSLogs/144218288521/CloudTrail-Digest/us-east-2/2015/09/01/144218288521_CloudTrail-Digest_us-east-2_example-trail-name_us-east-2_20150901T071728Z.json.gz valid [...] Log file s3://amzn-s3-demo-bucket/AWSLogs/111111111111/CloudTrail/us-east-2/2015/08/31/111111111111_CloudTrail_us-east-2_20150831T2245Z_mbJkEO5kNcDnVhGh.json.gz valid Log file s3://amzn-s3-demo-bucket/AWSLogs/111111111111/CloudTrail/us-east-2/2015/08/31/111111111111_CloudTrail_us-east-2_20150831T2225Z_IQ6kXy8sKU03RSPr.json.gz valid Log file s3://amzn-s3-demo-bucket/AWSLogs/111111111111/CloudTrail/us-east-2/2015/08/31/111111111111_CloudTrail_us-east-2_20150831T2230Z_eRPVRTxHQ5498ROA.json.gz valid Log file s3://amzn-s3-demo-bucket/AWSLogs/111111111111/CloudTrail/us-east-2/2015/08/31/111111111111_CloudTrail_us-east-2_20150831T2255Z_IlWawYZGvTWB5vYN.json.gz valid Digest file s3://amzn-s3-demo-bucket/AWSLogs/111111111111/CloudTrail-Digest/us-east-2/2015/08/31/111111111111_CloudTrail-Digest_us-east-2_example-trail-name_us-east-2_20150831T221728Z.json.gz valid Results requested for 2015-08-31T22:00:00Z to 2015-09-01T19:17:29Z Results found for 2015-08-31T22:17:28Z to 2015-09-01T20:17:28Z: 22/23 digest files valid, 1/23 digest files INVALID 63/63 log files valid ``` The following example `validate-logs` command uses the `--verbose` flag on a period when backfill digest files are present and produces the sample output that follows. Backfill digests appear with the `(backfill)` prefix and are validated separately from the standard digest chain. `[...]` indicates the sample output has been abbreviated. ``` aws cloudtrail validate-logs --trail-arn arn:aws:cloudtrail:us-east-2:111111111111:trail/example-trail-name --start-time 2024-07-31T22:00:00Z --end-time 2024-08-01T19:17:29Z --verbose ``` ``` Validating log files for trail arn:aws:cloudtrail:us-east-2:111111111111:trail/example-trail-name between 2024-07-31T22:00:00Z and 2024-08-01T19:17:29Z Digest file s3://amzn-s3-demo-bucket/AWSLogs/111111111111/CloudTrail-Digest/us-east-2/2024/08/01/111111111111_CloudTrail-Digest_us-east-2_example-trail-name_us-east-2_20240801T201728Z.json.gz valid Log file s3://amzn-s3-demo-bucket/AWSLogs/111111111111/CloudTrail/us-east-2/2024/08/01/111111111111_CloudTrail_us-east-2_20240801T1925Z_Xm3pK9vN2wQ5rT8h.json.gz valid Log file s3://amzn-s3-demo-bucket/AWSLogs/111111111111/CloudTrail/us-east-2/2024/08/01/111111111111_CloudTrail_us-east-2_20240801T1915Z_Bj7cL4nM6pR9sU2v.json.gz valid Log file s3://amzn-s3-demo-bucket/AWSLogs/111111111111/CloudTrail/us-east-2/2024/08/01/111111111111_CloudTrail_us-east-2_20240801T1930Z_Fy1dG8kN3qT6wX0z.json.gz valid Log file s3://amzn-s3-demo-bucket/AWSLogs/111111111111/CloudTrail/us-east-2/2024/08/01/111111111111_CloudTrail_us-east-2_20240801T1920Z_Hn5jM2pQ7sV9yB4e.json.gz valid Log file s3://amzn-s3-demo-bucket/AWSLogs/111111111111/CloudTrail/us-east-2/2024/08/01/111111111111_CloudTrail_us-east-2_20240801T1950Z_Kp8rN1tW4xZ7aC3f.json.gz valid Log file s3://amzn-s3-demo-bucket/AWSLogs/111111111111/CloudTrail/us-east-2/2024/08/01/111111111111_CloudTrail_us-east-2_20240801T1920Z_Mq6sP9uX2yB5dE8g.json.gz valid Log file s3://amzn-s3-demo-bucket/AWSLogs/111111111111/CloudTrail/us-east-2/2024/08/01/111111111111_CloudTrail_us-east-2_20240801T1915Z_Rt4vQ7wZ0aC3fG6h.json.gz valid Digest file s3://amzn-s3-demo-bucket/AWSLogs/111111111111/CloudTrail-Digest/us-east-2/2024/08/01/111111111111_CloudTrail-Digest_us-east-2_example-trail-name_us-east-2_20240801T191728Z.json.gz valid Log file s3://amzn-s3-demo-bucket/AWSLogs/111111111111/CloudTrail/us-east-2/2024/08/01/111111111111_CloudTrail_us-east-2_20240801T1910Z_Uw9xR2yB5dH8jK1m.json.gz valid [...] Log file s3://amzn-s3-demo-bucket/AWSLogs/144218288521/CloudTrail/us-east-2/2024/08/01/144218288521_CloudTrail_us-east-2_20240801T1055Z_Vz3aS6cE9fL2nP5q.json.gz valid Log file s3://amzn-s3-demo-bucket/AWSLogs/144218288521/CloudTrail/us-east-2/2024/08/01/144218288521_CloudTrail_us-east-2_20240801T1040Z_Xy7bT0dG3hM6pR9s.json.gz valid Digest file s3://amzn-s3-demo-bucket/AWSLogs/144218288521/CloudTrail-Digest/us-east-2/2024/08/01/144218288521_CloudTrail-Digest_us-east-2_example-trail-name_us-east-2_20240801T101728Z.json.gz INVALID: signature verification failed Digest file s3://amzn-s3-demo-bucket/AWSLogs/144218288521/CloudTrail-Digest/us-east-2/2024/08/01/144218288521_CloudTrail-Digest_us-east-2_example-trail-name_us-east-2_20240801T091728Z.json.gz valid Digest file s3://amzn-s3-demo-bucket/AWSLogs/144218288521/CloudTrail-Digest/us-east-2/2024/08/01/144218288521_CloudTrail-Digest_us-east-2_example-trail-name_us-east-2_20240801T081728Z.json.gz valid Digest file s3://amzn-s3-demo-bucket/AWSLogs/144218288521/CloudTrail-Digest/us-east-2/2024/08/01/144218288521_CloudTrail-Digest_us-east-2_example-trail-name_us-east-2_20240801T071728Z.json.gz valid [...] Digest file s3://amzn-s3-demo-bucket/AWSLogs/111111111111/CloudTrail-Digest/us-east-2/2024/07/31/111111111111_CloudTrail-Digest_us-east-2_example-trail-name_us-east-2_20240731T221728Z.json.gz valid (backfill) Digest file s3://amzn-s3-demo-bucket/AWSLogs/111111111111/CloudTrail-Digest/us-east-2/2024/08/01/111111111111_CloudTrail-Digest_us-east-2_example-trail-name_us-east-2_20240801T201728Z_backfill.json.gz valid (backfill) Digest file s3://amzn-s3-demo-bucket/AWSLogs/111111111111/CloudTrail-Digest/us-east-2/2024/08/01/111111111111_CloudTrail-Digest_us-east-2_example-trail-name_us-east-2_20240801T191728Z_backfill.json.gz valid [...] (backfill) Digest file s3://amzn-s3-demo-bucket/AWSLogs/144218288521/CloudTrail-Digest/us-east-2/2024/08/01/144218288521_CloudTrail-Digest_us-east-2_example-trail-name_us-east-2_20240801T101728Z_backfill.json.gz INVALID: signature verification failed (backfill) Digest file s3://amzn-s3-demo-bucket/AWSLogs/144218288521/CloudTrail-Digest/us-east-2/2024/08/01/144218288521_CloudTrail-Digest_us-east-2_example-trail-name_us-east-2_20240801T091728Z_backfill.json.gz valid Log file s3://amzn-s3-demo-bucket/AWSLogs/144218288521/CloudTrail/us-east-2/2024/08/01/144218288521_CloudTrail_us-east-2_20240801T0830Z_Rn6uk0wY5aD9fJ3n.json.gz valid (backfill) Digest file s3://amzn-s3-demo-bucket/AWSLogs/144218288521/CloudTrail-Digest/us-east-2/2024/08/01/144218288521_CloudTrail-Digest_us-east-2_example-trail-name_us-east-2_20240801T081728Z_backfill.json.gz valid (backfill) Digest file s3://amzn-s3-demo-bucket/AWSLogs/144218288521/CloudTrail-Digest/us-east-2/2024/08/01/144218288521_CloudTrail-Digest_us-east-2_example-trail-name_us-east-2_20240801T071728Z_backfill.json.gz valid [...] (backfill) Digest file s3://amzn-s3-demo-bucket/AWSLogs/111111111111/CloudTrail-Digest/us-east-2/2024/07/31/111111111111_CloudTrail-Digest_us-east-2_example-trail-name_us-east-2_20240731T221728Z_backfill.json.gz valid Log file s3://amzn-s3-demo-bucket/AWSLogs/111111111111/CloudTrail/us-east-2/2024/07/31/111111111111_CloudTrail_us-east-2_20240731T2145Z_Sp3vm7xZ2bE6gK0p.json.gz valid Log file s3://amzn-s3-demo-bucket/AWSLogs/111111111111/CloudTrail/us-east-2/2024/07/31/111111111111_CloudTrail_us-east-2_20240731T2125Z_Tq0wn4ya9cF3hL7q.json.gz valid Log file s3://amzn-s3-demo-bucket/AWSLogs/111111111111/CloudTrail/us-east-2/2024/07/31/111111111111_CloudTrail_us-east-2_20240731T2130Z_Ur7xp1zb6dG0jM4r.json.gz valid Log file s3://amzn-s3-demo-bucket/AWSLogs/111111111111/CloudTrail/us-east-2/2024/07/31/111111111111_CloudTrail_us-east-2_20240731T2155Z_Vs4yq8ac3eH7kN1s.json.gz valid Results requested for 2024-07-31T22:00:00Z to 2024-08-01T19:17:29Z Results found for 2024-07-31T22:17:28Z to 2024-08-01T20:17:28Z: 22/23 digest files valid, 1/23 digest files INVALID 22/23 backfill digest files valid, 1/23 backfill digest files INVALID 63/63 log files valid ``` ### Non-verbose The following example `validate-logs` command does not use the `--verbose` flag. In the sample output that follows, one error was found. Only the header, error, and summary information are returned. ``` aws cloudtrail validate-logs --trail-arn arn:aws:cloudtrail:us-east-2:111111111111:trail/example-trail-name --start-time 2015-08-31T22:00:00Z --end-time 2015-09-01T19:17:29Z ``` ``` Validating log files for trail arn:aws:cloudtrail:us-east-2:111111111111:trail/example-trail-name between 2015-08-31T22:00:00Z and 2015-09-01T19:17:29Z Digest file s3://amzn-s3-demo-bucket/AWSLogs/144218288521/CloudTrail-Digest/us-east-2/2015/09/01/144218288521_CloudTrail-Digest_us-east-2_example-trail-name_us-east-2_20150901T101728Z.json.gz INVALID: signature verification failed (backfill) Digest file s3://amzn-s3-demo-bucket/AWSLogs/144218288521/CloudTrail-Digest/us-east-2/2024/08/01/144218288521_CloudTrail-Digest_us-east-2_example-trail-name_us-east-2_20240801T101728Z_backfill.json.gz INVALID: signature verification failed Results requested for 2015-08-31T22:00:00Z to 2015-09-01T19:17:29Z Results found for 2015-08-31T22:17:28Z to 2015-09-01T20:17:28Z: 22/23 digest files valid, 1/23 digest files INVALID 22/23 backfill digest files valid, 1/23 backfill digest files INVALID 63/63 log files valid ``` ## Checking whether a particular file was delivered by CloudTrail To check if a particular file in your bucket was delivered by CloudTrail, run `validate-logs` in verbose mode for the time period that includes the file. If the file appears in the output of `validate-logs`, then the file was delivered by CloudTrail. # CloudTrail digest file structure Each digest file contains the names of the log files that were delivered to your Amazon S3 bucket during the last hour, the hash values for those log files, and the digital signature of the previous digest file. The signature for the current digest file is stored in the metadata properties of the digest file object. The digital signatures and hashes are used for validating the integrity of the log files and of the digest file itself. ## Digest file location Digest files are delivered to an Amazon S3 bucket location that follows this syntax. ``` s3://amzn-s3-demo-bucket/optional-prefix/AWSLogs/aws-account-id/CloudTrail-Digest/ region/digest-end-year/digest-end-month/digest-end-date/ aws-account-id_CloudTrail-Digest_region_trail-name_region_digest_end_timestamp.json.gz ``` **Note** For organization trails, the bucket location also includes the organization unit ID, as follows: ``` s3://amzn-s3-demo-bucket/optional-prefix/AWSLogs/O-ID/aws-account-id/CloudTrail-Digest/ region/digest-end-year/digest-end-month/digest-end-date/ aws-account-id_CloudTrail-Digest_region_trail-name_region_digest_end_timestamp.json.gz ``` **Note** Backfill digest files are delivered when CloudTrail needs to include log files that were not referenced in the original digest due to processing delays. Backfill digest files use the `_backfill` suffix in their filename, as follows: ``` s3://amzn-s3-demo-bucket/optional-prefix/AWSLogs/aws-account-id/CloudTrail-Digest/ region/digest-end-year/digest-end-month/digest-end-date/ aws-account-id_CloudTrail-Digest_region_trail-name_region_digest_end_timestamp_backfill.json.gz ``` ## Sample digest file contents The following example digest file contains information for a CloudTrail log. ``` { "awsAccountId": "111122223333", "digestStartTime": "2015-08-17T14:01:31Z", "digestEndTime": "2015-08-17T15:01:31Z", "digestS3Bucket": "amzn-s3-demo-bucket", "digestS3Object": "AWSLogs/111122223333/CloudTrail-Digest/us-east-2/2015/08/17/111122223333_CloudTrail-Digest_us-east-2_your-trail-name_us-east-2_20150817T150131Z.json.gz", "digestPublicKeyFingerprint": "31e8b5433410dfb61a9dc45cc65b22ff", "digestSignatureAlgorithm": "SHA256withRSA", "newestEventTime": "2015-08-17T14:52:27Z", "oldestEventTime": "2015-08-17T14:42:27Z", "previousDigestS3Bucket": "amzn-s3-demo-bucket", "previousDigestS3Object": "AWSLogs/111122223333/CloudTrail-Digest/us-east-2/2015/08/17/111122223333_CloudTrail-Digest_us-east-2_your-trail-name_us-east-2_20150817T140131Z.json.gz", "previousDigestHashValue": "97fb791cf91ffc440d274f8190dbdd9aa09c34432aba82739df18b6d3c13df2d", "previousDigestHashAlgorithm": "SHA-256", "previousDigestSignature": "50887ccffad4c002b97caa37cc9dc626e3c680207d41d27fa5835458e066e0d3652fc4dfc30937e4d5f4cc7f796e7a258fb50a43ac427f2237f6e505d4efaf373d156e15e3b68dea9f58111d395b62628d6bd367a9024d2183b5c5f6e19466d3a996b92df705bc997b8a0e13430f241d733cf95df4e41bb6c304c3f58363043572ea57a27085639ce187e679c0d81c7519b1184fa77fb7ab0b0e40a32dace6e1eefc3995c5ae182da49b62b26398cebb52a2201a6387b75b89c83e5570bcb9bba6c34a80f2f00a1c6ebe07d1ff149eccd812dc805bb3eeff6657db32a6cb48d2d096404eb76181877bc6ebb8cd0b23f823200155b2fd8848d428e46e8456328a", "logFiles": [ { "s3Bucket": "amzn-s3-demo-bucket", "s3Object": "AWSLogs/111122223333/CloudTrail/us-east-2/2015/08/17/111122223333_CloudTrail_us-east-2_20150817T1445Z_9nYN7gp2eWAJHIfT.json.gz", "hashValue": "9bb6196fc6b84d6f075a56548feca262bd99ba3c2de41b618e5b6e22c1fc71f6", "hashAlgorithm": "SHA-256", "newestEventTime": "2015-08-17T14:52:27Z", "oldestEventTime": "2015-08-17T14:42:27Z" } ] } ``` ## Digest file field descriptions The following are descriptions for each field in the digest file: `awsAccountId` The AWS account ID for which the digest file has been delivered. `digestStartTime` The starting UTC time range that the digest file covers, taking as a reference the time in which log files have been delivered by CloudTrail. This means that if the time range is [Ta, Tb], the digest will contain all the log files delivered to the customer between Ta and Tb. `digestEndTime` The ending UTC time range that the digest file covers, taking as a reference the time in which log files have been delivered by CloudTrail. This means that if the time range is [Ta, Tb], the digest will contain all the log files delivered to the customer between Ta and Tb. `digestS3Bucket` The name of the Amazon S3 bucket to which the current digest file has been delivered. `digestS3Object` The Amazon S3 object key (that is, the Amazon S3 bucket location) of the current digest file. The first two Regions in the string show the Region from which the digest file was delivered. The last Region (after `your-trail-name`) is the home Region of the trail. The home Region is the Region in which the trail was created. In the case of a multi-Region trail, this can be different from the Region from which the digest file was delivered. `newestEventTime` The UTC time of the most recent event among all of the events in the log files in the digest. `oldestEventTime` The UTC time of the oldest event among all of the events in the log files in the digest. If the digest file is delivered late, the value of `oldestEventTime` will be earlier than the value of `digestStartTime`. `previousDigestS3Bucket` The Amazon S3 bucket to which the previous digest file was delivered. `previousDigestS3Object` The Amazon S3 object key (that is, the Amazon S3 bucket location) of the previous digest file. `previousDigestHashValue` The hexadecimal encoded hash value of the uncompressed contents of the previous digest file. `previousDigestHashAlgorithm` The name of the hash algorithm that was used to hash the previous digest file. `publicKeyFingerprint` The hexadecimal encoded fingerprint of the public key that matches the private key used to sign this digest file. You can retrieve the public keys for the time range corresponding to the digest file by using the AWS CLI or the CloudTrail API. Of the public keys returned, the one whose fingerprint matches this value can be used for validating the digest file. For information about retrieving public keys for digest files, see the AWS CLI [https://docs.aws.amazon.com/cli/latest/reference/cloudtrail/list-public-keys.html](https://docs.aws.amazon.com/cli/latest/reference/cloudtrail/list-public-keys.html) command or the CloudTrail [https://docs.aws.amazon.com/awscloudtrail/latest/APIReference/API_ListPublicKeys.html](https://docs.aws.amazon.com/awscloudtrail/latest/APIReference/API_ListPublicKeys.html) API. CloudTrail uses different private/public key pairs per Region. Each digest file is signed with a private key unique to its Region. Therefore, when you validate a digest file from a particular Region, you must look in the same Region for its corresponding public key. `digestSignatureAlgorithm` The algorithm used to sign the digest file. `logFiles.s3Bucket` The name of the Amazon S3 bucket for the log file. `logFiles.s3Object` The Amazon S3 object key of the current log file. `logFiles.newestEventTime` The UTC time of the most recent event in the log file. This time also corresponds to the time stamp of the log file itself. `logFiles.oldestEventTime` The UTC time of the oldest event in the log file. `logFiles.hashValue` The hexadecimal encoded hash value of the uncompressed log file content. `logFiles.hashAlgorithm` The hash algorithm used to hash the log file. ## Starting digest file When log file integrity validation is started, a starting digest file will be generated. A starting digest file will also be generated when log file integrity validation is restarted (by either disabling and then reenabling log file integrity validation, or by stopping logging and then restarting logging with validation enabled). In a starting digest file, the following fields relating to the previous digest file will be null: + `previousDigestS3Bucket` + `previousDigestS3Object` + `previousDigestHashValue` + `previousDigestHashAlgorithm` + `previousDigestSignature` ## 'Empty' digest files CloudTrail will deliver a digest file even when there has been no API activity in your account during the one hour period that the digest file represents. This can be useful when you need to assert that no log files were delivered during the hour reported by the digest file. The following example shows the contents of a digest file that recorded an hour when no API activity occurred. Note that the `logFiles:[ ]` field at the end of the digest file contents is empty. ``` { "awsAccountId": "111122223333", "digestStartTime": "2015-08-20T17:01:31Z", "digestEndTime": "2015-08-20T18:01:31Z", "digestS3Bucket": "amzn-s3-demo-bucket", "digestS3Object": "AWSLogs/111122223333/CloudTrail-Digest/us-east-2/2015/08/20/111122223333_CloudTrail-Digest_us-east-2_example-trail-name_us-east-2_20150820T180131Z.json.gz", "digestPublicKeyFingerprint": "31e8b5433410dfb61a9dc45cc65b22ff", "digestSignatureAlgorithm": "SHA256withRSA", "newestEventTime": null, "oldestEventTime": null, "previousDigestS3Bucket": "amzn-s3-demo-bucket", "previousDigestS3Object": "AWSLogs/111122223333/CloudTrail-Digest/us-east-2/2015/08/20/111122223333_CloudTrail-Digest_us-east-2_example-trail-name_us-east-2_20150820T170131Z.json.gz", "previousDigestHashValue": "ed96c4bac9eaa8fe9716ca0e515da51938be651b1db31d781956416a9d05cdfa", "previousDigestHashAlgorithm": "SHA-256", "previousDigestSignature": "82705525fb0fe7f919f9434e5b7138cb41793c776c7414f3520c0242902daa8cc8286b29263d2627f2f259471c745b1654af76e2073264b2510fd45236b3aea4d80c0e8e6455223d7bd54ff80af0edf22a5f14fa856626daec919f0591479aa4f213787ba1e1076328dcf8ff624e03a977fa5612dcf58594c590fd8c1c5b48bddf43fc84ecc00b41bedd0ff7f293c3e2de8dcdc78f98b03e17577f5822ba842399d69eb79921c0429773509520e08c8b518702d987dfbb3a4e5d8c5f17673ce1f989dfff82d4becf24e452f20d3bcac94ad50131f93e57f10155536acb54c60efbe9d57228c2b930bc6082b2318e3ccd36834a8e835b8d112dbf32145f445c11", "logFiles": [] } ``` ## Signature of the digest file The signature information for a digest file is located in two object metadata properties of the Amazon S3 digest file object. Each digest file has the following metadata entries: + `x-amz-meta-signature` The hexadecimal encoded value of the digest file signature. The following is an example signature: ``` 3be472336fa2989ef34de1b3c1bf851f59eb030eaff3e2fb6600a082a23f4c6a82966565b994f9de4a5989d053d9d15d20fc5c43e66358652d93326550a4acc5c5f541bb52e9b455897ab723bd7cbabfe963a406a41d600f3658f7a3135e5ed9fcae7b79bb5857d1e5eb78fcce8595ce0ade2f3ad1d9f2d62be7bc4660d83166ce24586489b7da9ee9883eaf0b9efabb5dd3cbba565cc4aab5c9c46c9fa7e9cda310afcc5e8adcd9e48d0597ec5f8174a52c3bebb3e845eeb1d18904fbf4cc14cd117080098e10022ddf55e017a9431446acad8560de0ba1e477af9f8a3048bc6196350adad0cc0cb4ab99b5e7c9944437a3c674a038009220684ced7be07b4f 28f1cc237f372264a51b611c01da429565def703539f4e71009051769469231bc22232fa260df02740047af532229885ea2b0e95ecd353326b7104941e0cbddb076a391f1fcf2923c19565f4841770a78723451aeb732ff1b6162dc40e601fc6720bc5325987942ebd817783b322f0ac77698523bf742fdea7aa44f4911b3101221b7e1233387f16a52077610498f4a1254211258e37da0fb4cb207aef593b4c1baa13674e85acc52046b3adb889e63331a66abac5de7e42ffdd6952987c31ae871650e130bd2e63bfe145b22bbd39ea192210f6df64d49b888a321e02d3fc4cf126accae30d2857ccd6b2286a7c9feba6c35c44161b24147d645e6ca26844ba 05d3ffcb5d2dd5dc28f8bb5b7993938e8a5f912a82b448a367eccb2ec0f198ba71e23eb0b97278cf65f3c8d1e652c6de33a22ca8428821ffc95bf8b726ba9f37cfbc20c54dc5bd6159bdea1c4d951b68cb8e0528852c55bb0c5e499ea60560f7c2bb3af7f694407da863a2594f7a2f2838cb09254afbaf8003587746e719a0437f85eeffae534f283f3837eb939a9bccc3c71573500661245891051231b580ac92d9e0e68c6f47ad38975f493e2c40e7f303353c4adc7d563ef1e875977afac2e085f0c824045d998c9543d8a3293ad3c063b7a109d0bfd84b0b1e3f72c4f057e744e6a2cf9cc97727b08584f44bfa47799c5072b60f0b619aea88a17de585e9 ``` + `x-amz-meta-signature-algorithm` The following shows an example value of the algorithm used to generate the digest signature: `SHA256withRSA` + `x-amz-meta-backfill-generation-timestamp` The UTC timestamp when the backfill digest was generated. This metadata property is present only in backfill digest files and is used to identify the correct public key for signature validation. The following is an example timestamp: `2025-05-20T00:00:00.000Z` ## Digest file chaining The fact that each digest file contains a reference to its previous digest file enables a "chaining" that permits validation tools like the AWS CLI to detect if a digest file has been deleted. It also allows the digest files in a specified time range to be successively inspected, starting with the most recent first. **Note** When you disable log file integrity validation, the chain of digest files is broken after one hour. CloudTrail will not create digest files for log files that were delivered during a period in which log file integrity validation was disabled. For example, if you enable log file integrity validation at noon on January 1, disable it at noon on January 2, and re-enable it at noon on January 10, digest files will not be created for the log files delivered from noon on January 2 to noon on January 10. The same applies whenever you stop CloudTrail logging or delete a trail. If your trail's [S3 bucket policy](create-s3-bucket-policy-for-cloudtrail.md) is misconfigured or CloudTrail experiences an unexpected service disruption, you might not receive all or some digest files. To confirm if your trail has any digest delivery errors, run the [https://docs.aws.amazon.com/awscloudtrail/latest/APIReference/API_GetTrailStatus.html](https://docs.aws.amazon.com/awscloudtrail/latest/APIReference/API_GetTrailStatus.html) command and check the `LatestDigestDeliveryError` parameter for errors. After the delivery issue is resolved (for example, by fixing the bucket policy), CloudTrail will attempt to redeliver any missing digest files. During the redelivery period, the digest files might be delivered out of order, so the chain might temporarily appear to be broken. If logging is stopped or the trail is deleted, CloudTrail will deliver a final digest file. This digest file can contain information for any remaining log files that cover events up to and including the `StopLogging` event. # Custom implementations of CloudTrail log file integrity validation Because CloudTrail uses industry standard, openly available cryptographic algorithms and hash functions, you can create your own tools to validate the integrity of CloudTrail log files. When log file integrity validation is enabled, CloudTrail delivers digest files to your Amazon S3 bucket. You can use these files to implement your own validation solution. For more information about digest files, see [CloudTrail digest file structure](cloudtrail-log-file-validation-digest-file-structure.md). This topic describes how digest files are signed, and then details the steps that you will need to take to implement a solution that validates the digest files and the log files that they reference. ## Understanding how CloudTrail digest files are signed CloudTrail digest files are signed with RSA digital signatures. For each digest file, CloudTrail does the following: 1. Creates a string for data signing based on designated digest file fields (described in the next section). 1. Gets a private key unique to the Region. 1. Passes the SHA-256 hash of the string and the private key to the RSA signing algorithm, which produces a digital signature. 1. Encodes the byte code of the signature into hexadecimal format. 1. Puts the digital signature into the `x-amz-meta-signature` metadata property of the Amazon S3 digest file object. ### Contents of the data signing string The following CloudTrail objects are included in the string for data signing: + The ending timestamp of the digest file in UTC extended format (for example, `2015-05-08T07:19:37Z`) + The current digest file S3 path + The hexadecimal-encoded SHA-256 hash of the current digest file + The hexadecimal-encoded signature of the previous digest file The format for calculating this string and an example string are provided later in this document. ## Custom validation implementation steps When implementing a custom validation solution, you will need to validate the digest file first, and then the log files that it references. ### Validate the digest file To validate a digest file, you need its signature, the public key whose private key was used to signed it, and a data signing string that you compute. 1. Get the digest file. 1. Verify that the digest file has been retrieved from its original location. 1. Get the hexadecimal-encoded signature of the digest file. 1. Get the hexadecimal-encoded fingerprint of the public key whose private key was used to sign the digest file. 1. Retrieve the public keys for the time range corresponding to the digest file. 1. From among the public keys retrieved, choose the public key whose fingerprint matches the fingerprint in the digest file. 1. Using the digest file hash and other digest file fields, recreate the data signing string used to verify the digest file signature. 1. Validate the signature by passing in the SHA-256 hash of the string, the public key, and the signature as parameters to the RSA signature verification algorithm. If the result is true, the digest file is valid. ### Validate the log files If the digest file is valid, validate each of the log files that the digest file references. 1. To validate the integrity of a log file, compute its SHA-256 hash value on its uncompressed content and compare the results with the hash for the log file recorded in hexadecimal in the digest. If the hashes match, the log file is valid. 1. By using the information about the previous digest file that is included in the current digest file, validate the previous digest files and their corresponding log files in succession. The following sections describe these steps in detail. ### A. Get the digest file The first steps are to get the most recent digest file, verify that you have retrieved it from its original location, verify its digital signature, and get the fingerprint of the public key. 1. Using S3 [https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetObject.html](https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetObject.html) or the AmazonS3Client class (for example), get the most recent digest file from your Amazon S3 bucket for the time range that you want to validate. 1. Check that the S3 bucket and S3 object used to retrieve the file match the S3 bucket S3 object locations that are recorded in the digest file itself. 1. Next, get the digital signature of the digest file from the `x-amz-meta-signature` metadata property of the digest file object in Amazon S3. 1. In the digest file, get the fingerprint of the public key whose private key was used to sign the digest file from the `digestPublicKeyFingerprint` field. ### B. Retrieve the public key for validating the digest file To get the public key to validate the digest file, you can use either the AWS CLI or the CloudTrail API. In both cases, you specify a time range (that is, a start time and end time) for the digest files that you want to validate. One or more public keys may be returned for the time range that you specify. The returned keys may have validity time ranges that overlap. **Note** Because CloudTrail uses different private/public key pairs per Region, each digest file is signed with a private key unique to its Region. Therefore, when you validate a digest file from a particular Region, you must retrieve its public key from the same Region. #### Use the AWS CLI to retrieve public keys To retrieve public keys for digest files by using the AWS CLI, use the `cloudtrail list-public-keys` command. The command has the following format: `aws cloudtrail list-public-keys [--start-time ] [--end-time ]` The start-time and end-time parameters are UTC timestamps and are optional. If not specified, the current time is used, and the currently active public key or keys are returned. **Sample Response** The response will be a list of JSON objects representing the key (or keys) returned: ``` { "publicKeyList": [ { "ValidityStartTime": "1436317441.0", "ValidityEndTime": "1438909441.0", "Value": "MIIBCgKCAQEAn11L2YZ9h7onug2ILi1MWyHiMRsTQjfWE+pHVRLk1QjfWhirG+lpOa8NrwQ/r7Ah5bNL6HepznOU9XTDSfmmnP97mqyc7z/upfZdS/AHhYcGaz7n6Wc/RRBU6VmiPCrAUojuSk6/GjvA8iOPFsYDuBtviXarvuLPlrT9kAd4Lb+rFfR5peEgBEkhlzc5HuWO7S0y+KunqxX6jQBnXGMtxmPBPP0FylgWGNdFtks/4YSKcgqwH0YDcawP9GGGDAeCIqPWIXDLG1jOjRRzWfCmD0iJUkz8vTsn4hq/5ZxRFE7UBAUiVcGbdnDdvVfhF9C3dQiDq3k7adQIziLT0cShgQIDAQAB", "Fingerprint": "8eba5db5bea9b640d1c96a77256fe7f2" }, { "ValidityStartTime": "1434589460.0", "ValidityEndTime": "1437181460.0", "Value": "MIIBCgKCAQEApfYL2FiZhpN74LNWVUzhR+VheYhwhYm8w0n5Gf6i95ylW5kBAWKVEmnAQG7BvS5g9SMqFDQx52fW7NWV44IvfJ2xGXT+wT+DgR6ZQ+6yxskQNqV5YcXj4Aa5Zz4jJfsYjDuO2MDTZNIzNvBNzaBJ+r2WIWAJ/Xq54kyF63B6WE38vKuDE7nSd1FqQuEoNBFLPInvgggYe2Ym1Refe2z71wNcJ2kY+q0h1BSHrSM8RWuJIw7MXwF9iQncg9jYzUlNJomozQzAG5wSRfbplcCYNY40xvGd/aAmO0m+Y+XFMrKwtLCwseHPvj843qVno6x4BJN9bpWnoPo9sdsbGoiK3QIDAQAB", "Fingerprint": "8933b39ddc64d26d8e14ffbf6566fee4" }, { "ValidityStartTime": "1434589370.0", "ValidityEndTime": "1437181370.0", "Value": "MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAqlzPJbvZJ42UdcmLfPUqXYNfOs6I8lCfao/tOs8CmzPOEdtLWugB9xoIUz78qVHdKIqxbaG4jWHfJBiOSSFBM0lt8cdVo4TnRa7oG9io5pysS6DJhBBAeXsicufsiFJR+wrUNh8RSLxL4k6G1+BhLX20tJkZ/erT97tDGBujAelqseGg3vPZbTx9SMfOLN65PdLFudLP7Gat0Z9p5jw/rjpclKfo9Bfc3heeBxWGKwBBOKnFAaN9V57pOaosCvPKmHd9bg7jsQkI9Xp22IzGLsTFJZYVA3KiTAElDMu80iFXPHEq9hKNbt9e4URFam+1utKVEiLkR2disdCmPTK0VQIDAQAB", "Fingerprint": "31e8b5433410dfb61a9dc45cc65b22ff" } ] } ``` #### Use the CloudTrail API to retrieve public keys To retrieve public keys for digest files by using the CloudTrail API, pass in start time and end time values to the `ListPublicKeys` API. The `ListPublicKeys` API returns the public keys whose private keys were used to sign digest files within the specified time range. For each public key, the API also returns the corresponding fingerprint. ##### `ListPublicKeys` This section describes the request parameters and response elements for the `ListPublicKeys` API. **Note** The encoding for the binary fields for `ListPublicKeys` is subject to change. **Request Parameters** **** | Name | Description | | --- | --- | | StartTime | Optionally specifies, in UTC, the start of the time range to look up public keys for CloudTrail digest files. If StartTime is not specified, the current time is used, and the current public key is returned. Type: DateTime | | EndTime | Optionally specifies, in UTC, the end of the time range to look up public keys for CloudTrail digest files. If EndTime is not specified, the current time is used. Type: DateTime | **Response Elements** `PublicKeyList`, an array of `PublicKey` objects that contains: **** | | | | --- |--- | | Name | Description | | Value | The DER encoded public key value in PKCS \$11 format. Type: Blob | | ValidityStartTime | The starting time of validity of the public key. Type: DateTime | | ValidityEndTime | The ending time of validity of the public key. Type: DateTime | | Fingerprint | The fingerprint of the public key. The fingerprint can be used to identify the public key that you must use to validate the digest file. Type: String | ### C. Choose the public key to use for validation From among the public keys retrieved by `list-public-keys` or `ListPublicKeys`, choose the public key returned whose fingerprint matches the fingerprint recorded in the `digestPublicKeyFingerprint` field of the digest file. This is the public key that you will use to validate the digest file. ### D. Recreate the data signing string Now that you have the signature of the digest file and associated public key, you need to calculate the data signing string. After you have calculated the data signing string, you will have the inputs needed to verify the signature. The data signing string has the following format: ``` Data_To_Sign_String = Digest_End_Timestamp_in_UTC_Extended_format + '\n' + Current_Digest_File_S3_Path + '\n' + Hex(Sha256(current-digest-file-content)) + '\n' + Previous_digest_signature_in_hex ``` An example `Data_To_Sign_String` follows. ``` 2015-08-12T04:01:31Z amzn-s3-demo-bucket/AWSLogs/111122223333/CloudTrail-Digest/us-east-2/2015/08/12/111122223333_us-east-2_CloudTrail-Digest_us-east-2_20150812T040131Z.json.gz 4ff08d7c6ecd6eb313257e839645d20363ee3784a2328a7d76b99b53cc9bcacd 6e8540b83c3ac86a0312d971a225361d28ed0af20d70c211a2d405e32abf529a8145c2966e3bb47362383a52441545ed091fb81 d4c7c09dd152b84e79099ce7a9ec35d2b264eb92eb6e090f1e5ec5d40ec8a0729c02ff57f9e30d5343a8591638f8b794972ce15bb3063a01972 98b0aee2c1c8af74ec620261529265e83a9834ebef6054979d3e9a6767dfa6fdb4ae153436c567d6ae208f988047ccfc8e5e41f7d0121e54ed66b1b904f80fb2ce304458a2a6b91685b699434b946c52589e9438f8ebe5a0d80522b2f043b3710b87d2cda43e5c1e0db921d8d540b9ad5f6d4$31b1f4a8ef2d758424329583897339493a082bb36e782143ee5464b4e3eb4ef6 ``` After you recreate this string, you can validate the digest file. ### E. Validate the digest file Pass the SHA-256 hash of the recreated data signing string, digital signature, and public key to the RSA signature verification algorithm. If the output is true, the signature of the digest file is verified and the digest file is valid. ### F. Validate the log files After you have validated the digest file, you can validate the log files it references. The digest file contains the SHA-256 hashes of the log files. If one of the log files was modified after CloudTrail delivered it, the SHA-256 hashes will change, and the signature of digest file will not match. The following shows how validate the log files: 1. Do an `S3 Get` of the log file using the S3 location information in the digest file's `logFiles.s3Bucket` and `logFiles.s3Object` fields. 1. If the `S3 Get` operation is successful, iterate through the log files listed in the digest file's logFiles array using the following steps: 1. Retrieve the original hash of the file from the `logFiles.hashValue` field of the corresponding log in the digest file. 1. Hash the uncompressed contents of the log file with the hashing algorithm specified in `logFiles.hashAlgorithm`. 1. Compare the hash value that you generated with the one for the log in the digest file. If the hashes match, the log file is valid. ### G. Validate additional digest and log files In each digest file, the following fields provide the location and signature of the previous digest file: + `previousDigestS3Bucket` + `previousDigestS3Object` + `previousDigestSignature` Use this information to visit previous digest files sequentially, validating the signature of each and the log files that they reference by using the steps in the previous sections. The only difference is that for previous digest files, you do not need to retrieve the digital signature from the digest file object's Amazon S3 metadata properties. The signature for the previous digest file is provided for you in the `previousDigestSignature` field. You can go back until the starting digest file is reached, or until the chain of digest files is broken, whichever comes first. ## Validating digest and log files offline When validating digest and log files offline, you can generally follow the procedures described in the previous sections. However, you must take into account the following areas: ### Handling the most recent digest file The digital signature of the most recent (that is, "current") digest file is in the Amazon S3 metadata properties of the digest file object. In an offline scenario, the digital signature for the current digest file will not be available. Two possible ways of handling this are: + Since the digital signature for the previous digest file is in the current digest file, start validating from the next-to-last digest file. With this method, the most recent digest file cannot be validated. + As a preliminary step, obtain the signature for the current digest file from the digest file object's metadata properties and then store it securely offline. This would allow the current digest file to be validated in addition to the previous files in the chain. ### Path resolution Fields in the downloaded digest files like `s3Object` and `previousDigestS3Object` will still be pointing to Amazon S3 online locations for log files and digest files. An offline solution must find a way to reroute these to the current path of the downloaded log and digest files. ### Public keys In order to validate offline, all of the public keys that you need for validating log files in a given time range must first be obtained online (by calling `ListPublicKeys`, for example) and then stored securely offline. This step must be repeated whenever you want to validate additional files outside the initial time range that you specified. ## Sample validation snippet The following sample snippet provides skeleton code for validating CloudTrail digest and log files. The skeleton code is online/offline agnostic; that is, it is up to you to decide whether to implement it with or without online connectivity to AWS. The suggested implementation uses the [Java Cryptography Extension (JCE)](https://en.wikipedia.org/wiki/Java_Cryptography_Extension) and [Bouncy Castle](https://www.bouncycastle.org/) as a security provider. The sample snippet shows: + How to create the data signing string used to validate the digest file signature. + How to verify the digest file signature. + How to verify the log file hashes. + A code structure for validating a chain of digest files. ``` import java.util.Arrays; import java.security.MessageDigest; import java.security.KeyFactory; import java.security.PublicKey; import java.security.Security; import java.security.Signature; import java.security.spec.X509EncodedKeySpec; import org.json.JSONObject; import org.bouncycastle.jce.provider.BouncyCastleProvider; import org.apache.commons.codec.binary.Hex; public class DigestFileValidator { public void validateDigestFile(String digestS3Bucket, String digestS3Object, String digestSignature) { // Using the Bouncy Castle provider as a JCE security provider - http://www.bouncycastle.org/ Security.addProvider(new BouncyCastleProvider()); // Load the digest file from S3 (using Amazon S3 Client) or from your local copy JSONObject digestFile = loadDigestFileInMemory(digestS3Bucket, digestS3Object); // Check that the digest file has been retrieved from its original location if (!digestFile.getString("digestS3Bucket").equals(digestS3Bucket) || !digestFile.getString("digestS3Object").equals(digestS3Object)) { System.err.println("Digest file has been moved from its original location."); } else { // Compute digest file hash MessageDigest messageDigest = MessageDigest.getInstance("SHA-256"); messageDigest.update(convertToByteArray(digestFile)); byte[] digestFileHash = messageDigest.digest(); messageDigest.reset(); // Compute the data to sign String dataToSign = String.format("%s%n%s/%s%n%s%n%s", digestFile.getString("digestEndTime"), digestFile.getString("digestS3Bucket"), digestFile.getString("digestS3Object"), // Constructing the S3 path of the digest file as part of the data to sign Hex.encodeHexString(digestFileHash), digestFile.getString("previousDigestSignature")); byte[] signatureContent = Hex.decodeHex(digestSignature); /* NOTE: To find the right public key to verify the signature, call CloudTrail ListPublicKey API to get a list of public keys, then match by the publicKeyFingerprint in the digest file. Also, the public key bytes returned from ListPublicKey API are DER encoded in PKCS#1 format: PublicKeyInfo ::= SEQUENCE { algorithm AlgorithmIdentifier, PublicKey BIT STRING } AlgorithmIdentifier ::= SEQUENCE { algorithm OBJECT IDENTIFIER, parameters ANY DEFINED BY algorithm OPTIONAL } */ pkcs1PublicKeyBytes = getPublicKey(digestFile.getString("digestPublicKeyFingerprint"))); // Transform the PKCS#1 formatted public key to x.509 format. RSAPublicKey rsaPublicKey = RSAPublicKey.getInstance(pkcs1PublicKeyBytes); AlgorithmIdentifier rsaEncryption = new AlgorithmIdentifier(PKCSObjectIdentifiers.rsaEncryption, null); SubjectPublicKeyInfo publicKeyInfo = new SubjectPublicKeyInfo(rsaEncryption, rsaPublicKey); // Create the PublicKey object needed for the signature validation PublicKey publicKey = KeyFactory.getInstance("RSA", "BC").generatePublic(new X509EncodedKeySpec(publicKeyInfo.getEncoded())); // Verify signature Signature signature = Signature.getInstance("SHA256withRSA", "BC"); signature.initVerify(publicKey); signature.update(dataToSign.getBytes("UTF-8")); if (signature.verify(signatureContent)) { System.out.println("Digest file signature is valid, validating log files…"); for (int i = 0; i < digestFile.getJSONArray("logFiles").length(); i++) { JSONObject logFileMetadata = digestFile.getJSONArray("logFiles").getJSONObject(i); // Compute log file hash byte[] logFileContent = loadUncompressedLogFileInMemory( logFileMetadata.getString("s3Bucket"), logFileMetadata.getString("s3Object") ); messageDigest.update(logFileContent); byte[] logFileHash = messageDigest.digest(); messageDigest.reset(); // Retrieve expected hash for the log file being processed byte[] expectedHash = Hex.decodeHex(logFileMetadata.getString("hashValue")); boolean signaturesMatch = Arrays.equals(expectedHash, logFileHash); if (!signaturesMatch) { System.err.println(String.format("Log file: %s/%s hash doesn't match.\tExpected: %s Actual: %s", logFileMetadata.getString("s3Bucket"), logFileMetadata.getString("s3Object"), Hex.encodeHexString(expectedHash), Hex.encodeHexString(logFileHash))); } else { System.out.println(String.format("Log file: %s/%s hash match", logFileMetadata.getString("s3Bucket"), logFileMetadata.getString("s3Object"))); } } } else { System.err.println("Digest signature failed validation."); } System.out.println("Digest file validation completed."); if (chainValidationIsEnabled()) { // This enables the digests' chain validation validateDigestFile( digestFile.getString("previousDigestS3Bucket"), digestFile.getString("previousDigestS3Object"), digestFile.getString("previousDigestSignature")); } } } } ```