# Create Amazon Data Lifecycle Manager custom policy for EBS snapshots The following procedure shows you how to use Amazon Data Lifecycle Manager to automate Amazon EBS snapshot lifecycles. **Topics** + [Create a snapshot lifecycle policy](#create-snap-policy) + [Considerations for snapshot lifecycle policies](#snapshot-considerations) + [Additional resources](#snapshot-additional-resources) + [Automate application-consistent snapshots](automate-app-consistent-backups.md) + [Other use cases for pre and post scripts](script-other-use-cases.md) + [How pre and post scripts work](script-flow.md) + [Identify snapshots created with pre and post scripts](dlm-script-tags.md) + [Monitor pre and post scripts](dlm-script-monitoring.md) ## Create a snapshot lifecycle policy Use one of the following procedures to create a snapshot lifecycle policy. ------ #### [ Console ] **To create a snapshot policy** 1. Open the Amazon EC2 console at [https://console.aws.amazon.com/ec2/](https://console.aws.amazon.com/ec2/). 1. In the navigation pane, choose **Elastic Block Store**, **Lifecycle Manager**, and then choose **Create lifecycle policy**. 1. On the **Select policy type** screen, choose **EBS snapshot policy** and then choose **Next**. 1. In the **Target resources** section, do the following: 1. For **Target resource types**, choose the type of resource to back up. Choose `Volume` to create snapshots of individual volumes, or choose `Instance` to create multi-volume snapshots from the volumes attached to an instance. 1. (*Outpost and Local Zone customers only*) Specify where the target resources are located. For **Target resource location**, specify where the target resources are located. + To target resources in a Region, choose **AWS Region**. Amazon Data Lifecycle Manager will back up all resources of the specified type that have matching target tags in the current Region only. Snapshots are created in the same Region. + To target resources in Local Zones, choose **AWS Local Zones**. Amazon Data Lifecycle Manager will back up all resources of the specified type that have matching target tags across all Local Zones in the current Region only. Snapshots can be created in the same Local Zone as the source resource, or in its parent Region. + To target resources an Outpost, choose **AWS Outpost**. Amazon Data Lifecycle Manager will back up all resources of the specified type that have matching target tags across all Outposts in your account. Snapshots can be created on the same Outpost as the source resource, or in its parent Region. 1. For **Target resource tags**, choose the resource tags that identify the volumes or instances to back up. Only resources that have the specified tag key and value pairs are backed up by the policy. 1. For **Description**, enter a brief description for the policy. 1. For **IAM role**, choose the IAM role that has permissions to manage snapshots and to describe volumes and instances. To use the default role provided by Amazon Data Lifecycle Manager. choose **Default role**. Alternatively, to use a custom IAM role that you previously created, choose **Choose another role** and then select the role to use. 1. For **Policy tags**, add the tags to apply to the lifecycle policy. You can use these tags to identify and categorize your policies. 1. For **Policy status**, choose **Enable** to start the policy runs at the next scheduled time, or **Disable policy** to prevent the policy from running. If you do not enable the policy now, it will not start creating snapshots until you manually enable it after creation. 1. (*Policies that target instances only*) Exclude volumes from multi-volume snapshot sets. By default, Amazon Data Lifecycle Manager will create snapshots of all the volumes attached to targeted instances. However, you can choose to create snapshots of a subset of the attached volumes. In the **Parameters** section, do the following: + If you do not want to create snapshots of the root volumes attached to the targeted instances, select **Exclude root volume**. If you select this option, only the data (non-root) volumes that are attached to targeted instances will be included in the multi-volume snapshot sets. + If you want to create snapshots of a subset of the data (non-root) volumes attached to the targeted instances, select **Exclude specific data volumes**, and then specify the tags that are to be used to identify the data volumes that should not be snapshotted. Amazon Data Lifecycle Manager will not create snapshots of data volumes that have any of the specified tags. Amazon Data Lifecycle Manager will create snapshots only of data volumes that do not have any of the specified tags. 1. Choose **Next**. 1. On the **Configure schedule** screen, configure the policy schedules. A policy can have up to 4 schedules. Schedule 1 is mandatory. Schedules 2, 3, and 4 are optional. For each policy schedule that you add, do the following: 1. In the **Schedule details** section do the following: 1. For **Schedule name**, specify a descriptive name for the schedule. 1. For **Frequency** and the related fields, configure the interval between policy runs. You can configure policy runs on a daily, weekly, monthly, or yearly schedule. Alternatively, choose **Custom cron expression** to specify an interval of up to one year. For more information, see [Cron and rate expressions](https://docs.aws.amazon.com/eventbridge/latest/userguide/eb-scheduled-rule-pattern.html) in the *Amazon EventBridge User Guide*. **Note** If you need to enable **snapshot archiving** for the schedule, then you must select either the **monthly** or **yearly** frequency, or you must specify a cron expression with a creation frequency of at least 28 days. If specify a monthly frequency that creates snapshots on a specific day in a specific week (for example, the second Thursday of the month), then for count-based schedule, the retention count for the archive tier must be 4 or more. 1. For **Starting at**, specify the time at which the policy runs are scheduled to start. The first policy run starts within an hour after the scheduled time. The time must be entered in the `hh:mm` UTC format. 1. For **Retention type**, specify the retention policy for snapshots created by the schedule. You can retain snapshots based on either their total count or their age. + Count-based retention + With snapshot archiving disabled, the range is `1` to `1000`. When the retention threshold is reached, the oldest snapshot is permanently deleted. + With snapshot archiving enabled, the range is `0` (archive immediately after creation) to `1000`. When the retention threshold is reached, the oldest snapshot is converted to a full snapshot and it is moved to the archive tier. + Age-based retention + With snapshot archiving disabled, the range is `1` day to `100` years. When the retention threshold is reached, the oldest snapshot is permanently deleted. + With snapshot archiving enabled, the range is `0` days (archive immediately after creation) to `100` years. When the retention threshold is reached, the oldest snapshot is converted to a full snapshot and it is moved to the archive tier. **Note** All schedules must have the same retention type (age-based or count-based). You can specify the retention type for Schedule 1 only. Schedules 2, 3, and 4 inherit the retention type from Schedule 1. Each schedule can have its own retention count or period. If you enable fast snapshot restore, cross-Region copy, or snapshot sharing, then you must specify a retention count of `1` or more, or a retention period of `1` day or longer. 1. (*AWS Outposts and Local Zone customers only*) Specify the snapshot destination. For **Snapshot destination**, specify the destination for snapshots created by the policy. + If the policy targets resources in a Region, snapshots must be created in the same Region. AWS Region is selected for you. + If the policy targets resources in a Local Zone, you can create snapshots in the same Local Zone as the source resource, or in its parent Region. + If the policy targets resources on an Outpost, you can create snapshots on the same Outpost as the source resource, or in its parent Region. 1. Configure tagging for snapshots. In the **Tagging** section, do the following: 1. To copy all of the user-defined tags from the source volume to the snapshots created by the schedule, select **Copy tags from source**. 1. To specify additional tags to assign to snapshots created by this schedule, choose **Add tags**. 1. Configure pre and post scripts for application-consistent snapshots. For more information, see [Automate application-consistent snapshots with Data Lifecycle Manager](automate-app-consistent-backups.md). 1. (*Policies that target volumes only*) Configure snapshot archiving. In the **Snapshot archiving** section, do the following: **Note** You can enable snapshot archiving for only one schedule in a policy. 1. To enable snapshot archiving for the schedule, select **Archive snapshots created by this schedule**. **Note** You can enable snapshot archiving only if the snapshot creation frequency is monthly or yearly, or if you specify a cron expression with a creation frequency of at least 28 days. 1. Specify the retention rule for snapshots in the archive tier. + For **count-based schedules**, specify the number of snapshots to retain in the archive tier. When the retention threshold is reached, the oldest snapshot is permanently deleted from the archive tier. For example, if you specify 3, the schedule will retain a maximum of 3 snapshots in the archive tier. When the fourth snapshot is archived, the oldest of the three existing snapshots in the archive tier is deleted. + For **age-based schedules**, specify the time period for which to retain snapshots in the archive tier. When the retention threshold is reached, the oldest snapshot is permanently deleted from the archive tier. For example, if you specify 120 days, the schedule will automatically delete snapshots from the archive tier when they reach that age. **Important** The minimum retention period for archived snapshots is 90 days. You must specify a retention rule that retains the snapshot for at least 90 days. 1. Enable fast snapshot restore. To enable fast snapshot restore for snapshots created by the schedule, in the **Fast snapshot restore** section, select **Enable fast snapshot restore**. If you enable fast snapshot restore, you must choose the Availability Zones in which to enable it. If the schedule uses an age-based retention schedule, you must specify the period for which to enable fast snapshot restore for each snapshot. If the schedule uses count-based retention, you must specify the maximum number of snapshots to enable for fast snapshot restore. If the schedule creates snapshots on an Outpost, you can't enable fast snapshot restore. Fast snapshot restore is not supported with local snapshots that are stored on an Outpost. **Note** You are billed for each minute that fast snapshot restore is enabled for a snapshot in a particular Availability Zone. Charges are pro-rated with a minimum of one hour. 1. Configure cross-Region copy. To copy snapshots created by the schedule to an Outpost or to a different Region, in the **Cross-Region copy** section, select **Enable cross-Region copy**. If the schedule creates snapshots in a Region, you can copy the snapshots to up to three additional Regions or Outposts in your account. You must specify a separate cross-Region copy rule for each destination Region or Outpost. For each Region or Outpost, you can choose different retention policies and you can choose whether to copy all tags or no tags. If the source snapshot is encrypted, or if encryption by default is enabled, the copied snapshots are encrypted. If the source snapshot is unencrypted, you can enable encryption. If you do not specify a KMS key, the snapshots are encrypted using the default KMS key for EBS encryption in each destination Region. If you specify a KMS key for the destination Region, then the selected IAM role must have access to the KMS key. **Note** You must ensure that you do not exceed the number of concurrent snapshot copies per Region. If the policy creates snapshots on an Outpost, then you can't copy the snapshots to a Region or to another Outpost and the cross-Region copy settings are not available. 1. Configure cross-account sharing. In the **Cross-account sharing**, configure the policy to automatically share the snapshots created by the schedule with other AWS accounts. Do the following: 1. To enable sharing with other AWS accounts, select **Enable cross-account sharing**. 1. To add the accounts with which to share the snapshots, choose **Add account**, enter the 12-digit AWS account ID, and choose **Add**. 1. To automatically unshare shared snapshots after a specific period, select**Unshare automatically**. If you choose to automatically unshare shared snapshots, the period after which to automatically unshare the snapshots cannot be longer than the period for which the policy retains its snapshots. For example, if the policy's retention configuration retains snapshots for a period of 5 days, you can configure the policy to automatically unshare shared snapshots after periods up to 4 days. This applies to policies with age-based and count-based snapshot retention configurations. If you do not enable automatic unsharing, the snapshot is shared until it is deleted. **Note** You can only share snapshots that are unencrypted or that are encrypted using a customer managed key. You can't share snapshots that are encrypted with the default EBS encryption KMS key. If you share encrypted snapshots, then you must also share the KMS key that was used to encrypt the source volume with the target accounts. For more information, see [ Allowing users in other accounts to use a KMS key](https://docs.aws.amazon.com//kms/latest/developerguide/key-policy-modifying-external-accounts.html) in the *AWS Key Management Service Developer Guide*. 1. To add additional schedules, choose **Add another schedule**, which is located at the top of the screen. For each additional schedule, complete the fields as described previously in this topic. 1. After you have added the required schedules, choose **Review policy**. 1. Review the policy summary, and then choose **Create policy**. **Note** If you get the `Role with name AWSDataLifecycleManagerDefaultRole already exists` error, see [Troubleshoot Amazon Data Lifecycle Manager issues](dlm-troubleshooting.md) for more information. ------ #### [ Command line ] Use the [create-lifecycle-policy](https://docs.aws.amazon.com/cli/latest/reference/dlm/create-lifecycle-policy.html) command to create a snapshot lifecycle policy. For `PolicyType`, specify `EBS_SNAPSHOT_MANAGEMENT`. **Note** To simplify the syntax, the following examples use a JSON file, `policyDetails.json`, that includes the policy details. **Example 1—Snapshot lifecycle policy with two schedules** This example creates a snapshot lifecycle policy that creates snapshots of all volumes that have a tag key of `costcenter` with a value of `115`. The policy includes two schedules. The first schedule creates a snapshot every day at 03:00 UTC. The second schedule creates a weekly snapshot every Friday at 17:00 UTC. ``` aws dlm create-lifecycle-policy \ --description "My volume policy" \ --state ENABLED \ --execution-role-arn arn:aws:iam::12345678910:role/AWSDataLifecycleManagerDefaultRole \ --policy-details file://policyDetails.json ``` The following is an example of the `policyDetails.json` file. ``` { "PolicyType": "EBS_SNAPSHOT_MANAGEMENT", "ResourceTypes": [ "VOLUME" ], "TargetTags": [{ "Key": "costcenter", "Value": "115" }], "Schedules": [{ "Name": "DailySnapshots", "TagsToAdd": [{ "Key": "type", "Value": "myDailySnapshot" }], "CreateRule": { "Interval": 24, "IntervalUnit": "HOURS", "Times": [ "03:00" ] }, "RetainRule": { "Count": 5 }, "CopyTags": false }, { "Name": "WeeklySnapshots", "TagsToAdd": [{ "Key": "type", "Value": "myWeeklySnapshot" }], "CreateRule": { "CronExpression": "cron(0 17 ? * FRI *)" }, "RetainRule": { "Count": 5 }, "CopyTags": false } ]} ``` If the request succeeds, the command returns the ID of the newly created policy. The following is example output. ``` { "PolicyId": "policy-0123456789abcdef0" } ``` **Example 2—Snapshot lifecycle policy that targets instances and creates snapshots of a subset of data (non-root) volumes** This example creates a snapshot lifecycle policy that creates multi-volume snapshot sets from instances tagged with `code=production`. The policy includes only one schedule. The schedule does not create snapshots of the data volumes that are tagged with `code=temp`. ``` aws dlm create-lifecycle-policy \ --description "My volume policy" \ --state ENABLED \ --execution-role-arn arn:aws:iam::12345678910:role/AWSDataLifecycleManagerDefaultRole \ --policy-details file://policyDetails.json ``` The following is an example of the `policyDetails.json` file. ``` { "PolicyType": "EBS_SNAPSHOT_MANAGEMENT", "ResourceTypes": [ "INSTANCE" ], "TargetTags": [{ "Key": "code", "Value": "production" }], "Parameters": { "ExcludeDataVolumeTags": [{ "Key": "code", "Value": "temp" }] }, "Schedules": [{ "Name": "DailySnapshots", "TagsToAdd": [{ "Key": "type", "Value": "myDailySnapshot" }], "CreateRule": { "Interval": 24, "IntervalUnit": "HOURS", "Times": [ "03:00" ] }, "RetainRule": { "Count": 5 }, "CopyTags": false } ]} ``` If the request succeeds, the command returns the ID of the newly created policy. The following is example output. ``` { "PolicyId": "policy-0123456789abcdef0" } ``` **Example 3—Snapshot lifecycle policy that automates local snapshots of Outpost resources** This example creates a snapshot lifecycle policy that creates snapshots of volumes tagged with `team=dev` across all of your Outposts. The policy creates the snapshots on the same Outposts as the source volumes. The policy creates snapshots every `12` hours starting at `00:00` UTC. ``` aws dlm create-lifecycle-policy \ --description "My local snapshot policy" \ --state ENABLED \ --execution-role-arn arn:aws:iam::12345678910:role/AWSDataLifecycleManagerDefaultRole \ --policy-details file://policyDetails.json ``` The following is an example of the `policyDetails.json` file. ``` { "PolicyType": "EBS_SNAPSHOT_MANAGEMENT", "ResourceTypes": "VOLUME", "ResourceLocations": "OUTPOST", "TargetTags": [{ "Key": "team", "Value": "dev" }], "Schedules": [{ "Name": "on-site backup", "CreateRule": { "Interval": 12, "IntervalUnit": "HOURS", "Times": [ "00:00" ], "Location": [ "OUTPOST_LOCAL" ] }, "RetainRule": { "Count": 1 }, "CopyTags": false } ]} ``` **Example 4—Snapshot lifecycle policy that creates snapshots in a Region and copies them to an Outpost** The following example policy creates snapshots of volumes that are tagged with `team=dev`. Snapshots are created in the same Region as the source volume. Snapshots are created every `12` hours starting at `00:00` UTC, and retains a maximum of `1` snapshot. The policy also copies the snapshots to Outpost `arn:aws:outposts:us-east-1:123456789012:outpost/op-1234567890abcdef0`, encrypts the copied snapshots using the default encryption KMS key, and retains the copies for `1` month. ``` aws dlm create-lifecycle-policy \ --description "Copy snapshots to Outpost" \ --state ENABLED \ --execution-role-arn arn:aws:iam::12345678910:role/AWSDataLifecycleManagerDefaultRole \ --policy-details file://policyDetails.json ``` The following is an example of the `policyDetails.json` file. ``` { "PolicyType": "EBS_SNAPSHOT_MANAGEMENT", "ResourceTypes": "VOLUME", "ResourceLocations": "CLOUD", "TargetTags": [{ "Key": "team", "Value": "dev" }], "Schedules": [{ "Name": "on-site backup", "CopyTags": false, "CreateRule": { "Interval": 12, "IntervalUnit": "HOURS", "Times": [ "00:00" ], "Location": "CLOUD" }, "RetainRule": { "Count": 1 }, "CrossRegionCopyRules" : [ { "Target": "arn:aws:outposts:us-east-1:123456789012:outpost/op-1234567890abcdef0", "Encrypted": true, "CopyTags": true, "RetainRule": { "Interval": 1, "IntervalUnit": "MONTHS" } }] } ]} ``` **Example 5—Snapshot lifecycle policy with an archive-enabled, age-based schedule** This example creates a snapshot lifecycle policy that targets volumes tagged with `Name=Prod`. The policy has one age-based schedule that creates snapshots on the first day of each month at 09:00. The schedule retains each snapshot in the standard tier for one day, after which it moves them to the archive tier. Snapshots are stored in the archive tier for 90 days before being deleted. ``` aws dlm create-lifecycle-policy \ --description "Copy snapshots to Outpost" \ --state ENABLED \ --execution-role-arn arn:aws:iam::12345678910:role/AWSDataLifecycleManagerDefaultRole \ --policy-details file://policyDetails.json ``` The following is an example of the `policyDetails.json` file. ``` { "ResourceTypes": [ "VOLUME"], "PolicyType": "EBS_SNAPSHOT_MANAGEMENT", "Schedules" : [ { "Name": "sched1", "TagsToAdd": [ {"Key":"createdby","Value":"dlm"} ], "CreateRule": { "CronExpression": "cron(0 9 1 * ? *)" }, "CopyTags": true, "RetainRule":{ "Interval": 1, "IntervalUnit": "DAYS" }, "ArchiveRule": { "RetainRule":{ "RetentionArchiveTier": { "Interval": 90, "IntervalUnit": "DAYS" } } } } ], "TargetTags": [ { "Key": "Name", "Value": "Prod" } ] } ``` **Example 6—Snapshot lifecycle policy with an archive-enabled, count-based schedule** This example creates a snapshot lifecycle policy that targets volumes tagged with `Purpose=Test`. The policy has one count-based schedule that creates snapshots on the first day of each month at 09:00. The schedule archives snapshots immediately after creation and retains a maximum of three snapshots in the archive tier. ``` aws dlm create-lifecycle-policy \ --description "Copy snapshots to Outpost" \ --state ENABLED \ --execution-role-arn arn:aws:iam::12345678910:role/AWSDataLifecycleManagerDefaultRole \ --policy-details file://policyDetails.json ``` The following is an example of the `policyDetails.json` file. ``` { "ResourceTypes": [ "VOLUME"], "PolicyType": "EBS_SNAPSHOT_MANAGEMENT", "Schedules" : [ { "Name": "sched1", "TagsToAdd": [ {"Key":"createdby","Value":"dlm"} ], "CreateRule": { "CronExpression": "cron(0 9 1 * ? *)" }, "CopyTags": true, "RetainRule":{ "Count": 0 }, "ArchiveRule": { "RetainRule":{ "RetentionArchiveTier": { "Count": 3 } } } } ], "TargetTags": [ { "Key": "Purpose", "Value": "Test" } ] } ``` ------ ## Considerations for snapshot lifecycle policies The following **general considerations** apply to snapshot lifecycle policies: + Snapshot lifecycle policies target only instances or volumes that are in the same Region as the policy. + The first snapshot creation operation starts within one hour after the specified start time. Subsequent snapshot creation operations start within one hour of their scheduled time. + You can create multiple policies to back up a volume or instance. For example, if a volume has two tags, where tag *A* is the target for policy *A* to create a snapshot every 12 hours, and tag *B* is the target for policy *B* to create a snapshot every 24 hours, Amazon Data Lifecycle Manager creates snapshots according to the schedules for both policies. Alternatively, you can achieve the same result by creating a single policy that has multiple schedules. For example, you can create a single policy that targets only tag *A*, and specify two schedules — one for every 12 hours and one for every 24 hours. + Target resource tags are case sensitive. + If you remove the target tags from a resource that is targeted by a policy, Amazon Data Lifecycle Manager no longer manages existing snapshots in the standard tier and archive tier; you must manually delete them if they are no longer needed. + If you create a policy that targets instances, and new volumes are attached to a target instance after the policy has been created, the newly-added volumes are included in the backup at the next policy run. All volumes attached to the instance at the time of the policy run are included. + If you create a policy with a custom cron-based schedule that is configured to create only one snapshot, the policy will not automatically delete that snapshot when the retention threshold is reached. You must manually delete the snapshot if it is no longer needed. + If you create an age-based policy where the retention period is shorter than the creation frequency, Amazon Data Lifecycle Manager will always retain the last snapshot until the next one is created. For example, if an age-based policy creates one snapshot every month with a retention period of seven days, Amazon Data Lifecycle Manager will retain each snapshot for one month, even though the retention period is seven days. The following considerations apply to **[snapshot archiving](snapshot-archive.md)**: + You can enable snapshot archiving only for snapshot policies that target volumes. + You can specify an archiving rule for only one schedule for each policy. + If you are using the console, you can enable snapshot archiving only if the schedule has a monthly or yearly creation frequency, or if the schedule has a cron expression with a creation frequency of at least 28 days. If you are using the AWS CLI, AWS API, or AWS SDK, you can enable snapshot archiving only if the schedule has a cron expression with a creation frequency of at least 28 days. + The minimum retention period in the archive tier is 90 days. + When a snapshot is archived, it is converted to a full snapshot when it is moved to the archive tier. This could result in higher snapshot storage costs. For more information, see [Pricing and billing for archiving Amazon EBS snapshots](snapshot-archive-pricing.md). + Fast snapshot restore and snapshot sharing are disabled for snapshots when they are archived. + If, in the case of a leap year, your retention rule results in an archive retention period of less than 90 days, Amazon Data Lifecycle Manager ensures that snapshots are retained for the minimum 90-day period. + If you manually archive a snapshot created by Amazon Data Lifecycle Manager, and the snapshot is still archived when the schedule's retention threshold is reached, Amazon Data Lifecycle Manager no longer manages that snapshot. However, if you restore the snapshot to the standard tier before the schedule's retention threshold is reached, the schedule will continue to manage the snapshot as per the retention rules. + If you permanently or temporarily restore a snapshot archived by Amazon Data Lifecycle Manager to the standard tier, and the snapshot is still in the standard tier when the schedule's retention threshold is reached, Amazon Data Lifecycle Manager no longer manages the snapshot. However, if you re-archive the snapshot before the schedule's retention threshold is reached, the schedule will delete the snapshot when the retention threshold is met. + Snapshots archived by Amazon Data Lifecycle Manager count towards your `Archived snapshots per volume` and `In-progress snapshot archives per account` quotas. + If a schedule is unable to archive a snapshot after retrying for 24 hours, the snapshot remains in the standard tier and it is scheduled for deletion based on the time that it would have been deleted from the archive tier. For example, if the schedule archives snapshots for 120 days, it remains in the standard tier for 120 days after the failed archiving before being permanently deleted. For count-based schedules, the snapshot does not count towards the schedule's retention count. + Snapshots must be archived in the same Region in which they were created. If you enabled cross-Region copy and snapshot archiving, Amazon Data Lifecycle Manager does not archive the snapshot copy. + Snapshots archived by Amazon Data Lifecycle Manager are tagged with the `aws:dlm:archived=true` system tag. Additionally, snapshots created by an archive-enabled, age-based schedule are tagged with the `aws:dlm:expirationTime` system tag, which indicates the date and time at which the snapshot is scheduled to be archived. The following considerations apply to **excluding root volumes and data (non-root) volumes**: + If you choose to exclude boot volumes and you specify tags that consequently exclude all of the additional data volumes attached to an instance, then Amazon Data Lifecycle Manager will not create any snapshots for the affected instance, and it will emit a `SnapshotsCreateFailed` CloudWatch metric. For more information, see [Monitor policies using CloudWatch](monitor-dlm-cw-metrics.md). The following considerations apply to **deleting volumes or terminating instances targeted by snapshot lifecycle policies**: + If you delete a volume or terminate an instance targeted by a policy with a count-based retention schedule, Amazon Data Lifecycle Manager no longer manages snapshots in the standard tier and archive tier that were created from the deleted volume or instance. You must manually delete those earlier snapshots if they are no longer needed. + If you delete a volume or terminate an instance targeted by a policy with an age-based retention schedule, the policy continues to delete snapshots from the standard tier and archive tier that were created from the deleted volume or instance on the defined schedule, up to, but not including, the last snapshot. You must manually delete the last snapshot if it is no longer needed. The following considerations apply to snapshot lifecycle policies and ** [fast snapshot restore](ebs-fast-snapshot-restore.md)**: + Amazon Data Lifecycle Manager can enable fast snapshot restore only for snapshots with a size of 16 TiB or less. For more information, see [Amazon EBS fast snapshot restore](ebs-fast-snapshot-restore.md). + A snapshot that is enabled for fast snapshot restore remains enabled even if you delete or disable the policy, disable fast snapshot restore for the policy, or disable fast snapshot restore for the Availability Zone. You must disable fast snapshot restore for these snapshots manually. + If you enable fast snapshot restore for a policy and you exceed the maximum number of snapshots that can be enabled for fast snapshot restore, Amazon Data Lifecycle Manager creates snapshots as scheduled but does not enable them for fast snapshot restore. After a snapshot that is enabled for fast snapshot restore is deleted, the next snapshot that Amazon Data Lifecycle Manager creates is enabled for fast snapshot restore. + When fast snapshot restore is enabled for a snapshot, it takes 60 minutes per TiB to optimize the snapshot. We recommend that you configure your schedules so that each snapshot is fully optimized before Amazon Data Lifecycle Manager creates the next snapshot. + If you enable fast snapshot restore for a policy that targets instances, Amazon Data Lifecycle Manager enables fast snapshot restore for each snapshot in the multi-volume snapshot set individually. If Amazon Data Lifecycle Manager fails to enable fast snapshot restore for one of the snapshots in the multi-volume snapshot set, it will still attempt to enable fast snapshot restore for the remaining snapshots in the snapshot set. + You are billed for each minute that fast snapshot restore is enabled for a snapshot in a particular Availability Zone. Charges are pro-rated with a minimum of one hour. For more information, see [Pricing and Billing](ebs-fast-snapshot-restore.md#fsr-pricing). **Note** Depending on the configuration of your lifecycle policies, you could have multiple snapshots enabled for fast snapshot restore in multiple Availability Zones simultaneously. The following considerations apply to snapshot lifecycle policies and ** [Multi-Attach](ebs-volumes-multi.md) enabled volumes**: + When creating a lifecycle policy that targets instances that have the same Multi-Attach enabled volume, Amazon Data Lifecycle Manager initiates a snapshot of the volume for each attached instance. Use the *timestamp* tag to identify the set of time-consistent snapshots that are created from the attached instances. The following considerations apply to **sharing snapshots across accounts**: + You can only share snapshots that are unencrypted or that are encrypted using a customer managed key. + You can't share snapshots that are encrypted with the default EBS encryption KMS key. + If you share encrypted snapshots, you must also share the KMS key that was used to encrypt the source volume with the target accounts. For more information, see [Allowing users in other accounts to use a KMS key](https://docs.aws.amazon.com//kms/latest/developerguide/key-policy-modifying-external-accounts.html) in the *AWS Key Management Service Developer Guide*. The following considerations apply to snapshots policies and ** [snapshot archiving](snapshot-archive.md)**: + If you manually archive a snapshot that was created by a policy, and that snapshot is in the archive tier when the policy’s retention threshold is reached, Amazon Data Lifecycle Manager will not delete the snapshot. Amazon Data Lifecycle Manager does not manage snapshots while they are stored in the archive tier. If you no longer need snapshots that are stored in the archive tier, you must manually delete them. The following considerations apply to snapshot policies and [Recycle Bin](recycle-bin.md): + If Amazon Data Lifecycle Manager deletes a snapshot and sends it to the Recycle Bin when the policy's retention threshold is reached, and you manually restore the snapshot from the Recycle Bin, you must manually delete that snapshot when it is no longer needed. Amazon Data Lifecycle Manager will no longer manage the snapshot. + If you manually delete a snapshot that was created by a policy, and that snapshot is in the Recycle Bin when the policy’s retention threshold is reached, Amazon Data Lifecycle Manager will not delete the snapshot. Amazon Data Lifecycle Manager does not manage the snapshots while they are stored in the Recycle Bin. If the snapshot is restored from the Recycle Bin before the policy's retention threshold is reached, Amazon Data Lifecycle Manager will delete the snapshot when the policy's retention threshold is reached. If the snapshot is restored from the Recycle Bin after the policy's retention threshold is reached, Amazon Data Lifecycle Manager will no longer delete the snapshot. You must manually delete the snapshot when it is no longer needed. The following considerations apply to snapshot lifecycle policies that are in the **error** state: + For policies with age-based retention schedules, snapshots that are set to expire while the policy is in the `error` state are retained indefinitely. You must delete the snapshots manually. When you re-enable the policy, Amazon Data Lifecycle Manager resumes deleting snapshots as their retention periods expire. + For policies with count-based retention schedules, the policy stops creating and deleting snapshots while it is in the `error` state. When you re-enable the policy, Amazon Data Lifecycle Manager resumes creating snapshots, and it resumes deleting snapshots as the retention threshold is met. The following considerations apply to snapshot policies and **[snapshot lock](ebs-snapshot-lock.md)**: + If you manually lock a snapshot created by Amazon Data Lifecycle Manager, and that snapshot is still locked when its retention threshold is reached, Amazon Data Lifecycle Manager no longer manages that snapshot. You must manually delete the snapshot if it is no longer needed. + If you manually lock a snapshot that was created and enabled for fast snapshot restore by Amazon Data Lifecycle Manager, and the snapshot is still locked when its retention threshold is reached, Amazon Data Lifecycle Manager will not disable fast snapshot restore or delete the snapshot. You must manually disable fast snapshot restore and delete the snapshot if it is no longer needed. + If you manually register a snapshot that was created by Amazon Data Lifecycle Manager with an AMI and then lock that snapshot, and that snapshot is still locked and associated with the AMI when its retention threshold is reached, Amazon Data Lifecycle Manager will continue to attempt to delete that snapshot. When the AMI is deregistered and the snapshot is unlocked, Amazon Data Lifecycle Manager will automatically delete the snapshot. ## Additional resources For more information, see the [ Automating Amazon EBS snapshot and AMI management using Amazon Data Lifecycle Manager](https://aws.amazon.com/blogs/storage/automating-amazon-ebs-snapshot-and-ami-management-using-amazon-dlm/) AWS storage blog. # Automate application-consistent snapshots with Data Lifecycle Manager You can automate application-consistent snapshots with Amazon Data Lifecycle Manager by enabling pre and post scripts in your snapshot lifecycle policies that target instances. Amazon Data Lifecycle Manager integrates with AWS Systems Manager (Systems Manager) to support application-consistent snapshots. Amazon Data Lifecycle Manager uses Systems Manager (SSM) command documents that include pre and post scripts to automate the actions needed to complete application-consistent snapshots. Before Amazon Data Lifecycle Manager initiates snapshot creation, it runs the commands in the pre script to freeze and flush I/O. After Amazon Data Lifecycle Manager initiates snapshot creation, it runs the commands in the post script to thaw I/O. Using Amazon Data Lifecycle Manager, you can automate application-consistent snapshots of the following: + Windows applications using Volume Shadow Copy Service (VSS) + SAP HANA using an AWS managed SSDM document. For more information, see [ Amazon EBS snapshots for SAP HANA](https://docs.aws.amazon.com/sap/latest/sap-hana/ebs-sap-hana.html). + Self-managed databases, such as MySQL, PostgreSQL or InterSystems IRIS, using SSM document templates **Topics** + [Requirements for using pre and post scripts](#app-consistent-prereqs) + [Getting started with application-consistent snapshots](#app-consistent-get-started) + [Considerations for VSS Backups with Amazon Data Lifecycle Manager](#app-consistent-vss) + [Shared responsibility for application-consistent snapshots](#shared-responsibility) ## Requirements for using pre and post scripts The following table outlines the requirements for using pre and post scripts with Amazon Data Lifecycle Manager. | | Application-consistent snapshots | | | --- |--- |--- | | Requirement | VSS Backup | Custom SSM document | Other use cases | | --- |--- |--- |--- | | SSM Agent installed and running on target instances | ✓ | ✓ | ✓ | | VSS system requirements met on target instances | ✓ | | | | VSS enabled instance profile associated with target instances | ✓ | | | | VSS components installed on target instances | ✓ | | | | Prepare SSM document with pre and post script commands | | ✓ | ✓ | | Prepare Amazon Data Lifecycle Manager IAM role run pre and post scripts | ✓ | ✓ | ✓ | | Create snapshot policy that targets instances and is configured for pre and post scripts | ✓ | ✓ | ✓ | ## Getting started with application-consistent snapshots This section explains the steps you need to follow to automate application-consistent snapshots using Amazon Data Lifecycle Manager. ### Step 1: Prepare target instances You need to prepare the targeted instances for application-consistent snapshots using Amazon Data Lifecycle Manager. Do one of the following, depending on your use case. ------ #### [ Prepare for VSS Backups ] **To prepare your target instances for VSS backups** 1. Install the SSM Agent on your target instances, if it is not already installed. If SSM Agent is already installed on your target instances, skip this step. For more information, see [ Working with SSM Agent on EC2 instances for Windows server](https://docs.aws.amazon.com/systems-manager/latest/userguide/ssm-agent-windows.html). 1. Ensure that the SSM Agent is running. For more information, see [ Checking SSM Agent status and starting the agent](https://docs.aws.amazon.com/systems-manager/latest/userguide/ssm-agent-status-and-restart.html). 1. Set up Systems Manager for Amazon EC2 instances. For more information, see [Setting up Systems Manager for Amazon EC2 instances](https://docs.aws.amazon.com/systems-manager/latest/userguide/systems-manager-setting-up-ec2.html) in the *AWS Systems Manager User Guide*. 1. [ Ensure the system requirements for VSS backups are met](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/application-consistent-snapshots-prereqs.html). 1. [ Attach a VSS-enabled instance profile to the target instances](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/vss-iam-reqs.html). 1. [ Install the VSS components](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/application-consistent-snapshots-getting-started.html). ------ #### [ Prepare for SAP HANA backups ] **To prepare your target instances for SAP HANA backups** 1. Prepare the SAP HANA environment on your target instances. 1. Set up your instance with SAP HANA. If you don't already have an existing SAP HANA environment, then you can refer to the [ SAP HANA Environment Setup on AWS](https://docs.aws.amazon.com/sap/latest/sap-hana/std-sap-hana-environment-setup.html). 1. Login to the SystemDB as a suitable administrator user. 1. Create a database backup user to be used with Amazon Data Lifecycle Manager. ``` CREATE USER username PASSWORD password NO FORCE_FIRST_PASSWORD_CHANGE; ``` For example, the following command creates a user named `dlm_user` with password `password`. ``` CREATE USER dlm_user PASSWORD password NO FORCE_FIRST_PASSWORD_CHANGE; ``` 1. Assign the `BACKUP OPERATOR` role to the database backup user that you created in the previous step. ``` GRANT BACKUP OPERATOR TO username ``` For example, the following command assigns the role to a user named `dlm_user`. ``` GRANT BACKUP OPERATOR TO dlm_user ``` 1. Log in to the operating system as the administrator, for example `sidadm`. 1. Create an `hdbuserstore` entry to store connection information so that the SAP HANA SSM document can connect to SAP HANA without users having to enter the information. ``` hdbuserstore set DLM_HANADB_SNAPSHOT_USER localhost:3hana_instance_number13 username password ``` For example: ``` hdbuserstore set DLM_HANADB_SNAPSHOT_USER localhost:30013 dlm_user password ``` 1. Test the connection. ``` hdbsql -U DLM_HANADB_SNAPSHOT_USER "select * from dummy" ``` 1. Install the SSM Agent on your target instances, if it is not already installed. If SSM Agent is already installed on your target instances, skip this step. For more information, see [ Manually installing SSM Agent on EC2 instances for Linux](https://docs.aws.amazon.com/systems-manager/latest/userguide/manually-install-ssm-agent-linux.html). 1. Ensure that the SSM Agent is running. For more information, see [ Checking SSM Agent status and starting the agent](https://docs.aws.amazon.com/systems-manager/latest/userguide/ssm-agent-status-and-restart.html). 1. Set up Systems Manager for Amazon EC2 instances. For more information, see [Setting up Systems Manager for Amazon EC2 instances](https://docs.aws.amazon.com/systems-manager/latest/userguide/systems-manager-setting-up-ec2.html) in the *AWS Systems Manager User Guide*. ------ #### [ Prepare for custom SSM documents ] **To prepare your target instances custom SSM documents** 1. Install the SSM Agent on your target instances, if it is not already installed. If SSM Agent is already installed on your target instances, skip this step. + (Linux instances) [ Manually installing SSM Agent on EC2 instances for Linux](https://docs.aws.amazon.com/systems-manager/latest/userguide/manually-install-ssm-agent-linux.html) + (Windows instances) [ Working with SSM Agent on EC2 instances for Windows Server](https://docs.aws.amazon.com/systems-manager/latest/userguide/ssm-agent-windows.html) 1. Ensure that the SSM Agent is running. For more information, see [ Checking SSM Agent status and starting the agent](https://docs.aws.amazon.com/systems-manager/latest/userguide/ssm-agent-status-and-restart.html). 1. Set up Systems Manager for Amazon EC2 instances. For more information, see [Setting up Systems Manager for Amazon EC2 instances](https://docs.aws.amazon.com/systems-manager/latest/userguide/systems-manager-setting-up-ec2.html) in the *AWS Systems Manager User Guide*. ------ ### Step 2: Prepare SSM document **Note** This step is required only for custom SSM documents. It is not required for VSS Backup or SAP HANA. For VSS Backups and SAP HANA, Amazon Data Lifecycle Manager uses the AWS managed SSM document. If you are automating application-consistent snapshots for a self-managed database, such as MySQL, PostgreSQL, or InterSystems IRIS, you must create an SSM command document that includes a pre script to freeze and flush I/O before snapshot creation is initiated, and a post script to thaw I/O after snapshot creation is initiated. If your MySQL, PostgreSQL, or InterSystems IRIS database uses standard configurations, you can create an SSM command document using the sample SSM document content below. If your MySQL, PostgreSQL, or InterSystems IRIS database uses a non-standard configuration, you can use the sample content below as a starting point for your SSM command document and then customize it to meet your requirements. Alternatively, if you want to create a new SSM document from scratch, you can use the empty SSM document template below and add your pre and post commands in the appropriate document sections. **Note the following:** It is your responsibility to ensure that the SSM document performs the correct and required actions for your database configuration. Snapshots are guaranteed to be application-consistent only if the pre and post scripts in your SSM document can successfully freeze, flush, and thaw I/O. The SSM document must include required fields for `allowedValues`, including `pre-script`, `post-script`, and `dry-run`. Amazon Data Lifecycle Manager will execute commands on your instance based on the contents of those sections. If your SSM document does not have those sections, then Amazon Data Lifecycle Manager will treat it as a failed execution. ------ #### [ MySQL sample document content ] ``` ###===============================================================================### # Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. # Permission is hereby granted, free of charge, to any person obtaining a copy of this # software and associated documentation files (the "Software"), to deal in the Software # without restriction, including without limitation the rights to use, copy, modify, # merge, publish, distribute, sublicense, and/or sell copies of the Software, and to # permit persons to whom the Software is furnished to do so. # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, # INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A # PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT # HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION # OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE # SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. ###===============================================================================### schemaVersion: '2.2' description: Amazon Data Lifecycle Manager Pre/Post script for MySQL databases parameters: executionId: type: String default: None description: (Required) Specifies the unique identifier associated with a pre and/or post execution allowedPattern: ^(None|[a-fA-F0-9]{8}-[a-fA-F0-9]{4}-[a-fA-F0-9]{4}-[a-fA-F0-9]{4}-[a-fA-F0-9]{12})$ command: # Data Lifecycle Manager will trigger the pre-script and post-script actions during policy execution. # 'dry-run' option is intended for validating the document execution without triggering any commands # on the instance. The following allowedValues will allow Data Lifecycle Manager to successfully # trigger pre and post script actions. type: String default: 'dry-run' description: (Required) Specifies whether pre-script and/or post-script should be executed. allowedValues: - pre-script - post-script - dry-run mainSteps: - action: aws:runShellScript description: Run MySQL Database freeze/thaw commands name: run_pre_post_scripts precondition: StringEquals: - platformType - Linux inputs: runCommand: - | #!/bin/bash ###===============================================================================### ### Error Codes ###===============================================================================### # The following Error codes will inform Data Lifecycle Manager of the type of error # and help guide handling of the error. # The Error code will also be emitted via AWS Eventbridge events in the 'cause' field. # 1 Pre-script failed during execution - 201 # 2 Post-script failed during execution - 202 # 3 Auto thaw occurred before post-script was initiated - 203 # 4 Pre-script initiated while post-script was expected - 204 # 5 Post-script initiated while pre-script was expected - 205 # 6 Application not ready for pre or post-script initiation - 206 ###=================================================================### ### Global variables ###=================================================================### START=$(date +%s) # For testing this script locally, replace the below with OPERATION=$1. OPERATION={{ command }} FS_ALREADY_FROZEN_ERROR='freeze failed: Device or resource busy' FS_ALREADY_THAWED_ERROR='unfreeze failed: Invalid argument' FS_BUSY_ERROR='mount point is busy' # Auto thaw is a fail safe mechanism to automatically unfreeze the application after the # duration specified in the global variable below. Choose the duration based on your # database application's tolerance to freeze. export AUTO_THAW_DURATION_SECS="60" # Add all pre-script actions to be performed within the function below execute_pre_script() { echo "INFO: Start execution of pre-script" # Check if filesystem is already frozen. No error code indicates that filesystem # is not currently frozen and that the pre-script can proceed with freezing the filesystem. check_fs_freeze # Execute the DB commands to flush the DB in preparation for snapshot snap_db # Freeze the filesystem. No error code indicates that filesystem was succefully frozen freeze_fs echo "INFO: Schedule Auto Thaw to execute in ${AUTO_THAW_DURATION_SECS} seconds." $(nohup bash -c execute_schedule_auto_thaw >/dev/null 2>&1 &) } # Add all post-script actions to be performed within the function below execute_post_script() { echo "INFO: Start execution of post-script" # Unfreeze the filesystem. No error code indicates that filesystem was successfully unfrozen. unfreeze_fs thaw_db } # Execute Auto Thaw to automatically unfreeze the application after the duration configured # in the AUTO_THAW_DURATION_SECS global variable. execute_schedule_auto_thaw() { sleep ${AUTO_THAW_DURATION_SECS} execute_post_script } # Disable Auto Thaw if it is still enabled execute_disable_auto_thaw() { echo "INFO: Attempting to disable auto thaw if enabled" auto_thaw_pgid=$(pgrep -f execute_schedule_auto_thaw | xargs -i ps -hp {} -o pgid) if [ -n "${auto_thaw_pgid}" ]; then echo "INFO: execute_schedule_auto_thaw process found with pgid ${auto_thaw_pgid}" sudo pkill -g ${auto_thaw_pgid} rc=$? if [ ${rc} != 0 ]; then echo "ERROR: Unable to kill execute_schedule_auto_thaw process. retval=${rc}" else echo "INFO: Auto Thaw has been disabled" fi fi } # Iterate over all the mountpoints and check if filesystem is already in freeze state. # Return error code 204 if any of the mount points are already frozen. check_fs_freeze() { for target in $(lsblk -nlo MOUNTPOINTS) do # Freeze of the root and boot filesystems is dangerous and pre-script does not freeze these filesystems. # Hence, we will skip the root and boot mountpoints while checking if filesystem is in freeze state. if [ $target == '/' ]; then continue; fi if [[ "$target" == *"/boot"* ]]; then continue; fi error_message=$(sudo mount -o remount,noatime $target 2>&1) # Remount will be a no-op without a error message if the filesystem is unfrozen. # However, if filesystem is already frozen, remount will fail with busy error message. if [ $? -ne 0 ];then # If the filesystem is already in frozen, return error code 204 if [[ "$error_message" == *"$FS_BUSY_ERROR"* ]];then echo "ERROR: Filesystem ${target} already frozen. Return Error Code: 204" exit 204 fi # If the check filesystem freeze failed due to any reason other than the filesystem already frozen, return 201 echo "ERROR: Failed to check_fs_freeze on mountpoint $target due to error - $errormessage" exit 201 fi done } # Iterate over all the mountpoints and freeze the filesystem. freeze_fs() { for target in $(lsblk -nlo MOUNTPOINTS) do # Freeze of the root and boot filesystems is dangerous. Hence, skip filesystem freeze # operations for root and boot mountpoints. if [ $target == '/' ]; then continue; fi if [[ "$target" == *"/boot"* ]]; then continue; fi echo "INFO: Freezing $target" error_message=$(sudo fsfreeze -f $target 2>&1) if [ $? -ne 0 ];then # If the filesystem is already in frozen, return error code 204 if [[ "$error_message" == *"$FS_ALREADY_FROZEN_ERROR"* ]]; then echo "ERROR: Filesystem ${target} already frozen. Return Error Code: 204" sudo mysql -e 'UNLOCK TABLES;' exit 204 fi # If the filesystem freeze failed due to any reason other than the filesystem already frozen, return 201 echo "ERROR: Failed to freeze mountpoint $targetdue due to error - $errormessage" thaw_db exit 201 fi echo "INFO: Freezing complete on $target" done } # Iterate over all the mountpoints and unfreeze the filesystem. unfreeze_fs() { for target in $(lsblk -nlo MOUNTPOINTS) do # Freeze of the root and boot filesystems is dangerous and pre-script does not freeze these filesystems. # Hence, will skip the root and boot mountpoints during unfreeze as well. if [ $target == '/' ]; then continue; fi if [[ "$target" == *"/boot"* ]]; then continue; fi echo "INFO: Thawing $target" error_message=$(sudo fsfreeze -u $target 2>&1) # Check if filesystem is already unfrozen (thawed). Return error code 204 if filesystem is already unfrozen. if [ $? -ne 0 ]; then if [[ "$error_message" == *"$FS_ALREADY_THAWED_ERROR"* ]]; then echo "ERROR: Filesystem ${target} is already in thaw state. Return Error Code: 205" exit 205 fi # If the filesystem unfreeze failed due to any reason other than the filesystem already unfrozen, return 202 echo "ERROR: Failed to unfreeze mountpoint $targetdue due to error - $errormessage" exit 202 fi echo "INFO: Thaw complete on $target" done } snap_db() { # Run the flush command only when MySQL DB service is up and running sudo systemctl is-active --quiet mysqld.service if [ $? -eq 0 ]; then echo "INFO: Execute MySQL Flush and Lock command." sudo mysql -e 'FLUSH TABLES WITH READ LOCK;' # If the MySQL Flush and Lock command did not succeed, return error code 201 to indicate pre-script failure if [ $? -ne 0 ]; then echo "ERROR: MySQL FLUSH TABLES WITH READ LOCK command failed." exit 201 fi sync else echo "INFO: MySQL service is inactive. Skipping execution of MySQL Flush and Lock command." fi } thaw_db() { # Run the unlock command only when MySQL DB service is up and running sudo systemctl is-active --quiet mysqld.service if [ $? -eq 0 ]; then echo "INFO: Execute MySQL Unlock" sudo mysql -e 'UNLOCK TABLES;' else echo "INFO: MySQL service is inactive. Skipping execution of MySQL Unlock command." fi } export -f execute_schedule_auto_thaw export -f execute_post_script export -f unfreeze_fs export -f thaw_db # Debug logging for parameters passed to the SSM document echo "INFO: ${OPERATION} starting at $(date) with executionId: ${EXECUTION_ID}" # Based on the command parameter value execute the function that supports # pre-script/post-script operation case ${OPERATION} in pre-script) execute_pre_script ;; post-script) execute_post_script execute_disable_auto_thaw ;; dry-run) echo "INFO: dry-run option invoked - taking no action" ;; *) echo "ERROR: Invalid command parameter passed. Please use either pre-script, post-script, dry-run." exit 1 # return failure ;; esac END=$(date +%s) # Debug Log for profiling the script time echo "INFO: ${OPERATION} completed at $(date). Total runtime: $((${END} - ${START})) seconds." ``` ------ #### [ PostgreSQL sample document content ] ``` ###===============================================================================### # Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. # Permission is hereby granted, free of charge, to any person obtaining a copy of this # software and associated documentation files (the "Software"), to deal in the Software # without restriction, including without limitation the rights to use, copy, modify, # merge, publish, distribute, sublicense, and/or sell copies of the Software, and to # permit persons to whom the Software is furnished to do so. # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, # INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A # PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT # HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION # OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE # SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. ###===============================================================================### schemaVersion: '2.2' description: Amazon Data Lifecycle Manager Pre/Post script for PostgreSQL databases parameters: executionId: type: String default: None description: (Required) Specifies the unique identifier associated with a pre and/or post execution allowedPattern: ^(None|[a-fA-F0-9]{8}-[a-fA-F0-9]{4}-[a-fA-F0-9]{4}-[a-fA-F0-9]{4}-[a-fA-F0-9]{12})$ command: # Data Lifecycle Manager will trigger the pre-script and post-script actions during policy execution. # 'dry-run' option is intended for validating the document execution without triggering any commands # on the instance. The following allowedValues will allow Data Lifecycle Manager to successfully # trigger pre and post script actions. type: String default: 'dry-run' description: (Required) Specifies whether pre-script and/or post-script should be executed. allowedValues: - pre-script - post-script - dry-run mainSteps: - action: aws:runShellScript description: Run PostgreSQL Database freeze/thaw commands name: run_pre_post_scripts precondition: StringEquals: - platformType - Linux inputs: runCommand: - | #!/bin/bash ###===============================================================================### ### Error Codes ###===============================================================================### # The following Error codes will inform Data Lifecycle Manager of the type of error # and help guide handling of the error. # The Error code will also be emitted via AWS Eventbridge events in the 'cause' field. # 1 Pre-script failed during execution - 201 # 2 Post-script failed during execution - 202 # 3 Auto thaw occurred before post-script was initiated - 203 # 4 Pre-script initiated while post-script was expected - 204 # 5 Post-script initiated while pre-script was expected - 205 # 6 Application not ready for pre or post-script initiation - 206 ###===============================================================================### ### Global variables ###===============================================================================### START=$(date +%s) OPERATION={{ command }} FS_ALREADY_FROZEN_ERROR='freeze failed: Device or resource busy' FS_ALREADY_THAWED_ERROR='unfreeze failed: Invalid argument' FS_BUSY_ERROR='mount point is busy' # Auto thaw is a fail safe mechanism to automatically unfreeze the application after the # duration specified in the global variable below. Choose the duration based on your # database application's tolerance to freeze. export AUTO_THAW_DURATION_SECS="60" # Add all pre-script actions to be performed within the function below execute_pre_script() { echo "INFO: Start execution of pre-script" # Check if filesystem is already frozen. No error code indicates that filesystem # is not currently frozen and that the pre-script can proceed with freezing the filesystem. check_fs_freeze # Execute the DB commands to flush the DB in preparation for snapshot snap_db # Freeze the filesystem. No error code indicates that filesystem was succefully frozen freeze_fs echo "INFO: Schedule Auto Thaw to execute in ${AUTO_THAW_DURATION_SECS} seconds." $(nohup bash -c execute_schedule_auto_thaw >/dev/null 2>&1 &) } # Add all post-script actions to be performed within the function below execute_post_script() { echo "INFO: Start execution of post-script" # Unfreeze the filesystem. No error code indicates that filesystem was successfully unfrozen unfreeze_fs } # Execute Auto Thaw to automatically unfreeze the application after the duration configured # in the AUTO_THAW_DURATION_SECS global variable. execute_schedule_auto_thaw() { sleep ${AUTO_THAW_DURATION_SECS} execute_post_script } # Disable Auto Thaw if it is still enabled execute_disable_auto_thaw() { echo "INFO: Attempting to disable auto thaw if enabled" auto_thaw_pgid=$(pgrep -f execute_schedule_auto_thaw | xargs -i ps -hp {} -o pgid) if [ -n "${auto_thaw_pgid}" ]; then echo "INFO: execute_schedule_auto_thaw process found with pgid ${auto_thaw_pgid}" sudo pkill -g ${auto_thaw_pgid} rc=$? if [ ${rc} != 0 ]; then echo "ERROR: Unable to kill execute_schedule_auto_thaw process. retval=${rc}" else echo "INFO: Auto Thaw has been disabled" fi fi } # Iterate over all the mountpoints and check if filesystem is already in freeze state. # Return error code 204 if any of the mount points are already frozen. check_fs_freeze() { for target in $(lsblk -nlo MOUNTPOINTS) do # Freeze of the root and boot filesystems is dangerous and pre-script does not freeze these filesystems. # Hence, we will skip the root and boot mountpoints while checking if filesystem is in freeze state. if [ $target == '/' ]; then continue; fi if [[ "$target" == *"/boot"* ]]; then continue; fi error_message=$(sudo mount -o remount,noatime $target 2>&1) # Remount will be a no-op without a error message if the filesystem is unfrozen. # However, if filesystem is already frozen, remount will fail with busy error message. if [ $? -ne 0 ];then # If the filesystem is already in frozen, return error code 204 if [[ "$error_message" == *"$FS_BUSY_ERROR"* ]];then echo "ERROR: Filesystem ${target} already frozen. Return Error Code: 204" exit 204 fi # If the check filesystem freeze failed due to any reason other than the filesystem already frozen, return 201 echo "ERROR: Failed to check_fs_freeze on mountpoint $target due to error - $errormessage" exit 201 fi done } # Iterate over all the mountpoints and freeze the filesystem. freeze_fs() { for target in $(lsblk -nlo MOUNTPOINTS) do # Freeze of the root and boot filesystems is dangerous. Hence, skip filesystem freeze # operations for root and boot mountpoints. if [ $target == '/' ]; then continue; fi if [[ "$target" == *"/boot"* ]]; then continue; fi echo "INFO: Freezing $target" error_message=$(sudo fsfreeze -f $target 2>&1) if [ $? -ne 0 ];then # If the filesystem is already in frozen, return error code 204 if [[ "$error_message" == *"$FS_ALREADY_FROZEN_ERROR"* ]]; then echo "ERROR: Filesystem ${target} already frozen. Return Error Code: 204" exit 204 fi # If the filesystem freeze failed due to any reason other than the filesystem already frozen, return 201 echo "ERROR: Failed to freeze mountpoint $targetdue due to error - $errormessage" exit 201 fi echo "INFO: Freezing complete on $target" done } # Iterate over all the mountpoints and unfreeze the filesystem. unfreeze_fs() { for target in $(lsblk -nlo MOUNTPOINTS) do # Freeze of the root and boot filesystems is dangerous and pre-script does not freeze these filesystems. # Hence, will skip the root and boot mountpoints during unfreeze as well. if [ $target == '/' ]; then continue; fi if [[ "$target" == *"/boot"* ]]; then continue; fi echo "INFO: Thawing $target" error_message=$(sudo fsfreeze -u $target 2>&1) # Check if filesystem is already unfrozen (thawed). Return error code 204 if filesystem is already unfrozen. if [ $? -ne 0 ]; then if [[ "$error_message" == *"$FS_ALREADY_THAWED_ERROR"* ]]; then echo "ERROR: Filesystem ${target} is already in thaw state. Return Error Code: 205" exit 205 fi # If the filesystem unfreeze failed due to any reason other than the filesystem already unfrozen, return 202 echo "ERROR: Failed to unfreeze mountpoint $targetdue due to error - $errormessage" exit 202 fi echo "INFO: Thaw complete on $target" done } snap_db() { # Run the flush command only when PostgreSQL DB service is up and running sudo systemctl is-active --quiet postgresql if [ $? -eq 0 ]; then echo "INFO: Execute Postgres CHECKPOINT" # PostgreSQL command to flush the transactions in memory to disk sudo -u postgres psql -c 'CHECKPOINT;' # If the PostgreSQL Command did not succeed, return error code 201 to indicate pre-script failure if [ $? -ne 0 ]; then echo "ERROR: Postgres CHECKPOINT command failed." exit 201 fi sync else echo "INFO: PostgreSQL service is inactive. Skipping execution of CHECKPOINT command." fi } export -f execute_schedule_auto_thaw export -f execute_post_script export -f unfreeze_fs # Debug logging for parameters passed to the SSM document echo "INFO: ${OPERATION} starting at $(date) with executionId: ${EXECUTION_ID}" # Based on the command parameter value execute the function that supports # pre-script/post-script operation case ${OPERATION} in pre-script) execute_pre_script ;; post-script) execute_post_script execute_disable_auto_thaw ;; dry-run) echo "INFO: dry-run option invoked - taking no action" ;; *) echo "ERROR: Invalid command parameter passed. Please use either pre-script, post-script, dry-run." exit 1 # return failure ;; esac END=$(date +%s) # Debug Log for profiling the script time echo "INFO: ${OPERATION} completed at $(date). Total runtime: $((${END} - ${START})) seconds." ``` ------ #### [ InterSystems IRIS sample document content ] ``` ###===============================================================================### # MIT License # # Copyright (c) 2024 InterSystems # # Permission is hereby granted, free of charge, to any person obtaining a copy # of this software and associated documentation files (the "Software"), to deal # in the Software without restriction, including without limitation the rights # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell # copies of the Software, and to permit persons to whom the Software is # furnished to do so, subject to the following conditions: # # The above copyright notice and this permission notice shall be included in all # copies or substantial portions of the Software. # # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE # AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER # LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE # SOFTWARE. ###===============================================================================### schemaVersion: '2.2' description: SSM Document Template for Amazon Data Lifecycle Manager Pre/Post script feature for InterSystems IRIS. parameters: executionId: type: String default: None description: Specifies the unique identifier associated with a pre and/or post execution allowedPattern: ^(None|[a-fA-F0-9]{8}-[a-fA-F0-9]{4}-[a-fA-F0-9]{4}-[a-fA-F0-9]{4}-[a-fA-F0-9]{12})$ command: type: String # Data Lifecycle Manager will trigger the pre-script and post-script actions. You can also use this SSM document with 'dry-run' for manual testing purposes. default: 'dry-run' description: (Required) Specifies whether pre-script and/or post-script should be executed. #The following allowedValues will allow Data Lifecycle Manager to successfully trigger pre and post script actions. allowedValues: - pre-script - post-script - dry-run mainSteps: - action: aws:runShellScript description: Run InterSystems IRIS Database freeze/thaw commands name: run_pre_post_scripts precondition: StringEquals: - platformType - Linux inputs: runCommand: - | #!/bin/bash ###===============================================================================### ### Global variables ###===============================================================================### DOCKER_NAME=iris LOGDIR=./ EXIT_CODE=0 OPERATION={{ command }} START=$(date +%s) # Check if Docker is installed # By default if Docker is present, script assumes that InterSystems IRIS is running in Docker # Leave only the else block DOCKER_EXEC line, if you run InterSystems IRIS non-containerised (and Docker is present). # Script assumes irissys user has OS auth enabled, change the OS user or supply login/password depending on your configuration. if command -v docker &> /dev/null then DOCKER_EXEC="docker exec $DOCKER_NAME" else DOCKER_EXEC="sudo -i -u irissys" fi # Add all pre-script actions to be performed within the function below execute_pre_script() { echo "INFO: Start execution of pre-script" # find all iris running instances iris_instances=$($DOCKER_EXEC iris qall 2>/dev/null | tail -n +3 | grep '^up' | cut -c5- | awk '{print $1}') echo "`date`: Running iris instances $iris_instances" # Only for running instances for INST in $iris_instances; do echo "`date`: Attempting to freeze $INST" # Detailed instances specific log LOGFILE=$LOGDIR/$INST-pre_post.log #check Freeze status before starting $DOCKER_EXEC irissession $INST -U '%SYS' "##Class(Backup.General).IsWDSuspendedExt()" freeze_status=$? if [ $freeze_status -eq 5 ]; then echo "`date`: ERROR: $INST IS already FROZEN" EXIT_CODE=204 else echo "`date`: $INST is not frozen" # Freeze # Docs: https://docs.intersystems.com/irislatest/csp/documatic/%25CSP.Documatic.cls?LIBRARY=%25SYS&CLASSNAME=Backup.General#ExternalFreeze $DOCKER_EXEC irissession $INST -U '%SYS' "##Class(Backup.General).ExternalFreeze(\"$LOGFILE\",,,,,,600,,,300)" status=$? case $status in 5) echo "`date`: $INST IS FROZEN" ;; 3) echo "`date`: $INST FREEZE FAILED" EXIT_CODE=201 ;; *) echo "`date`: ERROR: Unknown status code: $status" EXIT_CODE=201 ;; esac echo "`date`: Completed freeze of $INST" fi done echo "`date`: Pre freeze script finished" } # Add all post-script actions to be performed within the function below execute_post_script() { echo "INFO: Start execution of post-script" # find all iris running instances iris_instances=$($DOCKER_EXEC iris qall 2>/dev/null | tail -n +3 | grep '^up' | cut -c5- | awk '{print $1}') echo "`date`: Running iris instances $iris_instances" # Only for running instances for INST in $iris_instances; do echo "`date`: Attempting to thaw $INST" # Detailed instances specific log LOGFILE=$LOGDIR/$INST-pre_post.log #check Freeze status befor starting $DOCKER_EXEC irissession $INST -U '%SYS' "##Class(Backup.General).IsWDSuspendedExt()" freeze_status=$? if [ $freeze_status -eq 5 ]; then echo "`date`: $INST is in frozen state" # Thaw # Docs: https://docs.intersystems.com/irislatest/csp/documatic/%25CSP.Documatic.cls?LIBRARY=%25SYS&CLASSNAME=Backup.General#ExternalFreeze $DOCKER_EXEC irissession $INST -U%SYS "##Class(Backup.General).ExternalThaw(\"$LOGFILE\")" status=$? case $status in 5) echo "`date`: $INST IS THAWED" $DOCKER_EXEC irissession $INST -U%SYS "##Class(Backup.General).ExternalSetHistory(\"$LOGFILE\")" ;; 3) echo "`date`: $INST THAW FAILED" EXIT_CODE=202 ;; *) echo "`date`: ERROR: Unknown status code: $status" EXIT_CODE=202 ;; esac echo "`date`: Completed thaw of $INST" else echo "`date`: ERROR: $INST IS already THAWED" EXIT_CODE=205 fi done echo "`date`: Post thaw script finished" } # Debug logging for parameters passed to the SSM document echo "INFO: ${OPERATION} starting at $(date) with executionId: ${EXECUTION_ID}" # Based on the command parameter value execute the function that supports # pre-script/post-script operation case ${OPERATION} in pre-script) execute_pre_script ;; post-script) execute_post_script ;; dry-run) echo "INFO: dry-run option invoked - taking no action" ;; *) echo "ERROR: Invalid command parameter passed. Please use either pre-script, post-script, dry-run." # return failure EXIT_CODE=1 ;; esac END=$(date +%s) # Debug Log for profiling the script time echo "INFO: ${OPERATION} completed at $(date). Total runtime: $((${END} - ${START})) seconds." exit $EXIT_CODE ``` For more information, see the [ GitHub repository](https://github.com/intersystems-community/aws/blob/master/README.md). ------ #### [ Empty document template ] ``` ###===============================================================================### # Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. # Permission is hereby granted, free of charge, to any person obtaining a copy of this # software and associated documentation files (the "Software"), to deal in the Software # without restriction, including without limitation the rights to use, copy, modify, # merge, publish, distribute, sublicense, and/or sell copies of the Software, and to # permit persons to whom the Software is furnished to do so. # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, # INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A # PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT # HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION # OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE # SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. ###===============================================================================### schemaVersion: '2.2' description: SSM Document Template for Amazon Data Lifecycle Manager Pre/Post script feature parameters: executionId: type: String default: None description: (Required) Specifies the unique identifier associated with a pre and/or post execution allowedPattern: ^(None|[a-fA-F0-9]{8}-[a-fA-F0-9]{4}-[a-fA-F0-9]{4}-[a-fA-F0-9]{4}-[a-fA-F0-9]{12})$ command: # Data Lifecycle Manager will trigger the pre-script and post-script actions during policy execution. # 'dry-run' option is intended for validating the document execution without triggering any commands # on the instance. The following allowedValues will allow Data Lifecycle Manager to successfully # trigger pre and post script actions. type: String default: 'dry-run' description: (Required) Specifies whether pre-script and/or post-script should be executed. allowedValues: - pre-script - post-script - dry-run mainSteps: - action: aws:runShellScript description: Run Database freeze/thaw commands name: run_pre_post_scripts precondition: StringEquals: - platformType - Linux inputs: runCommand: - | #!/bin/bash ###===============================================================================### ### Error Codes ###===============================================================================### # The following Error codes will inform Data Lifecycle Manager of the type of error # and help guide handling of the error. # The Error code will also be emitted via AWS Eventbridge events in the 'cause' field. # 1 Pre-script failed during execution - 201 # 2 Post-script failed during execution - 202 # 3 Auto thaw occurred before post-script was initiated - 203 # 4 Pre-script initiated while post-script was expected - 204 # 5 Post-script initiated while pre-script was expected - 205 # 6 Application not ready for pre or post-script initiation - 206 ###===============================================================================### ### Global variables ###===============================================================================### START=$(date +%s) # For testing this script locally, replace the below with OPERATION=$1. OPERATION={{ command }} # Add all pre-script actions to be performed within the function below execute_pre_script() { echo "INFO: Start execution of pre-script" } # Add all post-script actions to be performed within the function below execute_post_script() { echo "INFO: Start execution of post-script" } # Debug logging for parameters passed to the SSM document echo "INFO: ${OPERATION} starting at $(date) with executionId: ${EXECUTION_ID}" # Based on the command parameter value execute the function that supports # pre-script/post-script operation case ${OPERATION} in pre-script) execute_pre_script ;; post-script) execute_post_script ;; dry-run) echo "INFO: dry-run option invoked - taking no action" ;; *) echo "ERROR: Invalid command parameter passed. Please use either pre-script, post-script, dry-run." exit 1 # return failure ;; esac END=$(date +%s) # Debug Log for profiling the script time echo "INFO: ${OPERATION} completed at $(date). Total runtime: $((${END} - ${START})) seconds." ``` ------ Once you have your SSM document content, use one of the following procedures to create the custom SSM document. ------ #### [ Console ] **To create the SSM command document** 1. Open the AWS Systems Manager console at [ https://console.aws.amazon.com//systems-manager/](https://console.aws.amazon.com//systems-manager/). 1. In the navigation pane, choose **Documents**, then choose **Create document**, **Command or Session**. 1. For **Name**, enter a descriptive name for the document. 1. For **Target type**, select **/AWS::EC2::Instance**. 1. For **Document type**, select **Command**. 1. In the **Content** field, select **YAML** and then paste the document content. 1. In the **Document tags** section, add a tag with a tag key of `DLMScriptsAccess`, and a tag value of `true`. **Important** The `DLMScriptsAccess:true` tag is required by the **AWSDataLifecycleManagerSSMFullAccess** AWS managed policy used in *Step 3: Prepare Amazon Data Lifecycle Manager IAM role*. The policy uses the `aws:ResourceTag` condition key to restrict access to SSM documents that have this tag. 1. Choose **Create document**. ------ #### [ AWS CLI ] **To create the SSM command document** Use the [ create-document](https://docs.aws.amazon.com/cli/latest/reference/ssm/create-document.html) command. For `--name`, specify a descriptive name for the document. For `--document-type`, specify `Command`. For `--content`, specify the path to the .yaml file with the SSM document content. For `--tags`, specify `"Key=DLMScriptsAccess,Value=true"`. ``` $ aws ssm create-document \ --content file://path/to/file/documentContent.yaml \ --name "document_name" \ --document-type "Command" \ --document-format YAML \ --tags "Key=DLMScriptsAccess,Value=true" ``` ------ ### Step 3: Prepare Amazon Data Lifecycle Manager IAM role **Note** This step is needed if: You create or update a pre/post script-enabled snapshot policy that uses a custom IAM role. You use the command line to create or update a pre/post script-enabled snapshot policy that uses the default. If you use the console to create or update a pre/post script-enabled snapshot policy that uses the default role for managing snapshots (**AWSDataLifecycleManagerDefaultRole**), skip this step. In this case, we automatically attach the **AWSDataLifecycleManagerSSMFullAccess** policy to that role. You must ensure that the IAM role that you use for policy grants Amazon Data Lifecycle Manager permission to perform the SSM actions required to run pre and post scripts on instances targeted by the policy. Amazon Data Lifecycle Manager provides a managed policy (**AWSDataLifecycleManagerSSMFullAccess**) that includes the required permissions. You can attach this policy to your IAM role for managing snapshots to ensure that it includes the permissions. **Important** The AWSDataLifecycleManagerSSMFullAccess managed policy uses the `aws:ResourceTag` condition key to restrict access to specific SSM documents when using pre and post scripts. To allow Amazon Data Lifecycle Manager to access the SSM documents, you must ensure that your SSM documents are tagged with `DLMScriptsAccess:true`. Alternatively, you can manually create a custom policy or assign the required permissions directly to the IAM role that you use. You can use the same permissions that are defined in the AWSDataLifecycleManagerSSMFullAccess managed policy, however, the `aws:ResourceTag` condition key is optional. If you decide to not include that condition key, then you do not need to tag your SSM documents with `DLMScriptsAccess:true`. Use one of the following methods to add the **AWSDataLifecycleManagerSSMFullAccess** policy to your IAM role. ------ #### [ Console ] **To attach the managed policy to your custom role** 1. Open the IAM console at [https://console.aws.amazon.com/iam/](https://console.aws.amazon.com/iam/). 1. In the navigation panel, choose **Roles**. 1. Search for and select your custom role for managing snapshots. 1. On the **Permissions** tab, choose **Add permissions**, **Attach policies**. 1. Search for and select the **AWSDataLifecycleManagerSSMFullAccess** managed policy, and then choose **Add permissions**. ------ #### [ AWS CLI ] **To attach the managed policy to your custom role** Use the [ attach-role-policy](https://docs.aws.amazon.com/cli/latest/reference/iam/attach-role-policy.html) command. For `---role-name`, specify the name of your custom role. For `--policy-arn`, specify `arn:aws:iam::aws:policy/AWSDataLifecycleManagerSSMFullAccess`. ``` $ aws iam attach-role-policy \ --policy-arn arn:aws:iam::aws:policy/AWSDataLifecycleManagerSSMFullAccess \ --role-name your_role_name ``` ------ ### Step 4: Create snapshot lifecycle policy To automate application-consistent snapshots, you must create a snapshot lifecycle policy that targets instances, and configure pre and post scripts for that policy. ------ #### [ Console ] **To create the snapshot lifecycle policy** 1. Open the Amazon EC2 console at [https://console.aws.amazon.com/ec2/](https://console.aws.amazon.com/ec2/). 1. In the navigation pane, choose **Elastic Block Store**, **Lifecycle Manager**, and then choose **Create lifecycle policy**. 1. On the **Select policy type** screen, choose **EBS snapshot policy** and then choose **Next**. 1. In the **Target resources** section, do the following: 1. For **Target resource types**, choose `Instance`. 1. For **Target resource tags**, specify the resource tags that identify the instances to back up. Only resources that have the specified tags will be backed up. 1. For **IAM role**, either choose ** AWSDataLifecycleManagerDefaultRole** (the default role for managing snapshots), or choose a custom role that you created and prepared for pre and post scripts. 1. Configure the schedules and additional options as needed. We recommend that you schedule snapshot creation times for time periods that match your workload, such as during maintenance windows. For SAP HANA, we recommend that you enable fast snapshot restore. **Note** If you enable a schedule for VSS Backups, you can't enable **Exclude specific data volumes** or **Copy tags from source**. 1. In the **Pre and post scripts** section, select **Enable pre and post scripts**, and then do the following, depending on your workload: + To create application-consistent snapshots of your Windows applications, select **VSS Backup**. + To create application-consistent snapshots of your SAP HANA workloads, select **SAP HANA**. + To create application-consistent snapshots of all other databases and workloads, including your self-managed MySQL, PostgreSQL, or InterSystems IRIS databases, using a custom SSM document, select **Custom SSM document**. 1. For **Automate option**, choose **Pre and post scripts**. 1. For **SSM document**, select the SSM document that you prepared. 1. Depending on the option you selected, configure the following additional options: + **Script timeout** — (*Custom SSM document only*) The timeout period after which Amazon Data Lifecycle Manager fails the script run attempt if it has not completed. If a script does not complete within its timeout period, Amazon Data Lifecycle Manager fails the attempt. The timeout period applies to the pre and post scripts individually. The minimum and default timeout period is 10 seconds. And the maximum timeout period is 120 seconds. + **Retry failed scripts** — Select this option to retry scripts that do not complete within their timeout period. If the pre script fails, Amazon Data Lifecycle Manager retries entire snapshot creation process, including running the pre and post scripts. If the post script fails, Amazon Data Lifecycle Manager retries the post script only; in this case, the pre script will have completed and the snapshot might have been created. + **Default to crash-consistent snapshots** — Select this option to default to crash-consistent snapshots if the pre script fails to run. This is the default snapshot creation behavior for Amazon Data Lifecycle Manager if pre and post scripts is not enabled. If you enabled retries, Amazon Data Lifecycle Manager will default to crash-consistent snapshots only after all retry attempts have been exhausted. If the pre script fails and you do not default to crash-consistent snapshots, Amazon Data Lifecycle Manager will not create snapshots for the instance during that schedule run. **Note** If you are creating snapshots for SAP HANA, then you might want to disabled this option. Crash-consistent snapshots of SAP HANA workloads can't restored in the same manner. 1. Choose **Create default policy**. **Note** If you get the `Role with name AWSDataLifecycleManagerDefaultRole already exists` error, see [Troubleshoot Amazon Data Lifecycle Manager issues](dlm-troubleshooting.md) for more information. ------ #### [ AWS CLI ] **To create the snapshot lifecycle policy** Use the [ create-lifecycle-policy](https://docs.aws.amazon.com/cli/latest/reference/dlm/create-lifecycle-policy.html) command, and include the `Scripts` parameters in `CreateRule`. For more information about the parameters, see the [https://docs.aws.amazon.com/dlm/latest/APIReference/API_Script.html](https://docs.aws.amazon.com/dlm/latest/APIReference/API_Script.html). ``` $ aws dlm create-lifecycle-policy \ --description "policy_description" \ --state ENABLED \ --execution-role-arn iam_role_arn \ --policy-details file://policyDetails.json ``` Where `policyDetails.json` includes one of the following, depending on your use case: + **VSS Backup** ``` { "PolicyType": "EBS_SNAPSHOT_MANAGEMENT", "ResourceTypes": [ "INSTANCE" ], "TargetTags": [{ "Key": "tag_key", "Value": "tag_value" }], "Schedules": [{ "Name": "schedule_name", "CreateRule": { "CronExpression": "cron_for_creation_frequency", "Scripts": [{ "ExecutionHandler":"AWS_VSS_BACKUP", "ExecuteOperationOnScriptFailure":true|false, "MaximumRetryCount":retries (0-3) }] }, "RetainRule": { "Count": retention_count } }] } ``` + **SAP HANA backups** ``` { "PolicyType": "EBS_SNAPSHOT_MANAGEMENT", "ResourceTypes": [ "INSTANCE" ], "TargetTags": [{ "Key": "tag_key", "Value": "tag_value" }], "Schedules": [{ "Name": "schedule_name", "CreateRule": { "CronExpression": "cron_for_creation_frequency", "Scripts": [{ "Stages": ["PRE","POST"], "ExecutionHandlerService":"AWS_SYSTEMS_MANAGER", "ExecutionHandler":"AWSSystemsManagerSAP-CreateDLMSnapshotForSAPHANA", "ExecuteOperationOnScriptFailure":true|false, "ExecutionTimeout":timeout_in_seconds (10-120), "MaximumRetryCount":retries (0-3) }] }, "RetainRule": { "Count": retention_count } }] } ``` + **Custom SSM document** ``` { "PolicyType": "EBS_SNAPSHOT_MANAGEMENT", "ResourceTypes": [ "INSTANCE" ], "TargetTags": [{ "Key": "tag_key", "Value": "tag_value" }], "Schedules": [{ "Name": "schedule_name", "CreateRule": { "CronExpression": "cron_for_creation_frequency", "Scripts": [{ "Stages": ["PRE","POST"], "ExecutionHandlerService":"AWS_SYSTEMS_MANAGER", "ExecutionHandler":"ssm_document_name|arn", "ExecuteOperationOnScriptFailure":true|false, "ExecutionTimeout":timeout_in_seconds (10-120), "MaximumRetryCount":retries (0-3) }] }, "RetainRule": { "Count": retention_count } }] } ``` ------ ## Considerations for VSS Backups with Amazon Data Lifecycle Manager With Amazon Data Lifecycle Manager, you can back up and restore VSS (Volume Shadow Copy Service)-enabled Windows applications running on Amazon EC2 instances. If the application has a VSS writer registered with Windows VSS, then Amazon Data Lifecycle Manager creates a snapshot that will be application-consistent for that application. **Note** Amazon Data Lifecycle Manager currently supports application-consistent snapshots of resources running on Amazon EC2 only, specifically for backup scenarios where application data can be restored by replacing an existing instance with a new instance created from the backup. Not all instance types or applications are supported for VSS backups. For more information, see [ Application-consistent Windows VSS snapshots](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/application-consistent-snapshots.html) in the *Amazon EC2 User Guide*. **Unsupported instance types** The following Amazon EC2 instance types are not supported for VSS backups. If your policy targets one of these instance types, Amazon Data Lifecycle Manager might still create VSS backups, but the snapshots might not be tagged with the required system tags. Without these tags, the snapshots will not be managed by Amazon Data Lifecycle Manager after creation. You might need to manually delete these snapshots. + T3: `t3.nano` \$1 `t3.micro` + T3a: `t3a.nano` \$1 `t3a.micro` + T2: `t2.nano` \$1 `t2.micro` ## Shared responsibility for application-consistent snapshots **You must ensure that:** + The SSM Agent is installed, up-to-date, and running on your target instances + Systems Manager has permissions to perform the required actions on the target instances + Amazon Data Lifecycle Manager has permissions to perform the Systems Manager actions required to run pre and post scripts on the target instances. + For custom workloads, such as self-managed MySQL, PostgreSQL, or InterSystems IRIS databases, the SSM document that you use includes the correct and required actions for freezing, flushing, and thawing I/O for your database configuration. + Snapshot creation times align with your workload schedule. For example, try to schedule snapshot creation during scheduled maintenance windows. **Amazon Data Lifecycle Manager ensures that:** + Snapshot creation is initiated within 60 minutes of the scheduled snapshot creation time. + Pre scripts run before the snapshot creation is initiated. + Post scripts run after the pre script succeeds and the snapshot creation has been initiated. Amazon Data Lifecycle Manager runs the post script only if the pre script succeeds. If the pre script fails, Amazon Data Lifecycle Manager will not run the post script. + Snapshots are tagged with the appropriate tags on creation. + CloudWatch metrics and events are emitted when scripts are initiated, and when they fail or succeed. # Other use cases for Data Lifecycle Manager pre and post scripts In addition to using pre and post scripts for automating application-consistent snapshots, you can use pre and post scripts together, or individually, to automate other administrative tasks before or after snapshot creation. For example: + Using a pre script to apply patches before creating snapshots. This can help you create snapshots after applying your regular weekly or monthly software updates. **Note** If you choose to run a pre script only, **Default to crash-consistent snapshots** is enabled by default. + Using a post script to apply patches after creating snapshots. This can help you create snapshots before applying your regular weekly or monthly software updates. ## Getting started for other use cases This section explains the steps you need perform when using pre and/or post scripts for **uses cases other than application-consistent snapshots**. ### Step 1: Prepare target instances **To prepare your target instances for pre and/or post scripts** 1. Install the SSM Agent on your target instances, if it is not already installed. If SSM Agent is already installed on your target instances, skip this step. + (Linux instances) [ Manually installing SSM Agent on EC2 instances for Linux](https://docs.aws.amazon.com/systems-manager/latest/userguide/manually-install-ssm-agent-linux.html) + (Windows instances) [ Working with SSM Agent on EC2 instances for Windows Server](https://docs.aws.amazon.com/systems-manager/latest/userguide/ssm-agent-windows.html) 1. Ensure that the SSM Agent is running. For more information, see [ Checking SSM Agent status and starting the agent](https://docs.aws.amazon.com/systems-manager/latest/userguide/ssm-agent-status-and-restart.html). 1. Set up Systems Manager for Amazon EC2 instances. For more information, see [Setting up Systems Manager for Amazon EC2 instances](https://docs.aws.amazon.com/systems-manager/latest/userguide/systems-manager-setting-up-ec2.html) in the *AWS Systems Manager User Guide*. ### Step 2: Prepare SSM document You must create an SSM command document that includes the pre and/or post scripts with the commands you want to run. You can create an SSM document using the empty SSM document template below and add your pre and post script commands in the appropriate document sections. **Note the following:** It is your responsibility to ensure that the SSM document performs the correct and required actions for your workload. The SSM document must include required fields for `allowedValues`, including `pre-script`, `post-script`, and `dry-run`. Amazon Data Lifecycle Manager will execute commands on your instance based on the contents of those sections. If your SSM document does not have those sections, then Amazon Data Lifecycle Manager will treat it as a failed execution. ``` ###===============================================================================### # Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. # Permission is hereby granted, free of charge, to any person obtaining a copy of this # software and associated documentation files (the "Software"), to deal in the Software # without restriction, including without limitation the rights to use, copy, modify, # merge, publish, distribute, sublicense, and/or sell copies of the Software, and to # permit persons to whom the Software is furnished to do so. # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, # INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A # PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT # HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION # OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE # SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. ###===============================================================================### schemaVersion: '2.2' description: SSM Document Template for Amazon Data Lifecycle Manager Pre/Post script feature parameters: executionId: type: String default: None description: (Required) Specifies the unique identifier associated with a pre and/or post execution allowedPattern: ^(None|[a-fA-F0-9]{8}-[a-fA-F0-9]{4}-[a-fA-F0-9]{4}-[a-fA-F0-9]{4}-[a-fA-F0-9]{12})$ command: # Data Lifecycle Manager will trigger the pre-script and post-script actions during policy execution. # 'dry-run' option is intended for validating the document execution without triggering any commands # on the instance. The following allowedValues will allow Data Lifecycle Manager to successfully # trigger pre and post script actions. type: String default: 'dry-run' description: (Required) Specifies whether pre-script and/or post-script should be executed. allowedValues: - pre-script - post-script - dry-run mainSteps: - action: aws:runShellScript description: Run Database freeze/thaw commands name: run_pre_post_scripts precondition: StringEquals: - platformType - Linux inputs: runCommand: - | #!/bin/bash ###===============================================================================### ### Error Codes ###===============================================================================### # The following Error codes will inform Data Lifecycle Manager of the type of error # and help guide handling of the error. # The Error code will also be emitted via AWS Eventbridge events in the 'cause' field. # 1 Pre-script failed during execution - 201 # 2 Post-script failed during execution - 202 # 3 Auto thaw occurred before post-script was initiated - 203 # 4 Pre-script initiated while post-script was expected - 204 # 5 Post-script initiated while pre-script was expected - 205 # 6 Application not ready for pre or post-script initiation - 206 ###===============================================================================### ### Global variables ###===============================================================================### START=$(date +%s) # For testing this script locally, replace the below with OPERATION=$1. OPERATION={{ command }} # Add all pre-script actions to be performed within the function below execute_pre_script() { echo "INFO: Start execution of pre-script" } # Add all post-script actions to be performed within the function below execute_post_script() { echo "INFO: Start execution of post-script" } # Debug logging for parameters passed to the SSM document echo "INFO: ${OPERATION} starting at $(date) with executionId: ${EXECUTION_ID}" # Based on the command parameter value execute the function that supports # pre-script/post-script operation case ${OPERATION} in pre-script) execute_pre_script ;; post-script) execute_post_script ;; dry-run) echo "INFO: dry-run option invoked - taking no action" ;; *) echo "ERROR: Invalid command parameter passed. Please use either pre-script, post-script, dry-run." exit 1 # return failure ;; esac END=$(date +%s) # Debug Log for profiling the script time echo "INFO: ${OPERATION} completed at $(date). Total runtime: $((${END} - ${START})) seconds." ``` ### Step 3: Prepare Amazon Data Lifecycle Manager IAM role **Note** This step is needed if: You create or update a pre/post script-enabled snapshot policy that uses a custom IAM role. You use the command line to create or update a pre/post script-enabled snapshot policy that uses the default. If you use the console to create or update a pre/post script-enabled snapshot policy that uses the default role for managing snapshots (**AWSDataLifecycleManagerDefaultRole**), skip this step. In this case, we automatically attach the **AWSDataLifecycleManagerSSMFullAccess** policy to that role. You must ensure that that IAM role that you use for the policy grants Amazon Data Lifecycle Manager permission to perform the SSM actions required to run pre and post scripts on instances targeted by the policy. Amazon Data Lifecycle Manager provides a managed policy (**AWSDataLifecycleManagerSSMFullAccess**) that includes the required permissions. You can attach this policy to your IAM role for managing snapshots to ensure that it includes the permissions. **Important** The AWSDataLifecycleManagerSSMFullAccess managed policy uses the `aws:ResourceTag` condition key to restrict access to specific SSM documents when using pre and post scripts. To allow Amazon Data Lifecycle Manager to access the SSM documents, you must ensure that your SSM documents are tagged with `DLMScriptsAccess:true`. Alternatively, you can manually create a custom policy or assign the required permissions directly to the IAM role that you use. You can use the same permissions that are defined in the AWSDataLifecycleManagerSSMFullAccess managed policy, however, the `aws:ResourceTag` condition key is optional. If you decide to not use that condition key, then you do not need to tag your SSM documents with `DLMScriptsAccess:true`. Use one of the following methods to add the **AWSDataLifecycleManagerSSMFullAccess** policy to your IAM role. ------ #### [ Console ] **To attach the managed policy to your custom role** 1. Open the IAM console at [https://console.aws.amazon.com/iam/](https://console.aws.amazon.com/iam/). 1. In the navigation panel, choose **Roles**. 1. Search for and select your custom role for managing snapshots. 1. On the **Permissions** tab, choose **Add permissions**, **Attach policies**. 1. Search for and select the **AWSDataLifecycleManagerSSMFullAccess** managed policy, and then choose **Add permissions**. ------ #### [ AWS CLI ] **To attach the managed policy to your custom role** Use the [ attach-role-policy](https://docs.aws.amazon.com/cli/latest/reference/iam/attach-role-policy.html) command. For `---role-name`, specify the name of your custom role. For `--policy-arn`, specify `arn:aws:iam::aws:policy/AWSDataLifecycleManagerSSMFullAccess`. ``` $ aws iam attach-role-policy \ --policy-arn arn:aws:iam::aws:policy/AWSDataLifecycleManagerSSMFullAccess \ --role-name your_role_name ``` ------ ### Create snapshot lifecycle policy ------ #### [ Console ] **To create the snapshot lifecycle policy** 1. Open the Amazon EC2 console at [https://console.aws.amazon.com/ec2/](https://console.aws.amazon.com/ec2/). 1. In the navigation pane, choose **Elastic Block Store**, **Lifecycle Manager**, and then choose **Create lifecycle policy**. 1. On the **Select policy type** screen, choose **EBS snapshot policy** and then choose **Next**. 1. In the **Target resources** section, do the following: 1. For **Target resource types**, choose `Instance`. 1. For **Target resource tags**, specify the resource tags that identify the instances to back up. Only resources that have the specified tags will be backed up. 1. For **IAM role**, either choose **AWSDataLifecycleManagerDefaultRole** (the default role for managing snapshots), or choose a custom role that you created and prepared for pre and post scripts. 1. Configure the schedules and additional options as needed. We recommend that you schedule snapshot creation times for time periods that match your workload, such as during maintenance windows. 1. In the **Pre and post scripts** section, select **Enable pre and post scripts** and then do the following: 1. Select **Custom SSM document**. 1. For **Automate option**, choose the option that matches the scripts you want to run. 1. For **SSM document**, select the SSM document that you prepared. 1. Configure the following additional options if needed: + **Script timeout** — The timeout period after which Amazon Data Lifecycle Manager fails the script run attempt if it has not completed. If a script does not complete within its timeout period, Amazon Data Lifecycle Manager fails the attempt. The timeout period applies to the pre and post scripts individually. The minimum and default timeout period is 10 seconds. And the maximum timeout period is 120 seconds. + **Retry failed scripts** — Select this option to retry scripts that do not complete within their timeout period. If the pre script fails, Amazon Data Lifecycle Manager retries entire snapshot creation process, including running the pre and post scripts. If the post script fails, Amazon Data Lifecycle Manager retries the post script only; in this case, the pre script will have completed and the snapshot might have been created. + **Default to crash-consistent snapshots** — Select this option to default to crash-consistent snapshots if the pre script fails to run. This is the default snapshot creation behavior for Amazon Data Lifecycle Manager if pre and post scripts is not enabled. If you enabled retries, Amazon Data Lifecycle Manager will default to crash-consistent snapshots only after all retry attempts have been exhausted. If the pre script fails and you do not default to crash-consistent snapshots, Amazon Data Lifecycle Manager will not create snapshots for the instance during that schedule run. 1. Choose **Create default policy**. **Note** If you get the `Role with name AWSDataLifecycleManagerDefaultRole already exists` error, see [Troubleshoot Amazon Data Lifecycle Manager issues](dlm-troubleshooting.md) for more information. ------ #### [ AWS CLI ] **To create the snapshot lifecycle policy** Use the [ create-lifecycle-policy](https://docs.aws.amazon.com/cli/latest/reference/dlm/create-lifecycle-policy.html) command, and include the `Scripts` parameters in `CreateRule`. For more information about the parameters, see the [https://docs.aws.amazon.com/dlm/latest/APIReference/API_Script.html](https://docs.aws.amazon.com/dlm/latest/APIReference/API_Script.html). ``` $ aws dlm create-lifecycle-policy \ --description "policy_description" \ --state ENABLED \ --execution-role-arn iam_role_arn \ --policy-details file://policyDetails.json ``` Where `policyDetails.json` includes the following. ``` { "PolicyType": "EBS_SNAPSHOT_MANAGEMENT", "ResourceTypes": [ "INSTANCE" ], "TargetTags": [{ "Key": "tag_key", "Value": "tag_value" }], "Schedules": [{ "Name": "schedule_name", "CreateRule": { "CronExpression": "cron_for_creation_frequency", "Scripts": [{ "Stages": ["PRE" | "POST" | "PRE","POST"], "ExecutionHandlerService":"AWS_SYSTEMS_MANAGER", "ExecutionHandler":"ssm_document_name|arn", "ExecuteOperationOnScriptFailure":true|false, "ExecutionTimeout":timeout_in_seconds (10-120), "MaximumRetryCount":retries (0-3) }] }, "RetainRule": { "Count": retention_count } }] } ``` ------ # How Amazon Data Lifecycle Manager pre and post scripts work The following image shows the process flow for pre and post scripts when using custom SSM documents. This does not apply to VSS Backups. ![\[Amazon Data Lifecycle Manager pre and post script process flow\]](http://docs.aws.amazon.com/ebs/latest/userguide/images/dlm-scripts.png) At the scheduled snapshot creation time, the following actions and cross-service interactions occur. 1. Amazon Data Lifecycle Manager initiates the pre script action by calling the SSM document and passing the `pre-script` parameter. **Note** Steps 1 to 3 occur only if you run pre scripts. If you run post scripts only, steps 1 to 3 are skipped. 1. Systems Manager sends pre script commands to the SSM Agent running on the target instances. The SSM Agent runs the commands on the instance, and sends status information back to Systems Manager. For example, if the SSM document is used to create application-consistent snapshots, the pre script might freeze and flush I/O to ensure that all buffered data is written to the volume before the snapshot is taken. 1. Systems Manager sends pre script command status updates to Amazon Data Lifecycle Manager. If the pre script fails, Amazon Data Lifecycle Manager takes one of the following actions, depending on how you configure the pre and post script options: [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/ebs/latest/userguide/script-flow.html) 1. Amazon Data Lifecycle Manager initiates snapshot creation. 1. Amazon Data Lifecycle Manager initiates the post script action by calling the SSM document and passing the `post-script` parameter. **Note** Steps 5 to 7 occur only if you run pre scripts. If you run post scripts only, steps 1 to 3 are skipped. 1. Systems Manager sends post script commands to the SSM Agent running on the target instances. The SSM Agent runs the commands on the instance, and sends status information back to Systems Manager. For example, if the SSM document enables application-consistent snapshots, this post script might thaw I/O to ensure that your databases resume normal I/O operations after the snapshot has been taken. 1. If you run a post script and Systems Manager indicates that it completed successfully, the process completes. If the post script fails, Amazon Data Lifecycle Manager takes one of the following actions, depending on how you configure the pre and post script options: [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/ebs/latest/userguide/script-flow.html) Keep in mind that if the post script fails, the pre script (if enabled) will have completed successfully, and the snapshots might have been created. You might need to take further action on the instance to ensure that it is operating as expected. For example if the pre script paused and flushed I/O, but the post script failed to thaw I/O, you might need to configure your database to auto-thaw I/O or you need to manually thaw I/O. 1. The snapshot creation process might complete after the post script completes. The time taken to complete the snapshot depends on the snapshot size. # Identify snapshots created with Data Lifecycle Manager pre and post scripts Amazon Data Lifecycle Manager automatically assigns the following system tags to snapshots created with pre and post scripts. + Key: `aws:dlm:pre-script`; Value: `SUCCESS`\$1`FAILED` A tag value of `SUCCESS` indicates that the pre script executed successfully. A tag value of `FAILED` indicates that the pre script did not execute successfully. + Key: `aws:dlm:post-script`; Value: `SUCCESS`\$1`FAILED` A tag value of `SUCCESS` indicates that the post script executed successfully. A tag value of `FAILED` indicates that the post script did not execute successfully. For custom SSM documents and SAP HANA backups, you can infer successful application-consistent snapshot creation if the snapshot is tagged with both `aws:dlm:pre-script:SUCCESS` and `aws:dlm:post-script:SUCCESS`. Additionally, application-consistent snapshots created using VSS backup are automatically tagged with: + Key: `AppConsistent tag`; Value: `true`\$1`false` A tag value of `true` indicates that the VSS backup succeeded and that the snapshots are application-consistent. A tag value of `false` indicates that the VSS backup did not succeed and that the snapshots are not application-consistent. # Monitor Amazon Data Lifecycle Manager pre and post scripts **Amazon CloudWatch metrics** Amazon Data Lifecycle Manager publishes the following CloudWatch metrics when pre and post scripts fail and succeed and when VSS backups fail and succeed. + `PreScriptStarted` + `PreScriptCompleted` + `PreScriptFailed` + `PostScriptStarted` + `PostScriptCompleted` + `PostScriptFailed` + `VSSBackupStarted` + `VSSBackupCompleted` + `VSSBackupFailed` For more information, see [Monitor Data Lifecycle Manager policies using CloudWatch](monitor-dlm-cw-metrics.md). **Amazon EventBridge** Amazon Data Lifecycle Manager emits the following Amazon EventBridge event when a pre or post script is initiated, succeeds, or fails + `DLM Pre Post Script Notification` For more information, see [Monitor Data Lifecycle Manager policies using EventBridge](monitor-cloudwatch-events.md).