# Increase or decrease compute capacity of your application with scaling
*Scaling* is the ability to increase or decrease the compute capacity of your application. Scaling starts with an event, or scaling action, which instructs an Auto Scaling group to either launch or terminate Amazon EC2 instances.
Amazon EC2 Auto Scaling provides a number of ways to adjust scaling to best meet the needs of your applications. As a result, it's important that you have a good understanding of your application. Keep the following considerations in mind:
+ What role should Amazon EC2 Auto Scaling play in your application's architecture? It's common to think about automatic scaling primarily as a way to increase and decrease capacity, but it's also useful for maintaining a steady number of servers.
+ What cost constraints are important to you? Because Amazon EC2 Auto Scaling uses EC2 instances, you pay only for the resources that you use. Knowing your cost constraints helps you decide when to scale your applications, and by how much.
+ What metrics are important to your application? Amazon CloudWatch supports a number of different metrics that you can use with your Auto Scaling group.
**Topics**
+ [Choose your scaling method](scaling-overview.md)
+ [Set scaling limits](asg-capacity-limits.md)
+ [Set the default instance warmup](ec2-auto-scaling-default-instance-warmup.md)
+ [Manual scaling](ec2-auto-scaling-scaling-manually.md)
+ [Scheduled scaling](ec2-auto-scaling-scheduled-scaling.md)
+ [Dynamic scaling](as-scale-based-on-demand.md)
+ [Predictive scaling](ec2-auto-scaling-predictive-scaling.md)
+ [Control instance termination](as-instance-termination.md)
+ [Suspend-resume processes](as-suspend-resume-processes.md)
# Choose your scaling method
Amazon EC2 Auto Scaling provides several ways for you to scale your Auto Scaling group.
**Maintain a fixed number of instances**
The default for an Auto Scaling group is to not have any attached scaling policies or scheduled actions, which causes it to maintain a fixed size. After you create your Auto Scaling group, it starts by launching enough instances to meet its desired capacity. If there are no scaling conditions attached to the group, it continues to maintain its desired capacity even if an instance becomes unhealthy. Amazon EC2 Auto Scaling monitors the health of each instance in your Auto Scaling group. When it finds that an instance has become unhealthy, it replaces it with a new instance. You can read a more in-depth description of this process in [Health checks for instances in an Auto Scaling group](ec2-auto-scaling-health-checks.md).
**Scale manually**
Manual scaling is the most basic way to scale your Auto Scaling group. You can either update the desired capacity of the Auto Scaling group or terminate instances in the Auto Scaling group. For more information, see [Manual scaling for Amazon EC2 Auto Scaling](ec2-auto-scaling-scaling-manually.md).
**Scale based on a schedule**
Scaling by schedule means that scaling actions are performed automatically as a function of date and time. This is useful when you know exactly when to increase or decrease the number of instances in your group, simply because the need arises on a predictable schedule. For more information, see [Scheduled scaling for Amazon EC2 Auto Scaling](ec2-auto-scaling-scheduled-scaling.md).
**Scale dynamically based on demand**
A more advanced way to scale your resources, using dynamic scaling, lets you define a scaling policy that dynamically resizes your Auto Scaling group to meet changes in demand. For example, let's say that you have a web application that currently runs on two instances and you want the CPU utilization of the Auto Scaling group to stay at around 50 percent when the load on the application changes. This method is useful for scaling as traffic changes occur, when you don't know when the traffic will change. You can configure scaling policies to respond for you. There are multiple policy types (or a combination of them) that you can use to scale in response to traffic changes. For more information, see [Dynamic scaling for Amazon EC2 Auto Scaling](as-scale-based-on-demand.md).
**Scale proactively**
You can also combine predictive scaling and dynamic scaling (proactive and reactive approaches, respectively) to scale your EC2 capacity faster. Use predictive scaling to increase the number of EC2 instances in your Auto Scaling group in advance of daily and weekly patterns in traffic flows. For more information, see [Predictive scaling for Amazon EC2 Auto Scaling](ec2-auto-scaling-predictive-scaling.md).
# Set scaling limits for your Auto Scaling group
Scaling limits represent the minimum and maximum group size that you want for your Auto Scaling group. You set limits separately for the minimum and maximum size.
The group's desired capacity can be resized to a number that's within the range of your minimum and maximum size limits. The desired capacity must be equal to or greater than the minimum group size, and equal to or less than the maximum group size.
+ **Desired capacity**: Represents the initial capacity of the Auto Scaling group at the time of creation. An Auto Scaling group attempts to maintain the desired capacity. It starts by launching the number of instances that are specified for the desired capacity, and maintains this number of instances as long as there are no scaling policies or scheduled actions attached to the Auto Scaling group.
+ **Minimum capacity**: Represents the minimum group size. When scaling policies are set, they cannot decrease the group's desired capacity lower than the minimum capacity.
+ **Maximum capacity**: Represents the maximum group size. When scaling policies are set, they cannot increase the group's desired capacity higher than the maximum capacity.
The minimum and maximum size limits also apply in the following scenarios:
+ When you manually scale your Auto Scaling group by updating its desired capacity.
+ When scheduled actions run that update the desired capacity. If a scheduled action runs without specifying new minimum and maximum size limits for the group, then the group's current minimum and maximum size limits apply.
An Auto Scaling group always tries to maintain its desired capacity. In cases where an instance terminates unexpectedly (for example, because of a Spot Instance interruption, a health check failure, or human action), the group automatically launches a new instance to maintain its desired capacity.
**To manage these settings in the console**
1. Open the Amazon EC2 console at [https://console.aws.amazon.com/ec2/](https://console.aws.amazon.com/ec2/).
1. On the navigation pane, under **Auto Scaling**, choose **Auto Scaling Groups**.
1. On the **Auto Scaling groups** page, select the check box next to your Auto Scaling group.
A split pane opens up in the bottom of the page.
1. In the lower pane, in the **Details** tab, view or change the current settings for the group's desired, minimum, and maximum capacity. For more information, see [Change the desired capacity of an existing Auto Scaling group](ec2-auto-scaling-scaling-manually.md#change-desired-capacity).
Above the **Details** pane, you can find information such as the current number of instances in the Auto Scaling group, the desired, minimum, and maximum capacity, and a status column. If the Auto Scaling group uses instance weights, you can also find the number of capacity units contributed to the desired capacity.
To add or remove columns from the list, choose the settings icon at the top of the page. Then, for **Auto Scaling groups attributes**, turn each column on or off, and choose **Confirm**.
**To verify the size of your Auto Scaling group after making changes**
The **Instances** column shows the number of instances that are currently running. While an instance is being launched or terminated, the **Status** column displays a status of *Updating capacity*, as shown in the following image.
![\[Updating the capacity of an Auto Scaling group.\]](http://docs.aws.amazon.com/autoscaling/ec2/userguide/images/asg-console-updating-capacity.png)
Wait for a few minutes, and then refresh the view to see the latest status. After a scaling activity completes, the **Instances** column shows an updated value.
You can view the number of instances and the status of the currently running instances from the **Instance management** tab, under **Instances**.
# Set the default instance warmup for an Auto Scaling group
CloudWatch collects and aggregates usage data, such as CPU and network I/O, across your Auto Scaling instances. You use these metrics to create scaling policies that adjust the number of instances in your Auto Scaling group as the selected metric's value increases and decreases.
You can specify how long after an instance reaches the `InService` state it waits before contributing usage data to the aggregated metrics. This specified time is called the *default instance warmup*. This keeps dynamic scaling from being affected by metrics for individual instances that aren't yet handling application traffic and that might be experiencing temporarily high usage of compute resources.
To optimize the performance of your target tracking and step scaling policies, we strongly recommend that you enable and configure the default instance warmup. It is not enabled or configured by default.
When you enable the default instance warmup, keep in mind that if your Auto Scaling group is set to use an instance maintenance policy, or you use an instance refresh to replace instances, you can prevent instances from being counted toward the minimum healthy percentage before they have finished initializing.
**Topics**
+ [Scaling performance considerations](#scaling-performance-considerations)
+ [Choose the default instance warmup time](#choose-the-default-instance-warmup)
+ [Enable the default instance warmup for a group](enable-default-instance-warmup.md)
+ [Verify the default instance warmup time for a group](verify-default-instance-warmup.md)
+ [Find scaling policies with a previously set instance warmup time](find-policies-with-a-previously-set-instance-warmup.md)
+ [Clear the previously set instance warmup for a scaling policy](clearing-the-previously-set-instance-warmup.md)
## Scaling performance considerations
It's useful for most applications to have one default instance warmup time that applies to all features, rather than different warmup times for different features. For example, if you don't set a default instance warmup, the instance refresh feature uses the health check grace period as the default warmup time. If you have any target tracking and step scaling policies, they use the value set for the default cooldown as the default warmup time. If you have any predictive scaling policies, they have no default warmup time.
While instances are warming up, your dynamic scaling policies scale out only if the metric value from instances that are not warming up is greater than the policy's alarm high threshold (or the target utilization of a target tracking scaling policy). If demand decreases, dynamic scaling becomes more conservative to protect your application's availability. This blocks the scale in activities for dynamic scaling until the new instances finish warming up.
While scaling out, Amazon EC2 Auto Scaling considers instances that are warming up as part of the capacity of the group when deciding how many instances to add to the group. Therefore, multiple alarm breaches that require a similar amount of capacity to be added result in a single scaling activity. The intention is to continuously scale out, without doing so excessively.
If default instance warmup is not enabled, the amount of time an instance waits before sending metrics to CloudWatch and counting it towards the current capacity will vary from instance to instance. So, there is the potential for your scaling policies to perform unpredictably compared to the actual workload that is occurring.
For example, consider an application with a recurring on-and-off workload pattern. A predictive scaling policy is used to make recurring decisions about whether to increase the number of instances. Because there is no default warmup time for predictive scaling policies, the instances start contributing to the aggregated metrics immediately. If these instances have higher resource usage on startup, then adding instances could cause the aggregated metrics to spike. Depending on how long it takes for usage to stabilize, this could impact any dynamic scaling policies that use these metrics. If a dynamic scaling policy's alarm high threshold is breached, then the group increases in size again. While the new instances are warming up, scale in activities will be blocked.
## Choose the default instance warmup time
The key to setting the default instance warmup is determining how long your instances need to finish initializing and for resource consumption to stabilize after they reach the `InService` state. When choosing the instance warmup time, try to keep an optimal balance between collecting usage data for legitimate traffic, and minimizing data collection associated with temporary usage spikes on startup.
Suppose you have an Auto Scaling group attached to an Elastic Load Balancing load balancer. When new instances finish launching, they're registered to the load balancer before they enter the `InService` state. After the instances enter the `InService` state, resource consumption can still experience temporary spikes and need time to stabilize. For example, resource consumption for an application server that must download and cache large assets takes longer to stabilize than a lightweight web server with no large assets to download. The instance warmup provides the time delay necessary for resource consumption to stabilize.
**Important**
If you're not sure how much time you need for the warmup time, you could start with 300 seconds. Then gradually decrease or increase it until you get the best scaling performance for your application. You might need to do this a few times to get it right. Alternatively, if you have any scaling policies that have their own warmup time (`EstimatedInstanceWarmup`), you could use this value to start. For more information, see [Find scaling policies with a previously set instance warmup time](find-policies-with-a-previously-set-instance-warmup.md).
Consider using lifecycle hooks for use cases where you have configuration tasks or scripts to run on startup. Lifecycle hooks can delay new instances from being put in service until they have finished initializing. They are particularly useful if you have bootstrapping scripts that take a while to complete. If you add a lifecycle hook, you can reduce the value of the default instance warmup. For more information about using lifecycle hooks, see [Amazon EC2 Auto Scaling lifecycle hooks](lifecycle-hooks.md).
# Enable the default instance warmup for a group
You can enable the default instance warmup when you create an Auto Scaling group. You can also enable it for existing groups.
By enabling the default instance warmup feature, you no longer have to specify values for warmup parameters for the following features:
+ [Instance refresh](instance-refresh-overview.md#instance-refresh-core-concepts)
+ [Target tracking scaling](as-scaling-target-tracking.md#as-target-tracking-scaling-warmup)
+ [Step scaling](as-scaling-simple-step.md#as-step-scaling-warmup)
------
#### [ Console ]
**To enable the default instance warmup for a new group (console)**
When you create the Auto Scaling group, on the **Configure advanced options** page, under **Additional settings**, select the **Enable default instance warmup** option. Choose the warmup time that you need for your application.
------
#### [ AWS CLI ]
**To enable the default instance warmup for a new group (AWS CLI)**
To enable the default instance warmup for an Auto Scaling group, add the `--default-instance-warmup` option and specify a value, in seconds, from 0 to 3600. After it's enabled, a value of `-1` will turn this setting off.
The following [create-auto-scaling-group](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/autoscaling/create-auto-scaling-group.html) command creates an Auto Scaling group with the name *my-asg* and enables the default instance warmup with a value of *120* seconds.
```
aws autoscaling create-auto-scaling-group --auto-scaling-group-name my-asg --default-instance-warmup 120 ...
```
**Tip**
If this command throws an error, make sure that you have updated the AWS CLI locally to the latest version.
------
------
#### [ Console ]
**To enable the default instance warmup for an existing group (console)**
1. Open the Amazon EC2 console at [https://console.aws.amazon.com/ec2/](https://console.aws.amazon.com/ec2/), and choose **Auto Scaling Groups** from the navigation pane.
1. On the navigation bar at the top of the screen, choose the AWS Region that you created your Auto Scaling group in.
1. Select the check box next to the Auto Scaling group.
A split pane opens up in the bottom of the page.
1. On the **Details** tab, choose **Advanced configurations**, **Edit**.
1. For **Default instance warmup**, choose the warmup time that you need for your application.
1. Choose **Update**.
------
#### [ AWS CLI ]
**To enable the default instance warmup for an existing group (AWS CLI)**
The following example uses the [update-auto-scaling-group](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/autoscaling/update-auto-scaling-group.html) command to enable the default instance warmup with a value of *120* seconds for an existing Auto Scaling group named *my-asg*.
```
aws autoscaling update-auto-scaling-group --auto-scaling-group-name my-asg --default-instance-warmup 120
```
**Tip**
If this command throws an error, make sure that you have updated the AWS CLI locally to the latest version.
------
# Verify the default instance warmup time for a group
Use the following procedure to verify the default instance warmup time for an Auto Scaling group using the AWS CLI.
**To verify the default instance warmup time for an Auto Scaling group**
Use the following [describe-auto-scaling-groups](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/autoscaling/describe-auto-scaling-groups.html) command. Replace *my-asg* with the name of your Auto Scaling group.
```
aws autoscaling describe-auto-scaling-groups --auto-scaling-group-name my-asg
```
The following is an example response.
```
{
"AutoScalingGroups": [
{
"AutoScalingGroupName": "my-asg",
"AutoScalingGroupARN": "arn",
...
"DefaultInstanceWarmup": 120
}
]
}
```
# Find scaling policies with a previously set instance warmup time
To identify whether you have policies that have their own warmup time for `EstimatedInstanceWarmup`, run the following [describe-policies](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/autoscaling/describe-policies.html) command using the AWS CLI. Replace *my-asg* with the name of your Auto Scaling group.
```
aws autoscaling describe-policies --auto-scaling-group-name my-asg
--query 'ScalingPolicies[?EstimatedInstanceWarmup!=`null`]'
```
The following is example output.
```
[
{
"AutoScalingGroupName":"my-asg",
"PolicyName":"cpu50-target-tracking-scaling-policy",
"PolicyARN":"arn",
"PolicyType":"TargetTrackingScaling",
"StepAdjustments":[],
"EstimatedInstanceWarmup":120,
"Alarms":[{
"AlarmARN": "arn:aws:cloudwatch:us-west-2:123456789012:alarm:TargetTracking-my-asg-AlarmHigh-fc0e4183-23ac-497e-9992-691c9980c38e",
"AlarmName": "TargetTracking-my-asg-AlarmHigh-fc0e4183-23ac-497e-9992-691c9980c38e"
},
{
"AlarmARN": "arn:aws:cloudwatch:us-west-2:123456789012:alarm:TargetTracking-my-asg-AlarmLow-61a39305-ed0c-47af-bd9e-471a352ee1a2",
"AlarmName": "TargetTracking-my-asg-AlarmLow-61a39305-ed0c-47af-bd9e-471a352ee1a2"
}],
"TargetTrackingConfiguration":{
"PredefinedMetricSpecification":{
"PredefinedMetricType":"ASGAverageCPUUtilization"
},
"TargetValue":50.0,
"DisableScaleIn":false
},
"Enabled":true
},
... additional policies ...
]
```
# Clear the previously set instance warmup for a scaling policy
After enabling the default instance warmup, update any scaling policies that have still their own warmup time to clear the previously set value. Otherwise, it will override the default instance warmup.
You can update scaling policies using the console, AWS CLI, or AWS SDKs. This section covers the steps for the console. If you use the AWS CLI or AWS SDKs, make sure you preserve the existing policy configuration, but remove the `EstimatedInstanceWarmup` property. When you update an existing scaling policy, the policy will be replaced with what you specify when you programmatically call [PutScalingPolicy](https://docs.aws.amazon.com/autoscaling/ec2/APIReference/API_PutScalingPolicy.html). The original values are not kept.
**To clear the previously set instance warmup for a scaling policy (console)**
1. Open the Amazon EC2 console at [https://console.aws.amazon.com/ec2/](https://console.aws.amazon.com/ec2/), and choose **Auto Scaling Groups** from the navigation pane.
1. Select the check box next to the Auto Scaling group.
A split pane opens up in the bottom of the page.
1. On the **Automatic scaling** tab, in **Dynamic scaling policies**, choose the policy you're interested in, and then choose **Actions**, **Edit**.
1. For **Instance warmup**, clear the instance warmup value to use the default instance warmup value instead.
1. Choose **Update**.
# Manual scaling for Amazon EC2 Auto Scaling
You can manually adjust the number of EC2 instances in your Auto Scaling group at any time. This process of changing the instance count manually is referred to as *manual scaling*. Manual scaling is an alternative to auto scaling, especially if you want to make one-time capacity changes.
After you manually scale your group, Amazon EC2 Auto Scaling resumes normal auto scaling activities based on the scaling policies and scheduled actions that you defined. For groups with default instance warmup enabled, any new instances go through a warmup period before they start contributing to the metrics used for auto scaling. This warmup period assists in stabilizing the group at the new capacity. For more information, see [Set the default instance warmup for an Auto Scaling group](ec2-auto-scaling-default-instance-warmup.md).
Sometimes, you may want to temporarily disable scaling policies and scheduled actions before manually scaling a group. Doing so prevents conflicts from arising between manual scaling actions and automated scaling activities. For more information, see [Turn off scaling activities](CHAP_Troubleshooting.md#turn-off-scaling-activities).
**Topics**
+ [Change the desired capacity of an existing Auto Scaling group](#change-desired-capacity)
+ [Terminate an instance in your Auto Scaling group (AWS CLI)](#terminate-an-instance-aws-cli)
## Change the desired capacity of an existing Auto Scaling group
When you change the desired capacity of your Auto Scaling group, Amazon EC2 Auto Scaling manages the process of launching and terminating instances to reach the new desired size.
------
#### [ Console ]
**To change the size of your Auto Scaling group**
1. Open the Amazon EC2 console at [https://console.aws.amazon.com/ec2/](https://console.aws.amazon.com/ec2/), and choose **Auto Scaling Groups** from the navigation pane.
1. Select the check box next to your Auto Scaling group.
A split pane displays at the bottom of the page.
1. On the **Details** tab, choose **Group details**, **Edit**.
1. For **Desired capacity**, increase or decrease the desired capacity. For example, to increase the size of the group by one, if the current value is `1`, enter `2`.
If your new value for **Desired capacity** is greater than **Min desired capacity** and **Max desired capacity**, the **Max desired capacity** is automatically increased to the new desired capacity value.
1. When you are finished, choose **Update**.
Verify that the group size that you specified resulted in the same amount of instances being launched. For example, if you increased the size of the group by one, verify that your Auto Scaling group has launched one additional instance.
**To verify that the size of your Auto Scaling group has changed**
1. On the **Activity** tab, in **Activity history**, you can view the progress of activities that are associated with the Auto Scaling group. The **Status** column shows the current status of your instance. While your instance is launching, the status column shows `Not yet in service`. The status changes to `Successful` after the instance is launched. You can also use the refresh icon to see the current status of your instance. For more information, see [Verify a scaling activity for an Auto Scaling group](as-verify-scaling-activity.md).
1. On the **Instance management** tab, in **Instances**, you can view the status of the instance. It takes a short time for an instance to launch.
+ The **Lifecycle** column shows the state of your instance. Initially, your instance is in the `Pending` state. After an instance is ready to receive traffic, its state is `InService`.
+ The **Health status** column shows the result of the Amazon EC2 Auto Scaling health checks on your instance.
------
#### [ AWS CLI ]
The following example assumes that you've created an Auto Scaling group with a minimum size of 1 and a maximum size of 5. Therefore, the group currently has one running instance.
**To change the size of your Auto Scaling group**
Use the [set-desired-capacity](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/autoscaling/set-desired-capacity.html) command to change the size of your Auto Scaling group, as shown in the following example.
```
aws autoscaling set-desired-capacity --auto-scaling-group-name my-asg \
--desired-capacity 2
```
If you choose to honor the default cooldown period for your Auto Scaling group, you must specify the `–-honor-cooldown` option as shown in the following example. For more information, see [Scaling cooldowns for Amazon EC2 Auto Scaling](ec2-auto-scaling-scaling-cooldowns.md).
```
aws autoscaling set-desired-capacity --auto-scaling-group-name my-asg \
--desired-capacity 2 --honor-cooldown
```
**To verify the size of your Auto Scaling group**
Use the [describe-auto-scaling-groups](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/autoscaling/describe-auto-scaling-groups.html) command to confirm that the size of your Auto Scaling group has changed, as in the following example.
```
aws autoscaling describe-auto-scaling-groups --auto-scaling-group-name my-asg
```
The following is example output, which provides details about the group and instances launched.
```
{
"AutoScalingGroups": [
{
"AutoScalingGroupName": "my-asg",
"AutoScalingGroupARN": "arn",
"LaunchTemplate": {
"LaunchTemplateName": "my-launch-template",
"Version": "1",
"LaunchTemplateId": "lt-050555ad16a3f9c7f"
},
"MinSize": 1,
"MaxSize": 5,
"DesiredCapacity": 2,
"DefaultCooldown": 300,
"AvailabilityZones": [
"us-west-2a"
],
"LoadBalancerNames": [],
"TargetGroupARNs": [],
"HealthCheckType": "EC2",
"HealthCheckGracePeriod": 300,
"Instances": [
{
"ProtectedFromScaleIn": false,
"AvailabilityZone": "us-west-2a",
"LaunchTemplate": {
"LaunchTemplateName": "my-launch-template",
"Version": "1",
"LaunchTemplateId": "lt-050555ad16a3f9c7f"
},
"InstanceId": "i-05b4f7d5be44822a6",
"InstanceType": "t3.micro",
"HealthStatus": "Healthy",
"LifecycleState": "Pending"
},
{
"ProtectedFromScaleIn": false,
"AvailabilityZone": "us-west-2a",
"LaunchTemplate": {
"LaunchTemplateName": "my-launch-template",
"Version": "1",
"LaunchTemplateId": "lt-050555ad16a3f9c7f"
},
"InstanceId": "i-0c20ac468fa3049e8",
"InstanceType": "t3.micro",
"HealthStatus": "Healthy",
"LifecycleState": "InService"
}
],
"CreatedTime": "2019-03-18T23:30:42.611Z",
"SuspendedProcesses": [],
"VPCZoneIdentifier": "subnet-c87f2be0",
"EnabledMetrics": [],
"Tags": [],
"TerminationPolicies": [
"Default"
],
"NewInstancesProtectedFromScaleIn": false,
"ServiceLinkedRoleARN": "arn",
"TrafficSources": []
}
]
}
```
Notice that `DesiredCapacity` shows the new value. Your Auto Scaling group has launched an additional instance.
------
## Terminate an instance in your Auto Scaling group (AWS CLI)
There are times when you might want to manually scale in your Auto Scaling group but want to terminate a specific instance. You can manually scale in your Auto Scaling group by using the [terminate-instance-in-auto-scaling-group](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/autoscaling/terminate-instance-in-auto-scaling-group.html) command and specifying the ID of the instance you want to terminate and the `--should-decrement-desired-capacity` option as shown in the following example.
```
aws autoscaling terminate-instance-in-auto-scaling-group \
--instance-id i-026e4c9f62c3e448c --should-decrement-desired-capacity
```
The following is example output, which provides details about the scaling activity.
```
{
"Activities": [
{
"ActivityId": "b8d62b03-10d8-9df4-7377-e464ab6bd0cb",
"AutoScalingGroupName": "my-asg",
"Description": "Terminating EC2 instance: i-026e4c9f62c3e448c",
"Cause": "At 2023-09-23T06:39:59Z instance i-026e4c9f62c3e448c was taken out of service in response to a user request, shrinking the capacity from 1 to 0.",
"StartTime": "2023-09-23T06:39:59.015000+00:00",
"StatusCode": "InProgress",
"Progress": 0,
"Details": "{\"Subnet ID\":\"subnet-6194ea3b\",\"Availability Zone\":\"us-west-2c\"}"
}
]
}
```
This option is not available in the console. However, you can use the **Instances** page of the Amazon EC2 console to terminate an instance in your Auto Scaling group. When you do so, Amazon EC2 Auto Scaling detects that the instance is no longer running and replaces it automatically as part of the health check process. It takes a minute or two after you terminate the instance before a new instance launches. For information about how to terminate an instance, see [Terminate an instance](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/terminating-instances.html) in the *Amazon EC2 User Guide*.
If you terminate instances in your group and that causes uneven distribution across Availability Zones, Amazon EC2 Auto Scaling rebalances the group to re-establish an even distribution unless you suspend the `AZRebalance` process. For more information, see [Suspend and resume Amazon EC2 Auto Scaling processes](as-suspend-resume-processes.md).
# Scheduled scaling for Amazon EC2 Auto Scaling
With scheduled scaling, you can set up automatic scaling for your application based on predictable load changes. You create scheduled actions that increase or decrease your group's desired capacity at specific times.
For example, you experience a regular weekly traffic pattern where load increases midweek and declines toward the end of the week. You can configure a scaling schedule in Amazon EC2 Auto Scaling that aligns with this pattern:
+ On Wednesday morning, one scheduled action increases capacity by increasing the previously set desired capacity of the Auto Scaling group.
+ On Friday evening, another scheduled action decreases capacity by decreasing the previously set desired capacity of the Auto Scaling group.
These scheduled scaling actions allow you to optimize costs and performance. Your application has sufficient capacity to handle the midweek traffic peak, but does not over-provision unneeded capacity at other times.
You can use scheduled scaling and scaling policies together to get the benefits of both approaches to scaling. After a scheduled scaling action runs, the scaling policy can continue to make decisions about whether to further scale capacity. This helps you ensure that you have sufficient capacity to handle the load for your application. While your application scales to match demand, current capacity must fall within the minimum and maximum capacity that was set by your scheduled action.
**Topics**
+ [How scheduled scaling works](#scheduled-scaling-how-it-works)
+ [Recurring schedules](#scheduled-scaling-recurring-schedules)
+ [Time zone](#scheduled-scaling-time-zone)
+ [Considerations](#scheduled-scaling-considerations)
+ [Limitations](#scheduled-scaling-limitations)
+ [Create a scheduled action](scheduled-scaling-create-scheduled-action.md)
+ [View scheduled action details](scheduled-scaling-view-scheduled-actions.md)
+ [Delete a scheduled action](scheduled-scaling-delete-scheduled-action.md)
## How scheduled scaling works
To use scheduled scaling, create *scheduled actions*, which tell Amazon EC2 Auto Scaling to perform scaling activities at specific times. When you create a scheduled action, you specify the Auto Scaling group, when the scaling activity should occur, the new desired capacity, and optionally a new minimum capacity and a new maximum capacity. You can create scheduled actions that scale one time only or that scale on a recurring schedule.
At the specified time, Amazon EC2 Auto Scaling scales based on the new capacity values, by comparing current capacity to the specified desired capacity.
+ If current capacity is less than the specified desired capacity, Amazon EC2 Auto Scaling scales out, or adds instances, to the specified desired capacity.
+ If current capacity is greater than the specified desired capacity, Amazon EC2 Auto Scaling scales in, or removes instances, to the specified desired capacity.
A scheduled action sets the group's desired, minimum, and maximum capacity at the date and time specified. You can create a scheduled action for only one of these capacities at a time, for example, desired capacity. However, there are some cases where you must include the minimum and maximum capacity to ensure that the desired capacity that you specified in the action is not outside of these limits.
## Recurring schedules
To create a recurring schedule using the AWS CLI or an SDK, specify a cron expression and a time zone to describe when that scheduled action is to recur. You can optionally specify a date and time for the start time, the end time, or both.
To create a recurring schedule using the AWS Management Console, specify the recurrence pattern, time zone, start time, and optional end time of your scheduled action. All of the recurrence pattern options are based on cron expressions. Alternatively, you can write your own custom cron expression.
The supported cron expression format consists of five fields separated by white spaces: [Minute] [Hour] [Day\$1of\$1Month] [Month\$1of\$1Year] [Day\$1of\$1Week]. For example, the cron expression `30 6 * * 2` configures a scheduled action that recurs every Tuesday at 6:30 AM. The asterisk is used as a wildcard to match all values for a field. For other examples of cron expressions, see [https://crontab.guru/examples.html](https://crontab.guru/examples.html). For information about writing your own cron expressions in this format, see [Crontab](http://crontab.org).
Choose your start and end times carefully. Keep the following in mind:
+ If you specify a start time, Amazon EC2 Auto Scaling performs the action at this time, and then performs the action based on the specified recurrence.
+ If you specify an end time, the action stops repeating after this time. A scheduled action does not persist in your account once it has reached its end time.
+ If a recurrence time exactly matches the end time, Amazon EC2 Auto Scaling will not perform the scheduled action at the end time.
+ The start time and end time must be set in UTC when you use the AWS CLI or an SDK.
## Time zone
By default, the recurring schedules that you set are in Coordinated Universal Time (UTC). You can change the time zone to correspond to your local time zone or a time zone for another part of your network. When you specify a time zone that observes Daylight Saving Time (DST), the action automatically adjusts for DST.
The valid values are the canonical names for time zones from the Internet Assigned Numbers Authority (IANA) Time Zone database. For example, US Eastern time is canonically identified as `America/New_York`. For more information, see [https://www.iana.org/time-zones](https://www.iana.org/time-zones).
Location-based time zones such as `America/New_York` automatically adjust for DST. However, a UTC-based time zone such as `Etc/UTC` is an absolute time and will not adjust for DST.
For example, you have a recurring schedule whose time zone is `America/New_York`. The first scaling action happens in the `America/New_York` time zone before DST starts. The next scaling action happens in the `America/New_York` time zone after DST starts. The first action starts at 8:00 AM UTC-5 in local time, while the second time starts at 8:00 AM UTC-4 in local time.
If you create a scheduled action using the AWS Management Console and specify a time zone that observes DST, both the recurring schedule and the start and end times automatically adjust for DST.
## Considerations
When you create a scheduled action, keep the following in mind:
+ The order of execution for scheduled actions is guaranteed within the same group, but not for scheduled actions across groups.
+ A scheduled action generally runs within seconds. However, the action might be delayed for up to two minutes from the scheduled start time. Because scheduled actions within an Auto Scaling group are executed in the order that they are specified, actions with scheduled start times close to each other can take longer to execute.
+ You can temporarily turn off scheduled scaling for an Auto Scaling group by suspending the `ScheduledActions` process. This helps you prevent scheduled actions from being active without having to delete them. You can then resume scheduled scaling when you want to use it again. For more information, see [Suspend and resume Amazon EC2 Auto Scaling processes](as-suspend-resume-processes.md).
+ After creating a scheduled action, you can update any of its settings except the name.
+ When multiple scheduled actions within the same Auto Scaling group have identical cron expressions, the execution order becomes arbitrary and undefined. To ensure predictable behavior, you can use unique scheduled start times for each scheduled action.
## Limitations
+ The names of scheduled actions must be unique per Auto Scaling group.
+ A scheduled action must have a unique time value. If you attempt to schedule an activity at a time when another scaling activity is already scheduled, the call is rejected and returns an error indicating that a scheduled action with this scheduled start time already exists.
+ You can create a maximum of 125 scheduled actions per Auto Scaling group.
# Create a scheduled action
To create a scheduled action for your Auto Scaling group, use one of the following methods:
------
#### [ Console ]
**To create a scheduled action**
1. Open the Amazon EC2 console at [https://console.aws.amazon.com/ec2/](https://console.aws.amazon.com/ec2/), and choose **Auto Scaling Groups** from the navigation pane.
1. Select the check box next to your Auto Scaling group.
A split pane opens up in the bottom of the page.
1. On the **Automatic scaling** tab, in **Scheduled actions**, choose **Create scheduled action**.
1. Enter a **Name** for the scheduled action.
1. For **Desired capacity**, **Min**, **Max**, choose the new desired capacity of the group and the new minimum and maximum size limits. The desired capacity must be equal to or greater than the minimum group size, and equal to or less than the maximum group size.
1. For **Recurrence**, choose one of the available options.
+ If you want to scale on a recurring schedule, choose how often Amazon EC2 Auto Scaling should run the scheduled action.
+ If you choose an option that begins with **Every**, the cron expression is created for you.
+ If you choose **Cron**, enter a cron expression that specifies when to perform the action.
+ If you want to scale only once, choose **Once**.
1. For **Time zone**, choose a time zone. The default is `Etc/UTC`.
All of the time zones listed are from the IANA Time Zone database. For more information, see [https://en.wikipedia.org/wiki/List\$1of\$1tz\$1database\$1time\$1zones](https://en.wikipedia.org/wiki/List_of_tz_database_time_zones).
1. Define a date and time for **Specific start time**.
+ If you chose a recurring schedule, the start time defines when the first scheduled action in the recurring series runs.
+ If you chose **Once** as the recurrence, the start time defines the date and time for the schedule action to run.
1. (Optional) For recurring schedules, you can specify an end time by choosing **Set End Time** and then choosing a date and time for **End by**.
1. Choose **Create**. The console displays the scheduled actions for the Auto Scaling group.
------
#### [ AWS CLI ]
To create a scheduled action, you can use one of the following example commands. Replace each *user input placeholder* with your own information.
**Example: To scale one time only**
Use the following [put-scheduled-update-group-action](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/autoscaling/put-scheduled-update-group-action.html) command with the `--start-time "YYYY-MM-DDThh:mm:ssZ"` and `--desired-capacity` options.
```
aws autoscaling put-scheduled-update-group-action --scheduled-action-name my-one-time-action \
--auto-scaling-group-name my-asg --start-time "2021-03-31T08:00:00Z" --desired-capacity 3
```
**Example: To schedule scaling on a recurring schedule**
Use the following [put-scheduled-update-group-action](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/autoscaling/put-scheduled-update-group-action.html) command with the `--recurrence "cron expression"` and `--desired-capacity` options.
```
aws autoscaling put-scheduled-update-group-action --scheduled-action-name my-recurring-action \
--auto-scaling-group-name my-asg --recurrence "0 9 * * *" --desired-capacity 3
```
By default, Amazon EC2 Auto Scaling runs the specified recurrence schedule based on the UTC time zone. To specify a different time zone, include the `--time-zone` option and the name of the IANA time zone, as in the following example.
```
--time-zone "America/New_York"
```
For more information, see [https://en.wikipedia.org/wiki/List\$1of\$1tz\$1database\$1time\$1zones](https://en.wikipedia.org/wiki/List_of_tz_database_time_zones).
------
# View scheduled action details
To view details of upcoming scheduled actions for your Auto Scaling group, use one of the following methods:
------
#### [ Console ]
**To view scheduled action details**
1. Open the Amazon EC2 console at [https://console.aws.amazon.com/ec2/](https://console.aws.amazon.com/ec2/), and choose **Auto Scaling Groups** from the navigation pane.
1. Select your Auto Scaling group.
1. On the **Automatic scaling** tab, in the **Scheduled actions** section, you can view upcoming scheduled actions.
Note that the console shows the values for **Start time** and **End time** in your local time with the UTC offset in effect at the specified date and time. The UTC offset is the difference, in hours and minutes, from local time to UTC. The value for **Time zone** shows your requested time zone, for example, `America/New_York`.
------
#### [ AWS CLI ]
Use the following [describe-scheduled-actions](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/autoscaling/describe-scheduled-actions.html) command.
```
aws autoscaling describe-scheduled-actions --auto-scaling-group-name my-asg
```
If successful, this command returns output similar to the following.
```
{
"ScheduledUpdateGroupActions": [
{
"AutoScalingGroupName": "my-asg",
"ScheduledActionName": "my-recurring-action",
"Recurrence": "30 0 1 1,6,12 *",
"ScheduledActionARN": "arn:aws:autoscaling:us-west-2:123456789012:scheduledUpdateGroupAction:8e86b655-b2e6-4410-8f29-b4f094d6871c:autoScalingGroupName/my-asg:scheduledActionName/my-recurring-action",
"StartTime": "2020-12-01T00:30:00Z",
"Time": "2020-12-01T00:30:00Z",
"MinSize": 1,
"MaxSize": 6,
"DesiredCapacity": 4
}
]
}
```
------
## Verify scaling activities
To verify the scaling activities associated with scheduled scaling, see [Verify a scaling activity for an Auto Scaling group](as-verify-scaling-activity.md).
# Delete a scheduled action
To delete a scheduled action, use one of the following methods:
------
#### [ Console ]
**To delete a scheduled action**
1. Open the Amazon EC2 console at [https://console.aws.amazon.com/ec2/](https://console.aws.amazon.com/ec2/), and choose **Auto Scaling Groups** from the navigation pane.
1. Select your Auto Scaling group.
1. On the **Automatic scaling** tab, in **Scheduled actions**, select a scheduled action.
1. Choose **Actions**, **Delete**.
1. When prompted for confirmation, choose **Yes, Delete**.
------
#### [ AWS CLI ]
Use the following [delete-scheduled-action](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/autoscaling/delete-scheduled-action.html) command.
```
aws autoscaling delete-scheduled-action --auto-scaling-group-name my-asg \
--scheduled-action-name my-recurring-action
```
------
# Dynamic scaling for Amazon EC2 Auto Scaling
Dynamic scaling scales the capacity of your Auto Scaling group as traffic changes occur.
Amazon EC2 Auto Scaling supports the following types of dynamic scaling policies:
+ **Target tracking scaling**—Increase and decrease the current capacity of the group based on a Amazon CloudWatch metric and a target value. It works similar to the way that your thermostat maintains the temperature of your home—you select a temperature and the thermostat does the rest.
+ **Step scaling**—Increase and decrease the current capacity of the group based on a set of scaling adjustments, known as *step adjustments*, that vary based on the size of the alarm breach.
+ **Simple scaling**—Increase and decrease the current capacity of the group based on a single scaling adjustment, with a cooldown period between each scaling activity.
We strongly recommend that you use target tracking scaling policies and choose a metric that changes inversely proportional to a change in the capacity of your Auto Scaling group. So if you double the size of your Auto Scaling group, the metric decreases by 50 percent. This allows the metric data to accurately trigger proportional scaling events. Included are metrics like average CPU utilization or average request count per target.
With target tracking, your Auto Scaling group scales in direct proportion to the actual load on your application. That means that in addition to meeting the immediate need for capacity in response to load changes, a target tracking policy can also adapt to load changes that take place over time, for example, due to seasonal variations.
Target tracking policies also remove the need to manually define CloudWatch alarms and scaling adjustments. Amazon EC2 Auto Scaling handles this automatically based on the target you set.
**Topics**
+ [How dynamic scaling policies work](#as-how-scaling-policies-work)
+ [Multiple dynamic scaling policies](#multiple-scaling-policy-resolution)
+ [Target tracking scaling policies for Amazon EC2 Auto Scaling](as-scaling-target-tracking.md)
+ [Step and simple scaling policies for Amazon EC2 Auto Scaling](as-scaling-simple-step.md)
+ [Scaling cooldowns for Amazon EC2 Auto Scaling](ec2-auto-scaling-scaling-cooldowns.md)
+ [Scaling policy based on Amazon SQS](as-using-sqs-queue.md)
+ [Verify a scaling activity for an Auto Scaling group](as-verify-scaling-activity.md)
+ [Disable a scaling policy for an Auto Scaling group](as-enable-disable-scaling-policy.md)
+ [Delete a scaling policy for an Auto Scaling group](deleting-scaling-policy.md)
+ [Example scaling policies for the AWS CLI](examples-scaling-policies.md)
## How dynamic scaling policies work
A dynamic scaling policy instructs Amazon EC2 Auto Scaling to track a specific CloudWatch metric, and it defines what action to take when the associated CloudWatch alarm is in ALARM. The metrics that are used to invoke the alarm state are an aggregation of metrics coming from all of the instances in the Auto Scaling group. (For example, let's say you have an Auto Scaling group with two instances where one instance is at 60 percent CPU and the other is at 40 percent CPU. On average, they are at 50 percent CPU.) When the policy is in effect, Amazon EC2 Auto Scaling adjusts the group's desired capacity up or down when the threshold of an alarm is breached.
When a dynamic scaling policy is invoked, if the capacity calculation produces a number outside of the minimum and maximum size range of the group, Amazon EC2 Auto Scaling ensures that the new capacity never goes outside of the minimum and maximum size limits. Capacity is measured in one of two ways: using the same units that you chose when you set the desired capacity in terms of instances, or using capacity units (if [instance weights](ec2-auto-scaling-mixed-instances-groups-instance-weighting.md) are applied).
+ Example 1: An Auto Scaling group has a maximum capacity of 3, a current capacity of 2, and a dynamic scaling policy that adds 3 instances. When invoking this policy, Amazon EC2 Auto Scaling adds only 1 instance to the group to prevent the group from exceeding its maximum size.
+ Example 2: An Auto Scaling group has a minimum capacity of 2, a current capacity of 3, and a dynamic scaling policy that removes 2 instances. When invoking this policy, Amazon EC2 Auto Scaling removes only 1 instance from the group to prevent the group from becoming less than its minimum size.
When the desired capacity reaches the maximum size limit, scaling out stops. If demand drops and capacity decreases, Amazon EC2 Auto Scaling can scale out again.
The exception is when you use instance weights. In this case, Amazon EC2 Auto Scaling can scale out above the maximum size limit, but only by up to your maximum instance weight. Its intention is to get as close to the new desired capacity as possible but still adhere to the allocation strategies that are specified for the group. The allocation strategies determine which instance types to launch. The weights determine how many capacity units each instance contributes to the desired capacity of the group based on its instance type.
+ Example 3: An Auto Scaling group has a maximum capacity of 12, a current capacity of 10, and a dynamic scaling policy that adds 5 capacity units. Instance types have one of three weights assigned: 1, 4, or 6. When invoking the policy, Amazon EC2 Auto Scaling chooses to launch an instance type with a weight of 6 based on the allocation strategy. The result of this scale-out event is a group with a desired capacity of 12 and a current capacity of 16.
## Multiple dynamic scaling policies
In most cases, a target tracking scaling policy is sufficient to configure your Auto Scaling group to scale out and scale in automatically. A target tracking scaling policy allows you to select a desired outcome and have the Auto Scaling group add and remove instances as needed to achieve that outcome.
For an advanced scaling configuration, your Auto Scaling group can have more than one scaling policy. For example, you can define one or more target tracking scaling policies, one or more step scaling policies, or both. This provides greater flexibility to cover multiple scenarios.
To illustrate how multiple dynamic scaling policies work together, consider an application that uses an Auto Scaling group and an Amazon SQS queue to send requests to a single EC2 instance. To help ensure that the application performs at optimum levels, there are two policies that control when the Auto Scaling group should scale out. One is a target tracking policy that uses a custom metric to add and remove capacity based on the number of SQS messages in the queue. The other is a step scaling policy that uses the Amazon CloudWatch `CPUUtilization` metric to add capacity when the instance exceeds 90 percent utilization for a specified length of time.
When there are multiple policies in force at the same time, there's a chance that each policy could instruct the Auto Scaling group to scale out (or in) at the same time. For example, it's possible that the `CPUUtilization` metric spikes and breaches the threshold of the CloudWatch alarm at the same time that the SQS custom metric spikes and breaches the threshold of the custom metric alarm.
When these situations occur, Amazon EC2 Auto Scaling chooses the policy that provides the largest capacity for both scale out and scale in. Suppose, for example, that the policy for `CPUUtilization` launches one instance, while the policy for the SQS queue launches two instances. If the scale-out criteria for both policies are met at the same time, Amazon EC2 Auto Scaling gives precedence to the SQS queue policy. This results in the Auto Scaling group launching two instances.
The approach of giving precedence to the policy that provides the largest capacity applies even when the policies use different criteria for scaling in. For example, if one policy terminates three instances, another policy decreases the number of instances by 25 percent, and the group has eight instances at the time of scale in, Amazon EC2 Auto Scaling gives precedence to the policy that provides the largest number of instances for the group. This results in the Auto Scaling group terminating two instances (25 percent of 8 = 2). The intention is to prevent Amazon EC2 Auto Scaling from removing too many instances.
We recommend caution, however, when using target tracking scaling policies with step scaling policies because conflicts between these policies can cause undesirable behavior. For example, if the step scaling policy initiates a scale in activity before the target tracking policy is ready to scale in, the scale in activity will not be blocked. After the scale in activity completes, the target tracking policy could instruct the group to scale out again.
# Target tracking scaling policies for Amazon EC2 Auto Scaling
A target tracking scaling policy automatically scales the capacity of your Auto Scaling group based on a target metric value. It automatically adapts to the unique usage patterns of your individual applications. This allows your application to maintain optimal performance and high utilization for your EC2 instances for better cost efficiency without manual intervention.
With target tracking, you select a metric and a target value to represent the ideal average utilization or throughput level for your application. Amazon EC2 Auto Scaling creates and manages the CloudWatch alarms that invoke scaling events when the metric deviates from the target. As an example, this is similar to how a thermostat maintains a target temperature.
For example, let's say that you currently have an application that runs on two instances, and you want the CPU utilization of the Auto Scaling group to stay at around 50 percent when the load on the application changes. This gives you extra capacity to handle traffic spikes without maintaining an excessive number of idle resources.
You can meet this need by creating a target tracking scaling policy that targets an average CPU utilization of 50 percent. Then, your Auto Scaling group will scale out, or increase capacity, when CPU exceeds 50 percent to handle increased load. It will scale in, or decrease capacity, when CPU drops below 50 percent to optimize costs during periods of low utilization.
**Topics**
+ [Multiple target tracking scaling policies](#target-tracking-multiple-policies)
+ [Choose metrics](#target-tracking-choose-metrics)
+ [Define target value](#target-tracking-define-target-value)
+ [Define instance warmup time](#as-target-tracking-scaling-warmup)
+ [Considerations](#target-tracking-considerations)
+ [Create a target tracking scaling policy](policy_creating.md)
+ [Create a target tracking policy using high-resolution metrics for faster response](policy-creating-high-resolution-metrics.md)
+ [Create a target tracking scaling policy using metric math](ec2-auto-scaling-target-tracking-metric-math.md)
## Multiple target tracking scaling policies
To help optimize scaling performance, you can use multiple target tracking scaling policies together, provided that each of them uses a different metric. For example, utilization and throughput can influence each other. Whenever one of these metrics changes, it usually implies that other metrics will also be impacted. The use of multiple metrics therefore provides additional information about the load that your Auto Scaling group is under. This can help Amazon EC2 Auto Scaling make more informed decisions when determining how much capacity to add to your group.
The intention of Amazon EC2 Auto Scaling is to always prioritize availability. It will scale out the Auto Scaling group if any of the target tracking policies are ready to scale out. It will scale in only if all of the target tracking policies (with the scale in portion enabled) are ready to scale in.
## Choose metrics
You can create target tracking scaling policies with either predefined metrics or custom metrics. Predefined metrics provide you easier access to the most commonly used metrics for scaling. Custom metrics allow you to scale on other available CloudWatch metrics including [high-resolution metrics](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/cloudwatch_concepts.html#Resolution_definition) that are published at finer intervals in the order of a few seconds. You can publish your own high-resolution metrics or metrics that other AWS services publish.
For more information about creating target tracking policies using high resolution metrics, see [Create a target tracking policy using high-resolution metrics for faster response](policy-creating-high-resolution-metrics.md).
Target tracking supports the following predefined metrics:
+ `ASGAverageCPUUtilization`—Average CPU utilization of the Auto Scaling group.
+ `ASGAverageNetworkIn`—Average number of bytes received on all network interfaces by the Auto Scaling group.
+ `ASGAverageNetworkOut`—Average number of bytes sent out on all network interfaces by the Auto Scaling group.
+ `ALBRequestCountPerTarget`—Average Application Load Balancer request count per target for your Auto Scaling group.
**Important**
Other valuable information about the metrics for CPU utilization, network I/O, and Application Load Balancer request count per target can be found in the [List the available CloudWatch metrics for your instances](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/viewing_metrics_with_cloudwatch.html) topic in the *Amazon EC2 User Guide* and the [CloudWatch metrics for your Application Load Balancer](https://docs.aws.amazon.com/elasticloadbalancing/latest/application/load-balancer-cloudwatch-metrics.html) topic in the *User Guide for Application Load Balancers*, respectively.
You can choose other available CloudWatch metrics or your own metrics in CloudWatch by specifying a custom metric. For an example that specifies a customized metric specification for a target tracking scaling policy using the AWS CLI, see [Example scaling policies for the AWS CLI](examples-scaling-policies.md).
Keep the following in mind when choosing a metric:
+ We recommend that you only use metrics that are available at one-minute or lower intervals to help you scale faster in response to utilization changes. Metrics that are published at lower intervals allow the target tracking policy to detect and respond faster to changes in the utilization of your Auto Scaling group.
+ If you choose predefined metrics that are published by Amazon EC2, such as CPU utilization, we recommend that you enable detailed monitoring. By default, all Amazon EC2 metrics are published in five-minute intervals, but they are configurable to a lower interval of one minute by enabling detailed monitoring. For information on how to enable detailed monitoring, see [Configure monitoring for Auto Scaling instances](enable-as-instance-metrics.md).
+ Not all custom metrics work for target tracking. The metric must be a valid utilization metric and describe how busy an instance is. The metric value must increase or decrease proportionally to the number of instances in the Auto Scaling group. That's so the metric data can be used to proportionally scale out or in the number of instances. For example, the CPU utilization of an Auto Scaling group works (that is, the Amazon EC2 metric `CPUUtilization` with the metric dimension `AutoScalingGroupName`), if the load on the Auto Scaling group is distributed across the instances.
+ The following metrics do not work for target tracking:
+ The number of requests received by the load balancer fronting the Auto Scaling group (that is, the Elastic Load Balancing metric `RequestCount`). The number of requests received by the load balancer doesn't change based on the utilization of the Auto Scaling group.
+ Load balancer request latency (that is, the Elastic Load Balancing metric `Latency`). Request latency can increase based on increasing utilization, but doesn't necessarily change proportionally.
+ The CloudWatch Amazon SQS queue metric `ApproximateNumberOfMessagesVisible`. The number of messages in a queue might not change proportionally to the size of the Auto Scaling group that processes messages from the queue. However, a custom metric that measures the number of messages in the queue per EC2 instance in the Auto Scaling group can work. For more information, see [Scaling policy based on Amazon SQS](as-using-sqs-queue.md).
+ To use the `ALBRequestCountPerTarget` metric, you must specify the `ResourceLabel` parameter to identify the load balancer target group that is associated with the metric. For an example that specifies the `ResourceLabel` parameter for a target tracking scaling policy using the AWS CLI, see [Example scaling policies for the AWS CLI](examples-scaling-policies.md).
+ When a metric emits real 0 values to CloudWatch (for example, `ALBRequestCountPerTarget`), an Auto Scaling group can scale in to 0 when there is no traffic to your application for a sustained period of time. To have your Auto Scaling group scale in to 0 when no requests are routed it, the group's minimum capacity must be set to 0.
+ Instead of publishing new metrics to use in your scaling policy, you can use metric math to combine existing metrics. For more information, see [Create a target tracking scaling policy using metric math](ec2-auto-scaling-target-tracking-metric-math.md).
## Define target value
When you create a target tracking scaling policy, you must specify a target value. The target value represents the optimal average utilization or throughput for the Auto Scaling group. To use resources cost efficiently, set the target value as high as possible with a reasonable buffer for unexpected traffic increases. When your application is optimally scaled out for a normal traffic flow, the actual metric value should be at or just below the target value.
When a scaling policy is based on throughput, such as the request count per target for an Application Load Balancer, network I/O, or other count metrics, the target value represents the optimal average throughput from a single instance, for a one-minute period.
## Define instance warmup time
You can optionally specify the number of seconds that it takes for a newly launched instance to warm up. Until its specified warmup time has expired, an instance is not counted toward the aggregated EC2 instance metrics of the Auto Scaling group.
While instances are in the warmup period, your scaling policies only scale out if the metric value from instances that are not warming up is greater than the policy's target utilization.
If the group scales out again, the instances that are still warming up are counted as part of the desired capacity for the next scale-out activity. The intention is to continuously (but not excessively) scale out.
While the scale-out activity is in progress, all scale in activities initiated by scaling policies are blocked until the instances finish warming up. When the instances finish warming up, if a scale in event occurs, any instances currently in the process of terminating will be counted towards the current capacity of the group when calculating the new desired capacity. Therefore, we don't remove more instances from the Auto Scaling group than necessary.
**Default value**
If no value is set, then the scaling policy will use the default value, which is the value for the [default instance warmup](ec2-auto-scaling-default-instance-warmup.md) defined for the group. If the default instance warmup is null, then it falls back to the value of the [default cooldown](ec2-auto-scaling-scaling-cooldowns.md#set-default-cooldown). We recommend using the default instance warmup to make it easier to update all scaling policies when the warmup time changes.
## Considerations
The following considerations apply when working with target tracking scaling policies:
+ Do not create, edit, or delete the CloudWatch alarms that are used with a target tracking scaling policy. Amazon EC2 Auto Scaling creates and manages the CloudWatch alarms that are associated with your target tracking scaling policies and can edit, replace, or delete them when necessary to customize the scaling experience for your applications and their changing utilization patterns.
+ A target tracking scaling policy prioritizes availability during periods of fluctuating traffic levels by scaling in more gradually when traffic is decreasing. If you want greater control, a step scaling policy might be the better option. You can temporarily disable the scale-in portion of a target tracking policy. This helps maintain a minimum number of instances for successful deployments.
+ If the metric is missing data points, this causes the CloudWatch alarm state to change to `INSUFFICIENT_DATA`. When this happens, Amazon EC2 Auto Scaling cannot scale your group until new data points are found.
+ If the metric is sparsely reported by design, metric math can be helpful. For example, to use the most recent values, then use the `FILL(m1,REPEAT)` function where `m1` is the metric.
+ You might see gaps between the target value and the actual metric data points. This is because we act conservatively by rounding up or down when determining how many instances to add or remove. This prevents us from adding an insufficient number of instances or removing too many instances. However, for smaller Auto Scaling groups with fewer instances, the utilization of the group might seem far from the target value.
For example, suppose that you set a target value of 50 percent for CPU utilization and your Auto Scaling group then exceeds the target. We might determine that adding 1.5 instances will decrease the CPU utilization to close to 50 percent. Because it is not possible to add 1.5 instances, we round up and add two instances. This might decrease the CPU utilization to a value below 50 percent, but it ensures that your application has enough resources to support it. Similarly, if we determine that removing 0.5 instances increases your CPU utilization to above 50 percent, we will choose not to scale-in until the metric lowers enough that we think scaling in won't cause oscillation.
For larger Auto Scaling groups with more instances, the utilization is spread over a larger number of instances, in which case adding or removing instances causes less of a gap between the target value and the actual metric data points.
+ A target tracking scaling policy assumes that it should scale out your Auto Scaling group when the specified metric is above the target value. You can't use a target tracking scaling policy to scale out your Auto Scaling group when the specified metric is below the target value.
# Create a target tracking scaling policy
To create a target tracking scaling policy for your Auto Scaling group, use one of the following methods.
Before you begin, confirm that your preferred metric is available at 1-minute intervals (compared to the default 5-minute interval of Amazon EC2 metrics).
------
#### [ Console ]
**To create a target tracking scaling policy for a new Auto Scaling group**
1. Open the Amazon EC2 console at [https://console.aws.amazon.com/ec2/](https://console.aws.amazon.com/ec2/), and choose **Auto Scaling Groups** from the navigation pane.
1. Choose **Create Auto Scaling group**.
1. In Steps 1, 2, and 3, choose the options as desired and proceed to **Step 4: Configure group size and scaling policies**.
1. Under **Scaling**, specify the range that you want to scale between by updating the **Min desired capacity** and **Max desired capacity**. These two settings allow your Auto Scaling group to scale dynamically. For more information, see [Set scaling limits for your Auto Scaling group](asg-capacity-limits.md).
1. Under **Automatic scaling**, choose **Target tracking scaling policy**.
1. To define a policy, do the following:
1. Specify a name for the policy.
1. For **Metric type**, choose a metric.
If you chose **Application Load Balancer request count per target**, choose a target group in **Target group**.
1. Specify a **Target value** for the metric.
1. (Optional) For **Instance warmup**, update the instance warmup value as needed.
1. (Optional) Select **Disable scale in to create only a scale-out policy**. This allows you to create a separate scale-in policy of a different type if wanted.
1. Proceed to create the Auto Scaling group. Your scaling policy will be created after the Auto Scaling group has been created.
**To create a target tracking scaling policy for an existing Auto Scaling group**
1. Open the Amazon EC2 console at [https://console.aws.amazon.com/ec2/](https://console.aws.amazon.com/ec2/), and choose **Auto Scaling Groups** from the navigation pane.
1. Select the check box next to your Auto Scaling group.
A split pane opens up in the bottom of the page.
1. Verify that the scaling limits are appropriately set. For example, if your group's desired capacity is already at its maximum, you need to specify a new maximum in order to scale out. For more information, see [Set scaling limits for your Auto Scaling group](asg-capacity-limits.md).
1. On the **Automatic scaling** tab, in **Dynamic scaling policies**, choose **Create dynamic scaling policy**.
1. To define a policy, do the following:
1. For **Policy type**, keep the default of **Target tracking scaling**.
1. Specify a name for the policy.
1. For **Metric type**, choose a metric. You can choose only one metric type. To use more than one metric, create multiple policies.
If you chose **Application Load Balancer request count per target**, choose a target group in **Target group**.
1. Specify a **Target value** for the metric.
1. (Optional) For **Instance warmup**, update the instance warmup value as needed.
1. (Optional) Select **Disable scale in to create only a scale-out policy**. This allows you to create a separate scale-in policy of a different type if wanted.
1. Choose **Create**.
------
#### [ AWS CLI ]
To create a target tracking scaling policy, you can use the following example to help you get started. Replace each *user input placeholder* with your own information.
**Note**
For more examples, see [Example scaling policies for the AWS CLI](examples-scaling-policies.md).
**To create a target tracking scaling policy (AWS CLI)**
1. Use the following `cat` command to store a target value for your scaling policy and a predefined metric specification in a JSON file named `config.json` in your home directory. The following is an example target tracking configuration that keeps the average CPU utilization at 50 percent.
```
$ cat ~/config.json
{
"TargetValue": 50.0,
"PredefinedMetricSpecification":
{
"PredefinedMetricType": "ASGAverageCPUUtilization"
}
}
```
For more information, see [PredefinedMetricSpecification](https://docs.aws.amazon.com/autoscaling/ec2/APIReference/API_PredefinedMetricSpecification.html) in the *Amazon EC2 Auto Scaling API Reference*.
1. Use the [put-scaling-policy](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/autoscaling/put-scaling-policy.html) command, along with the `config.json` file that you created in the previous step, to create your scaling policy.
```
aws autoscaling put-scaling-policy --policy-name cpu50-target-tracking-scaling-policy \
--auto-scaling-group-name my-asg --policy-type TargetTrackingScaling \
--target-tracking-configuration file://config.json
```
If successful, this command returns the ARNs and names of the two CloudWatch alarms created on your behalf.
```
{
"PolicyARN": "arn:aws:autoscaling:us-west-2:123456789012:scalingPolicy:228f02c2-c665-4bfd-aaac-8b04080bea3c:autoScalingGroupName/my-asg:policyName/cpu50-target-tracking-scaling-policy",
"Alarms": [
{
"AlarmARN": "arn:aws:cloudwatch:us-west-2:123456789012:alarm:TargetTracking-my-asg-AlarmHigh-fc0e4183-23ac-497e-9992-691c9980c38e",
"AlarmName": "TargetTracking-my-asg-AlarmHigh-fc0e4183-23ac-497e-9992-691c9980c38e"
},
{
"AlarmARN": "arn:aws:cloudwatch:us-west-2:123456789012:alarm:TargetTracking-my-asg-AlarmLow-61a39305-ed0c-47af-bd9e-471a352ee1a2",
"AlarmName": "TargetTracking-my-asg-AlarmLow-61a39305-ed0c-47af-bd9e-471a352ee1a2"
}
]
}
```
------
# Create a target tracking policy using high-resolution metrics for faster response
Target tracking supports high-resolution CloudWatch metrics with seconds-level data points that are published at lower intervals than one minute. Configure target tracking policies to monitor utilization through high-resolution CloudWatch metrics for applications that have volatile demand patterns, such as client-serving APIs, live streaming services, ecommerce websites, and on-demand data processing. To achieve higher precision in matching capacity with demand, target tracking uses this fine-grained monitoring to detect and respond to changing demand and utilization of your EC2 instances more quickly.
For more information about how to publish your metrics at high resolution, see [Publish custom metrics](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/publishingMetrics.html) in the *Amazon CloudWatch User Guide*. To access and publish EC2 metrics, such as CPU utilization at high resolution, you might want to use [CloudWatch agent](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/Install-CloudWatch-Agent.html).
## AWS Regions
Target tracking using high-resolution metrics is available in all AWS Regions except the AWS GovCloud (US) Regions.
## How target tracking policy with high-resolution metrics works
You create target tracking policies by defining the metric that you want to track and the target value that you want to maintain for the metric. To scale on a high-resolution metric, specify the name of the metric and set the metric period at which the target tracking observes this metric to a value lower than 60 seconds. Currently the lowest supported interval is 10 seconds. You can publish your metric at lower intervals than this.
**Note**
A metric period greater than 60 isn't supported.
You can configure target tracking on a single CloudWatch metric or query multiple CloudWatch metrics and use math expressions to create new single time series based on these metrics. Both options allow you to define the metric period.
## Examples
**Example 1**
The following example creates a target tracking policy based on a high-resolution CloudWatch metric. The metric is published at 10 seconds resolution. By defining the period, you can enable target tracking to monitor this metric at 10-second granularity. Replace each *user input placeholder* with your own information.
```
$ cat ~/config.json
{
"TargetValue": 100.0,
"CustomizedMetricSpecification": {
"MetricName": "MyHighResolutionMetric",
"Namespace": "MyNamespace",
"Dimensions": [
{
"Name": "MyOptionalDimensionName",
"Value": "MyOptionalMetricDimensionValue"
}
],
"Statistic": "Average",
"Unit": "None"
"Period": "10
}
}
```
**Example 2**
You can use metric math expressions to combine multiple metrics into a single time series for scaling. Metric math is particularly useful to convert existing metrics into average per-instance. Converting metrics is essential because target tracking assumes that the metric is inversely proportional to the capacity of the Auto Scaling group. So when capacity increases, the metric should decrease by nearly the same proportion.
For example, suppose you have a metric that represents the pending jobs to be processed by your application. You can use metric math to divide the pending jobs by the running capacity of your Auto Scaling group. Auto Scaling publishes the capacity metric at 1-minute granularity, so there won't be any value for this metric for sub-minute intervals. If you want to use higher resolution for scaling, this can lead to a period mismatch between capacity and pending job metric. To avoid this mismatch, we recommend that you use the FILL expression to fill the missing values with the capacity number recorded in the previous minute timestamp.
The following example uses metric math to divide the pending jobs metric by the capacity. For period, we are setting both metrics at 10 seconds. Because the metric is published at 1-minute intervals, we are using the FILL operation on the capacity metric.
To use metric math to modify multiple metrics
```
{
"CustomizedMetricSpecification": {
"Metrics": [
{
"Label": "Pending jobs to be processed",
"Id": "m1",
"MetricStat": {
"Metric": {
"MetricName": "MyPendingJobsMetric",
"Namespace": "Custom",
},
"Stat": "Sum"
"Period": 10
},
"ReturnData": false
},
{
"Label": "Get the running instance capacity (matching the period to that of the m1)",
"Id": "m2",
"MetricStat": {
"Metric": {
"MetricName": "GroupInServiceInstances",
"Namespace": "AWS/AutoScaling",
"Dimensions": [
{
"Name": "AutoScalingGroupName",
"Value": "my-asg"
}
]
},
"Stat": "Average"
"Period": 10
},
"ReturnData": false
},
{
"Label": "Calculate the pending job per capacity (note the use of the FILL expression)",
"Id": "e1",
"Expression": "m1 / FILL(m2,REPEAT)",
"ReturnData": true
}
]
},
"TargetValue": 100
}
```
## Considerations
Consider the following when using target tracking and high-resolution metrics.
+ To make sure that you don’t have missing data points that could lead to undesired automatic scaling results, your CloudWatch metric must be published at the same or higher resolution than the period that you specify.
+ Define the target value as the per-instance-per-minute metric value that you want to maintain for your Auto Scaling group. Setting an appropriate target value is crucial if you use a metric whose value can multiply based on the period of the metric. For example, any count-based metrics such as request counts or pending jobs that use the SUM statistic will have a different metric value depending on the chosen period. You should still assume that you are setting a target against the per-minute average.
+ Although there are no additional fees for using Amazon EC2 Auto Scaling, you must pay for the resources such as Amazon EC2 instances, CloudWatch metrics, and CloudWatch alarms. The high-resolution alarms created in preceding example are priced differently than standard CloudWatch alarms. For more information about CloudWatch pricing, see [Amazon CloudWatch Pricing](https://aws.amazon.com/cloudwatch/pricing/).
+ Target tracking requires that metrics represent the average per-instance utilization of your EC2 instances. To achieve this, you can use [metric math operations](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/using-metric-math.html) as part of your target tracking policy configuration. Divide your metric by the running capacity of your Auto Scaling group. Make sure that the same metric period is defined for each of the metrics that you use to create a single time series. If these metrics publish at different intervals, use the FILL operation on the metric with the higher interval to fill in the missing data points.
# Create a target tracking scaling policy using metric math
Using metric math, you can query multiple CloudWatch metrics and use math expressions to create new time series based on these metrics. You can visualize the resulting time series in the CloudWatch console and add them to dashboards. For more information about metric math, see [Using metric math](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/using-metric-math.html) in the *Amazon CloudWatch User Guide*.
The following considerations apply to metric math expressions:
+ You can query any available CloudWatch metric. Each metric is a unique combination of metric name, namespace, and zero or more dimensions.
+ You can use any arithmetic operator (\$1 - \$1 / ^), statistical function (such as AVG or SUM), or other function that CloudWatch supports.
+ You can use both metrics and the results of other math expressions in the formulas of the math expression.
+ Any expressions used in a metric specification must eventually return a single time series.
+ You can verify that a metric math expression is valid by using the CloudWatch console or the CloudWatch [GetMetricData](https://docs.aws.amazon.com/AmazonCloudWatch/latest/APIReference/API_GetMetricData.html) API.
## Example: Amazon SQS queue backlog per instance
To calculate the Amazon SQS queue backlog per instance, take the approximate number of messages available for retrieval from the queue and divide that number by the Auto Scaling group's running capacity, which is the number of instances in the `InService` state. For more information, see [Scaling policy based on Amazon SQS](as-using-sqs-queue.md).
The logic for the expression is this:
`sum of (number of messages in the queue)/(number of InService instances)`
Then your CloudWatch metric information is the following.
| ID | CloudWatch metric | Statistic | Period |
| --- | --- | --- | --- |
| m1 | ApproximateNumberOfMessagesVisible | Sum | 1 minute |
| m2 | GroupInServiceInstances | Average | 1 minute |
Your metric math ID and expression are the following.
| ID | Expression |
| --- | --- |
| e1 | (m1)/(m2) |
The following diagram illustrates the architecture for this metric:
![\[Amazon EC2 Auto Scaling using queues architectural diagram\]](http://docs.aws.amazon.com/autoscaling/ec2/userguide/images/sqs-as-custom-metric-diagram.png)
**To use this metric math to create a target tracking scaling policy (AWS CLI)**
1. Store the metric math expression as part of a customized metric specification in a JSON file named `config.json`.
Use the following example to help you get started. Replace each *user input placeholder* with your own information.
```
{
"CustomizedMetricSpecification": {
"Metrics": [
{
"Label": "Get the queue size (the number of messages waiting to be processed)",
"Id": "m1",
"MetricStat": {
"Metric": {
"MetricName": "ApproximateNumberOfMessagesVisible",
"Namespace": "AWS/SQS",
"Dimensions": [
{
"Name": "QueueName",
"Value": "my-queue"
}
]
},
"Stat": "Sum"
},
"ReturnData": false
},
{
"Label": "Get the group size (the number of InService instances)",
"Id": "m2",
"MetricStat": {
"Metric": {
"MetricName": "GroupInServiceInstances",
"Namespace": "AWS/AutoScaling",
"Dimensions": [
{
"Name": "AutoScalingGroupName",
"Value": "my-asg"
}
]
},
"Stat": "Average"
},
"ReturnData": false
},
{
"Label": "Calculate the backlog per instance",
"Id": "e1",
"Expression": "m1 / m2",
"ReturnData": true
}
]
},
"TargetValue": 100
}
```
For more information, see [TargetTrackingConfiguration](https://docs.aws.amazon.com/autoscaling/ec2/APIReference/API_TargetTrackingConfiguration.html) in the *Amazon EC2 Auto Scaling API Reference*.
**Note**
Following are some additional resources that can help you find metric names, namespaces, dimensions, and statistics for CloudWatch metrics:
For information about the available metrics for AWS services, see [AWS services that publish CloudWatch metrics](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/aws-services-cloudwatch-metrics.html) in the *Amazon CloudWatch User Guide*.
To get the exact metric name, namespace, and dimensions (if applicable) for a CloudWatch metric with the AWS CLI, see [list-metrics](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/cloudwatch/list-metrics.html).
1. To create this policy, run the [put-scaling-policy](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/autoscaling/put-scaling-policy.html) command using the JSON file as input, as demonstrated in the following example.
```
aws autoscaling put-scaling-policy --policy-name sqs-backlog-target-tracking-scaling-policy \
--auto-scaling-group-name my-asg --policy-type TargetTrackingScaling \
--target-tracking-configuration file://config.json
```
If successful, this command returns the policy's Amazon Resource Name (ARN) and the ARNs of the two CloudWatch alarms created on your behalf.
```
{
"PolicyARN": "arn:aws:autoscaling:us-west-2:123456789012:scalingPolicy:228f02c2-c665-4bfd-aaac-8b04080bea3c:autoScalingGroupName/my-asg:policyName/sqs-backlog-target-tracking-scaling-policy",
"Alarms": [
{
"AlarmARN": "arn:aws:cloudwatch:us-west-2:123456789012:alarm:TargetTracking-my-asg-AlarmHigh-fc0e4183-23ac-497e-9992-691c9980c38e",
"AlarmName": "TargetTracking-my-asg-AlarmHigh-fc0e4183-23ac-497e-9992-691c9980c38e"
},
{
"AlarmARN": "arn:aws:cloudwatch:us-west-2:123456789012:alarm:TargetTracking-my-asg-AlarmLow-61a39305-ed0c-47af-bd9e-471a352ee1a2",
"AlarmName": "TargetTracking-my-asg-AlarmLow-61a39305-ed0c-47af-bd9e-471a352ee1a2"
}
]
}
```
**Note**
If this command throws an error, make sure that you have updated the AWS CLI locally to the latest version.
# Step and simple scaling policies for Amazon EC2 Auto Scaling
Step scaling and simple scaling policies scale the capacity of your Auto Scaling group in predefined increments based on CloudWatch alarms. You can define separate scaling policies to handle scaling out (increasing capacity) and scaling in (decreasing capacity) when an alarm threshold is breached.
Auto Scaling group capacity is measured in terms of instances or capacity units if you are using [instance weights](ec2-auto-scaling-mixed-instances-groups-instance-weighting.md). Also, there is a difference between desired capacity and current capacity.
+ Desired capacity – The number of instances (or capacity units) that you want to have in your group. Desired capacity can be adjusted manually or automatically using scaling policies.
+ Current capacity – The number of instances (or capacity units) in your group that have passed their warmup and cooldown periods and are running and ready to be used.
With step scaling and simple scaling, you create and manage the CloudWatch alarms that invoke the scaling process. When an alarm is breached, Amazon EC2 Auto Scaling initiates the scaling policy associated with that alarm.
We strongly recommend that you use target tracking scaling policies to scale on metrics like average CPU utilization or average request count per target. Metrics that decrease when capacity increases and increase when capacity decreases can be used to proportionally scale out or in the number of instances using target tracking. This helps ensure that Amazon EC2 Auto Scaling follows the demand curve for your applications closely. For more information, see [Target tracking scaling policies](as-scaling-target-tracking.md).
**Contents**
+ [How step scaling policies work](#step-scaling-how-it-works)
+ [Step adjustments for step scaling](#as-scaling-steps)
+ [Scaling adjustment types](#as-scaling-adjustment)
+ [Instance warmup](#as-step-scaling-warmup)
+ [Considerations](#step-scaling-considerations)
+ [Create a step scaling policy for scale out](step-scaling-create-scale-out-policy.md)
+ [Create a step scaling policy for scale in](step-scaling-create-scale-in-policy.md)
+ [Simple scaling policies](simple-scaling-policies.md)
## How step scaling policies work
To use step scaling, you first create a CloudWatch alarm that monitors a metric for your Auto Scaling group. Define the metric, threshold value, and number of evaluation periods that determine an alarm breach. Then, create a step scaling policy that defines how to scale your group when the alarm threshold is breached. You can use a percentage of the current capacity of your Auto Scaling group or capacity units for the scaling adjustment type. For more information, see [Scaling adjustment types](#as-scaling-adjustment).
Add the step adjustments in the policy. You can define different step adjustments based on the breach size of the alarm. For example:
+ Scale out by 10 instances if the alarm metric reaches 60 percent
+ Scale out by 30 instances if the alarm metric reaches 75 percent
+ Scale out by 40 instances if the alarm metric reaches 85 percent
When the alarm threshold is breached for the specified number of evaluation periods, Amazon EC2 Auto Scaling will apply the step adjustments defined in the policy. The adjustments can continue for additional alarm breaches until the alarm state returns to `OK`.
Each instance has a warmup period to prevent scaling activities from being too reactive to changes that occur over short periods of time. You can optionally configure the warmup period for your scaling policy. However, we recommend using the default instance warmup to make it easier to update all scaling policies when the warmup time changes. For more information, see [Set the default instance warmup for an Auto Scaling group](ec2-auto-scaling-default-instance-warmup.md).
Simple scaling policies are similar to step scaling policies, except they're based on a single scaling adjustment, with a cooldown period between each scaling activity. For more information, see [Simple scaling policies](simple-scaling-policies.md).
## Step adjustments for step scaling
When you create a step scaling policy, you specify one or more step adjustments that automatically scale the number of instances dynamically based on the size of the alarm breach. Each step adjustment specifies the following:
+ A lower bound for the metric value
+ An upper bound for the metric value
+ The amount by which to scale, based on the scaling adjustment type
CloudWatch aggregates metric data points based on the statistic for the metric that's associated with your CloudWatch alarm. When the alarm is breached, the appropriate scaling policy is invoked. Amazon EC2 Auto Scaling applies the aggregation type to the most recent metric data points from CloudWatch (as opposed to the raw metric data). It compares this aggregated metric value against the upper and lower bounds defined by the step adjustments to determine which step adjustment to perform.
You specify the upper and lower bounds relative to the breach threshold. For example, let's say you made a CloudWatch alarm and a scale-out policy for when the metric is above 50 percent. You then made a second alarm and a scale-in policy for when the metric is below 50 percent. You made a set of step adjustments with an adjustment type of `PercentChangeInCapacity` (or **Percent of group** in the console) for each policy:
**Example: Step adjustments for scale-out policy**
| **Lower bound** | **Upper bound** | **Adjustment** |
| --- | --- | --- |
| 0 | 10 | 0 |
| 10 | 20 | 10 |
| 20 | null | 30 |
**Example: Step adjustments for scale-in policy**
| **Lower bound** | **Upper bound** | **Adjustment** |
| --- | --- | --- |
| -10 | 0 | 0 |
| -20 | -10 | -10 |
| null | -20 | -30 |
This creates the following scaling configuration.
```
Metric value
-infinity 30% 40% 60% 70% infinity
-----------------------------------------------------------------------
-30% | -10% | Unchanged | +10% | +30%
-----------------------------------------------------------------------
```
Now, let's say that you use this scaling configuration on an Auto Scaling group that has both a current capacity and a desired capacity of 10. The following points summarize the behavior of the scaling configuration in relation to the desired and current capacity of the group:
+ The desired and current capacity is maintained while the aggregated metric value is greater than 40 and less than 60.
+ If the metric value gets to 60, the desired capacity of the group increases by 1 instance, to 11 instances, based on the second step adjustment of the scale-out policy (add 10 percent of 10 instances). After the new instance is running and its specified warmup time has expired, the current capacity of the group increases to 11 instances. If the metric value rises to 70 even after this increase in capacity, the desired capacity of the group increases by another 3 instances, to 14 instances. This is based on the third step adjustment of the scale-out policy (add 30 percent of 11 instances, 3.3 instances, rounded down to 3 instances).
+ If the metric value gets to 40, the desired capacity of the group decreases by 1 instance, to 13 instances, based on the second step adjustment of the scale-in policy (remove 10 percent of 14 instances, 1.4 instances, rounded down to 1 instance). If the metric value falls to 30 even after this decrease in capacity, the desired capacity of the group decreases by another 3 instances, to 10 instances. This is based on the third step adjustment of the scale-in policy (remove 30 percent of 13 instances, 3.9 instances, rounded down to 3 instances).
When you specify the step adjustments for your scaling policy, note the following:
+ If you use the AWS Management Console, you specify the upper and lower bounds as absolute values. If you use the AWS CLI or an SDK, you specify the upper and lower bounds relative to the breach threshold.
+ The ranges of your step adjustments can't overlap or have a gap.
+ Only one step adjustment can have a null lower bound (negative infinity). If one step adjustment has a negative lower bound, then there must be a step adjustment with a null lower bound.
+ Only one step adjustment can have a null upper bound (positive infinity). If one step adjustment has a positive upper bound, then there must be a step adjustment with a null upper bound.
+ The upper and lower bound can't be null in the same step adjustment.
+ If the metric value is above the breach threshold, the lower bound is inclusive and the upper bound is exclusive. If the metric value is below the breach threshold, the lower bound is exclusive and the upper bound is inclusive.
## Scaling adjustment types
You can define a scaling policy that performs the optimal scaling action, based on the scaling adjustment type that you choose. You can specify the adjustment type as a percentage of the current capacity of your Auto Scaling group, or in capacity units. Normally a capacity unit means one instance, unless you are using the instance weights feature.
Amazon EC2 Auto Scaling supports the following adjustment types for step scaling and simple scaling:
+ `ChangeInCapacity` — Increment or decrement the current capacity of the group by the specified value. A positive value increases the capacity and a negative adjustment value decreases the capacity. For example: If the current capacity of the group is 3 and the adjustment is 5, then when this policy is performed, we add 5 capacity units to the capacity for a total of 8 capacity units.
+ `ExactCapacity` — Change the current capacity of the group to the specified value. Specify a non-negative value with this adjustment type. For example: If the current capacity of the group is 3 and the adjustment is 5, then when this policy is performed, we change the capacity to 5 capacity units.
+ `PercentChangeInCapacity` — Increment or decrement the current capacity of the group by the specified percentage. A positive value increases the capacity and a negative value decreases the capacity. For example: If the current capacity is 10 and the adjustment is 10 percent, then when this policy is performed, we add 1 capacity unit to the capacity for a total of 11 capacity units.
**Note**
If the resulting value is not an integer, it is rounded as follows:
Values greater than 1 are rounded down. For example, `12.7` is rounded to `12`.
Values between 0 and 1 are rounded to 1. For example, `.67` is rounded to `1`.
Values between 0 and -1 are rounded to -1. For example, `-.58` is rounded to `-1`.
Values less than -1 are rounded up. For example, `-6.67` is rounded to `-6`.
With `PercentChangeInCapacity`, you can also specify the minimum number of instances to scale using the `MinAdjustmentMagnitude` parameter. For example, suppose that you create a policy that adds 25 percent and you specify a minimum increment of 2 instances. If you have an Auto Scaling group with 4 instances and the scaling policy is executed, 25 percent of 4 is 1 instance. However, because you specified a minimum increment of 2, there are 2 instances added.
When you use [instance weights](ec2-auto-scaling-mixed-instances-groups-instance-weighting.md), the effect of setting the `MinAdjustmentMagnitude` parameter to a non-zero value changes. The value is in capacity units. To set the minimum number of instances to scale, set this parameter to a value that is at least as large as your largest instance weight.
If you use instance weights, keep in mind that the current capacity of your Auto Scaling group can exceed the desired capacity as needed. If your absolute number to decrement, or the amount that the percentage says to decrement, is less than the difference between current and desired capacity, no scaling action is taken. You must take these behaviors into account when you look at the outcome of a scaling policy when a threshold alarm is in breach. For example, suppose that the desired capacity is 30 and the current capacity is 32. When the alarm is in breach, if the scaling policy decrements the desired capacity by 1, then no scaling action is taken.
## Instance warmup
For step scaling, you can optionally specify the number of seconds that it takes for a newly launched instance to warm up. Until its specified warmup time has expired, an instance is not counted toward the aggregated EC2 instance metrics of the Auto Scaling group.
While instances are in the warmup period, your scaling policies only scale out if the metric value from instances that are not warming up is greater than the policy's alarm high threshold.
If the group scales out again, the instances that are still warming up are counted as part of the desired capacity for the next scale-out activity. Therefore, multiple alarm breaches that fall in the range of the same step adjustment result in a single scaling activity. The intention is to continuously (but not excessively) scale out.
For example, let's say that you create a policy with two steps. The first step adds 10 percent when the metric gets to 60, and the second step adds 30 percent when the metric gets to 70 percent. Your Auto Scaling group has a desired and current capacity of 10. The desired and current capacity do not change while the aggregated metric value is less than 60. Suppose that the metric gets to 60, so 1 instance is added (10 percent of 10 instances). Then, the metric gets to 62 while the new instance is still warming up. The scaling policy calculates the new desired capacity based on the current capacity, which is still 10. However, the desired capacity of the group has already increased to 11 instances, so the scaling policy does not increase the desired capacity further. If the metric gets to 70 while the new instance is still warming up, we should add 3 instances (30 percent of 10 instances). However, the desired capacity of the group is already 11, so we add only 2 instances, for a new desired capacity of 13 instances.
While the scale-out activity is in progress, all scale-in activities initiated by scaling policies are blocked until the instances finish warming up. When the instances finish warming up, if a scale-in event occurs, any instances currently in the process of terminating will be counted towards the current capacity of the group when calculating the new desired capacity. Therefore, we don't remove more instances from the Auto Scaling group than necessary. For example, while an instance is already terminating, if an alarm is in breach in the range of the same step adjustment that decremented the desired capacity by 1, then no scaling action is taken.
**Default value**
If no value is set, then the scaling policy will use the default value, which is the value for the [default instance warmup](ec2-auto-scaling-default-instance-warmup.md) defined for the group. If the default instance warmup is null, then it falls back to the value of the [default cooldown](ec2-auto-scaling-scaling-cooldowns.md#set-default-cooldown).
## Considerations
The following considerations apply when working with step and simple scaling policies:
+ Consider whether you can predict the step adjustments on the application accurately enough to use step scaling. If your scaling metric increases or decreases proportionally to the capacity of the scalable target, we recommend that you use a target tracking scaling policy instead. You still have the option to use step scaling as an additional policy for a more advanced configuration. For example, you can configure a more aggressive response when utilization reaches a certain level.
+ Make sure to choose an adequate margin between the scale-out and scale-in thresholds to prevent flapping. Flapping is an infinite loop of scaling in and scaling out. That is, if a scaling action is taken, the metric value would change and start another scaling action in the reverse direction.
# Create a step scaling policy for scale out
To create a step scaling policy for scale out for your Auto Scaling group, use one of the following methods:
------
#### [ Console ]
**Step 1: Create a CloudWatch alarm for the metric high threshold**
1. Open the CloudWatch console at [https://console.aws.amazon.com/cloudwatch/](https://console.aws.amazon.com/cloudwatch/).
1. If necessary, change the Region. From the navigation bar, select the Region where your Auto Scaling group resides.
1. In the navigation pane, choose **Alarms, All alarms** and then choose **Create alarm**.
1. Choose **Select metric**.
1. On the **All metrics** tab, choose **EC2**, **By Auto Scaling Group**, and enter the Auto Scaling group's name in the search field. Then, select `CPUUtilization` and choose **Select metric**. The **Specify metric and conditions** page appears, showing a graph and other information about the metric.
1. For **Period**, choose the evaluation period for the alarm, for example, 1 minute. When evaluating the alarm, each period is aggregated into one data point.
**Note**
A shorter period creates a more sensitive alarm.
1. Under **Conditions**, do the following:
+ For **Threshold type**, choose **Static**.
+ For **Whenever `CPUUtilization` is**, specify whether you want the value of the metric to be greater than or greater than or equal to the threshold to breach the alarm. Then, under **than**, enter the threshold value that you want to breach the alarm.
1. Under **Additional configuration**, do the following:
+ For **Datapoints to alarm**, enter the number of data points (evaluation periods) during which the metric value must meet the threshold conditions for the alarm. For example, two consecutive periods of 5 minutes would take 10 minutes to invoke the alarm state.
+ For **Missing data treatment**, choose **Treat missing data as bad (breaching threshold)**. For more information, see [Configuring how CloudWatch alarms treat missing data](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/AlarmThatSendsEmail.html#alarms-and-missing-data) in the *Amazon CloudWatch User Guide*.
1. Choose **Next**.
The **Configure actions** page appears.
1. Under **Notification**, select an Amazon SNS topic to notify when the alarm is in `ALARM` state, `OK` state, or `INSUFFICIENT_DATA` state.
To have the alarm send multiple notifications for the same alarm state or for different alarm states, choose **Add notification**.
To have the alarm not send notifications, choose **Remove**.
1. You can leave the other sections of the **Configure actions** page empty. Leaving the other sections empty creates an alarm without associating it to a scaling policy. You can then associate the alarm with a scaling policy from the Amazon EC2 Auto Scaling console.
1. Choose **Next**.
1. Enter a name (for example, `Step-Scaling-AlarmHigh-AddCapacity`) and, optionally, a description for the alarm, and then choose **Next**.
1. Choose **Create alarm**.
Use the following procedure to continue where you left off after creating your CloudWatch alarm.
**Step 2: Create a step scaling policy for scale out**
1. Open the Amazon EC2 console at [https://console.aws.amazon.com/ec2/](https://console.aws.amazon.com/ec2/), and choose **Auto Scaling Groups** from the navigation pane.
1. Select the check box next to your Auto Scaling group.
A split pane opens up in the bottom of the page.
1. Verify that the scaling limits are appropriately set. For example, if your group's desired capacity is already at its maximum, you need to specify a new maximum in order to scale out. For more information, see [Set scaling limits for your Auto Scaling group](asg-capacity-limits.md).
1. On the **Automatic scaling** tab, in **Dynamic scaling policies**, choose **Create dynamic scaling policy**.
1. For **Policy type**, choose **Step scaling**, and then specify a name for the policy.
1. For **CloudWatch alarm**, choose your alarm. If you haven't already created an alarm, choose **Create a CloudWatch alarm** and complete step 4 through step 14 in the previous procedure to create an alarm.
1. Specify the change in the current group size that this policy will make when executed using **Take the action**. You can add a specific number of instances or a percentage of the existing group size, or set the group to an exact size.
For example, to create a scale-out policy that increases the capacity of the group by 30 percent, choose `Add`, enter `30` in the next field, and then choose `percent of group`. By default, the lower bound for this step adjustment is the alarm threshold and the upper bound is positive (\$1) infinity.
1. To add another step, choose **Add step** and then define the amount by which to scale and the lower and upper bounds of the step relative to the alarm threshold.
1. To set a minimum number of instances to scale, update the number field in **Add capacity units in increments of at least** `1` **capacity units**.
1. (Optional) For **Instance warmup**, update the instance warmup value as needed.
1. Choose **Create**.
------
#### [ AWS CLI ]
To create a step scaling policy for scale out (increase capacity), you can use the following example commands. Replace each *user input placeholder* with your own information.
When you use the AWS CLI, you first create a step scaling policy that provides instructions to Amazon EC2 Auto Scaling about how to scale out when a metric's value is increasing. Then, you create the alarm by identifying the metric to watch, defining the metric high threshold and other details for the alarms, and associating the alarm with the scaling policy.
**Step 1: Create a policy for scale out**
Use the following [put-scaling-policy](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/autoscaling/put-scaling-policy.html) command to create a step scaling policy named `my-step-scale-out-policy`, with an adjustment type of `PercentChangeInCapacity` that increases the capacity of the group based on the following step adjustments (assuming a CloudWatch alarm threshold of 60 percent):
+ Increase the instance count by 10 percent when the value of the metric is greater than or equal to 60 percent but less than 75 percent
+ Increase the instance count by 20 percent when the value of the metric is greater than or equal to 75 percent but less than 85 percent
+ Increase the instance count by 30 percent when the value of the metric is greater than or equal to 85 percent
```
aws autoscaling put-scaling-policy \
--auto-scaling-group-name my-asg \
--policy-name my-step-scale-out-policy \
--policy-type StepScaling \
--adjustment-type PercentChangeInCapacity \
--metric-aggregation-type Average \
--step-adjustments MetricIntervalLowerBound=0.0,MetricIntervalUpperBound=15.0,ScalingAdjustment=10 \
MetricIntervalLowerBound=15.0,MetricIntervalUpperBound=25.0,ScalingAdjustment=20 \
MetricIntervalLowerBound=25.0,ScalingAdjustment=30 \
--min-adjustment-magnitude 1
```
Record the policy's Amazon Resource Name (ARN). You need it to create a CloudWatch alarm for the policy.
```
{
"PolicyARN": "arn:aws:autoscaling:region:123456789012:scalingPolicy:4ee9e543-86b5-4121-b53b-aa4c23b5bbcc:autoScalingGroupName/my-asg:policyName/my-step-scale-in-policy
}
```
**Step 2: Create a CloudWatch alarm for the metric high threshold**
Use the following CloudWatch [put-metric-alarm](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/cloudwatch/put-metric-alarm.html) command to create an alarm that increases the size of the Auto Scaling group based on an average CPU threshold value of 60 percent for at least two consecutive evaluation periods of two minutes. To use your own custom metric, specify its name in `--metric-name` and its namespace in `--namespace`.
```
aws cloudwatch put-metric-alarm --alarm-name Step-Scaling-AlarmHigh-AddCapacity \
--metric-name CPUUtilization --namespace AWS/EC2 --statistic Average \
--period 120 --evaluation-periods 2 --threshold 60 \
--comparison-operator GreaterThanOrEqualToThreshold \
--dimensions "Name=AutoScalingGroupName,Value=my-asg" \
--alarm-actions PolicyARN
```
------
# Create a step scaling policy for scale in
To create a step scaling policy for scale in for your Auto Scaling group, use one of the following methods:
------
#### [ Console ]
**Step 1: Create a CloudWatch alarm for the metric low threshold**
1. Open the CloudWatch console at [https://console.aws.amazon.com/cloudwatch/](https://console.aws.amazon.com/cloudwatch/).
1. If necessary, change the Region. From the navigation bar, select the Region where your Auto Scaling group resides.
1. In the navigation pane, choose **Alarms, All alarms** and then choose **Create alarm**.
1. Choose **Select metric**.
1. On the **All metrics** tab, choose **EC2**, **By Auto Scaling Group**, and enter the Auto Scaling group's name in the search field. Then, select `CPUUtilization` and choose **Select metric**. The **Specify metric and conditions** page appears, showing a graph and other information about the metric.
1. For **Period**, choose the evaluation period for the alarm, for example, 1 minute. When evaluating the alarm, each period is aggregated into one data point.
**Note**
A shorter period creates a more sensitive alarm.
1. Under **Conditions**, do the following:
+ For **Threshold type**, choose **Static**.
+ For **Whenever `CPUUtilization` is**, specify whether you want the value of the metric to be less than or less than or equal to the threshold to breach the alarm. Then, under **than**, enter the threshold value that you want to breach the alarm.
**Important**
For an alarm to use with a scale in policy (metric low), make sure you do not choose greater than or greater than or equal to the threshold.
1. Under **Additional configuration**, do the following:
+ For **Datapoints to alarm**, enter the number of data points (evaluation periods) during which the metric value must meet the threshold conditions for the alarm. For example, two consecutive periods of 5 minutes would take 10 minutes to invoke the alarm state.
+ For **Missing data treatment**, choose **Treat missing data as bad (breaching threshold)**. For more information, see [Configuring how CloudWatch alarms treat missing data](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/AlarmThatSendsEmail.html#alarms-and-missing-data) in the *Amazon CloudWatch User Guide*.
1. Choose **Next**.
The **Configure actions** page appears.
1. Under **Notification**, select an Amazon SNS topic to notify when the alarm is in `ALARM` state, `OK` state, or `INSUFFICIENT_DATA` state.
To have the alarm send multiple notifications for the same alarm state or for different alarm states, choose **Add notification**.
To have the alarm not send notifications, choose **Remove**.
1. You can leave the other sections of the **Configure actions** page empty. Leaving the other sections empty creates an alarm without associating it to a scaling policy. You can then associate the alarm with a scaling policy from the Amazon EC2 Auto Scaling console.
1. Choose **Next**.
1. Enter a name (for example, `Step-Scaling-AlarmLow-RemoveCapacity`) and, optionally, a description for the alarm, and then choose **Next**.
1. Choose **Create alarm**.
Use the following procedure to continue where you left off after creating your CloudWatch alarm.
**Step 2: Create a step scaling policy for scale in**
1. Open the Amazon EC2 console at [https://console.aws.amazon.com/ec2/](https://console.aws.amazon.com/ec2/), and choose **Auto Scaling Groups** from the navigation pane.
1. Select the check box next to your Auto Scaling group.
A split pane opens up in the bottom of the page.
1. Verify that the scaling limits are appropriately set. For example, if your group's desired capacity is already at its minimum, you need to specify a new minimum in order to scale in. For more information, see [Set scaling limits for your Auto Scaling group](asg-capacity-limits.md).
1. On the **Automatic scaling** tab, in **Dynamic scaling policies**, choose **Create dynamic scaling policy**.
1. For **Policy type**, choose **Step scaling**, and then specify a name for the policy.
1. For **CloudWatch alarm**, choose your alarm. If you haven't already created an alarm, choose **Create a CloudWatch alarm** and complete step 4 through step 14 in the previous procedure to create an alarm.
1. Specify the change in the current group size that this policy will make when executed using **Take the action**. You can remove a specific number of instances or a percentage of the existing group size, or set the group to an exact size.
For example, to create a scale in policy that decreases the capacity of the group by two instances, choose `Remove`, enter `2` in the next field, and then choose `capacity units`. By default, the upper bound for this step adjustment is the alarm threshold and the lower bound is negative (-) infinity.
1. To add another step, choose **Add step** and then define the amount by which to scale and the lower and upper bounds of the step relative to the alarm threshold.
1. Choose **Create**.
------
#### [ AWS CLI ]
To create a step scaling policy for scale in (decrease capacity), you can use the following example commands. Replace each *user input placeholder* with your own information.
When you use the AWS CLI, you first create a step scaling policy that provides instructions to Amazon EC2 Auto Scaling about how to scale in when a metric's value is decreasing. Then, you create the alarm by identifying the metric to watch, defining the metric low threshold and other details for the alarms, and associating the alarm with the scaling policy.
**Step 1: Create a policy for scale in**
Use the following [put-scaling-policy](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/autoscaling/put-scaling-policy.html) command to create a step scaling policy named `my-step-scale-in-policy`, with an adjustment type of `ChangeInCapacity` that decreases the capacity of the group by 2 instances when the associated CloudWatch alarm breaches the metric low threshold value.
```
aws autoscaling put-scaling-policy \
--auto-scaling-group-name my-asg \
--policy-name my-step-scale-in-policy \
--policy-type StepScaling \
--adjustment-type ChangeInCapacity \
--step-adjustments MetricIntervalUpperBound=0.0,ScalingAdjustment=-2
```
Record the policy's Amazon Resource Name (ARN). You need it to create the CloudWatch alarm for the policy.
```
{
"PolicyARN": "arn:aws:autoscaling:region:123456789012:scalingPolicy:ac542982-cbeb-4294-891c-a5a941dfa787:autoScalingGroupName/my-asg:policyName/my-step-scale-out-policy
}
```
**Step 2: Create a CloudWatch alarm for the metric low threshold**
Use the following CloudWatch [put-metric-alarm](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/cloudwatch/put-metric-alarm.html) command to create an alarm that decreases the size of the Auto Scaling group based on average CPU threshold value of 40 percent for at least two consecutive evaluation periods of two minutes. To use your own custom metric, specify its name in `--metric-name` and its namespace in `--namespace`.
```
aws cloudwatch put-metric-alarm --alarm-name Step-Scaling-AlarmLow-RemoveCapacity \
--metric-name CPUUtilization --namespace AWS/EC2 --statistic Average \
--period 120 --evaluation-periods 2 --threshold 40 \
--comparison-operator LessThanOrEqualToThreshold \
--dimensions "Name=AutoScalingGroupName,Value=my-asg" \
--alarm-actions PolicyARN
```
------
# Simple scaling policies
The following examples show how you can use CLI commands to create simple scaling policies. They remain in this document as a reference for any customers who want to use them, but we recommend that you use target tracking or step scaling policies instead.
Similar to step scaling policies, simple scaling policies require you to create CloudWatch alarms for your scaling policies. In the policies that you create, you must also define whether to add or remove instances, and how many, or set the group to an exact size.
One of the main differences between step scaling policies and simple scaling policies is the step adjustments that you get with step scaling policies. With step scaling, you can make bigger or smaller changes to the size of the group based on the step adjustments that you specify.
A simple scaling policy must also wait for an in-progress scaling activity or health check replacement to complete and a [cooldown period](ec2-auto-scaling-scaling-cooldowns.md) to end before it responds to additional alarms. In contrast, with step scaling, the policy continues to respond to additional alarms, even while a scaling activity or health check replacement is in progress. This means that Amazon EC2 Auto Scaling evaluates all alarm breaches as it receives the alarm messages. Because of this, we recommend that you use step scaling policies instead, even if you have only a single scaling adjustment.
Amazon EC2 Auto Scaling originally supported only simple scaling policies. If you created your scaling policy before target tracking and step scaling policies were introduced, your policy is treated as a simple scaling policy.
## Create a simple scaling policy for scale out
Use the following [put-scaling-policy](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/autoscaling/put-scaling-policy.html) command to create a simple scaling policy named `my-simple-scale-out-policy`, with an adjustment type of `PercentChangeInCapacity` that increases the capacity of the group by 30 percent when the associated CloudWatch alarm breaches the metric high threshold value.
```
aws autoscaling put-scaling-policy --policy-name my-simple-scale-out-policy \
--auto-scaling-group-name my-asg --scaling-adjustment 30 \
--adjustment-type PercentChangeInCapacity
```
Record the policy's Amazon Resource Name (ARN). You need it to create the CloudWatch alarm for the policy.
## Create a simple scaling policy for scale in
Use the following [put-scaling-policy](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/autoscaling/put-scaling-policy.html) command to create a simple scaling policy named `my-simple-scale-in-policy`, with an adjustment type of `ChangeInCapacity` that decreases the capacity of the group by one instance when the associated CloudWatch alarm breaches the metric low threshold value.
```
aws autoscaling put-scaling-policy --policy-name my-simple-scale-in-policy \
--auto-scaling-group-name my-asg --scaling-adjustment -1 \
--adjustment-type ChangeInCapacity --cooldown 180
```
Record the policy's Amazon Resource Name (ARN). You need it to create the CloudWatch alarm for the policy.
# Scaling cooldowns for Amazon EC2 Auto Scaling
**Important**
As a best practice, we recommend that you do not use simple scaling policies and scaling cooldowns. A target tracking scaling policy or a step scaling policy is better for scaling performance. For a scaling policy that changes the size of your Auto Scaling group proportionally as the value of the scaling metric decreases or increases, we recommend [target tracking](as-scaling-target-tracking.md) over either simple scaling or step scaling.
When you create simple scaling policies for your Auto Scaling group, we recommend that you configure the scaling cooldown at the same time.
After your Auto Scaling group launches or terminates instances, it waits for a cooldown period to end before any further scaling activities initiated by simple scaling policies can start. The intention of the cooldown period is to let your Auto Scaling group stabilize and prevent it from launching or terminating additional instances before the effects of the previous scaling activity are visible.
Suppose, for example, that a simple scaling policy for CPU utilization recommends launching two instances. Amazon EC2 Auto Scaling launches two instances and then pauses the scaling activities until the cooldown period ends. After the cooldown period ends, any scaling activities initiated by simple scaling policies can resume. If CPU utilization breaches the alarm high threshold again, the Auto Scaling group scales out again, and the cooldown period takes effect again. However, if two instances were enough to bring the metric value back down, the group remains at its current size.
**Topics**
+ [Considerations](#cooldown-considerations)
+ [Lifecycle hooks can cause additional delays](#cooldowns-lifecycle-hooks)
+ [Change the default cooldown period](#set-default-cooldown)
+ [Set a cooldown period for specific simple scaling policies](#cooldowns-scaling-specific)
## Considerations
The following considerations apply when working with simple scaling policies and scaling cooldowns:
+ Target tracking and step scaling policies can initiate a scale-out activity immediately without waiting for the cooldown period to end. Instead, whenever your Auto Scaling group launches instances, the individual instances have a warmup period. For more information, see [Set the default instance warmup for an Auto Scaling group](ec2-auto-scaling-default-instance-warmup.md).
+ While a scale-out activity is in progress, all target tracking and step scaling scale-in activities are blocked until the instances finish warming up. Scale-in activities can also be delayed when an Auto Scaling group is in a cooldown period.
+ When a scheduled action starts at the scheduled time, it can also initiate a scaling activity immediately without waiting for the cooldown period to end.
+ If an instance becomes unhealthy, Amazon EC2 Auto Scaling does not wait for the cooldown period to end before replacing the unhealthy instance.
+ When multiple instances launch or terminate, the cooldown period (either the default cooldown or the scaling policy-specific cooldown) takes effect starting when the last instance finishes launching or terminating.
+ When you manually scale your Auto Scaling group, the default is not to wait for a cooldown to end. However, you can override this behavior and honor the default cooldown when you use the AWS CLI or an SDK to manually scale.
+ By default, Elastic Load Balancing waits 300 seconds to complete the deregistration (connection draining) process. If the group is behind an Elastic Load Balancing load balancer, it will wait for the terminating instances to deregister before starting the cooldown period.
## Lifecycle hooks can cause additional delays
If a [lifecycle hook](lifecycle-hooks.md) is invoked, the cooldown period begins after you complete the lifecycle action or after the timeout period ends. For example, consider an Auto Scaling group that has a lifecycle hook for instance launch. When the application experiences an increase in demand, the group launches an instance to add capacity. Because there is a lifecycle hook, the instance is put into a wait state and scaling activities due to simple scaling policies are paused. When the instance enters the `InService` state, the cooldown period starts. When the cooldown period ends, simple scaling policy activities are resumed.
When Elastic Load Balancing is enabled, for the purposes of scaling in, the cooldown period starts when the instance that's selected for termination starts connection draining (deregistration delay). The cooldown period doesn't wait for connection draining to finish or the lifecycle hook to complete its action. This means that any scaling activities due to simple scaling policies can resume as soon as the result of the scale in event is reflected in the capacity of the group. Otherwise, waiting to complete all three activities—connection draining, a lifecycle hook, and a cooldown period— would significantly increase the amount of time that the Auto Scaling group needs to pause scaling.
## Change the default cooldown period
You can't set the default cooldown when you initially create an Auto Scaling group in the Amazon EC2 Auto Scaling console. By default, this cooldown period is set to 300 seconds (5 minutes). If needed, you can update this after the group is created.
**To change the default cooldown period (console)**
After creating the Auto Scaling group, on the **Details** tab, choose **Advanced configurations**, **Edit**. For **Default cooldown**, choose the amount of time that you want based on your instance startup time or other application needs.
**To change the default cooldown period (AWS CLI)**
Use the following commands to change the default cooldown for new or existing Auto Scaling groups. If the default cooldown is not defined, the default value of 300 seconds is used.
+ [create-auto-scaling-group](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/autoscaling/create-auto-scaling-group.html)
+ [update-auto-scaling-group](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/autoscaling/update-auto-scaling-group.html)
To confirm the value of the default cooldown, use the [describe-auto-scaling-groups](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/autoscaling/describe-auto-scaling-groups.html) command.
## Set a cooldown period for specific simple scaling policies
By default, all simple scaling policies use the default cooldown period that is defined for the Auto Scaling group. To set a cooldown period for specific simple scaling policies, use the optional cooldown parameter when you create or update the policy. When a cooldown period is specified for a policy, it overrides the default cooldown.
One common use for a scaling policy-specific cooldown period is with a scale in policy. Because this policy terminates instances, Amazon EC2 Auto Scaling needs less time to determine whether to terminate additional instances. Terminating instances should be a much quicker operation than launching instances. The default cooldown period of 300 seconds is therefore too long. In this case, a scaling policy-specific cooldown period with a lower value for your scale in policy can help you reduce costs by allowing the group to scale in faster.
To create or update simple scaling policies in the console, choose the **Automatic scaling** tab after you create the group. To create or update simple scaling policies using the AWS CLI, use the [put-scaling-policy](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/autoscaling/put-scaling-policy.html) command. For more information, see [Step and simple scaling policies](as-scaling-simple-step.md).
# Scaling policy based on Amazon SQS
**Important**
The following information and steps shows you how to calculate the Amazon SQS queue backlog per instance using the `ApproximateNumberOfMessages` queue attribute before publishing it as a custom metric to CloudWatch. However, you can now save the cost and effort put into publishing your own metric by using metric math. For more information, see [Create a target tracking scaling policy using metric math](ec2-auto-scaling-target-tracking-metric-math.md).
You can scale your Auto Scaling group in response to changes in system load in an Amazon Simple Queue Service (Amazon SQS) queue. To learn more about how you can use Amazon SQS, see the [Amazon Simple Queue Service Developer Guide](https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/).
There are some scenarios where you might think about scaling in response to activity in an Amazon SQS queue. For example, suppose that you have a web app that lets users upload images and use them online. In this scenario, each image requires resizing and encoding before it can be published. The app runs on EC2 instances in an Auto Scaling group, and it's configured to handle your typical upload rates. Unhealthy instances are terminated and replaced to maintain current instance levels at all times. The app places the raw bitmap data of the images in an SQS queue for processing. It processes the images and then publishes the processed images where they can be viewed by users. The architecture for this scenario works well if the number of image uploads doesn't vary over time. But if the number of uploads changes over time, you might consider using dynamic scaling to scale the capacity of your Auto Scaling group.
**Topics**
+ [Use target tracking with the right metric](#scale-sqs-queue-custom-metric)
+ [Limitations](#scale-sqs-queue-limitations)
+ [Configure scaling based on Amazon SQS](scale-sqs-queue-cli.md)
+ [Amazon SQS and instance scale-in protection](#scale-sqs-queue-scale-in-protection)
## Use target tracking with the right metric
If you use a target tracking scaling policy based on a custom Amazon SQS queue metric, dynamic scaling can adjust to the demand curve of your application more effectively. For more information about choosing metrics for target tracking, see [Choose metrics](as-scaling-target-tracking.md#target-tracking-choose-metrics).
The issue with using a CloudWatch Amazon SQS metric like `ApproximateNumberOfMessagesVisible` for target tracking is that the number of messages in the queue might not change proportionally to the size of the Auto Scaling group that processes messages from the queue. That's because the number of messages in your SQS queue does not solely define the number of instances needed. The number of instances in your Auto Scaling group can be driven by multiple factors, including how long it takes to process a message and the acceptable amount of latency (queue delay).
The solution is to use a *backlog per instance* metric with the target value being the *acceptable backlog per instance* to maintain. You can calculate these numbers as follows:
+ **Backlog per instance**: To calculate your backlog per instance, start with the `ApproximateNumberOfMessages` queue attribute to determine the length of the SQS queue (number of messages available for retrieval from the queue). Divide that number by the fleet's running capacity, which for an Auto Scaling group is the number of instances in the `InService` state, to get the backlog per instance.
+ **Acceptable backlog per instance**: To calculate your target value, first determine what your application can accept in terms of latency. Then, take the acceptable latency value and divide it by the average time that an EC2 instance takes to process a message.
As an example, let's say that you currently have an Auto Scaling group with 10 instances and the number of visible messages in the queue (`ApproximateNumberOfMessages`) is 1500. If the average processing time is 0.1 seconds for each message and the longest acceptable latency is 10 seconds, then the acceptable backlog per instance is 10 / 0.1, which equals 100 messages. This means that 100 is the target value for your target tracking policy. When the backlog per instance reaches the target value, a scale-out event will happen. Because the backlog per instance is already 150 messages (1500 messages / 10 instances), your group scales out, and it scales out by five instances to maintain proportion to the target value.
The following procedures demonstrate how to publish the custom metric and create the target tracking scaling policy that configures your Auto Scaling group to scale based on these calculations.
**Important**
Remember, to reduce costs, use metric math instead. For more information, see [Create a target tracking scaling policy using metric math](ec2-auto-scaling-target-tracking-metric-math.md).
There are three main parts to this configuration:
+ An Auto Scaling group to manage EC2 instances for the purposes of processing messages from an SQS queue.
+ A custom metric to send to Amazon CloudWatch that measures the number of messages in the queue per EC2 instance in the Auto Scaling group.
+ A target tracking policy that configures your Auto Scaling group to scale based on the custom metric and a set target value. CloudWatch alarms invoke the scaling policy.
The following diagram illustrates the architecture of this configuration.
![\[Amazon EC2 Auto Scaling using queues architectural diagram\]](http://docs.aws.amazon.com/autoscaling/ec2/userguide/images/sqs-as-custom-metric-diagram.png)
## Limitations
You must use the AWS CLI or an SDK to publish your custom metric to CloudWatch. You can then monitor your metric with the AWS Management Console.
In the following sections you use the AWS CLI for the tasks you need to perform. For example, to get metric data that reflects the present use of the queue, you use the SQS [get-queue-attributes](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/sqs/get-queue-attributes.html) command.
# Configure scaling based on Amazon SQS
The following procedure describes how to configure automatic scaling based on Amazon SQS. You'll learn how to create a CloudWatch custom metric, how to set up a target tracking policy using the AWS CLI, and how to test your configuration.
Before you begin, make sure that you have the AWS CLI [installed](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html) and [configured](https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-configure.html). Also, you must have an Amazon SQS queue to use. The following tasks assume that you already have a queue (standard or FIFO), an Auto Scaling group, and EC2 instances running the application that uses the queue.
For more information about Amazon SQS, see the [Amazon Simple Queue Service Developer Guide](https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/).
**Topics**
+ [Step 1: Create a CloudWatch custom metric](#create-sqs-cw-alarms-cli)
+ [Step 2: Create a target tracking scaling policy](#create-sqs-policies-cli)
+ [Step 3: Test your scaling policy](#validate-sqs-scaling-cli)
## Step 1: Create a CloudWatch custom metric
A custom metric is defined using a metric name and namespace of your choosing. Namespaces for custom metrics cannot start with `AWS/`. For more information about publishing custom metrics, see the [Publish custom metrics](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/publishingMetrics.html) topic in the *Amazon CloudWatch User Guide*.
Follow this procedure to create the custom metric by first reading information from your AWS account. Then, calculate the backlog per instance metric, as recommended in an earlier section. Lastly, publish this number to CloudWatch at a 1-minute granularity. Whenever possible, we strongly recommend that you scale on metrics with a 1-minute granularity to ensure a faster response to changes in system load.
**To create a CloudWatch custom metric (AWS CLI)**
1. Use the SQS [get-queue-attributes](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/sqs/get-queue-attributes.html) command to get the number of messages waiting in the queue (`ApproximateNumberOfMessages`).
```
aws sqs get-queue-attributes --queue-url https://sqs.region.amazonaws.com/123456789/MyQueue \
--attribute-names ApproximateNumberOfMessages
```
1. Use the [describe-auto-scaling-groups](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/autoscaling/describe-auto-scaling-groups.html) command to get the running capacity of the group, which is the number of instances in the `InService` lifecycle state. This command returns the instances of an Auto Scaling group along with their lifecycle state.
```
aws autoscaling describe-auto-scaling-groups --auto-scaling-group-names my-asg
```
1. Calculate the backlog per instance by dividing the approximate number of messages available for retrieval from the queue by the group's running capacity.
1. Create a script that runs every minute to retrieve the backlog per instance value and publish it to a CloudWatch custom metric. When you publish a custom metric, you specify the metric's name, namespace, unit, value, and zero or more dimensions. A dimension consists of a dimension name and a dimension value.
To publish your custom metric, replace placeholder values in *italics* with your preferred metric name, the metric's value, a namespace (as long as it doesn’t begin with "`AWS`"), and dimensions (optional), and then run the following [put-metric-data](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/cloudwatch/put-metric-data.html) command.
```
aws cloudwatch put-metric-data --metric-name MyBacklogPerInstance --namespace MyNamespace \
--unit None --value 20 --dimensions MyOptionalMetricDimensionName=MyOptionalMetricDimensionValue
```
After your application is emitting the desired metric, the data is sent to CloudWatch. The metric is visible in the CloudWatch console. You can access it by logging into the AWS Management Console and navigating to the CloudWatch page. Then, view the metric by navigating to the metrics page or by searching for it using the search box. For information about viewing metrics, see [View available metrics](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/viewing_metrics_with_cloudwatch.html) in the *Amazon CloudWatch User Guide*.
## Step 2: Create a target tracking scaling policy
The metric you created can now be added to a target tracking scaling policy.
**To create a target tracking scaling policy (AWS CLI)**
1. Use the following `cat` command to store a target value for your scaling policy and a customized metric specification in a JSON file named `config.json` in your home directory. Replace each *user input placeholder* with your own information. For the `TargetValue`, calculate the acceptable backlog per instance metric and enter it here. To calculate this number, decide on a normal latency value and divide it by the average time that it takes to process a message, as described in an earlier section.
If you didn't specify any dimensions for the metric you created in step 1, don't include any dimensions in the customized metric specification.
```
$ cat ~/config.json
{
"TargetValue":100,
"CustomizedMetricSpecification":{
"MetricName":"MyBacklogPerInstance",
"Namespace":"MyNamespace",
"Dimensions":[
{
"Name":"MyOptionalMetricDimensionName",
"Value":"MyOptionalMetricDimensionValue"
}
],
"Statistic":"Average",
"Unit":"None"
}
}
```
1. Use the [put-scaling-policy](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/autoscaling/put-scaling-policy.html) command, along with the `config.json` file that you created in the previous step, to create your scaling policy.
```
aws autoscaling put-scaling-policy --policy-name sqs100-target-tracking-scaling-policy \
--auto-scaling-group-name my-asg --policy-type TargetTrackingScaling \
--target-tracking-configuration file://~/config.json
```
This creates two alarms: one for scaling out and one for scaling in. It also returns the Amazon Resource Name (ARN) of the policy that is registered with CloudWatch, which CloudWatch uses to invoke scaling whenever the metric threshold is in breach.
## Step 3: Test your scaling policy
After your setup is complete, verify that your scaling policy is working. You can test it by increasing the number of messages in your SQS queue and then verifying that your Auto Scaling group has launched an additional EC2 instance. You can also test it by decreasing the number of messages in your SQS queue and then verifying that the Auto Scaling group has terminated an EC2 instance.
**To test the scale-out function**
1. Follow the steps in [Creating an Amazon SQS standard queue and sending a message](https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/creating-sqs-standard-queues.html) or [Creating an Amazon SQS FIFO queue and sending a message](https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/creating-sqs-fifo-queues.html) to add messages to your queue. Make sure that you have increased the number of messages in the queue so that the backlog per instance metric exceeds the target value.
It can take a few minutes for your changes to invoke the alarm.
1. Use the [describe-auto-scaling-groups](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/autoscaling/describe-auto-scaling-groups.html) command to verify that the group has launched an instance.
```
aws autoscaling describe-auto-scaling-groups --auto-scaling-group-name my-asg
```
**To test the scale in function**
1. Follow the steps in [Receive and delete a message (console)](https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/step-receive-delete-message.html) to delete messages from the queue. Make sure that you have decreased the number of messages in the queue so that the backlog per instance metric is below the target value.
It can take a few minutes for your changes to invoke the alarm.
1. Use the [describe-auto-scaling-groups](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/autoscaling/describe-auto-scaling-groups.html) command to verify that the group has terminated an instance.
```
aws autoscaling describe-auto-scaling-groups --auto-scaling-group-name my-asg
```
## Amazon SQS and instance scale-in protection
Messages that have not been processed at the time an instance is terminated are returned to the SQS queue where they can be processed by another instance that is still running. For applications where long running tasks are performed, you can optionally use instance scale-in protection to have control over which queue workers are terminated when your Auto Scaling group scales in.
The following pseudocode shows one way to protect long-running, queue-driven worker processes from scale-in termination.
```
while (true)
{
SetInstanceProtection(False);
Work = GetNextWorkUnit();
SetInstanceProtection(True);
ProcessWorkUnit(Work);
SetInstanceProtection(False);
}
```
For more information, see [Design your applications to gracefully handle instance termination](gracefully-handle-instance-termination.md).
# Verify a scaling activity for an Auto Scaling group
In the Amazon EC2 Auto Scaling section of the Amazon EC2 console, the **Activity history** for an Auto Scaling group lets you view the current status of a scaling activity that is currently in progress. When the scaling activity is finished, you can see whether it succeeds or not. This is particularly useful when you are creating Auto Scaling groups or you are adding scaling conditions to existing groups.
When you add a target tracking, step, or simple scaling policy to your Auto Scaling group, Amazon EC2 Auto Scaling immediately starts evaluating the policy against the metric. The metric alarm goes to ALARM state when the metric breaches the threshold for a specified number of evaluation periods. This means that a scaling policy could result in a scaling activity soon after it's created. After Amazon EC2 Auto Scaling adjusts the desired capacity in response to a scaling policy, you can verify the scaling activity in your account. If you want to receive email notification from Amazon EC2 Auto Scaling informing you about a scaling activity, follow the instructions in [Amazon SNS notification options for Amazon EC2 Auto Scaling](ec2-auto-scaling-sns-notifications.md).
**Tip**
In the following procedure, you look at the **Activity history** and **Instances** sections for the Auto Scaling group. In both, the named columns should already be displayed. To display hidden columns or change the number of rows shown, choose the gear icon on the top right corner of each section to open the preferences modal, update the settings as needed, and choose **Confirm**.
**To view the scaling activities for an Auto Scaling group (console)**
1. Open the Amazon EC2 console at [https://console.aws.amazon.com/ec2/](https://console.aws.amazon.com/ec2/), and choose **Auto Scaling Groups** from the navigation pane.
1. In the navigation bar at the top of the screen, select the Region your Auto Scaling group is in.
1. Select the check box next to the Auto Scaling group.
A split pane opens up in the bottom of the page.
1. On the **Activity** tab, under **Activity history**, the **Status** column shows whether your Auto Scaling group has successfully launched or terminated instances, or whether the scaling activity is still in progress.
1. (Optional) If you have a lot of scaling activities, you can choose the **>** icon at the top edge of the activity history to see the next page of scaling activities.
1. On the **Instance management** tab, under **Instances**, the **Lifecycle** column contains the state of your instances. After the instance starts and any lifecycle hooks have finished, its lifecycle state changes to `InService`. The **Health status** column shows the result of the EC2 instance health check on your instance.
**To view the scaling activities for an Auto Scaling group (AWS CLI)**
Use the following [describe-scaling-activities](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/autoscaling/describe-scaling-activities.html) command.
```
aws autoscaling describe-scaling-activities --auto-scaling-group-name my-asg
```
The following is example output.
Scaling activities are ordered by start time. Activities still in progress are described first.
```
{
"Activities": [
{
"ActivityId": "5e3a1f47-2309-415c-bfd8-35aa06300799",
"AutoScalingGroupName": "my-asg",
"Description": "Terminating EC2 instance: i-06c4794c2499af1df",
"Cause": "At 2020-02-11T18:34:10Z a monitor alarm TargetTracking-my-asg-AlarmLow-b9376cab-18a7-4385-920c-dfa3f7783f82 in state ALARM triggered policy my-target-tracking-policy changing the desired capacity from 3 to 2. At 2020-02-11T18:34:31Z an instance was taken out of service in response to a difference between desired and actual capacity, shrinking the capacity from 3 to 2. At 2020-02-11T18:34:31Z instance i-06c4794c2499af1df was selected for termination.",
"StartTime": "2020-02-11T18:34:31.268Z",
"EndTime": "2020-02-11T18:34:53Z",
"StatusCode": "Successful",
"Progress": 100,
"Details": "{\"Subnet ID\":\"subnet-5ea0c127\",\"Availability Zone\":\"us-west-2a\"...}",
"AutoScalingGroupARN": "arn"
},
...
]
}
```
For a description of the fields in the output, see [Activity](https://docs.aws.amazon.com/autoscaling/ec2/APIReference/API_Activity.html) in the *Amazon EC2 Auto Scaling API Reference*.
For help retrieving the scaling activities for a deleted group and for information about the types of errors that you may encounter and how to handle them, see [Troubleshoot issues in Amazon EC2 Auto Scaling](CHAP_Troubleshooting.md).
# Disable a scaling policy for an Auto Scaling group
This topic describes how to temporarily disable a scaling policy so it won't initiate changes to the number of instances the Auto Scaling group contains. When you disable a scaling policy, the configuration details are preserved, so you can quickly re-enable the policy. This is easier than temporarily deleting a policy when you don't need it, and recreating it later.
When a scaling policy is disabled, the Auto Scaling group does not scale out or scale in for the metric alarms that are breached while the scaling policy is disabled. However, any scaling activities still in progress are not stopped.
Note that disabled scaling policies still count toward your quotas on the number of scaling policies that you can add to an Auto Scaling group.
**To disable a scaling policy (console)**
1. Open the Amazon EC2 console at [https://console.aws.amazon.com/ec2/](https://console.aws.amazon.com/ec2/), and choose **Auto Scaling Groups** from the navigation pane.
1. Select the check box next to the Auto Scaling group.
A split pane opens up in the bottom of the page.
1. On the **Automatic scaling** tab, under **Dynamic scaling policies**, select the check box in the top right corner of the desired scaling policy.
1. Scroll to the top of the **Dynamic scaling policies** section, and choose **Actions**, **Disable**.
When you are ready to re-enable the scaling policy, repeat these steps and then choose **Actions**, **Enable**. After you re-enable a scaling policy, your Auto Scaling group may immediately initiate a scaling action if there are any alarms currently in ALARM state.
**To disable a scaling policy (AWS CLI)**
Use the [put-scaling-policy](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/autoscaling/put-scaling-policy.html) command with the `--no-enabled` option as follows. Specify all options in the command as you would specify them when creating the policy.
```
aws autoscaling put-scaling-policy --auto-scaling-group-name my-asg \
--policy-name my-scaling-policy --policy-type TargetTrackingScaling \
--estimated-instance-warmup 360 \
--target-tracking-configuration '{ "TargetValue": 70, "PredefinedMetricSpecification": { "PredefinedMetricType": "ASGAverageCPUUtilization" } }' \
--no-enabled
```
**To re-enable a scaling policy (AWS CLI)**
Use the [put-scaling-policy](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/autoscaling/put-scaling-policy.html) command with the `--enabled` option as follows. Specify all options in the command as you would specify them when creating the policy.
```
aws autoscaling put-scaling-policy --auto-scaling-group-name my-asg \
--policy-name my-scaling-policy --policy-type TargetTrackingScaling \
--estimated-instance-warmup 360 \
--target-tracking-configuration '{ "TargetValue": 70, "PredefinedMetricSpecification": { "PredefinedMetricType": "ASGAverageCPUUtilization" } }' \
--enabled
```
**To describe a scaling policy (AWS CLI)**
Use the [describe-policies](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/autoscaling/describe-policies.html) command to verify the enabled status of a scaling policy.
```
aws autoscaling describe-policies --auto-scaling-group-name my-asg \
--policy-names my-scaling-policy
```
The following is example output.
```
{
"ScalingPolicies": [
{
"AutoScalingGroupName": "my-asg",
"PolicyName": "my-scaling-policy",
"PolicyARN": "arn:aws:autoscaling:us-west-2:123456789012:scalingPolicy:1d52783a-b03b-4710-bb0e-549fd64378cc:autoScalingGroupName/my-asg:policyName/my-scaling-policy",
"PolicyType": "TargetTrackingScaling",
"StepAdjustments": [],
"Alarms": [
{
"AlarmName": "TargetTracking-my-asg-AlarmHigh-9ca53fdd-7cf5-4223-938a-ae1199204502",
"AlarmARN": "arn:aws:cloudwatch:us-west-2:123456789012:alarm:TargetTracking-my-asg-AlarmHigh-9ca53fdd-7cf5-4223-938a-ae1199204502"
},
{
"AlarmName": "TargetTracking-my-asg-AlarmLow-7010c83d-d55a-4a7a-abe0-1cf8b9de6d6c",
"AlarmARN": "arn:aws:cloudwatch:us-west-2:123456789012:alarm:TargetTracking-my-asg-AlarmLow-7010c83d-d55a-4a7a-abe0-1cf8b9de6d6c"
}
],
"TargetTrackingConfiguration": {
"PredefinedMetricSpecification": {
"PredefinedMetricType": "ASGAverageCPUUtilization"
},
"TargetValue": 70.0,
"DisableScaleIn": false
},
"Enabled": true
}
]
}
```
# Delete a scaling policy for an Auto Scaling group
After you no longer need a scaling policy, you can delete it. Depending on the type of scaling policy, you might also need to delete the CloudWatch alarms. Deleting a target tracking scaling policy also deletes any associated CloudWatch alarms. Deleting a step scaling policy or a simple scaling policy deletes the underlying alarm action, but it does not delete the CloudWatch alarm, even if it no longer has an associated action.
**To delete a scaling policy (console)**
1. Open the Amazon EC2 console at [https://console.aws.amazon.com/ec2/](https://console.aws.amazon.com/ec2/), and choose **Auto Scaling Groups** from the navigation pane.
1. Select the check box next to the Auto Scaling group.
A split pane opens up in the bottom of the page.
1. On the **Automatic scaling** tab, under **Dynamic scaling policies**, select the check box in the top right corner of the desired scaling policy.
1. Scroll to the top of the **Dynamic scaling policies** section, and choose **Actions**, **Delete**.
1. When prompted for confirmation, choose **Yes, Delete**.
1. (Optional) If you deleted a step scaling policy or a simple scaling policy, do the following to delete the CloudWatch alarm that was associated with the policy. You can skip these substeps to keep the alarm for future use.
1. Open the CloudWatch console at [https://console.aws.amazon.com/cloudwatch/](https://console.aws.amazon.com/cloudwatch/).
1. On the navigation pane, choose **Alarms**.
1. Choose the alarm (for example, `Step-Scaling-AlarmHigh-AddCapacity`) and choose **Action**, **Delete**.
1. When prompted for confirmation, choose **Delete**.
**To get the scaling policies for an Auto Scaling group (AWS CLI)**
Before you delete a scaling policy, use the following [describe-policies](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/autoscaling/describe-policies.html) command to see what scaling policies were created for the Auto Scaling group. You can use the output when deleting the policy and the CloudWatch alarms.
```
aws autoscaling describe-policies --auto-scaling-group-name my-asg
```
You can filter the results by the type of scaling policy using the `--query` parameter. This syntax for `query` works on Linux or macOS. On Windows, change the single quotes to double quotes.
```
aws autoscaling describe-policies --auto-scaling-group-name my-asg
--query 'ScalingPolicies[?PolicyType==`TargetTrackingScaling`]'
```
The following is example output.
```
[
{
"AutoScalingGroupName": "my-asg",
"PolicyName": "cpu50-target-tracking-scaling-policy",
"PolicyARN": "PolicyARN",
"PolicyType": "TargetTrackingScaling",
"StepAdjustments": [],
"Alarms": [
{
"AlarmARN": "arn:aws:cloudwatch:us-west-2:123456789012:alarm:TargetTracking-my-asg-AlarmHigh-fc0e4183-23ac-497e-9992-691c9980c38e",
"AlarmName": "TargetTracking-my-asg-AlarmHigh-fc0e4183-23ac-497e-9992-691c9980c38e"
},
{
"AlarmARN": "arn:aws:cloudwatch:us-west-2:123456789012:alarm:TargetTracking-my-asg-AlarmLow-61a39305-ed0c-47af-bd9e-471a352ee1a2",
"AlarmName": "TargetTracking-my-asg-AlarmLow-61a39305-ed0c-47af-bd9e-471a352ee1a2"
}
],
"TargetTrackingConfiguration": {
"PredefinedMetricSpecification": {
"PredefinedMetricType": "ASGAverageCPUUtilization"
},
"TargetValue": 50.0,
"DisableScaleIn": false
},
"Enabled": true
}
]
```
**To delete your scaling policy (AWS CLI)**
Use the following [delete-policy](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/autoscaling/delete-policy.html) command.
```
aws autoscaling delete-policy --auto-scaling-group-name my-asg \
--policy-name cpu50-target-tracking-scaling-policy
```
**To delete your CloudWatch alarm (AWS CLI)**
For step and simple scaling policies, use the [delete-alarms](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/cloudwatch/delete-alarms.html) command to delete the CloudWatch alarm that was associated with the policy. You can skip this step to keep the alarm for future use. You can delete one or more alarms at a time. For example, use the following command to delete the `Step-Scaling-AlarmHigh-AddCapacity` and `Step-Scaling-AlarmLow-RemoveCapacity` alarms.
```
aws cloudwatch delete-alarms --alarm-name Step-Scaling-AlarmHigh-AddCapacity Step-Scaling-AlarmLow-RemoveCapacity
```
# Example scaling policies for the AWS CLI
You can create scaling policies for Amazon EC2 Auto Scaling through the AWS Management Console, AWS Command Line Interface (AWS CLI), or SDKs.
The following examples show how you can create scaling policies for Amazon EC2 Auto Scaling with the AWS CLI [put-scaling-policy](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/autoscaling/put-scaling-policy.html) command. Replace each *user input placeholder* with your own information.
To get started with writing scaling policies using the AWS CLI, see the introductory exercises in [Target tracking scaling policies](as-scaling-target-tracking.md) and [Step and simple scaling policies](as-scaling-simple-step.md).
**Example 1: To apply a target tracking scaling policy with a predefined metric specification**
```
aws autoscaling put-scaling-policy --policy-name cpu50-target-tracking-scaling-policy \
--auto-scaling-group-name my-asg --policy-type TargetTrackingScaling \
--target-tracking-configuration file://config.json
{
"TargetValue": 50.0,
"PredefinedMetricSpecification": {
"PredefinedMetricType": "ASGAverageCPUUtilization"
}
}
```
For more information, see [PredefinedMetricSpecification](https://docs.aws.amazon.com/autoscaling/ec2/APIReference/API_PredefinedMetricSpecification.html) in the *Amazon EC2 Auto Scaling API Reference*.
**Note**
If the file is not in the current directory, type the full path to file. For more information about reading AWS CLI parameter values from a file, see [Loading AWS CLI parameters from a file](https://docs.aws.amazon.com/cli/latest/userguide/cli-usage-parameters-file.html) in the AWS Command Line Interface User Guide.
**Example 2: To apply a target tracking scaling policy with a customized metric specification**
```
aws autoscaling put-scaling-policy --policy-name sqs100-target-tracking-scaling-policy \
--auto-scaling-group-name my-asg --policy-type TargetTrackingScaling \
--target-tracking-configuration file://config.json
{
"TargetValue": 100.0,
"CustomizedMetricSpecification": {
"MetricName": "MyBacklogPerInstance",
"Namespace": "MyNamespace",
"Dimensions": [{
"Name": "MyOptionalMetricDimensionName",
"Value": "MyOptionalMetricDimensionValue"
}],
"Statistic": "Average",
"Unit": "None"
}
}
```
For more information, see [CustomizedMetricSpecification](https://docs.aws.amazon.com/autoscaling/ec2/APIReference/API_CustomizedMetricSpecification.html) in the *Amazon EC2 Auto Scaling API Reference*.
**Example 3: To apply a target tracking scaling policy for scale out only**
```
aws autoscaling put-scaling-policy --policy-name alb1000-target-tracking-scaling-policy \
--auto-scaling-group-name my-asg --policy-type TargetTrackingScaling \
--target-tracking-configuration file://config.json
{
"TargetValue": 1000.0,
"PredefinedMetricSpecification": {
"PredefinedMetricType": "ALBRequestCountPerTarget",
"ResourceLabel": "app/my-alb/778d41231b141a0f/targetgroup/my-alb-target-group/943f017f100becff"
},
"DisableScaleIn": true
}
```
**Example 4: To apply a step scaling policy for scale out**
```
aws autoscaling put-scaling-policy \
--auto-scaling-group-name my-asg \
--policy-name my-step-scale-out-policy \
--policy-type StepScaling \
--adjustment-type PercentChangeInCapacity \
--metric-aggregation-type Average \
--step-adjustments MetricIntervalLowerBound=10.0,MetricIntervalUpperBound=20.0,ScalingAdjustment=10 \
MetricIntervalLowerBound=20.0,MetricIntervalUpperBound=30.0,ScalingAdjustment=20 \
MetricIntervalLowerBound=30.0,ScalingAdjustment=30 \
--min-adjustment-magnitude 1
```
Record the policy's Amazon Resource Name (ARN). You need the ARN when you create the CloudWatch alarm.
**Example 5: To apply a step scaling policy for scale in**
```
aws autoscaling put-scaling-policy \
--auto-scaling-group-name my-asg \
--policy-name my-step-scale-in-policy \
--policy-type StepScaling \
--adjustment-type ChangeInCapacity \
--step-adjustments MetricIntervalUpperBound=0.0,ScalingAdjustment=-2
```
Record the policy's Amazon Resource Name (ARN). You need the ARN when you create the CloudWatch alarm.
**Example 6: To apply a simple scaling policy for scale out**
```
aws autoscaling put-scaling-policy --policy-name my-simple-scale-out-policy \
--auto-scaling-group-name my-asg --scaling-adjustment 30 \
--adjustment-type PercentChangeInCapacity --min-adjustment-magnitude 2
```
Record the policy's Amazon Resource Name (ARN). You need the ARN when you create the CloudWatch alarm.
**Example 7: To apply a simple scaling policy for scale in**
```
aws autoscaling put-scaling-policy --policy-name my-simple-scale-in-policy \
--auto-scaling-group-name my-asg --scaling-adjustment -1 \
--adjustment-type ChangeInCapacity --cooldown 180
```
Record the policy's Amazon Resource Name (ARN). You need the ARN when you create the CloudWatch alarm.
# Predictive scaling for Amazon EC2 Auto Scaling
Predictive scaling works by analyzing historical load data to detect daily or weekly patterns in traffic flows. It uses this information to forecast future capacity needs so Amazon EC2 Auto Scaling can proactively increase the capacity of your Auto Scaling group to match the anticipated load.
Predictive scaling is well suited for situations where you have:
+ Cyclical traffic, such as high use of resources during regular business hours and low use of resources during evenings and weekends
+ Recurring on-and-off workload patterns, such as batch processing, testing, or periodic data analysis
+ Applications that take a long time to initialize, causing a noticeable latency impact on application performance during scale-out events
In general, if you have regular patterns of traffic increases and applications that take a long time to initialize, you should consider using predictive scaling. Predictive scaling can help you scale faster by launching capacity in advance of forecasted load, compared to using only dynamic scaling, which is reactive in nature. Predictive scaling can also potentially save you money on your EC2 bill by helping you avoid the need to over provision capacity.
For example, consider an application that has high usage during business hours and low usage overnight. At the start of each business day, predictive scaling can add capacity before the first influx of traffic. This helps your application maintain high availability and performance when going from a period of lower utilization to a period of higher utilization. You don't have to wait for dynamic scaling to react to changing traffic. You also don't have to spend time reviewing your application's load patterns and trying to schedule the right amount of capacity using scheduled scaling.
**Topics**
+ [How predictive scaling works](predictive-scaling-policy-overview.md)
+ [Create a predictive scaling policy](predictive-scaling-create-policy.md)
+ [Evaluate your predictive scaling policies](predictive-scaling-graphs.md)
+ [Override the forecast](predictive-scaling-overriding-forecast-capacity.md)
+ [Use custom metrics](predictive-scaling-customized-metric-specification.md)
# How predictive scaling works
This topic explains how predictive scaling works and describes what to consider when you create a predictive scaling policy.
**Topics**
+ [How it works](#how-predictive-scaling-works)
+ [Maximum capacity limit](#predictive-scaling-maximum-capacity-limit)
+ [Considerations](#predictive-scaling-considerations)
+ [Supported Regions](#predictive-scaling-regions)
## How it works
To use predictive scaling, create a predictive scaling policy that specifies the CloudWatch metric to monitor and analyze. For predictive scaling to start forecasting future values, this metric must have at least 24 hours of data.
After you create the policy, predictive scaling starts analyzing metric data from up to the past 14 days to identify patterns. It uses this analysis to generate an hourly forecast of capacity requirements for the next 48 hours. The forecast is updated every 6 hours using the latest CloudWatch data. As new data comes in, predictive scaling is able to continuously improve the accuracy of future forecasts.
When you first enable predictive scaling, it runs in *forecast only* mode. In this mode, it generates capacity forecasts but does not actually scale your Auto Scaling group based on those forecasts. This allows you to evaluate the accuracy and suitability of the forecast. You can view forecast data by using the `GetPredictiveScalingForecast` API operation or the AWS Management Console.
After you review the forecast data and decide to start scaling based on that data, switch the scaling policy to *forecast and scale* mode. In this mode:
+ If the forecast expects an increase in load, Amazon EC2 Auto Scaling will increase capacity by scaling out.
+ If the forecast expects a decrease in load, it will not scale in to remove capacity. If you want to remove capacity that is no longer needed, you must create dynamic scaling policies.
By default, Amazon EC2 Auto Scaling scales your Auto Scaling group at the start of each hour based on the forecast for that hour. You can optionally specify an earlier start time by using the `SchedulingBufferTime` property in the `PutScalingPolicy` API operation or the **Pre-launch instances** setting in the AWS Management Console. This causes Amazon EC2 Auto Scaling to launch new instances ahead of the forecasted demand, giving them time to boot and become ready to handle traffic.
To support launching new instances ahead of the forecasted demand, we strongly recommend that you enable the *default instance warmup* for your Auto Scaling group. This specifies a time period after a scale-out activity during which Amazon EC2 Auto Scaling won't scale in, even if dynamic scaling policies indicate capacity should be decreased. This helps you ensure that newly launched instances have adequate time to start serving the increased traffic before being considered for scale-in operations. For more information, see [Set the default instance warmup for an Auto Scaling group](ec2-auto-scaling-default-instance-warmup.md).
## Maximum capacity limit
Auto Scaling groups have a maximum capacity setting that limits the maximum number of EC2 instances that can be launched for the group. By default, when scaling policies are set, they cannot increase capacity higher than its maximum capacity.
Alternatively, you can allow the group's maximum capacity to be automatically increased if the forecast capacity approaches or exceeds the maximum capacity of the Auto Scaling group. To enable this behavior, use the `MaxCapacityBreachBehavior` and `MaxCapacityBuffer` properties in the `PutScalingPolicy` API operation or the **Max capacity behavior** setting in the AWS Management Console.
**Warning**
Use caution when allowing the maximum capacity to be automatically increased. This can lead to more instances being launched than intended if the increased maximum capacity is not monitored and managed. The increased maximum capacity then becomes the new normal maximum capacity for the Auto Scaling group until you manually update it. The maximum capacity does not automatically decrease back to the original maximum.
## Considerations
+ Confirm whether predictive scaling is suitable for your workload. A workload is a good fit for predictive scaling if it exhibits recurring load patterns that are specific to the day of the week or the time of day. To check this, configure predictive scaling policies in *forecast only* mode and then refer to the recommendations in the console. Amazon EC2 Auto Scaling provides recommendations based on observations about potential policy performance. Evaluate the forecast and the recommendations before letting predictive scaling actively scale your application.
+ Predictive scaling needs at least 24 hours of historical data to start forecasting. However, forecasts are more effective if historical data spans two full weeks. If you update your application by creating a new Auto Scaling group and deleting the old one, then your new Auto Scaling group needs 24 hours of historical load data before predictive scaling can start generating forecasts again. You can use custom metrics to aggregate metrics across old and new Auto Scaling groups. Otherwise, you might have to wait a few days for a more accurate forecast.
+ Choose a load metric that accurately represents the full load on your application and is the aspect of your application that's most important to scale on.
+ Using dynamic scaling with predictive scaling helps you follow the demand curve for your application closely, scaling in during periods of low traffic and scaling out when traffic is higher than expected. When multiple scaling policies are active, each policy determines the desired capacity independently, and the desired capacity is set to the maximum of those. For example, if 10 instances are required to stay at the target utilization in a target tracking scaling policy, and 8 instances are required to stay at the target utilization in a predictive scaling policy, then the group's desired capacity is set to 10. If you are new to dynamic scaling, we recommend using target tracking scaling policies. For more information, see [Dynamic scaling for Amazon EC2 Auto Scaling](as-scale-based-on-demand.md).
+ A core assumption of predictive scaling is that the Auto Scaling group is homogenous and all instances are of equal capacity. If this isn’t true for your group, forecasted capacity can be inaccurate. Therefore, use caution when creating predictive scaling policies for [mixed instances groups](ec2-auto-scaling-mixed-instances-groups.md) because instances of different types can be provisioned that are of unequal capacity. Following are some examples where the forecasted capacity will be inaccurate:
+ Your predictive scaling policy is based on CPU utilization, but the number of vCPUs on each Auto Scaling instance varies between instance types.
+ Your predictive scaling policy is based on network in or network out, but the network bandwidth throughput for each Auto Scaling instance varies between instance types. For example, the M5 and M5n instance types are similar, but the M5n instance type delivers significantly higher network throughput.
## Supported Regions
+ US East (N. Virginia)
+ US East (Ohio)
+ US West (N. California)
+ US West (Oregon)
+ Africa (Cape Town)
+ Asia Pacific (Hong Kong)
+ Asia Pacific (Hyderabad)
+ Asia Pacific (Jakarta)
+ Asia Pacific (Melbourne)
+ Asia Pacific (Mumbai)
+ Asia Pacific (Osaka)
+ Asia Pacific (Seoul)
+ Asia Pacific (Singapore)
+ Asia Pacific (Sydney)
+ Asia Pacific (Tokyo)
+ Canada (Central)
+ Canada West (Calgary)
+ China (Beijing)
+ China (Ningxia)
+ Europe (Frankfurt)
+ Europe (Ireland)
+ Europe (London)
+ Europe (Milan)
+ Europe (Paris)
+ Europe (Spain)
+ Europe (Stockholm)
+ Europe (Zurich)
+ Israel (Tel Aviv)
+ Middle East (Bahrain)
+ Middle East (UAE)
+ South America (São Paulo)
+ AWS GovCloud (US-East)
+ AWS GovCloud (US-West)
# Create a predictive scaling policy for an Auto Scaling group
The following procedures help you create a predictive scaling policy using the AWS Management Console or AWS CLI.
If the Auto Scaling group is new, it must provide at least 24 hours of data before Amazon EC2 Auto Scaling can generate a forecast for it.
**Topics**
+ [Create a predictive scaling policy (console)](#predictive-scaling-policy-console)
+ [Create a predictive scaling policy (AWS CLI)](#predictive-scaling-policy-aws-cli)
## Create a predictive scaling policy (console)
If this is your first time creating a predictive scaling policy, we recommend using the console to create multiple predictive scaling policies in *forecast only* mode. This allows you to test the potential effects of different metrics and target values. You can create multiple predictive scaling policies for each Auto Scaling group, but only one of the policies can be used for active scaling.
### Create a predictive scaling policy in the console (predefined metrics)
Use the following procedure to create a predictive scaling policy using predefined metrics (CPU, network I/O, or Application Load Balancer request count per target). The easiest way to create a predictive scaling policy is to use predefined metrics. If you prefer to use custom metrics instead, see [Create a predictive scaling policy in the console (custom metrics)](#create-a-predictive-scaling-policy-in-the-console-custom-metrics).
**To create a predictive scaling policy**
1. Open the Amazon EC2 console at [https://console.aws.amazon.com/ec2/](https://console.aws.amazon.com/ec2/), and choose **Auto Scaling Groups** from the navigation pane.
1. Select the check box next to your Auto Scaling group.
A split pane opens up at the bottom of the page.
1. On the **Automatic scaling** tab, in **Scaling policies**, choose **Create predictive scaling policy**.
1. Enter a name for the policy.
1. Turn on **Scale based on forecast** to give Amazon EC2 Auto Scaling permission to start scaling right away.
To keep the policy in *forecast only* mode, keep **Scale based on forecast** turned off.
1. For **Metrics**, choose your metrics from the list of options. Options include **CPU**, **Network In**, **Network Out**, **Application Load Balancer request count**, and **Custom metric pair**.
If you chose **Application Load Balancer request count per target**, then choose a target group in **Target group**. **Application Load Balancer request count per target** is only supported if you have attached an Application Load Balancer target group to your Auto Scaling group.
If you chose **Custom metric pair**, choose individual metrics from the drop-down lists for **Load metric** and **Scaling metric**.
1. For **Target utilization**, enter the target value that Amazon EC2 Auto Scaling should maintain. Amazon EC2 Auto Scaling scales out your capacity until the average utilization is at the target utilization, or until it reaches the maximum number of instances you specified.
[\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/autoscaling/ec2/userguide/predictive-scaling-create-policy.html)
1. (Optional) For **Pre-launch instances**, choose how far in advance you want your instances launched before the forecast calls for the load to increase.
1. (Optional) For **Max capacity behavior**, choose whether to let Amazon EC2 Auto Scaling scale out higher than the group's maximum capacity when predicted capacity exceeds the defined maximum. Turning on this setting lets scale out occur during periods when your traffic is forecasted to be at its highest.
1. (Optional) For **Buffer maximum capacity above the forecasted capacity**, choose how much additional capacity to use when the predicted capacity is close to or exceeds the maximum capacity. The value is specified as a percentage relative to the predicted capacity. For example, if the buffer is 10, this means a 10 percent buffer. Therefore, if the predicted capacity is 50 and the maximum capacity is 40, the effective maximum capacity is 55.
If set to 0, Amazon EC2 Auto Scaling might scale capacity higher than the maximum capacity to equal but not exceed predicted capacity.
1. Choose **Create predictive scaling policy**.
### Create a predictive scaling policy in the console (custom metrics)
Use the following procedure to create a predictive scaling policy using custom metrics. Custom metrics can include other metrics provided by CloudWatch or metrics that you publish to CloudWatch. To use CPU, network I/O, or Application Load Balancer request count per target, see [Create a predictive scaling policy in the console (predefined metrics)](#create-a-predictive-scaling-policy-in-the-console).
To create a predictive scaling policy using custom metrics, you must do the following:
+ You must provide the raw queries that let Amazon EC2 Auto Scaling interact with the metrics in CloudWatch. For more information, see [Advanced predictive scaling policy using custom metrics](predictive-scaling-customized-metric-specification.md). To be sure that Amazon EC2 Auto Scaling can extract the metric data from CloudWatch, confirm that each query is returning data points. Confirm this by using the CloudWatch console or the CloudWatch [GetMetricData](https://docs.aws.amazon.com/AmazonCloudWatch/latest/APIReference/API_GetMetricData.html) API operation.
**Note**
We provide sample JSON payloads in the JSON editor in the Amazon EC2 Auto Scaling console. These examples give you a reference for the key-value pairs that are required to add other CloudWatch metrics provided by AWS or metrics that you previously published to CloudWatch. You can use them as a starting point, then customize them for your needs.
+ If you use any metric math, you must manually construct the JSON to fit your unique scenario. For more information, see [Use metric math expressions](using-math-expression-examples.md). Before using metric math in your policy, confirm that metric queries based on metric math expressions are valid and return a single time series. Confirm this by using the CloudWatch console or the CloudWatch [GetMetricData](https://docs.aws.amazon.com/AmazonCloudWatch/latest/APIReference/API_GetMetricData.html) API operation.
If you make an error in a query by providing incorrect data, such as the wrong Auto Scaling group name, the forecast won't have any data. For troubleshooting custom metric issues, see [Considerations for custom metrics in a predictive scaling policy](custom-metrics-troubleshooting.md).
**To create a predictive scaling policy**
1. Open the Amazon EC2 console at [https://console.aws.amazon.com/ec2/](https://console.aws.amazon.com/ec2/), and choose **Auto Scaling Groups** from the navigation pane.
1. Select the check box next to your Auto Scaling group.
A split pane opens up at the bottom of the page.
1. On the **Automatic scaling** tab, in **Scaling policies**, choose **Create predictive scaling policy**.
1. Enter a name for the policy.
1. Turn on **Scale based on forecast** to give Amazon EC2 Auto Scaling permission to start scaling right away.
To keep the policy in *forecast only* mode, keep **Scale based on forecast** turned off.
1. For **Metrics**, choose **Custom metric pair**.
1. For **Load metric**, choose **Custom CloudWatch metric** to use a custom metric. Construct the JSON payload that contains the load metric definition for the policy and paste it into the JSON editor box, replacing what is already in the box.
1. For **Scaling metric**, choose **Custom CloudWatch metric** to use a custom metric. Construct the JSON payload that contains the scaling metric definition for the policy and paste it into the JSON editor box, replacing what is already in the box.
1. (Optional) To add a custom capacity metric, select the check box for **Add custom capacity metric**. Construct the JSON payload that contains the capacity metric definition for the policy and paste it into the JSON editor box, replacing what is already in the box.
You only need to enable this option to create a new time series for capacity if your capacity metric data spans multiple Auto Scaling groups. In this case, you must use metric math to aggregate the data into a single time series.
1. For **Target utilization**, enter the target value that Amazon EC2 Auto Scaling should maintain. Amazon EC2 Auto Scaling scales out your capacity until the average utilization is at the target utilization, or until it reaches the maximum number of instances you specified.
1. (Optional) For **Pre-launch instances**, choose how far in advance you want your instances launched before the forecast calls for the load to increase.
1. (Optional) For **Max capacity behavior**, choose whether to let Amazon EC2 Auto Scaling scale out higher than the group's maximum capacity when predicted capacity exceeds the defined maximum. Turning on this setting lets scale out occur during periods when your traffic is forecasted to be at its highest.
1. (Optional) For **Buffer maximum capacity above the forecasted capacity**, choose how much additional capacity to use when the predicted capacity is close to or exceeds the maximum capacity. The value is specified as a percentage relative to the predicted capacity. For example, if the buffer is 10, this means a 10 percent buffer. Therefore, if the predicted capacity is 50 and the maximum capacity is 40, the effective maximum capacity is 55.
If set to 0, Amazon EC2 Auto Scaling might scale capacity higher than the maximum capacity to equal but not exceed predicted capacity.
1. Choose **Create predictive scaling policy**.
## Create a predictive scaling policy (AWS CLI)
Use the AWS CLI as follows to configure predictive scaling policies for your Auto Scaling group. Replace each *user input placeholder* with your own information.
For more information about the CloudWatch metrics you can specify, see [PredictiveScalingMetricSpecification](https://docs.aws.amazon.com/autoscaling/ec2/APIReference/API_PredictiveScalingMetricSpecification.html) in the *Amazon EC2 Auto Scaling API Reference*.
### Example 1: A predictive scaling policy that creates forecasts but doesn't scale
The following example policy shows a complete policy configuration that uses CPU utilization metrics for predictive scaling with a target utilization of `40`. `ForecastOnly` mode is used by default, unless you explicitly specify which mode to use. Save this configuration in a file named `config.json`.
```
{
"MetricSpecifications": [
{
"TargetValue": 40,
"PredefinedMetricPairSpecification": {
"PredefinedMetricType": "ASGCPUUtilization"
}
}
]
}
```
To create the policy from the command line, run the [put-scaling-policy](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/autoscaling/put-scaling-policy.html) command with the configuration file specified, as demonstrated in the following example.
```
aws autoscaling put-scaling-policy --policy-name cpu40-predictive-scaling-policy \
--auto-scaling-group-name my-asg --policy-type PredictiveScaling \
--predictive-scaling-configuration file://config.json
```
If successful, this command returns the policy's Amazon Resource Name (ARN).
```
{
"PolicyARN": "arn:aws:autoscaling:region:account-id:scalingPolicy:2f4f5048-d8a8-4d14-b13a-d1905620f345:autoScalingGroupName/my-asg:policyName/cpu40-predictive-scaling-policy",
"Alarms": []
}
```
### Example 2: A predictive scaling policy that forecasts and scales
For a policy that allows Amazon EC2 Auto Scaling to forecast and scale, add the property `Mode` with a value of `ForecastAndScale`. The following example shows a policy configuration that uses Application Load Balancer request count metrics. The target utilization is `1000`, and predictive scaling is set to `ForecastAndScale` mode.
```
{
"MetricSpecifications": [
{
"TargetValue": 1000,
"PredefinedMetricPairSpecification": {
"PredefinedMetricType": "ALBRequestCount",
"ResourceLabel": "app/my-alb/778d41231b141a0f/targetgroup/my-alb-target-group/943f017f100becff"
}
}
],
"Mode": "ForecastAndScale"
}
```
To create this policy, run the [put-scaling-policy](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/autoscaling/put-scaling-policy.html) command with the configuration file specified, as demonstrated in the following example.
```
aws autoscaling put-scaling-policy --policy-name alb1000-predictive-scaling-policy \
--auto-scaling-group-name my-asg --policy-type PredictiveScaling \
--predictive-scaling-configuration file://config.json
```
If successful, this command returns the policy's Amazon Resource Name (ARN).
```
{
"PolicyARN": "arn:aws:autoscaling:region:account-id:scalingPolicy:19556d63-7914-4997-8c81-d27ca5241386:autoScalingGroupName/my-asg:policyName/alb1000-predictive-scaling-policy",
"Alarms": []
}
```
### Example 3: A predictive scaling policy that can scale higher than maximum capacity
The following example shows how to create a policy that can scale higher than the group's maximum size limit when you need it to handle a higher than normal load. By default, Amazon EC2 Auto Scaling doesn't scale your EC2 capacity higher than your defined maximum capacity. However, it might be helpful to let it scale higher with slightly more capacity to avoid performance or availability issues.
To provide room for Amazon EC2 Auto Scaling to provision additional capacity when the capacity is predicted to be at or very close to your group's maximum size, specify the `MaxCapacityBreachBehavior` and `MaxCapacityBuffer` properties, as shown in the following example. You must specify `MaxCapacityBreachBehavior` with a value of `IncreaseMaxCapacity`. The maximum number of instances that your group can have depends on the value of `MaxCapacityBuffer`.
```
{
"MetricSpecifications": [
{
"TargetValue": 70,
"PredefinedMetricPairSpecification": {
"PredefinedMetricType": "ASGCPUUtilization"
}
}
],
"MaxCapacityBreachBehavior": "IncreaseMaxCapacity",
"MaxCapacityBuffer": 10
}
```
In this example, the policy is configured to use a 10 percent buffer (`"MaxCapacityBuffer": 10`), so if the predicted capacity is 50 and the maximum capacity is 40, then the effective maximum capacity is 55. A policy that can scale capacity higher than the maximum capacity to equal but not exceed predicted capacity would have a buffer of 0 (`"MaxCapacityBuffer": 0`).
To create this policy, run the [put-scaling-policy](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/autoscaling/put-scaling-policy.html) command with the configuration file specified, as demonstrated in the following example.
```
aws autoscaling put-scaling-policy --policy-name cpu70-predictive-scaling-policy \
--auto-scaling-group-name my-asg --policy-type PredictiveScaling \
--predictive-scaling-configuration file://config.json
```
If successful, this command returns the policy's Amazon Resource Name (ARN).
```
{
"PolicyARN": "arn:aws:autoscaling:region:account-id:scalingPolicy:d02ef525-8651-4314-bf14-888331ebd04f:autoScalingGroupName/my-asg:policyName/cpu70-predictive-scaling-policy",
"Alarms": []
}
```
# Evaluate your predictive scaling policies
Before you use a predictive scaling policy to scale your Auto Scaling group, review the recommendations and other data for your policy in the Amazon EC2 Auto Scaling console. This is important because you don't want a predictive scaling policy to scale your actual capacity until you know that its predictions are accurate.
If the Auto Scaling group is new, give Amazon EC2 Auto Scaling 24 hours to create the first forecast.
When Amazon EC2 Auto Scaling creates a forecast, it uses historical data. If your Auto Scaling group doesn't have much recent historical data yet, Amazon EC2 Auto Scaling might temporarily backfill the forecast with aggregates created from the currently available historical aggregates. Forecasts are backfilled for up to two weeks before a policy's creation date.
**Topics**
+ [View your recommendations](#view-predictive-scaling-recommendations)
+ [Review monitoring graphs](#review-predictive-scaling-monitoring-graphs)
+ [Monitor metrics with CloudWatch](monitor-predictive-scaling-cloudwatch.md)
## View your predictive scaling recommendations
For effective analysis, Amazon EC2 Auto Scaling should have at least two predictive scaling policies to compare. (However, you can still review the findings for a single policy.) When you create multiple policies, you can evaluate a policy that uses one metric against a policy that uses a different metric. You can also evaluate the impact of different target value and metric combinations. After the predictive scaling policies are created, Amazon EC2 Auto Scaling immediately starts evaluating which policy would do a better job of scaling your group.
**To view your recommendations in the Amazon EC2 Auto Scaling console**
1. Open the Amazon EC2 console at [https://console.aws.amazon.com/ec2/](https://console.aws.amazon.com/ec2/), and choose **Auto Scaling Groups** from the navigation pane.
1. Select the check box next to the Auto Scaling group.
A split pane opens up in the bottom of the page.
1. On the **Auto scaling** tab, under **Predictive scaling policies**, you can view details about a policy along with our recommendation. The recommendation tells you whether the predictive scaling policy does a better job than not using it.
If you're unsure whether a predictive scaling policy is appropriate for your group, review the **Availability impact** and **Cost impact** columns to choose the right policy. The information for each column tells you what the impact of the policy is.
+ **Availability impact**: Describes whether the policy would avoid negative impact to availability by provisioning enough instances to handle the workload, compared to not using the policy.
+ **Cost impact**: Describes whether the policy would avoid negative impact on your costs by not over-provisioning instances, compared to not using the policy. By over-provisioning too much, your instances are underutilized or idle, which only adds to the cost impact.
If you have multiple policies, then a **Best prediction** tag displays next to the name of the policy that gives the most availability benefits at lower cost. More weight is given to availability impact.
1. (Optional) To select the desired time period for recommendation results, choose your preferred value from the **Evaluation period** dropdown: **2 days**, **1 week**, **2 weeks**, **4 weeks**, **6 weeks**, or **8 weeks**. By default, the evaluation period is the last two weeks. A longer evaluation period provides more data points to the recommendation results. However, adding more data points might not improve the results if your load patterns have changed, such as after a period of exceptional demand. In this case, you can get a more focused recommendation by looking at more recent data.
**Note**
Recommendations are generated only for policies that are in **Forecast only** mode. The recommendations feature works better when a policy is in the **Forecast only** mode throughout the evaluation period. If you start a policy in **Forecast and scale** mode and switch it to **Forecast only** mode later, the findings for that policy are likely to be biased. This is because the policy has already contributed toward the actual capacity.
## Review predictive scaling monitoring graphs
In the Amazon EC2 Auto Scaling console, you can review the forecast of the previous days, weeks, or months to visualize how well the policy performs over time. You can also use this information to evaluate the accuracy of predictions when deciding whether to let a policy scale your actual capacity.
**To review predictive scaling monitoring graphs in the Amazon EC2 Auto Scaling console**
1. Choose a policy from the **Predictive scaling policies** list.
1. In the **Monitoring** section, you can view your policy's past and future forecasts for load and capacity against actual values. The **Load** graph shows load forecast and actual values for the load metric that you chose. The **Capacity** graph shows the number of instances predicted by the policy. It also includes the actual number of instances launched. The vertical line separates historical values from future forecasts. These graphs become available shortly after the policy is created.
1. (Optional) To change the amount of historical data shown in the chart, choose your preferred value from the **Evaluation period** dropdown at the top of the page. The evaluation period does not transform the data on this page in any way. It only changes the amount of historical data shown.
The following image shows the **Load** and **Capacity** graphs when forecasts have been applied multiple times. Predictive scaling forecasts load based on your historical load data. The load your application generates is represented as the sum of the CPU utilization, network in/out, received requests, or custom metric for each instance in the Auto Scaling group. Predictive scaling calculates future capacity needs based on the load forecast and the target utilization that you want to achieve for the scaling metric.
![\[Predictive scaling graphs\]](http://docs.aws.amazon.com/autoscaling/ec2/userguide/images/predictive-scaling-graphs.png)
**Compare data in the **Load** graph**
Each horizontal line represents a different set of data points reported in one-hour intervals:
1. **Actual observed load** uses the SUM statistic for your chosen load metric to show the past total hourly load.
1. **Load predicted by the policy** shows the hourly load prediction. This prediction is based on the previous two weeks of actual load observations.
**Compare data in the **Capacity** graph**
Each horizontal line represents a different set of data points reported in one-hour intervals:
1. **Actual observed capacity** shows your Auto Scaling group's past actual capacity when the [GroupTotalInstances](ec2-auto-scaling-metrics.md#as-group-metrics) metric is enabled. This capacity depends on your other scaling policies and the minimum group size during the selected time period.
1. **Capacity predicted by the policy** shows the baseline capacity that you can expect to have at the beginning of each hour when the policy is in **Forecast and scale** mode.
1. **Inferred required capacity** shows the ideal capacity to maintain the scaling metric at the target value you chose.
1. **Minimum capacity** shows the minimum capacity of the Auto Scaling group.
1. **Maximum capacity** shows the maximum capacity of the Auto Scaling group.
For the purpose of calculating the inferred required capacity, we begin by assuming that each instance is equally utilized at a specified target value. In practice, instances are not equally utilized. By assuming that utilization is uniformly spread between instances, however, we can make a likelihood estimate of the amount of capacity that is needed. The capacity requirement is then calculated to be inversely proportional to the scaling metric that you used for your predictive scaling policy. In other words, as capacity increases, the scaling metric decreases at the same rate. For example, if capacity doubles, the scaling metric must decrease by half.
The formula for the inferred required capacity:
`sum of (actualCapacityUnits*scalingMetricValue)/(targetUtilization)`
For example, we take the `actualCapacityUnits` (`10`) and the `scalingMetricValue` (`30`) for a given hour. We then take the `targetUtilization` that you specified in your predictive scaling policy (`60`) and calculate the inferred required capacity for the same hour. This returns a value of `5`. This means that five is the inferred amount of capacity required to maintain capacity in direct inverse proportion to the target value of the scaling metric.
**Note**
Various levers are available for you to adjust and improve the cost savings and availability of your application.
You use predictive scaling for the baseline capacity and dynamic scaling to handle additional capacity. Dynamic scaling works independently from predictive scaling, scaling in and out based on current utilization. First, Amazon EC2 Auto Scaling calculates the recommended number of instances for each dynamic scaling policy. Then, it scales based on the policy that provides the largest number of instances.
To allow scale in to occur when the load decreases, your Auto Scaling group should always have at least one dynamic scaling policy with the scale-in portion enabled.
You can improve scaling performance by making sure that your minimum and maximum capacity are not too restrictive. A policy with a recommended number of instances that does not fall within the minimum and maximum capacity range will be prevented from scaling in and out.
# Monitor predictive scaling metrics with CloudWatch
Depending on your needs, you might prefer to access monitoring data for predictive scaling from Amazon CloudWatch instead of the Amazon EC2 Auto Scaling console. After you create a predictive scaling policy, the policy collects data that is used to forecast your future load and capacity. After this data is collected, it is automatically stored in CloudWatch at regular intervals. Then, you can use CloudWatch to visualize how well the policy performs over time. You can also create CloudWatch alarms to notify you when performance indicators change beyond the limits that you define in CloudWatch.
**Topics**
+ [Visualize historical forecast data](#visualize-historical-forecast-data)
+ [Create accuracy metrics using metric math](#create-accuracy-metrics)
## Visualize historical forecast data
You can view the load and capacity forecast data for a predictive scaling policy in CloudWatch. This can be useful when visualizing forecasts against other CloudWatch metrics in a single graph. It can also help when viewing a broader time range so that you can see trends over time. You can access up to 15 months of historical metrics to get a better perspective on how your policy is performing.
For more information, see [Predictive scaling metrics and dimensions](ec2-auto-scaling-metrics.md#predictive-scaling-metrics).
**To view historical forecast data using the CloudWatch console**
1. Open the CloudWatch console at [https://console.aws.amazon.com/cloudwatch/](https://console.aws.amazon.com/cloudwatch/).
1. In the navigation pane, choose **Metrics** and then **All metrics**.
1. Choose the **Auto Scaling** metric namespace.
1. Choose one of the following options to view either the load forecast or capacity forecast metrics:
+ **Predictive Scaling Load Forecasts**
+ **Predictive Scaling Capacity Forecasts**
1. In the search field, enter the name of the predictive scaling policy or the name of the Auto Scaling group, and then press Enter to filter the results.
1. To graph a metric, select the check box next to the metric. To change the name of the graph, choose the pencil icon. To change the time range, select one of the predefined values or choose **custom**. For more information, see [Graphing a metric](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/graph_a_metric.html) in the *Amazon CloudWatch User Guide*.
1. To change the statistic, choose the **Graphed metrics** tab. Choose the column heading or an individual value, and then choose a different statistic. Although you can choose any statistic for each metric, not all statistics are useful for **PredictiveScalingLoadForecast** and **PredictiveScalingCapacityForecast** metrics. For example, the **Average**, **Minimum**, and **Maximum** statistics are useful, but the **Sum** statistic is not.
1. To add another metric to the graph, under **Browse**, choose **All**, find the specific metric, and then select the check box next to it. You can add up to 10 metrics.
For example, to add the actual values for CPU utilization to the graph, choose the **EC2** namespace and then choose **By Auto Scaling Group**. Then, select the check box for the **CPUUtilization** metric and the specific Auto Scaling group.
1. (Optional) To add the graph to a CloudWatch dashboard, choose **Actions**, **Add to dashboard**.
## Create accuracy metrics using metric math
With metric math, you can query multiple CloudWatch metrics and use math expressions to create new time series based on these metrics. You can visualize the resulting time series on the CloudWatch console and add them to dashboards. For more information about metric math, see [Using metric math](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/using-metric-math.html) in the *Amazon CloudWatch User Guide*.
Using metric math, you can graph the data that Amazon EC2 Auto Scaling generates for predictive scaling in different ways. This helps you monitor policy performance over time, and helps you understand whether your combination of metrics can be improved.
For example, you can use a metric math expression to monitor the [mean absolute percentage error](https://en.wikipedia.org/wiki/Mean_absolute_percentage_error) (MAPE). The MAPE metric helps monitor the difference between the forecasted values and the actual values observed during a given forecast window. Changes in the value of MAPE can indicate whether the policy's performance is degrading over time as the nature of your application changes. An increase in MAPE signals a wider gap between the forecasted values and the actual values.
**Example: Metric math expression**
To get started with this type of graph, you can create a metric math expression like the one shown in the following example.
```
{
"MetricDataQueries": [
{
"Expression": "TIME_SERIES(AVG(ABS(m1-m2)/m1))",
"Id": "e1",
"Period": 3600,
"Label": "MeanAbsolutePercentageError",
"ReturnData": true
},
{
"Id": "m1",
"Label": "ActualLoadValues",
"MetricStat": {
"Metric": {
"Namespace": "AWS/EC2",
"MetricName": "CPUUtilization",
"Dimensions": [
{
"Name": "AutoScalingGroupName",
"Value": "my-asg"
}
]
},
"Period": 3600,
"Stat": "Sum"
},
"ReturnData": false
},
{
"Id": "m2",
"Label": "ForecastedLoadValues",
"MetricStat": {
"Metric": {
"Namespace": "AWS/AutoScaling",
"MetricName": "PredictiveScalingLoadForecast",
"Dimensions": [
{
"Name": "AutoScalingGroupName",
"Value": "my-asg"
},
{
"Name": "PolicyName",
"Value": "my-predictive-scaling-policy"
},
{
"Name": "PairIndex",
"Value": "0"
}
]
},
"Period": 3600,
"Stat": "Average"
},
"ReturnData": false
}
]
}
```
Instead of a single metric, there is an array of metric data query structures for `MetricDataQueries`. Each item in `MetricDataQueries` gets a metric or performs a math expression. The first item, `e1`, is the math expression. The designated expression sets the `ReturnData` parameter to `true`, which ultimately produces a single time series. For all other metrics, the `ReturnData` value is `false`.
In the example, the designated expression uses the actual and forecasted values as input and returns the new metric (MAPE). `m1` is the CloudWatch metric that contains the actual load values (assuming CPU utilization is the load metric that was originally specified for the policy named `my-predictive-scaling-policy`). `m2` is the CloudWatch metric that contains the forecasted load values. The math syntax for the MAPE metric is as follows:
*Average of (abs ((Actual - Forecast)/(Actual)))*
### Visualize your accuracy metrics and set alarms
To visualize the accuracy metric data, select the **Metrics** tab in the CloudWatch console. You can graph the data from there. For more information, see [Adding a math expression to a CloudWatch graph](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/using-metric-math.html#adding-metrics-expression-console) in the *Amazon CloudWatch User Guide*.
You can also set an alarm on a metric that you're monitoring from the **Metrics** section. While on the **Graphed metrics** tab, select the **Create alarm** icon under the **Actions** column. The **Create alarm** icon is represented as a small bell. For more information and notification options, see [Creating a CloudWatch alarm based on a metric math expression](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/Create-alarm-on-metric-math-expression.html) and [Notifying users on alarm changes](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/Notify_Users_Alarm_Changes.html) in the *Amazon CloudWatch User Guide*.
Alternatively, you can use [GetMetricData](https://docs.aws.amazon.com/AmazonCloudWatch/latest/APIReference/API_GetMetricData.html) and [PutMetricAlarm](https://docs.aws.amazon.com/AmazonCloudWatch/latest/APIReference/API_PutMetricAlarm.html) to perform calculations using metric math and create alarms based on the output.
# Override forecast values using scheduled actions
Sometimes, you might have additional information about your future application requirements that the forecast calculation is unable to take into account. For example, forecast calculations might underestimate the capacity needed for an upcoming marketing event. You can use scheduled actions to temporarily override the forecast during future time periods. The scheduled actions can run on a recurring basis, or at a specific date and time when there are one-time demand fluctuations.
For example, you can create a scheduled action with a higher minimum capacity than what is forecasted. At runtime, Amazon EC2 Auto Scaling updates the minimum capacity of your Auto Scaling group. Because predictive scaling optimizes for capacity, a scheduled action with a minimum capacity that is higher than the forecast values is honored. This prevents capacity from being less than expected. To stop overriding the forecast, use a second scheduled action to return the minimum capacity to its original setting.
The following procedure outlines the steps for overriding the forecast during future time periods.
**Topics**
+ [Step 1: (Optional) Analyze time series data](#analyzing-time-series-data)
+ [Step 2: Create two scheduled actions](#scheduling-capacity)
**Important**
This topic assumes that you are trying to override the forecast to scale to a higher capacity than what is forecasted. If you need to temporarily decrease capacity without interference from a predictive scaling policy, use *forecast only* mode instead. While in forecast only mode, predictive scaling will continue to generate forecasts, but it will not automatically increase capacity. You can then monitor resource utilization and manually decrease the size of your group as needed. For more information about scaling manually, see [Manual scaling for Amazon EC2 Auto Scaling](ec2-auto-scaling-scaling-manually.md).
## Step 1: (Optional) Analyze time series data
Start by analyzing the forecast time series data. This is an optional step, but it is helpful if you want to understand the details of the forecast.
1. **Retrieve the forecast**
After the forecast is created, you can query for a specific time period in the forecast. The goal of the query is to get a complete view of the time series data for a specific time period.
Your query can include up to two days of future forecast data. If you have been using predictive scaling for a while, you can also access your past forecast data. However, the maximum time duration between the start and end time is 30 days.
To get the forecast using the [get-predictive-scaling-forecast](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/autoscaling/get-predictive-scaling-forecast.html) AWS CLI command, provide the following parameters in the command:
+ Enter the name of the Auto Scaling group in the `--auto-scaling-group-name` parameter.
+ Enter the name of the policy in the `--policy-name` parameter.
+ Enter the start time in the `--start-time` parameter to return only forecast data for after or at the specified time.
+ Enter the end time in the `--end-time` parameter to return only forecast data for before the specified time.
```
aws autoscaling get-predictive-scaling-forecast --auto-scaling-group-name my-asg \
--policy-name cpu40-predictive-scaling-policy \
--start-time "2021-05-19T17:00:00Z" \
--end-time "2021-05-19T23:00:00Z"
```
If successful, the command returns data similar to the following example.
```
{
"LoadForecast": [
{
"Timestamps": [
"2021-05-19T17:00:00+00:00",
"2021-05-19T18:00:00+00:00",
"2021-05-19T19:00:00+00:00",
"2021-05-19T20:00:00+00:00",
"2021-05-19T21:00:00+00:00",
"2021-05-19T22:00:00+00:00",
"2021-05-19T23:00:00+00:00"
],
"Values": [
153.0655799339254,
128.8288551285919,
107.1179447150675,
197.3601844551528,
626.4039934516954,
596.9441277518481,
677.9675713779869
],
"MetricSpecification": {
"TargetValue": 40.0,
"PredefinedMetricPairSpecification": {
"PredefinedMetricType": "ASGCPUUtilization"
}
}
}
],
"CapacityForecast": {
"Timestamps": [
"2021-05-19T17:00:00+00:00",
"2021-05-19T18:00:00+00:00",
"2021-05-19T19:00:00+00:00",
"2021-05-19T20:00:00+00:00",
"2021-05-19T21:00:00+00:00",
"2021-05-19T22:00:00+00:00",
"2021-05-19T23:00:00+00:00"
],
"Values": [
2.0,
2.0,
2.0,
2.0,
4.0,
4.0,
4.0
]
},
"UpdateTime": "2021-05-19T01:52:50.118000+00:00"
}
```
The response includes two forecasts: `LoadForecast` and `CapacityForecast`. `LoadForecast` shows the hourly load forecast. `CapacityForecast` shows forecast values for the capacity that is needed on an hourly basis to handle the forecasted load while maintaining a `TargetValue` of 40.0 (40% average CPU utilization).
1. **Identify the target time period**
Identify the hour or hours when the one-time demand fluctuation should take place. Remember that dates and times shown in the forecast are in UTC.
## Step 2: Create two scheduled actions
Next, create two scheduled actions for a specific time period when your application will have a higher than forecasted load. For example, if you have a marketing event that will drive traffic to your site for a limited period of time, you can schedule a one-time action to update the minimum capacity when it starts. Then, schedule another action to return the minimum capacity to the original setting when the event ends.
**To create two scheduled actions for one-time events (console)**
1. Open the Amazon EC2 console at [https://console.aws.amazon.com/ec2/](https://console.aws.amazon.com/ec2/), and choose **Auto Scaling Groups** from the navigation pane.
1. Select the check box next to your Auto Scaling group.
A split pane opens up in the bottom of the page.
1. On the **Automatic scaling** tab, in **Scheduled actions**, choose **Create scheduled action**.
1. Fill in the following scheduled action settings:
1. Enter a **Name** for the scheduled action.
1. For **Min**, enter the new minimum capacity for your Auto Scaling group. The **Min** must be less than or equal to the maximum size of the group. If your value for **Min** is greater than group's maximum size, you must update **Max**.
1. For **Recurrence**, choose **Once**.
1. For **Time zone**, choose a time zone. If no time zone is chosen, `ETC/UTC` is used by default.
1. Define a **Specific start time**.
1. Choose **Create**.
The console displays the scheduled actions for the Auto Scaling group.
1. Configure a second scheduled action to return the minimum capacity to the original setting at the end of the event. Predictive scaling can scale capacity only when the value you set for **Min** is lower than the forecast values.
**To create two scheduled actions for one-time events (AWS CLI)**
To use the AWS CLI to create the scheduled actions, use the [put-scheduled-update-group-action](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/autoscaling/put-scheduled-update-group-action.html) command.
For example, let's define a schedule that maintains a minimum capacity of three instances on May 19 at 5:00 PM for eight hours. The following commands show how to implement this scenario.
The first [put-scheduled-update-group-action](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/autoscaling/put-scheduled-update-group-action.html) command instructs Amazon EC2 Auto Scaling to update the minimum capacity of the specified Auto Scaling group at 5:00 PM UTC on May 19, 2021.
```
aws autoscaling put-scheduled-update-group-action --scheduled-action-name my-event-start \
--auto-scaling-group-name my-asg --start-time "2021-05-19T17:00:00Z" --minimum-capacity 3
```
The second command instructs Amazon EC2 Auto Scaling to set the group's minimum capacity to one at 1:00 AM UTC on May 20, 2021.
```
aws autoscaling put-scheduled-update-group-action --scheduled-action-name my-event-end \
--auto-scaling-group-name my-asg --start-time "2021-05-20T01:00:00Z" --minimum-capacity 1
```
After you add these scheduled actions to the Auto Scaling group, Amazon EC2 Auto Scaling does the following:
+ At 5:00 PM UTC on May 19, 2021, the first scheduled action runs. If the group currently has fewer than three instances, the group scales out to three instances. During this time and for the next eight hours, Amazon EC2 Auto Scaling can continue to scale out if the predicted capacity is higher than the actual capacity or if there is a dynamic scaling policy in effect.
+ At 1:00 AM UTC on May 20, 2021, the second scheduled action runs. This returns the minimum capacity to its original setting at the end of the event.
### Scaling based on recurring schedules
To override the forecast for the same time period every week, create two scheduled actions and provide the time and date logic using a cron expression.
The cron expression format consists of five fields separated by spaces: [Minute] [Hour] [Day\$1of\$1Month] [Month\$1of\$1Year] [Day\$1of\$1Week]. Fields can contain any allowed values, including special characters.
For example, the following cron expression runs the action every Tuesday at 6:30 AM. The asterisk is used as a wildcard to match all values for a field.
```
30 6 * * 2
```
### See also
For more information about how to create, list, edit, and delete scheduled actions, see [Scheduled scaling for Amazon EC2 Auto Scaling](ec2-auto-scaling-scheduled-scaling.md).
# Advanced predictive scaling policy using custom metrics
In a predictive scaling policy, you can use predefined or custom metrics. Custom metrics are useful when the predefined metrics (CPU, network I/O, and Application Load Balancer request count) do not sufficiently describe your application load.
When creating a predictive scaling policy with custom metrics, you can specify other CloudWatch metrics provided by AWS, or you can specify metrics that you define and publish yourself. You can also use metric math to aggregate and transform existing metrics into a new time series that AWS doesn't automatically track. When you combine values in your data, for example, by calculating new sums or averages, it's called *aggregating*. The resulting data is called an *aggregate*.
The following section contains best practices and examples of how to construct the JSON structure for the policy.
**Topics**
+ [Best practices](#custom-metrics-best-practices)
+ [Prerequisites](#custom-metrics-prerequisites)
+ [Constructing the JSON for custom metrics](construct-json-custom-metrics.md)
+ [Considerations for custom metrics in a predictive scaling policy](custom-metrics-troubleshooting.md)
+ [Limitations](#custom-metrics-limitations)
## Best practices
The following best practices can help you use custom metrics more effectively:
+ For the load metric specification, the most useful metric is a metric that represents the load on an Auto Scaling group as a whole, regardless of the group's capacity.
+ For the scaling metric specification, the most useful metric to scale by is an average throughput or utilization per instance metric.
+ The scaling metric must be inversely proportional to capacity. That is, if the number of instances in the Auto Scaling group increases, the scaling metric should decrease by roughly the same proportion. To ensure that predictive scaling behaves as expected, the load metric and the scaling metric must also correlate strongly with each other.
+ The target utilization must match the type of scaling metric. For a policy configuration that uses CPU utilization, this is a target percentage. For a policy configuration that uses throughput, such as the number of requests or messages, this is the target number of requests or messages per instance during any one-minute interval.
+ If these recommendations are not followed, the forecasted future values of the time series will probably be incorrect. To validate that the data is correct, you can view the forecasted values in the Amazon EC2 Auto Scaling console. Alternatively, after you create your predictive scaling policy, inspect the `LoadForecast` and `CapacityForecast` objects returned by a call to the [GetPredictiveScalingForecast](https://docs.aws.amazon.com/autoscaling/ec2/APIReference/API_GetPredictiveScalingForecast.html) API.
+ We strongly recommend that you configure predictive scaling in forecast only mode so that you can evaluate the forecast before predictive scaling starts actively scaling capacity.
## Prerequisites
To add custom metrics to your predictive scaling policy, you must have `cloudwatch:GetMetricData` permissions.
To specify your own metrics instead of the metrics that AWS provides, you must first publish your metrics to CloudWatch. For more information, see [Publishing custom metrics](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/publishingMetrics.html) in the *Amazon CloudWatch User Guide*.
If you publish your own metrics, make sure to publish the data points at a minimum frequency of five minutes. Amazon EC2 Auto Scaling retrieves the data points from CloudWatch based on the length of the period that it needs. For example, the load metric specification uses hourly metrics to measure the load on your application. CloudWatch uses your published metric data to provide a single data value for any one-hour period by aggregating all data points with timestamps that fall within each one-hour period.
# Constructing the JSON for custom metrics
The following section contains examples for how to configure predictive scaling to query data from CloudWatch. There are two different methods to configure this option, and the method that you choose affects which format you use to construct the JSON for your predictive scaling policy. When you use metric math, the format of the JSON varies further based on the metric math being performed.
1. To create a policy that gets data directly from other CloudWatch metrics provided by AWS or metrics that you publish to CloudWatch, see [Example predictive scaling policy with custom load and scaling metrics (AWS CLI)](#custom-metrics-ex1).
1. To create a policy that can query multiple CloudWatch metrics and use math expressions to create new time series based on these metrics, see [Use metric math expressions](using-math-expression-examples.md).
## Example predictive scaling policy with custom load and scaling metrics (AWS CLI)
To create a predictive scaling policy with custom load and scaling metrics with the AWS CLI, store the arguments for `--predictive-scaling-configuration` in a JSON file named `config.json`.
You start adding custom metrics by replacing the replaceable values in the following example with those of your metrics and your target utilization.
```
{
"MetricSpecifications": [
{
"TargetValue": 50,
"CustomizedScalingMetricSpecification": {
"MetricDataQueries": [
{
"Id": "scaling_metric",
"MetricStat": {
"Metric": {
"MetricName": "MyUtilizationMetric",
"Namespace": "MyNameSpace",
"Dimensions": [
{
"Name": "MyOptionalMetricDimensionName",
"Value": "MyOptionalMetricDimensionValue"
}
]
},
"Stat": "Average"
}
}
]
},
"CustomizedLoadMetricSpecification": {
"MetricDataQueries": [
{
"Id": "load_metric",
"MetricStat": {
"Metric": {
"MetricName": "MyLoadMetric",
"Namespace": "MyNameSpace",
"Dimensions": [
{
"Name": "MyOptionalMetricDimensionName",
"Value": "MyOptionalMetricDimensionValue"
}
]
},
"Stat": "Sum"
}
}
]
}
}
]
}
```
For more information, see [MetricDataQuery](https://docs.aws.amazon.com/autoscaling/ec2/APIReference/API_MetricDataQuery.html) in the *Amazon EC2 Auto Scaling API Reference*.
**Note**
Following are some additional resources that can help you find metric names, namespaces, dimensions, and statistics for CloudWatch metrics:
For information about the available metrics for AWS services, see [AWS services that publish CloudWatch metrics](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/aws-services-cloudwatch-metrics.html) in the *Amazon CloudWatch User Guide*.
To get the exact metric name, namespace, and dimensions (if applicable) for a CloudWatch metric with the AWS CLI, see [list-metrics](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/cloudwatch/list-metrics.html).
To create this policy, run the [put-scaling-policy](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/autoscaling/put-scaling-policy.html) command using the JSON file as input, as demonstrated in the following example.
```
aws autoscaling put-scaling-policy --policy-name my-predictive-scaling-policy \
--auto-scaling-group-name my-asg --policy-type PredictiveScaling \
--predictive-scaling-configuration file://config.json
```
If successful, this command returns the policy's Amazon Resource Name (ARN).
```
{
"PolicyARN": "arn:aws:autoscaling:region:account-id:scalingPolicy:2f4f5048-d8a8-4d14-b13a-d1905620f345:autoScalingGroupName/my-asg:policyName/my-predictive-scaling-policy",
"Alarms": []
}
```
# Use metric math expressions
The following section provides information and examples of predictive scaling policies that show how you can use metric math in your policy.
**Topics**
+ [Understand metric math](#custom-metrics-metric-math)
+ [Example predictive scaling policy that combines metrics using metric math (AWS CLI)](#custom-metrics-ex2)
+ [Example predictive scaling policy to use in a blue/green deployment scenario (AWS CLI)](#custom-metrics-ex3)
## Understand metric math
If all you want to do is aggregate existing metric data, CloudWatch metric math saves you the effort and cost of publishing another metric to CloudWatch. You can use any metric that AWS provides, and you can also use metrics that you define as part of your applications. For example, you might want to calculate the Amazon SQS queue backlog per instance. You can do this by taking the approximate number of messages available for retrieval from the queue and dividing that number by the Auto Scaling group's running capacity.
For more information, see [Using metric math](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/using-metric-math.html) in the *Amazon CloudWatch User Guide*.
If you choose to use a metric math expression in your predictive scaling policy, consider the following points:
+ Metric math operations use the data points of the unique combination of metric name, namespace, and dimension keys/value pairs of metrics.
+ You can use any arithmetic operator (\$1 - \$1 / ^), statistical function (such as AVG or SUM), or other function that CloudWatch supports.
+ You can use both metrics and the results of other math expressions in the formulas of the math expression.
+ Your metric math expressions can be made up of different aggregations. However, it's a best practice for the final aggregation result to use `Average` for the scaling metric and `Sum` for the load metric.
+ Any expressions used in a metric specification must eventually return a single time series.
To use metric math, do the following:
+ Choose one or more CloudWatch metrics. Then, create the expression. For more information, see [Using metric math](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/using-metric-math.html) in the *Amazon CloudWatch User Guide*.
+ Verify that the metric math expression is valid by using the CloudWatch console or the CloudWatch [GetMetricData](https://docs.aws.amazon.com/AmazonCloudWatch/latest/APIReference/API_GetMetricData.html) API.
## Example predictive scaling policy that combines metrics using metric math (AWS CLI)
Sometimes, instead of specifying the metric directly, you might need to first process its data in some way. For example, you might have an application that pulls work from an Amazon SQS queue, and you might want to use the number of items in the queue as criteria for predictive scaling. The number of messages in the queue does not solely define the number of instances that you need. Therefore, more work is needed to create a metric that can be used to calculate the backlog per instance. For more information, see [Scaling policy based on Amazon SQS](as-using-sqs-queue.md).
The following is an example predictive scaling policy for this scenario. It specifies scaling and load metrics that are based on the Amazon SQS `ApproximateNumberOfMessagesVisible` metric, which is the number of messages available for retrieval from the queue. It also uses the Amazon EC2 Auto Scaling `GroupInServiceInstances` metric and a math expression to calculate the backlog per instance for the scaling metric.
```
aws autoscaling put-scaling-policy --policy-name my-sqs-custom-metrics-policy \
--auto-scaling-group-name my-asg --policy-type PredictiveScaling \
--predictive-scaling-configuration file://config.json
{
"MetricSpecifications": [
{
"TargetValue": 100,
"CustomizedScalingMetricSpecification": {
"MetricDataQueries": [
{
"Label": "Get the queue size (the number of messages waiting to be processed)",
"Id": "queue_size",
"MetricStat": {
"Metric": {
"MetricName": "ApproximateNumberOfMessagesVisible",
"Namespace": "AWS/SQS",
"Dimensions": [
{
"Name": "QueueName",
"Value": "my-queue"
}
]
},
"Stat": "Sum"
},
"ReturnData": false
},
{
"Label": "Get the group size (the number of running instances)",
"Id": "running_capacity",
"MetricStat": {
"Metric": {
"MetricName": "GroupInServiceInstances",
"Namespace": "AWS/AutoScaling",
"Dimensions": [
{
"Name": "AutoScalingGroupName",
"Value": "my-asg"
}
]
},
"Stat": "Sum"
},
"ReturnData": false
},
{
"Label": "Calculate the backlog per instance",
"Id": "scaling_metric",
"Expression": "queue_size / running_capacity",
"ReturnData": true
}
]
},
"CustomizedLoadMetricSpecification": {
"MetricDataQueries": [
{
"Id": "load_metric",
"MetricStat": {
"Metric": {
"MetricName": "ApproximateNumberOfMessagesVisible",
"Namespace": "AWS/SQS",
"Dimensions": [
{
"Name": "QueueName",
"Value": "my-queue"
}
],
},
"Stat": "Sum"
},
"ReturnData": true
}
]
}
}
]
}
```
The example returns the policy's ARN.
```
{
"PolicyARN": "arn:aws:autoscaling:region:account-id:scalingPolicy:2f4f5048-d8a8-4d14-b13a-d1905620f345:autoScalingGroupName/my-asg:policyName/my-sqs-custom-metrics-policy",
"Alarms": []
}
```
## Example predictive scaling policy to use in a blue/green deployment scenario (AWS CLI)
A search expression provides an advanced option in which you can query for a metric from multiple Auto Scaling groups and perform math expressions on them. This is especially useful for blue/green deployments.
**Note**
A *blue/green deployment* is a deployment method in which you create two separate but identical Auto Scaling groups. Only one of the groups receives production traffic. User traffic is initially directed to the earlier ("blue") Auto Scaling group, while a new group ("green") is used for testing and evaluation of a new version of an application or service. User traffic is shifted to the green Auto Scaling group after a new deployment is tested and accepted. You can then delete the blue group after the deployment is successful.
When new Auto Scaling groups get created as part of a blue/green deployment, the metric history of each group can be automatically included in the predictive scaling policy without you having to change its metric specifications. For more information, see [Using EC2 Auto Scaling predictive scaling policies with Blue/Green deployments](https://aws.amazon.com/blogs/compute/retaining-metrics-across-blue-green-deployment-for-predictive-scaling/) on the AWS Compute Blog.
The following example policy shows how this can be done. In this example, the policy uses the `CPUUtilization` metric emitted by Amazon EC2. It uses the Amazon EC2 Auto Scaling `GroupInServiceInstances` metric and a math expression to calculate the value of the scaling metric per instance. It also specifies a capacity metric specification to get the `GroupInServiceInstances` metric.
The search expression finds the `CPUUtilization` of instances in multiple Auto Scaling groups based on the specified search criteria. If you later create a new Auto Scaling group that matches the same search criteria, the `CPUUtilization` of the instances in the new Auto Scaling group is automatically included.
```
aws autoscaling put-scaling-policy --policy-name my-blue-green-predictive-scaling-policy \
--auto-scaling-group-name my-asg --policy-type PredictiveScaling \
--predictive-scaling-configuration file://config.json
{
"MetricSpecifications": [
{
"TargetValue": 25,
"CustomizedScalingMetricSpecification": {
"MetricDataQueries": [
{
"Id": "load_sum",
"Expression": "SUM(SEARCH('{AWS/EC2,AutoScalingGroupName} MetricName=\"CPUUtilization\" ASG-myapp', 'Sum', 300))",
"ReturnData": false
},
{
"Id": "capacity_sum",
"Expression": "SUM(SEARCH('{AWS/AutoScaling,AutoScalingGroupName} MetricName=\"GroupInServiceInstances\" ASG-myapp', 'Average', 300))",
"ReturnData": false
},
{
"Id": "weighted_average",
"Expression": "load_sum / capacity_sum",
"ReturnData": true
}
]
},
"CustomizedLoadMetricSpecification": {
"MetricDataQueries": [
{
"Id": "load_sum",
"Expression": "SUM(SEARCH('{AWS/EC2,AutoScalingGroupName} MetricName=\"CPUUtilization\" ASG-myapp', 'Sum', 3600))"
}
]
},
"CustomizedCapacityMetricSpecification": {
"MetricDataQueries": [
{
"Id": "capacity_sum",
"Expression": "SUM(SEARCH('{AWS/AutoScaling,AutoScalingGroupName} MetricName=\"GroupInServiceInstances\" ASG-myapp', 'Average', 300))"
}
]
}
}
]
}
```
The example returns the policy's ARN.
```
{
"PolicyARN": "arn:aws:autoscaling:region:account-id:scalingPolicy:2f4f5048-d8a8-4d14-b13a-d1905620f345:autoScalingGroupName/my-asg:policyName/my-blue-green-predictive-scaling-policy",
"Alarms": []
}
```
# Considerations for custom metrics in a predictive scaling policy
If an issue occurs while using custom metrics, we recommend that you do the following:
+ If an error message is provided, read the message and resolve the issue it reports, if possible.
+ If an issue occurs when you are trying to use a search expression in a blue/green deployment scenario, first make sure that you understand how to create a search expression that looks for a partial match instead of an exact match. Also, check that your query finds only the Auto Scaling groups that are running the specific application. For more information about the search expression syntax, see [CloudWatch search expression syntax](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/search-expression-syntax.html) in the *Amazon CloudWatch User Guide*.
+ If you did not validate an expression in advance, the [put-scaling-policy](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/autoscaling/put-scaling-policy.html) command validates it when you create your scaling policy. However, there is a possibility that this command might fail to identify the exact cause of the detected errors. To fix the issues, troubleshoot the errors that you receive in a response from a request to the [get-metric-data](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/cloudwatch/get-metric-data.html) command. You can also troubleshoot the expression from the CloudWatch console.
+ When you view your **Load** and **Capacity** graphs in the console, the **Capacity** graph might not show any data. To ensure that the graphs have complete data, make sure that you consistently enable group metrics for your Auto Scaling groups. For more information, see [Enable Auto Scaling group metrics (console)](ec2-auto-scaling-metrics.md#as-enable-group-metrics).
+ The capacity metric specification is only useful for blue/green deployments when you have applications that run in different Auto Scaling groups over their lifetime. This custom metric lets you provide the total capacity of multiple Auto Scaling groups. Predictive scaling uses this to show historical data in the **Capacity** graphs in the console.
+ You must specify `false` for `ReturnData` if `MetricDataQueries` specifies the SEARCH() function on its own without a math function like SUM(). This is because search expressions might return multiple time series, and a metric specification based on an expression can return only one time series.
+ All metrics involved in a search expression should be of the same resolution.
## Limitations
+ You can query data points of up to 10 metrics in one metric specification.
+ For the purposes of this limit, one expression counts as one metric.
# Control which Auto Scaling instances terminate during scale in
Amazon EC2 Auto Scaling uses termination policies to decide the order for terminating instances. You can use a predefined policy or create a custom policy to meet your specific requirements. By using a custom policy or instance scale in protection, you can also prevent your Auto Scaling group from terminating instances that aren't yet ready to terminate.
**Topics**
+ [When Amazon EC2 Auto Scaling uses termination policies](#common-scenarios-termination)
+ [Configure termination policies for Amazon EC2 Auto Scaling](ec2-auto-scaling-termination-policies.md)
+ [Create a custom termination policy with Lambda](lambda-custom-termination-policy.md)
+ [Use instance scale-in protection to control instance termination](ec2-auto-scaling-instance-protection.md)
+ [Design your applications to gracefully handle instance termination](gracefully-handle-instance-termination.md)
## When Amazon EC2 Auto Scaling uses termination policies
The following sections describe the scenarios in which Amazon EC2 Auto Scaling uses termination policies.
**Topics**
+ [Scale in events](#common-scenarios-termination-scale-in)
+ [Instance refresh](#common-scenarios-termination-instance-refreshes)
+ [Availability Zone rebalancing](#common-scenarios-termination-rebalancing)
### Scale in events
A scale in event occurs when there is a new value for the desired capacity of an Auto Scaling group that is lower than the current capacity of the group.
Scale in events occur in the following scenarios:
+ When using dynamic scaling policies and the size of the group decreases as a result of changes in a metric's value
+ When using scheduled scaling and the size of the group decreases as a result of a scheduled action
+ When you manually decrease the size of the group
The following example shows how termination policies work when there is a scale in event.
1. The Auto Scaling group in this example has one instance type, two Availability Zones, and a desired capacity of two instances. It also has a dynamic scaling policy that adds and removes instances when resource utilization increases or decreases. The two instances in this group are distributed across the two Availability Zones as shown in the following diagram.
![\[A basic Auto Scaling group with two instances.\]](http://docs.aws.amazon.com/autoscaling/ec2/userguide/images/termination-policy-default-diagram.png)
1. When the Auto Scaling group scales out, Amazon EC2 Auto Scaling launches a new instance. The Auto Scaling group now has three instances, distributed across the two Availability Zones as shown in the following diagram.
![\[An Auto Scaling group after scaling out by one instance.\]](http://docs.aws.amazon.com/autoscaling/ec2/userguide/images/termination-policy-default-2-diagram.png)
1. When the Auto Scaling group scales in, Amazon EC2 Auto Scaling terminates one of the instances.
1. If you did not assign a specific termination policy to the group, Amazon EC2 Auto Scaling uses the default termination policy. It selects the Availability Zone with two instances, and terminates the instance that was launched from a launch configuration, a different launch template, or the oldest version of the current launch template. If the instances were launched from the same launch template and version, Amazon EC2 Auto Scaling selects the instance that is closest to the next billing hour and terminates it.
![\[An Auto Scaling group after scaling in by one instance.\]](http://docs.aws.amazon.com/autoscaling/ec2/userguide/images/termination-policy-default-3-diagram.png)
### Instance refresh
You can start an instance refresh to update the instances in your Auto Scaling group. During an instance refresh, Amazon EC2 Auto Scaling terminates instances in the group and then launches replacements for the terminated instances. The termination policy for the Auto Scaling group controls which instances are replaced first.
### Availability Zone rebalancing
Amazon EC2 Auto Scaling balances your capacity evenly across the Availability Zones enabled for your Auto Scaling group. This helps reduce the impact of an Availability Zone outage. If the distribution of capacity across Availability Zones becomes out of balance, Amazon EC2 Auto Scaling rebalances the Auto Scaling group by launching instances in the enabled Availability Zones with the fewest instances and terminating instances elsewhere. The termination policy controls which instances are prioritized for termination first.
There are a number of reasons why the distribution of instances across Availability Zones can become out of balance.
Removing instances
If you detach instances from your Auto Scaling group, you put instances on standby, or you explicitly terminate instances and decrement the desired capacity, which prevents replacement instances from launching, the group can become unbalanced. If this occurs, Amazon EC2 Auto Scaling compensates by rebalancing the Availability Zones.
Using different Availability Zones than originally specified
If you expand your Auto Scaling group to include additional Availability Zones, or you change which Availability Zones are used, Amazon EC2 Auto Scaling launches instances in the new Availability Zones and terminates instances in other zones to help ensure that your Auto Scaling group spans Availability Zones evenly.
Availability outage
Availability outages are rare. However, if one Availability Zone becomes unavailable and recovers later, your Auto Scaling group can become unbalanced between Availability Zones. Amazon EC2 Auto Scaling tries to gradually rebalance the group, and rebalancing might terminate instances in other zones.
For example, imagine that you have an Auto Scaling group that has one instance type, two Availability Zones, and a desired capacity of two instances. In a situation where one Availability Zone fails, Amazon EC2 Auto Scaling automatically launches a new instance in the healthy Availability Zone to replace the one in the unhealthy Availability Zone. Then, when the unhealthy Availability Zone returns to a healthy state later on, Amazon EC2 Auto Scaling automatically launches a new instance in this zone, which in turn terminates an instance in the unaffected zone.
**Note**
When rebalancing, Amazon EC2 Auto Scaling launches new instances before terminating the old ones, so that rebalancing does not compromise the performance or availability of your application.
Because Amazon EC2 Auto Scaling attempts to launch new instances before terminating the old ones, being at or near the specified maximum capacity could impede or completely stop rebalancing activities. To avoid this problem, the system can temporarily exceed the specified maximum capacity of a group by a 10 percent margin (or by a margin of one instance, whichever is greater) during a rebalancing activity. The margin is extended only if the group is at or near maximum capacity and needs rebalancing, either because of user-requested rezoning or to compensate for zone availability issues. The extension lasts only as long as needed to rebalance the group.
# Configure termination policies for Amazon EC2 Auto Scaling
A termination policy provides the criteria that Amazon EC2 Auto Scaling follows to terminate instances in a specific order. By default, Amazon EC2 Auto Scaling uses a termination policy that's designed to terminate instances that are using outdated configurations first. You can change the termination policy to control which instances are most important to terminate first.
When Amazon EC2 Auto Scaling terminates instances, it tries to maintain balance across the Availability Zones that are enabled for your Auto Scaling group. Maintaining Zonal balance takes precedence over the termination policy. If one Availability Zone has more instances than others, Amazon EC2 Auto Scaling applies the termination policy to the imbalanced zone first. If the Availability Zones are balanced, it applies the termination policy across all Zones.
**Topics**
+ [How the default termination policy works](#default-termination-policy)
+ [Default termination policy and mixed instances groups](#default-termination-policy-mixed-instances-groups)
+ [Predefined termination policies](#predefined-termination-policies)
+ [Change the termination policy for an Auto Scaling group](custom-termination-policy.md)
**Note**
Amazon EC2 Auto Scaling applies termination policies only to instances that are not considered unhealthy by the Auto Scaling group. As a result, instances marked as unhealthy by Auto Scaling health checks will bypass termination policy evaluation.
For more information, see [Design your applications to gracefully handle instance termination](gracefully-handle-instance-termination.md).
## How the default termination policy works
When Amazon EC2 Auto Scaling needs to terminate an instance, it first identifies which Availability Zone (or Zones) has the most instances and at least one instance that is not protected from scale in. Then, it proceeds to evaluate unprotected instances within the identified Availability Zone as follows:
**Instances that use outdated configurations**
+ **For groups that use a launch template** – Determine whether any of the instances use outdated configurations, prioritizing in this order:
1. First, check for instances launched with a launch configuration.
1. Then, check for instances launched using a different launch template instead of the current launch template.
1. Finally, check for instances using the oldest version of the current launch template.
+ **For groups that use a launch configuration** – Determine whether any of the instances use the oldest launch configuration.
If no instances with outdated configurations are found, or there are multiple instances to choose from, Amazon EC2 Auto Scaling considers the next criteria of instances approaching their next billing hour.
**Instances approaching next billing hour**
Determine whether any of the instances that meet the previous criteria are closest to the next billing hour. If multiple instances are equally close, terminate one at random. This helps you maximize the use of your instances that are billed hourly. However, most EC2 usage is now billed per second, so this optimization provides less benefit. For more information, see [Amazon EC2 pricing](https://aws.amazon.com/ec2/pricing/).
The following flow diagram illustrates how the default termination policy works for groups that use a launch template.
![\[A flowchart showing how an Auto Scaling group uses the default termination policy to terminate instances.\]](http://docs.aws.amazon.com/autoscaling/ec2/userguide/images/termination-policy-default-flowchart-diagram.png)
## Default termination policy and mixed instances groups
Amazon EC2 Auto Scaling applies additional criteria when terminating instances in [mixed instances groups](ec2-auto-scaling-mixed-instances-groups.md).
When Amazon EC2 Auto Scaling needs to terminate an instance, it first identifies which purchase option (Spot or On-Demand) should be terminated based on the settings of the group. This makes sure that the group trends toward the specified ratio of Spot and On-Demand instances over time.
It then applies the termination policy independently within each Availability Zone. It determines which Spot or On-Demand Instance in which Availability Zone to terminate to keep the Availability Zones balanced. The same logic applies to a mixed instances group with weights defined for the instance types.
Within each zone, the default termination policy works as follows to determine which unprotected instance within the identified purchase option can be terminated:
1. Determine whether any of the instances can be terminated to improve alignment with the specified [allocation strategy](allocation-strategies.md) for the Auto Scaling group. If no instances are identified for optimization, or there are multiple instances to choose from, the evaluation continues.
1. Determine whether any of the instances use outdated configurations, prioritizing in this order:
1. First, check for instances launched with a launch configuration.
1. Then, check for instances launched using a different launch template instead of the current launch template.
1. Finally, check for instances using the oldest version of the current launch template.
If no instances with outdated configurations are found, or there are multiple instances to choose from, the evaluation continues.
1. Determine whether any of the instances are closest to the next billing hour. If multiple instances are equally close, choose one at random.
## Predefined termination policies
You choose from the following predefined termination policies:
+ **`Default`** – Terminate instances according to the default termination policy.
+ **`AllocationStrategy`** – Terminate instances in the Auto Scaling group to align the remaining instances to the allocation strategy for the type of instance that is terminating (either a Spot Instance or an On-Demand Instance). This policy is useful when your preferred instance types have changed. If the Spot allocation strategy is `lowest-price`, you can gradually rebalance the distribution of Spot Instances across your N lowest priced Spot pools. If the Spot allocation strategy is `capacity-optimized`, you can gradually rebalance the distribution of Spot Instances across Spot pools where there is more available Spot capacity. You can also gradually replace On-Demand Instances of a lower priority type with On-Demand Instances of a higher priority type.
+ **`OldestLaunchTemplate`** – Terminate instances that have the oldest launch template. With this policy, instances that use the noncurrent launch template are terminated first, followed by instances that use the oldest version of the current launch template. This policy is useful when you're updating a group and phasing out the instances from a previous configuration.
+ **`OldestLaunchConfiguration`** – Terminate instances that have the oldest launch configuration. This policy is useful when you're updating a group and phasing out the instances from a previous configuration. With this policy, instances that use the noncurrent launch configuration are terminated first.
+ **`ClosestToNextInstanceHour`** – Terminate instances that are closest to the next billing hour. This policy helps you maximize the use of your instances that have an hourly charge.
+ **`NewestInstance`** – Terminate the newest instance in the group. This policy is useful when you're testing a new launch configuration but don't want to keep it in production.
+ **`OldestInstance`** – Terminate the oldest instance in the group. This option is useful when you're upgrading the instances in the Auto Scaling group to a new EC2 instance type. You can gradually replace instances of the old type with instances of the new type.
**Note**
Amazon EC2 Auto Scaling always balances instances across Availability Zones first, regardless of which termination policy is used. As a result, you might encounter situations in which some newer instances are terminated before older instances. For example, when there is a more recently added Availability Zone, or when one Availability Zone has more instances than the other Availability Zones that are used by the group.
# Change the termination policy for an Auto Scaling group
To change the termination policy for your Auto Scaling group, use one of the following methods.
------
#### [ Console ]
You can't change the termination policy when you initially create an Auto Scaling group in the Amazon EC2 Auto Scaling console. The default termination policy is used automatically. After your Auto Scaling group has been created, you can replace the default policy with a different termination policy or multiple termination policies listed in the order in which they should apply.
**To change the termination policy for an Auto Scaling group**
1. Open the Amazon EC2 console at [https://console.aws.amazon.com/ec2/](https://console.aws.amazon.com/ec2/), and choose **Auto Scaling Groups** from the navigation pane.
1. Select the check box next to the Auto Scaling group.
A split pane opens up in the bottom of the page.
1. On the **Details** tab, choose **Advanced configurations**, **Edit**.
1. For **Termination policies**, choose one or more termination policies. If you choose multiple policies, put them in the order that you want them evaluated in.
You can optionally choose **Custom termination policy** and then choose a Lambda function that meets your needs. If you have created versions and aliases for your Lambda function, you can choose a version or alias from the **Version/Alias** drop-down. To use the unpublished version of your Lambda function, keep **Version/Alias** set to its default. For more information, see [Create a custom termination policy with Lambda](lambda-custom-termination-policy.md).
**Note**
When using multiple policies, their order must be set correctly:
If you use the **Default** policy, it must be the last policy in the list.
If you use a **Custom termination policy**, it must be the first policy in the list.
1. Choose **Update**.
------
#### [ AWS CLI ]
The default termination policy is used automatically unless a different policy is specified.
**To change the termination policy for an Auto Scaling group**
Use one of the following commands:
+ [create-auto-scaling-group](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/autoscaling/create-auto-scaling-group.html)
+ [update-auto-scaling-group](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/autoscaling/update-auto-scaling-group.html)
You can use termination policies individually, or combine them into a list of policies. For example, use the following command to update an Auto Scaling group to use the `OldestLaunchConfiguration` policy first and then use the `ClosestToNextInstanceHour` policy.
```
aws autoscaling update-auto-scaling-group --auto-scaling-group-name my-asg --termination-policies "OldestLaunchConfiguration" "ClosestToNextInstanceHour"
```
If you use the `Default` termination policy, make it the last one in the list of termination policies. For example, `--termination-policies "OldestLaunchConfiguration" "Default"`.
To use a custom termination policy, you must first create your termination policy using AWS Lambda. To specify the Lambda function to use as your termination policy, make it the first one in the list of termination policies. For example, `--termination-policies "arn:aws:lambda:us-west-2:123456789012:function:HelloFunction:prod" "OldestLaunchConfiguration"`. For more information, see [Create a custom termination policy with Lambda](lambda-custom-termination-policy.md).
------
# Create a custom termination policy with Lambda
Amazon EC2 Auto Scaling uses termination policies to prioritize which instances to terminate first when decreasing the size of your Auto Scaling group (referred to as *scaling in*). Your Auto Scaling group uses a default termination policy, but you can optionally choose or create your own termination policies. For more information about choosing a predefined termination policy, see [Configure termination policies for Amazon EC2 Auto Scaling](ec2-auto-scaling-termination-policies.md).
In this topic, you learn how to create a custom termination policy using an AWS Lambda function that Amazon EC2 Auto Scaling invokes in response to certain events. The Lambda function that you create processes the information in the input data sent by Amazon EC2 Auto Scaling and returns a list of instances that are ready to terminate.
A custom termination policy provides better control over which instances are terminated, and when. For example, when your Auto Scaling group scales in, Amazon EC2 Auto Scaling cannot determine whether there are workloads running that should not be disrupted. With a Lambda function, you can validate the termination request and wait until the workload is done before returning the instance ID to Amazon EC2 Auto Scaling for termination.
**Topics**
+ [Input data](#lambda-custom-termination-policy-input-data)
+ [Response data](#lambda-custom-termination-policy-response-data)
+ [Considerations](#lambda-termination-policy-considerations)
+ [Create the Lambda function](#lambda-custom-termination-policy-create-function)
+ [Limitations](#lambda-custom-termination-policy-limitations)
## Input data
Amazon EC2 Auto Scaling generates a JSON payload for scale in events, and also does so when instances are about to be terminated as a result of the maximum instance lifetime or instance refresh features. It also generates a JSON payload for the scale in events that it can initiate when rebalancing your group across Availability Zones.
This payload contains information about the capacity Amazon EC2 Auto Scaling needs to terminate, a list of instances that it suggests for termination, and the event that initiated the termination.
The following is an example payload:
```
{
"AutoScalingGroupARN": "arn:aws:autoscaling:us-east-1::autoScalingGroup:d4738357-2d40-4038-ae7e-b00ae0227003:autoScalingGroupName/my-asg",
"AutoScalingGroupName": "my-asg",
"CapacityToTerminate": [
{
"AvailabilityZone": "us-east-1b",
"Capacity": 2,
"InstanceMarketOption": "on-demand"
},
{
"AvailabilityZone": "us-east-1b",
"Capacity": 1,
"InstanceMarketOption": "spot"
},
{
"AvailabilityZone": "us-east-1c",
"Capacity": 3,
"InstanceMarketOption": "on-demand"
}
],
"Instances": [
{
"AvailabilityZone": "us-east-1b",
"InstanceId": "i-0056faf8da3e1f75d",
"InstanceType": "t2.nano",
"InstanceMarketOption": "on-demand"
},
{
"AvailabilityZone": "us-east-1c",
"InstanceId": "i-02e1c69383a3ed501",
"InstanceType": "t2.nano",
"InstanceMarketOption": "on-demand"
},
{
"AvailabilityZone": "us-east-1c",
"InstanceId": "i-036bc44b6092c01c7",
"InstanceType": "t2.nano",
"InstanceMarketOption": "on-demand"
},
...
],
"Cause": "SCALE_IN"
}
```
The payload includes the name of the Auto Scaling group, its Amazon Resource Name (ARN), and the following elements:
+ `CapacityToTerminate` describes how much of your Spot or On-Demand capacity is set to be terminated in a given Availability Zone.
+ `Instances` represents the instances that Amazon EC2 Auto Scaling suggests for termination based on the information in `CapacityToTerminate`.
+ `Cause` describes the event that caused the termination: `SCALE_IN`, `INSTANCE_REFRESH`, `MAX_INSTANCE_LIFETIME`, or `REBALANCE`.
The following information outlines the most significant factors in how Amazon EC2 Auto Scaling generates the `Instances` in the input data:
+ Maintaining balance across Availability Zones takes precedence when an instance is terminating due to scale in events and instance refresh-based terminations. Therefore, if one Availability Zone has more instances than the other Availability Zones that are used by the group, the input data contains instances that are eligible for termination only from the imbalanced Availability Zone. If the Availability Zones used by the group are balanced, the input data contains instances from all of the Availability Zones for the group.
+ When using a [mixed instances policy](ec2-auto-scaling-mixed-instances-groups.md), maintaining your Spot and On-Demand capacities in balance based on your desired percentages for each purchase option also takes precedence. We first identify which of the two types (Spot or On-Demand) should be terminated. We then identify which instances (within the identified purchase option) in which Availability Zones we can terminate that will result in the Availability Zones being most balanced.
## Response data
The input data and response data work together to narrow down the list of instances to terminate.
With the given input, the response from your Lambda function should look like the following example:
```
{
"InstanceIDs": [
"i-02e1c69383a3ed501",
"i-036bc44b6092c01c7",
...
]
}
```
The `InstanceIDs` in the response represent the instances that are ready to terminate.
Alternatively, you can return a different set of instances that are ready to be terminated, which overrides the instances in the input data. If no instances are ready to terminate when your Lambda function is invoked, you can also choose not to return any instances.
When no instances are ready to terminate, the response from your Lambda function should look like the following example:
```
{
"InstanceIDs": [ ]
}
```
## Considerations
Note the following considerations when using a custom termination policy:
+ Returning an instance first in the response data does not guarantee its termination. If more than the required number of instances are returned when your Lambda function is invoked, Amazon EC2 Auto Scaling evaluates each instance against the other termination policies that you specified for your Auto Scaling group. When there are multiple termination policies, it tries to apply the next termination policy in the list, and if there are more instances than are required to terminate, it moves on to the next termination policy, and so on. If no other termination policies are specified, then the default termination policy is used to determine which instances to terminate.
+ If no instances are returned or your Lambda function times out, then Amazon EC2 Auto Scaling waits a short time before invoking your function again. For any scale in event, it keeps trying as long as the group's desired capacity is less than its current capacity. For instance refresh-based terminations, it keeps trying for an hour. After that, if it continues to fail to terminate any instances, the instance refresh operation fails. With maximum instance lifetime, Amazon EC2 Auto Scaling keeps trying to terminate the instance that is identified as exceeding its maximum lifetime.
+ Because your function is retried repeatedly, make sure to test and fix any permanent errors in your code before using a Lambda function as a custom termination policy.
+ If you override the input data with your own list of instances to terminate, and terminating these instances puts the Availability Zones out of balance, Amazon EC2 Auto Scaling gradually rebalances the distribution of capacity across Availability Zones. First, it invokes your Lambda function to see if there are instances that are ready to be terminated so that it can determine whether to start rebalancing. If there are instances ready to be terminated, it launches new instances first. When the instances finish launching, it then detects that your group's current capacity is higher than its desired capacity and initiates a scale in event.
+ A custom termination policy does not affect your ability to also use scale in protection to protect certain instances from being terminated. For more information, see [Use instance scale-in protection to control instance termination](ec2-auto-scaling-instance-protection.md).
+ If your Lambda function returns no instances, this does not automatically prevent all termination. Amazon EC2 Auto Scaling will still terminate instances that are considered unhealthy by the Auto Scaling health checks, regardless of termination policies.
## Create the Lambda function
Start by creating the Lambda function, so that you can specify its Amazon Resource Name (ARN) in the termination policies for your Auto Scaling group.
**To create a Lambda function (console)**
1. Open the [Functions page](https://console.aws.amazon.com/lambda/home#/functions) on the Lambda console.
1. On the navigation bar at the top of the screen, choose the same Region that you used when you created the Auto Scaling group.
1. Choose **Create function**, **Author from scratch**.
1. Under **Basic information**, for **Function name**, enter the name of your function.
1. Choose **Create function**. You are returned to the function's code and configuration.
1. With your function still open in the console, under **Function code**, paste your code into the editor.
1. Choose **Deploy**.
1. Optionally, create a published version of the Lambda function by choosing the **Versions** tab and then **Publish new version**. To learn more about versioning in Lambda, see [Lambda function versions](https://docs.aws.amazon.com/lambda/latest/dg/configuration-versions.html) in the *AWS Lambda Developer Guide*.
1. If you chose to publish a version, choose the **Aliases** tab if you want to associate an alias with this version of the Lambda function. To learn more about aliases in Lambda, see [Lambda function aliases](https://docs.aws.amazon.com/lambda/latest/dg/configuration-aliases.html) in the *AWS Lambda Developer Guide*
1. Next, choose the **Configuration** tab and then **Permissions**.
1. Scroll down to **Resource-based policy** and then choose **Add permissions**. A resource-based policy is used to grant permissions to invoke your function to the principal that is specified in the policy. In this case, the principal will be the [Amazon EC2 Auto Scaling service-linked role](https://docs.aws.amazon.com/autoscaling/ec2/userguide/autoscaling-service-linked-role.html) that is associated with the Auto Scaling group.
1. In the **Policy statement** section, configure your permissions:
1. Choose **AWS account**.
1. For **Principal**, enter the ARN of the calling service-linked role, for example, **arn:aws:iam:::role/aws-service-role/autoscaling.amazonaws.com/AWSServiceRoleForAutoScaling**.
1. For **Action**, choose **lambda:InvokeFunction**.
1. For **Statement ID**, enter a unique statement ID, such as **AllowInvokeByAutoScaling**.
1. Choose **Save**.
1. After you have followed these instructions, continue on to specify the ARN of your function in the termination policies for your Auto Scaling group as a next step. For more information, see [Change the termination policy for an Auto Scaling group](custom-termination-policy.md).
**Note**
For examples that you can use as a reference for developing your Lambda function, see the [GitHub repository](https://github.com/aws-samples/amazon-ec2-auto-scaling-group-examples) for Amazon EC2 Auto Scaling.
## Limitations
+ You can only specify one Lambda function in the termination policies for an Auto Scaling group. If there are multiple termination policies specified, the Lambda function must be specified first.
+ You can reference your Lambda function using either an unqualified ARN (without a suffix) or a qualified ARN that has either a version or an alias as its suffix. If an unqualified ARN is used (for example, `function:my-function`), your resource-based policy must be created on the unpublished version of your function. If a qualified ARN is used (for example, `function:my-function:1` or `function:my-function:prod`), your resource-based policy must be created on that specific published version of your function.
+ You cannot use a qualified ARN with the `$LATEST` suffix. If you try to add a custom termination policy that refers to a qualified ARN with the `$LATEST` suffix, it will result in an error.
+ The number of instances provided in the input data is limited to 30,000 instances. If there are more than 30,000 instances that could be terminated, the input data includes `"HasMoreInstances": true` to indicate that the maximum number of instances are returned.
+ The maximum run time for your Lambda function is two seconds (2000 milliseconds). As a best practice, you should set the timeout value of your Lambda function based on your expected run time. Lambda functions have a default timeout of three seconds, but this can be decreased.
+ If your runtime exceeds the 2-second limit, any scale in action will be on hold until the runtime falls below this threshold. For Lambda functions with consistently longer runtimes, find a way to reduce the runtime, such as by caching the results where they can be retrieved during subsequent Lambda invocations.
+ Cross-account Lambda functions aren't supported. The Lambda function used as a custom termination policy must be in the same AWS account as the Auto Scaling group. The setup instructions on this page specifically focus on same-account configurations.
+ Custom termination policies do not apply to unhealthy instances. Unhealthy instances are always considered eligible for termination, and the Lambda function only evaluates the termination order for the remaining instances.
# Use instance scale-in protection to control instance termination
Instance scale-in protection gives you control over which instances Amazon EC2 Auto Scaling can terminate. A common use case for this feature is scaling container-based workloads. For more information, see [Design your applications to gracefully handle instance termination](gracefully-handle-instance-termination.md).
By default, instance scale-in protection is disabled when you create an Auto Scaling group. This means that Amazon EC2 Auto Scaling can terminate any instance in the group.
You can protect instances as soon as they launch by enabling the instance scale-in protection setting on your Auto Scaling group. Instance scale-in protection starts when the instance state is `InService`. Then, to control which instances can terminate, disable the scale-in protection setting on individual instances within the Auto Scaling group. By doing so, you can continue to protect certain instances from unwanted terminations.
**Topics**
+ [Considerations](#instance-protection-considerations)
+ [Change scale-in protection for an Auto Scaling group](#instance-protection-group)
+ [Change scale-in protection for an instance](#instance-protection-instance)
## Considerations
The following are considerations when using instance scale-in protection:
+ If all instances in an Auto Scaling group are protected from scale in, and a scale in event occurs, its desired capacity is decremented. However, the Auto Scaling group can't terminate the required number of instances until their instance scale in protection settings are disabled. In the AWS Management Console, the **Activity history** for the Auto Scaling group includes the following message if all instances in an Auto Scaling group are protected from scale in when a scale in event occurs: `Could not scale to desired capacity because all remaining instances are protected from scale in.`
+ If you detach an instance that is protected from scale in, its instance scale in protection setting is lost. When you attach the instance to the group again, it inherits the current instance scale in protection setting of the group. When Amazon EC2 Auto Scaling launches a new instance or moves an instance from a warm pool into the Auto Scaling group, the instance inherits the instance scale in protection setting of the Auto Scaling group.
+ Instance scale-in protection does not protect Auto Scaling instances from the following:
+ Health check replacement if the instance fails health checks. For more information, see [Health checks for instances in an Auto Scaling group](ec2-auto-scaling-health-checks.md).
+ Spot Instance interruptions. A Spot Instance is terminated when capacity is no longer available or the Spot price exceeds your maximum price.
+ A Capacity Block reservation ends. Amazon EC2 reclaims the Capacity Block instances even if they are protected from scale in.
+ Manual termination through the `terminate-instance-in-auto-scaling-group` command. For more information, see [Terminate an instance in your Auto Scaling group (AWS CLI)](ec2-auto-scaling-scaling-manually.md#terminate-an-instance-aws-cli).
+ Manual termination through the Amazon EC2 console, CLI commands, and API operations. To protect Auto Scaling instances from manual termination, enable Amazon EC2 termination protection. (This does not prevent Amazon EC2 Auto Scaling from terminating instances or manual termination through the `terminate-instance-in-auto-scaling-group` command.) For information about enabling Amazon EC2 termination protection in a launch template, see [Create a launch template using advanced settings](advanced-settings-for-your-launch-template.md).
## Change scale-in protection for an Auto Scaling group
You can enable or disable the instance scale-in protection setting for an Auto Scaling group. When you enable it, all new instances launched by the group will have instance scale-in protection enabled.
Enabling or disabling this setting for an Auto Scaling group does not affect existing instances.
------
#### [ Console ]
**To enable scale-in protection for a new Auto Scaling group**
When you create the Auto Scaling group, on the **Configure group size and scaling policies** page, under **Instance scale-in protection**, select the **Enable instance scale-in protection** check box.
**To enable or disable scale-in protection for an existing group**
1. Open the Amazon EC2 console at [https://console.aws.amazon.com/ec2/](https://console.aws.amazon.com/ec2/), and choose **Auto Scaling Groups** from the navigation pane.
1. Select check box next to the Auto Scaling group.
A split pane opens up in the bottom of the page.
1. On the **Details** tab, choose **Advanced configurations**, **Edit**.
1. For **Instance scale-in protection**, select or clear the **Enable instance-scale protection** check box to enable or disable this option as required.
1. Choose **Update**.
------
#### [ AWS CLI ]
**To enable scale-in protection for a new Auto Scaling group**
Use the following [create-auto-scaling-group](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/autoscaling/create-auto-scaling-group.html) command to enable instance scale-in protection.
```
aws autoscaling create-auto-scaling-group --auto-scaling-group-name my-asg --new-instances-protected-from-scale-in ...
```
**To enable scale-in protection for an existing group**
Use the following [update-auto-scaling-group](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/autoscaling/update-auto-scaling-group.html) command to enable instance scale-in protection for the specified Auto Scaling group.
```
aws autoscaling update-auto-scaling-group --auto-scaling-group-name my-asg --new-instances-protected-from-scale-in
```
**To disable scale-in protection for an existing group**
Use the following command to disable instance scale-in protection for the specified group.
```
aws autoscaling update-auto-scaling-group --auto-scaling-group-name my-asg --no-new-instances-protected-from-scale-in
```
------
## Change scale-in protection for an instance
By default, an instance gets its instance scale-in protection setting from its Auto Scaling group. However, you can enable or disable instance scale-in protection for individual instances after they launch.
------
#### [ Console ]
**To enable or disable scale-in protection for an instance**
1. Open the Amazon EC2 console at [https://console.aws.amazon.com/ec2/](https://console.aws.amazon.com/ec2/), and choose **Auto Scaling Groups** from the navigation pane.
1. Select the check box next to your Auto Scaling group.
A split pane opens up in the bottom of the page.
1. On the **Instance management** tab, in **Instances**, select an instance.
1. To enable instance scale-in protection, choose **Actions**, **Set scale-in protection**. When prompted, choose **Set scale-in protection**.
1. To disable instance scale-in protection, choose **Actions**, **Remove scale-in protection**. When prompted, choose **Remove scale-in protection**.
------
#### [ AWS CLI ]
**To enable scale-in protection for an instance**
Use the following [set-instance-protection](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/autoscaling/set-instance-protection.html) command to enable instance scale-in protection for the specified instance.
```
aws autoscaling set-instance-protection --instance-ids i-5f2e8a0d --auto-scaling-group-name my-asg --protected-from-scale-in
```
**To disable scale-in protection for an instance**
Use the following command to disable instance scale-in protection for the specified instance.
```
aws autoscaling set-instance-protection --instance-ids i-5f2e8a0d --auto-scaling-group-name my-asg --no-protected-from-scale-in
```
------
**Note**
Remember, instance scale-in protection does not guarantee that instances won't be terminated in the event of a human error—for example, if someone manually terminates an instance using the Amazon EC2 console or AWS CLI. To protect your instance from accidental termination, you can use Amazon EC2 termination protection. However, even with termination protection and instance scale-in protection enabled, data saved to instance storage can be lost if a health check determines that an instance is unhealthy or if the group itself is accidentally deleted. As with any environment, a best practice is to back up your data frequently, or whenever it's appropriate for your business continuity requirements.
# Design your applications to gracefully handle instance termination
This topic describes features that you can use to prevent your Amazon EC2 Auto Scaling group from terminating Amazon EC2 instances that aren't yet ready to terminate. By default, Auto Scaling has no visibility into the applications running on your instances. It can terminate instances before your application is able to gracefully shut down or complete its assigned jobs. These features give your application time to complete in-progress work, transfer state, or perform cleanup before instance terminations. You can use them individually or in combination depending on your application's requirements.
These features are particularly useful for stateful workloads, where each instance in your fleet holds different data, jobs, or state than other instances. Terminating stateful instances without graceful shutdown could result in long-running jobs restarting from the beginning, reduced data redundancy or data loss, and interrupted in-progress transactions or computations. To gracefully shut down a stateful instance, its workload should be either drained (completing all currently assigned jobs) or transferred (moving jobs, data, or configuration to another active instance).
**Topics**
+ [Termination lifecycle hooks](#gracefully-handle-instance-termination-lifecycle-hooks)
+ [Instance scale-in protection](#gracefully-handle-instance-termination-scale-in-protection)
+ [Custom termination policy](#gracefully-handle-instance-termination-custom-termination-policy)
+ [Instance lifecycle policy](#gracefully-handle-instance-termination-instance-lifecycle-policy)
+ [Suspend terminations altogether](#gracefully-handle-instance-termination-suspend-terminate)
+ [Limitations](#gracefully-handle-instance-termination-limitations)
+ [Example scenarios](#gracefully-handle-instance-termination-examples)
## Termination lifecycle hooks
A termination lifecycle hook extends the life of your Amazon EC2 instance that's already selected for termination. It provides extra time to complete in-progress work currently assigned to the instance, or to save progress and transfer the work to another instance.
For many workloads, a termination lifecycle hook may be enough to gracefully shut down an application on an instance that's selected for termination. This is a best-effort approach and can't be used to prevent termination if a termination lifecycle action is abandoned. If your workload has a low tolerance for termination lifecycle action failures, configure [ instance lifecycle policies ](https://docs.aws.amazon.com/autoscaling/ec2/userguide/instance-lifecycle-policy.html) in combination with your termination lifecycle hooks to retain instances.
To use a termination lifecycle hook, you need to know when an instance is selected for termination. You have two ways to know this:
| Option | Description | Best used for | Link to documentation |
| --- | --- | --- | --- |
| Inside the instance | The Instance Metadata Service (IMDS) is a secure endpoint that you can poll for the status of an instance directly from the instance. If the metadata comes back with Terminated, then your instance is scheduled to be terminated. | Applications where you must perform an action on the instance before the instance is terminated. | [ Retrieve the target lifecycle state ](https://docs.aws.amazon.com/autoscaling/ec2/userguide/retrieving-target-lifecycle-state-through-imds.html) |
| Outside the instance | When an instance is terminating, an event notification is generated. You can create rules using Amazon EventBridge, Amazon SQS, Amazon SNS, or AWS Lambda to capture these events, and invoke a response such as with a Lambda function. | Applications that need to take action outside of the instance. | [ Configure a notification target ](https://docs.aws.amazon.com/autoscaling/ec2/userguide/prepare-for-lifecycle-notifications.html#lifecycle-hook-notification-target) |
To use a lifecycle hook, you also need to know when your instance is ready to be fully terminated. Amazon EC2 Auto Scaling will not terminate the instance until it receives a [ CompleteLifecycleAction ](https://docs.aws.amazon.com/autoscaling/ec2/APIReference/API_CompleteLifecycleAction.html) API call or the timeout elapses, whichever happens first.
By default, an instance can continue running for one hour (heartbeat timeout) due to a termination lifecycle hook. You can configure the default timeout if one hour is not enough time to complete the lifecycle action. When a lifecycle action is in progress, you can extend the timeout with [ RecordLifecycleActionHeartbeat ](https://docs.aws.amazon.com/autoscaling/ec2/APIReference/API_RecordLifecycleActionHeartbeat.html) API calls.
For more information, see [Amazon EC2 Auto Scaling lifecycle hooks](lifecycle-hooks.md).
## Instance scale-in protection
You can use instance scale-in protection to control which instances are selected for termination during scale-in events, especially to prevent an instance that is actively processing a long-running job from being terminated. For example, when running containerized workloads, it's common to want to protect all instances and remove protection only for instances with no current or scheduled tasks. Instances can continue polling for new jobs and re-enable protection when there are new jobs assigned.
You can enable scale-in protection at the Auto Scaling group level and instance level. When you enable scale-in protection at the Auto Scaling group level, only new instances are protected when they're created. For existing instances, you can enable protection individually.
Applications can set protection either from the instances themselves, or from a centralized control plane that manages whether each instance is terminable. We recommend the centralized approach for large fleets or when protection needs to be toggled frequently, as it allows you to batch calls to [ SetInstanceProtection ](https://docs.aws.amazon.com/autoscaling/ec2/APIReference/API_SetInstanceProtection.html) and avoid API throttling issues.
For more information, see [Use instance scale-in protection to control instance termination](ec2-auto-scaling-instance-protection.md).
## Custom termination policy
Like instance scale-in protection, a custom termination policy helps you prevent your Amazon EC2 Auto Scaling group from terminating specific EC2 instances. Unhealthy instances can still be terminated regardless of your custom termination policy.
Your Auto Scaling group uses a default termination policy to determine which Amazon EC2 instances it terminates first. If you want more control over which instances terminate first, you can implement a custom termination policy using a Lambda function. Auto Scaling calls this function whenever it needs to select an instance for termination, and will only terminate instances that the function returns. If the function errors, times out, or returns an empty list, Auto Scaling doesn't terminate any instances unless the instance is unhealthy.
A custom termination policy is useful when your application can identify which instances are idle or safe to terminate. This typically requires a control plane that tracks workload across the group.
For more information, see [Create a custom termination policy with Lambda](lambda-custom-termination-policy.md).
## Instance lifecycle policy
Instance lifecycle policies provide protection against Amazon EC2 Auto Scaling terminations when a termination lifecycle action is abandoned. Unlike lifecycle hooks alone, instance lifecycle policies are designed to ensure that instances move to a retained state when graceful shutdown procedures don't complete successfully.
When Auto Scaling selects an instance for termination, your configured termination lifecycle hooks are invoked and your application begins graceful shutdown procedures. If the termination lifecycle actions complete successfully with `CONTINUE`, the instance terminates normally. However, if a termination lifecycle action is abandoned for any reason, the instance lifecycle policy moves the instance to a retained state rather than terminating it. Retained instances don't count toward your Auto Scaling group's desired capacity, so replacement instances are launched automatically. You will incur standard Amazon EC2 charges for both the retained instance and its replacement until you manually terminate the retained instance using the [ TerminateInstanceInAutoScalingGroup ](https://docs.aws.amazon.com/autoscaling/ec2/APIReference/API_TerminateInstanceInAutoScalingGroup.html) API.
To use this feature, you must configure both an instance lifecycle policy with the `TerminateHookAbandon` retention trigger set to `retain`, as well as at least one termination lifecycle hook. Because retained instances incur ongoing Amazon EC2 costs and require manual action, monitoring is critical. You should enable CloudWatch metrics like `GroupTerminatingRetainedInstances` and create CloudWatch alarms to alert you when instances enter retained states.
For more information, see [Control instance retention with instance lifecycle policies](instance-lifecycle-policy.md).
## Suspend terminations altogether
If you require complete control over all EC2 instance terminations within your Amazon EC2 Auto Scaling group, suspend the `Terminate` process. We only recommend using this option if the above options do not offer you the control you need for your service. By calling [ SuspendProcesses ](https://docs.aws.amazon.com/autoscaling/ec2/APIReference/API_SuspendProcesses.html) to suspend the `Terminate` process, you prevent Auto Scaling from attempting terminations for any reason, except those initiated by a user request to the [ TerminateInstanceInAutoScalingGroup ](https://docs.aws.amazon.com/autoscaling/ec2/APIReference/API_TerminateInstanceInAutoScalingGroup.html) API.
For more information, see [Suspend and resume Amazon EC2 Auto Scaling processes](as-suspend-resume-processes.md).
## Limitations
**Important**
When designing your application on Amazon EC2 Auto Scaling to gracefully handle instance terminations, keep the following limitations in mind.
### Unhealthy instances bypass some protections
If an instance is unhealthy, Amazon EC2 Auto Scaling will start terminating it even if you have custom termination policies or scale-in protection in place. The only way to prevent replacement of unhealthy instances by Auto Scaling is to suspend the `HealthCheck`, `ReplaceUnhealthy`, or `Terminate` process. You can use lifecycle hooks and an instance lifecycle policy to allow the application to shut down gracefully or copy any data that you need to recover before the unhealthy instance is terminated.
| Feature | Controls healthy instances | Controls unhealthy instances |
| --- | --- | --- |
| Custom termination policies | ![\[alt text not found\]](http://docs.aws.amazon.com/autoscaling/ec2/userguide/images/success_icon.png) Yes | ![\[alt text not found\]](http://docs.aws.amazon.com/autoscaling/ec2/userguide/images/negative_icon.png) No |
| Scale-in protection | ![\[alt text not found\]](http://docs.aws.amazon.com/autoscaling/ec2/userguide/images/success_icon.png) Yes | ![\[alt text not found\]](http://docs.aws.amazon.com/autoscaling/ec2/userguide/images/negative_icon.png) No |
| Suspend HealthCheck, ReplaceUnhealthy, or Terminate process | ![\[alt text not found\]](http://docs.aws.amazon.com/autoscaling/ec2/userguide/images/success_icon.png) Yes | ![\[alt text not found\]](http://docs.aws.amazon.com/autoscaling/ec2/userguide/images/success_icon.png) Yes |
| Lifecycle hooks | ![\[alt text not found\]](http://docs.aws.amazon.com/autoscaling/ec2/userguide/images/success_icon.png) Yes | ![\[alt text not found\]](http://docs.aws.amazon.com/autoscaling/ec2/userguide/images/success_icon.png) Yes |
| Instance lifecycle policy | ![\[alt text not found\]](http://docs.aws.amazon.com/autoscaling/ec2/userguide/images/success_icon.png) Yes | ![\[alt text not found\]](http://docs.aws.amazon.com/autoscaling/ec2/userguide/images/success_icon.png) Yes |
### Lifecycle hooks alone do not guarantee graceful shutdown
By default, termination lifecycle hooks operate on a best-effort basis. If a termination lifecycle action is abandoned, Amazon EC2 Auto Scaling proceeds with terminating the instance immediately. You can combine termination lifecycle hooks with an instance lifecycle policy to retain instances when termination lifecycle actions are abandoned. With this combination:
+ Your termination lifecycle hooks attempt to gracefully shut down your application after Auto Scaling triggers an instance termination and the draining from any configured Elastic Load Balancing load balancers complete.
+ If a termination lifecycle action is abandoned for any reason, the instance moves to a retained state instead of being terminated.
+ The retained instance stays in its current Amazon EC2 state, allowing you to manually complete your shutdown procedures or investigate the failure.
+ You can manually terminate retained instances by calling the [ TerminateInstanceInAutoScalingGroup ](https://docs.aws.amazon.com/autoscaling/ec2/APIReference/API_TerminateInstanceInAutoScalingGroup.html) API after completing the necessary actions.
For more information, see [Control instance retention with instance lifecycle policies](instance-lifecycle-policy.md).
### Certain instance market options can be interrupted with limited notice
If you use instance market options such as Spot Instances and interruptible capacity reservations in your Auto Scaling group, Amazon EC2 can interrupt and reclaim your instances at any time. These interruptions bypass all Amazon EC2 Auto Scaling protection mechanisms, including:
+ Termination lifecycle hooks
+ Instance scale-in protection
+ Custom termination policies
+ Instance lifecycle policies
+ Suspended processes
When a Spot Instance receives an interruption notice, you have approximately two minutes to perform graceful shutdown tasks. While you can use termination lifecycle hooks to respond to Spot Instance interruptions, the instance will be forcibly terminated at the end of the two-minute window, even if the lifecycle hook is still in progress. Instance lifecycle policies also can't prevent Spot Instance interruptions.
For more information about handling Spot Instance interruptions, see [ Spot Instance interruptions ](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/spot-interruptions.html) and [ Best practices for Amazon EC2 Spot ](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/spot-best-practices.html) in the *Amazon EC2 User Guide*.
### Direct Amazon EC2 terminations bypass all protections
If you terminate an instance in your Auto Scaling group with the Amazon EC2 [ TerminateInstances ](https://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_TerminateInstances.html) API directly, the termination bypasses all Amazon EC2 Auto Scaling protection mechanisms.
To terminate instances in your Auto Scaling group while respecting your configured protections, use the [ TerminateInstanceInAutoScalingGroup ](https://docs.aws.amazon.com/autoscaling/ec2/APIReference/API_TerminateInstanceInAutoScalingGroup.html) API instead.
## Example scenarios
When you use Amazon EC2 Auto Scaling, you can choose how much fleet management Auto Scaling handles on your behalf versus how much direct control you retain over EC2 instance termination decisions. The more sensitive your workload is to instance terminations, the more control you may want to retain. The following examples describe workloads with different tolerance levels and the recommended configurations:
### Example 1: Distributed database nodes (Low tolerance)
You run a distributed database where each EC2 instance holds a partition of your data with a replication factor of 3. Losing multiple instances that hold replicas of the same partition could cause data loss or make that partition unavailable.
**Challenge:** Auto Scaling might terminate instances faster than data can be re-replicated to other nodes, and terminations might reduce your capacity below what's needed to maintain your replication factor.
Consider the following configurations:
+ Enable [ instance scale-in protection ](https://docs.aws.amazon.com/autoscaling/ec2/userguide/ec2-auto-scaling-instance-protection.html) on all database instances; remove it programmatically only after confirming data is safely replicated elsewhere.
+ Configure [ termination lifecycle hooks ](https://docs.aws.amazon.com/autoscaling/ec2/userguide/lifecycle-hooks.html) with extended timeouts in combination with an instance lifecycle policy to allow data transfer to complete.
+ Set an [ instance maintenance policy ](https://docs.aws.amazon.com/autoscaling/ec2/userguide/ec2-auto-scaling-instance-maintenance-policy.html) with a minimum healthy percentage of 100% to maintain your required capacity.
### Example 2: Long-running job processing (Medium tolerance)
You have an Amazon SQS queue that collects incoming messages for long-running jobs. When a new message arrives, an EC2 instance retrieves the message and starts a job that takes 3 hours to process. As the queue grows, Auto Scaling adds instances based on your scaling policies. As the queue shrinks, Auto Scaling terminates instances.
**Challenge:** Auto Scaling might terminate an instance that is 3 hours into processing a job rather than an idle instance. The job can be restarted on another instance, but you lose significant progress.
Consider the following configurations:
+ Configure a [ custom termination policy ](https://docs.aws.amazon.com/autoscaling/ec2/userguide/lambda-custom-termination-policy.html) that prioritizes terminating idle instances first.
+ Use [ termination lifecycle hooks ](https://docs.aws.amazon.com/autoscaling/ec2/userguide/lifecycle-hooks.html) to allow in-progress jobs to complete.
+ Enable [ instance scale-in protection ](https://docs.aws.amazon.com/autoscaling/ec2/userguide/ec2-auto-scaling-instance-protection.html) programmatically when an instance starts a job, and remove it when the job completes.
### Example 3: Worker fleet for test environments (High tolerance)
You run a fleet of EC2 instances that execute automated tests, CI/CD pipeline jobs, or development workloads. These worker instances pull tasks from a queue, and test results can be regenerated if a job fails.
**Challenge:** Test jobs may be interrupted during scale-in events, but since tests can be retried without impact, you want to optimize for cost and simplicity rather than zero-interruption availability.
Consider the following configurations:
+ Use [ termination lifecycle hooks ](https://docs.aws.amazon.com/autoscaling/ec2/userguide/lifecycle-hooks.html) to allow in-flight requests to complete.
+ Consider using [ Spot Instances ](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-spot-instances.html) with capacity-optimized allocation strategy to further reduce costs.
# Suspend and resume Amazon EC2 Auto Scaling processes
This topic describes how to suspend and then resume one or more of the processes for your Auto Scaling group to temporarily disable certain operations.
Suspending processes can be useful when you need to investigate or troubleshoot an issue without interference from scaling policies or scheduled actions. It also helps prevent Amazon EC2 Auto Scaling from marking instances unhealthy and replacing them while you are making changes to your Auto Scaling group.
**Topics**
+ [Types of processes](#process-types)
+ [Considerations](suspend-resume-considerations.md)
+ [Suspend processes](suspend-processes.md)
+ [Resume processes](resume-processes.md)
+ [How suspended processes affect other processes](understand-how-suspending-processes-affects-other-processes.md)
**Note**
In addition to suspensions that you initiate, Amazon EC2 Auto Scaling can also suspend processes for Auto Scaling groups that repeatedly fail to launch instances. This is known as an *administrative suspension*. An administrative suspension most commonly applies to Auto Scaling groups that have been trying to launch instances for over 24 hours but have not succeeded in launching any instances. You can resume processes that were suspended by Amazon EC2 Auto Scaling for administrative reasons.
## Types of processes
The suspend-resume feature supports the following processes:
+ `Launch` – Adds instances to the Auto Scaling group when the group scales out, or when Amazon EC2 Auto Scaling chooses to launch instances for other reasons, such as when it adds instances to a warm pool.
+ `Terminate` – Removes instances from the Auto Scaling group when the group scales in, or when Amazon EC2 Auto Scaling chooses to terminate instances for other reasons, such as when an instance is terminated for exceeding its maximum lifetime duration or failing a health check.
+ `AddToLoadBalancer` – Adds instances to the attached load balancer target group or Classic Load Balancer when they are launched. For more information, see [Use Elastic Load Balancing to distribute incoming application traffic in your Auto Scaling group](autoscaling-load-balancer.md).
+ `AlarmNotification` – Accepts notifications from CloudWatch alarms that are associated with dynamic scaling policies. For more information, see [Dynamic scaling for Amazon EC2 Auto Scaling](as-scale-based-on-demand.md).
+ `AZRebalance` – Balances the number of EC2 instances in the group evenly across all of the specified Availability Zones when the group becomes unbalanced, for example, when a previously unavailable Availability Zone returns to a healthy state. For more information, see [Rebalancing activities](auto-scaling-benefits.md#AutoScalingBehavior.InstanceUsage).
+ `HealthCheck` – Checks the health of the instances and marks an instance as unhealthy if Amazon EC2 or Elastic Load Balancing tells Amazon EC2 Auto Scaling that the instance is unhealthy. This process can override the health status of an instance that you set manually. For more information, see [Health checks for instances in an Auto Scaling group](ec2-auto-scaling-health-checks.md).
+ `InstanceRefresh` – Terminates and replaces instances using the instance refresh feature. For more information, see [Use an instance refresh to update instances in an Auto Scaling group](asg-instance-refresh.md).
+ `ReplaceUnhealthy` – Terminates instances that are marked as unhealthy and then creates new instances to replace them. For more information, see [Health checks for instances in an Auto Scaling group](ec2-auto-scaling-health-checks.md).
+ `ScheduledActions` – Performs the scheduled scaling actions that you create or that are created for you when you create an AWS Auto Scaling scaling plan and turn on predictive scaling. For more information, see [Scheduled scaling for Amazon EC2 Auto Scaling](ec2-auto-scaling-scheduled-scaling.md).
# Considerations for suspending processes
Consider the following before suspending processes:
+ Suspending `AlarmNotification` allows you to temporarily stop the group's target tracking, step, and simple scaling policies without deleting the scaling policies or their associated CloudWatch alarms. To temporarily stop individual scaling policies instead, see [Disable a scaling policy for an Auto Scaling group](as-enable-disable-scaling-policy.md).
+ You might choose to suspend the `HealthCheck` and `ReplaceUnhealthy` processes to reboot instances without Amazon EC2 Auto Scaling terminating the instances based on its health checks. However, if you need Amazon EC2 Auto Scaling to continue performing health checks on the remaining instances, use the standby feature instead. For more information, see [Temporarily remove instances from your Auto Scaling group](as-enter-exit-standby.md).
+ If you suspend the `Launch` and `Terminate` processes, or `AZRebalance`, and then you make changes to your Auto Scaling group, for example, by detaching instances or changing the Availability Zones that are specified, your group can become unbalanced between Availability Zones. If that happens, after you resume the suspended processes, Amazon EC2 Auto Scaling gradually redistributes instances evenly between the Availability Zones.
+ If you suspend the `Terminate` process, you can still force instances to be terminated by using the [delete-auto-scaling-group](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/autoscaling/delete-auto-scaling-group.html) command with the force delete option.
+ Suspending the `Terminate` process applies only to instances that are currently in the `InService` state. It does not prevent the termination of instances in other states, such as `Pending`, or instances that fail to resume properly from standby.
+ The `RemoveFromLoadBalancerLowPriority` process can be ignored when it is present in calls to describe Auto Scaling groups using the AWS CLI or SDKs. This process is deprecated and is retained only for backward compatibility.
# Suspend processes
To suspend a process for an Auto Scaling group, use one of the following methods:
------
#### [ Console ]
**To suspend a process**
1. Open the Amazon EC2 console at [https://console.aws.amazon.com/ec2/](https://console.aws.amazon.com/ec2/), and choose **Auto Scaling Groups** from the navigation pane.
1. Select the check box next to the Auto Scaling group.
A split pane opens up in the bottom of the page.
1. On the **Details** tab, choose **Advanced configurations**, **Edit**.
1. For **Suspended processes**, choose the process to suspend.
1. Choose **Update**.
------
#### [ AWS CLI ]
Use the following [suspend-processes](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/autoscaling/suspend-processes.html) command to suspend individual processes.
```
aws autoscaling suspend-processes --auto-scaling-group-name my-asg --scaling-processes HealthCheck ReplaceUnhealthy
```
To suspend all processes, omit the `--scaling-processes` option, as follows.
```
aws autoscaling suspend-processes --auto-scaling-group-name my-asg
```
------
# Resume processes
To resume a suspended process for an Auto Scaling group, use one of the following methods:
------
#### [ Console ]
**To resume a suspended process**
1. Open the Amazon EC2 console at [https://console.aws.amazon.com/ec2/](https://console.aws.amazon.com/ec2/), and choose **Auto Scaling Groups** from the navigation pane.
1. Select the check box next to the Auto Scaling group.
A split pane opens up in the bottom of the page.
1. On the **Details** tab, choose **Advanced configurations**, **Edit**.
1. For **Suspended processes**, remove the suspended process.
1. Choose **Update**.
------
#### [ AWS CLI ]
To resume a suspended process, use the following [resume-processes](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/autoscaling/resume-processes.html) command.
```
aws autoscaling resume-processes --auto-scaling-group-name my-asg --scaling-processes HealthCheck
```
To resume all suspended processes, omit the `--scaling-processes` option, as follows.
```
aws autoscaling resume-processes --auto-scaling-group-name my-asg
```
------
# How suspended processes affect other processes
The following sections describe what happens when different processes are suspended individually.
**Topics**
+ [`Launch` is suspended](#launch-is-suspended)
+ [`Terminate` is suspended](#terminate-is-suspended)
+ [`AddToLoadBalancer` is suspended](#addtoloadbalancer-is-suspended)
+ [`AlarmNotification` is suspended](#alarmnotification-is-suspended)
+ [`AZRebalance` is suspended](#azrebalance-is-suspended)
+ [`HealthCheck` is suspended](#healthcheck-is-suspended)
+ [`InstanceRefresh` is suspended](#instancerefresh-is-suspended)
+ [`ReplaceUnhealthy` is suspended](#replaceunhealthy-is-suspended)
+ [`ScheduledActions` is suspended](#scheduledactions-is-suspended)
+ [Additional considerations](#other-considerations)
## `Launch` is suspended
+ `AlarmNotification` is still active, but your Auto Scaling group can't initiate scale-out activities for alarms that are in breach.
+ `ScheduledActions` is active, but your Auto Scaling group can't initiate scale-out activities for any scheduled actions that occur.
+ `AZRebalance` stops rebalancing the group.
+ `ReplaceUnhealthy` continues to terminate unhealthy instances, but does not launch replacements. When you resume the `Launch` process, Amazon EC2 Auto Scaling immediately replaces any instances that it terminated during the time that `Launch` was suspended.
+ `InstanceRefresh` does not replace instances.
## `Terminate` is suspended
+ `AlarmNotification` is still active, but your Auto Scaling group can't initiate scale in activities for alarms that are in breach.
+ `ScheduledActions` is active, but your Auto Scaling group can't initiate scale in activities for any scheduled actions that occur.
+ `AZRebalance` is still active but does not function properly. It can launch new instances without terminating the old ones. This could cause your Auto Scaling group to grow up to 10 percent larger than its maximum size, because this is allowed temporarily during rebalancing activities. Your Auto Scaling group could remain above its maximum size until you resume the `Terminate` process.
+ `ReplaceUnhealthy` is inactive but not `HealthCheck`. When `Terminate` resumes, the `ReplaceUnhealthy` process immediately starts running. If any instances were marked as unhealthy while `Terminate` was suspended, they are immediately replaced.
+ `InstanceRefresh` does not replace instances.
## `AddToLoadBalancer` is suspended
+ Amazon EC2 Auto Scaling launches the instances but does not add them to the load balancer target group or Classic Load Balancer. When you resume the `AddToLoadBalancer` process, it resumes adding instances to the load balancer when they are launched. However, it does not add the instances that were launched while this process was suspended. You must register those instances manually.
## `AlarmNotification` is suspended
+ Amazon EC2 Auto Scaling does not invoke scaling policies when a CloudWatch alarm threshold is in breach. When you resume `AlarmNotification`, Amazon EC2 Auto Scaling considers policies with alarm thresholds that are currently in breach.
## `AZRebalance` is suspended
+ Amazon EC2 Auto Scaling does not attempt to redistribute instances after certain events. However, if a scale-out or scale in event occurs, the scaling process still tries to balance the Availability Zones. For example, during scale out, it launches instances in the Availability Zone with the fewest instances. If the group becomes unbalanced while `AZRebalance` is suspended and you resume it, Amazon EC2 Auto Scaling attempts to rebalance the group. It first calls `Launch` and then `Terminate`.
+ Warm pools are not affected when `AZRebalance` is suspended.
## `HealthCheck` is suspended
+ Amazon EC2 Auto Scaling stops marking instances unhealthy as a result of EC2 and Elastic Load Balancing health checks. Your custom health checks continue to function properly. After you suspend `HealthCheck`, if you need to, you can manually set the health state of instances in your group and have `ReplaceUnhealthy` replace them.
## `InstanceRefresh` is suspended
+ Amazon EC2 Auto Scaling stops replacing instances as a result of an instance refresh. If there is an instance refresh in progress, this pauses the operation without canceling it.
## `ReplaceUnhealthy` is suspended
+ Amazon EC2 Auto Scaling stops replacing instances that are marked as unhealthy. Instances that fail EC2 or Elastic Load Balancing health checks are still marked as unhealthy. As soon as you resume the `ReplaceUnhealthy` process, Amazon EC2 Auto Scaling replaces instances that were marked unhealthy while this process was suspended. The `ReplaceUnhealthy` process calls `Terminate` first and then `Launch`.
## `ScheduledActions` is suspended
+ Amazon EC2 Auto Scaling does not run scheduled actions that are scheduled to run during the suspension period. When you resume `ScheduledActions`, Amazon EC2 Auto Scaling only considers scheduled actions whose scheduled time has not yet passed.
## Additional considerations
In addition, when `Launch` or `Terminate` are suspended, the following features might not function correctly:
+ **Maximum instance lifetime** – When `Launch` or `Terminate` are suspended, the maximum instance lifetime feature can't replace any instances.
+ **Spot Instance interruptions** – If `Terminate` is suspended and your Auto Scaling group has Spot Instances, they can still terminate in the event that Spot capacity is no longer available. While `Launch` is suspended, Amazon EC2 Auto Scaling can't launch replacement instances from another Spot Instance pool or from the same Spot Instance pool when it is available again.
+ **Capacity Rebalancing** – If `Terminate` is suspended and you use Capacity Rebalancing to handle Spot Instance interruptions, the Amazon EC2 Spot service can still terminate instances in the event that Spot capacity is no longer available. If `Launch` is suspended, Amazon EC2 Auto Scaling can't launch replacement instances from another Spot Instance pool or from the same Spot Instance pool when it is available again.
+ **Attaching and detaching instances ** – When `Launch` and `Terminate` are suspended, you can detach instances that are attached to your Auto Scaling group, but while `Launch` is suspended, you can't attach new instances to the group.
+ **Standby instances** – When `Launch` and `Terminate` are suspended, you can put an instance in the `Standby` state, but while `Launch` is suspended, you can't return an instance in the `Standby` state to service.