# REL 1  How do you manage service quotas and constraints? For cloud-based workload architectures, there are service quotas (which are also referred to as service limits). These quotas exist to prevent accidentally provisioning more resources than you need and to limit request rates on API operations so as to protect services from abuse. There are also resource constraints, for example, the rate that you can push bits down a fiber-optic cable, or the amount of storage on a physical disk. **Topics** + [ # REL01-BP01 Aware of service quotas and constraints ](rel_manage_service_limits_aware_quotas_and_constraints.md) + [ # REL01-BP02 Manage service quotas across accounts and regions ](rel_manage_service_limits_limits_considered.md) + [ # REL01-BP03 Accommodate fixed service quotas and constraints through architecture ](rel_manage_service_limits_aware_fixed_limits.md) + [ # REL01-BP04 Monitor and manage quotas ](rel_manage_service_limits_monitor_manage_limits.md) + [ # REL01-BP05 Automate quota management ](rel_manage_service_limits_automated_monitor_limits.md) + [ # REL01-BP06 Ensure that a sufficient gap exists between the current quotas and the maximum usage to accommodate failover ](rel_manage_service_limits_suff_buffer_limits.md) # REL01-BP01 Aware of service quotas and constraints REL01-BP01 Aware of service quotas and constraints You are aware of your default quotas and quota increase requests for your workload architecture. You additionally know which resource constraints, such as disk or network, are potentially impactful. Service Quotas is an AWS service that helps you manage your quotas for over 100 AWS services from one location. Along with looking up the quota values, you can also request and track quota increases from the Service Quotas console or via the AWS SDK. AWS Trusted Advisor offers a service quotas check that displays your usage and quotas for some aspects of some services. The default service quotas per service are also in the AWS documentation per respective service, for example, see [Amazon VPC Quotas](https://docs.aws.amazon.com/vpc/latest/userguide/amazon-vpc-limits.html). Rate limits on throttled APIs are set within the API Gateway itself by configuring a usage plan. Other limits that are set as configuration on their respective services include Provisioned IOPS, RDS storage allocated, and EBS volume allocations. Amazon Elastic Compute Cloud (Amazon EC2) has its own service limits dashboard that can help you manage your instance, Amazon Elastic Block Store (Amazon EBS), and Elastic IP address limits. If you have a use case where service quotas impact your application’s performance and they are not adjustable to your needs, then contact AWS Support to see if there are mitigations. **Common anti-patterns:** + Deploying a workload with no regard of the service quotas on the AWS services used. + Designing a workload without investigating and accommodating for AWS services' design constraints. + Deploying a workload with significant use that replaces a known existing workload without configuring the necessary quotas or contacting AWS Support in advance. + Planning an event to drive traffic to your workload, but not configuring the necessary quotas or contacting AWS Support in advance. **Benefits of establishing this best practice:** Being aware of the service quotas, API throttling limits, and design constraints will allow you to account for these in your design, implementation, and operation of the workload. **Level of risk exposed if this best practice is not established:** High ## Implementation guidance Implementation guidance + Review AWS service quotas in the published documentation and Service Quotas + [AWS Service Quotas (formerly referred to as limits)](https://docs.aws.amazon.com/general/latest/gr/aws_service_limits.html) + Determine all the services your workload requires by looking at the deployment code. + Use AWS Config to find all AWS resources used in your AWS accounts. + [AWS Config Supported AWS Resource Types and Resource Relationships](https://docs.aws.amazon.com/config/latest/developerguide/resource-config-reference.html) + You can also use your AWS CloudFormation to determine your AWS resources used. Look at the resources that were created either in the AWS Management Console or via the list-stack-resources CLI command. You can also see resources configured to be deployed in the template itself. + [Viewing AWS CloudFormation Stack Data and Resources on the AWS Management Console](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/cfn-console-view-stack-data-resources.html) + [AWS CLI for CloudFormation: list-stack-resources](https://docs.aws.amazon.com/cli/latest/reference/cloudformation/list-stack-resources.html) + Determine the service quotas that apply. Use the programmatically accessible information via Trusted Advisor and Service Quotas. ## Resources Resources **Related documents:** + [AWS Marketplace: CMDB products that help track limits](https://aws.amazon.com/marketplace/search/results?searchTerms=CMDB) + [AWS Service Quotas (formerly referred to as service limits)](https://docs.aws.amazon.com/general/latest/gr/aws_service_limits.html) + [AWS Trusted Advisor Best Practice Checks (see the Service Limits section)](https://aws.amazon.com/premiumsupport/technology/trusted-advisor/best-practice-checklist/) + [AWS limit monitor on AWS answers](https://aws.amazon.com/answers/account-management/limit-monitor/) + [Amazon EC2 Service Limits](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-resource-limits.html) + [What is Service Quotas?](https://docs.aws.amazon.com/servicequotas/latest/userguide/intro.html) **Related videos:** + [AWS Live re:Inforce 2019 - Service Quotas](https://youtu.be/O9R5dWgtrVo) # REL01-BP02 Manage service quotas across accounts and regions REL01-BP02 Manage service quotas across accounts and regions If you are using multiple AWS accounts or AWS Regions, ensure that you request the appropriate quotas in all environments in which your production workloads run. Service quotas are tracked per account. Unless otherwise noted, each quota is AWS Region-specific. In addition to the production environments, also manage quotas in all applicable non-production environments, so that testing and development are not hindered. **Common anti-patterns:** + Allowing resource utilization in one isolation zone to grow with no mechanism to maintain capacity in the other ones. + Manually setting all quotas independently in isolation zones. + Not ensuring Regionally isolated deployments are sized to accommodate the increase in traffic from another Region if a deployment is lost. **Benefits of establishing this best practice:** Ensuring that you can handle your current load if an isolation zone is unavailable can help reduce the number of errors that occur during failover, instead of causing a denial of service to your customers. **Level of risk exposed if this best practice is not established:** High ## Implementation guidance Implementation guidance + Select relevant accounts and Regions based on your service requirements, latency, regulatory, and disaster recovery (DR) requirements. + Identify service quotas across all relevant accounts, Regions, and Availability Zones. The limits are scoped to account and Region. + [What is Service Quotas?](https://docs.aws.amazon.com/servicequotas/latest/userguide/intro.html) ## Resources Resources **Related documents:** + [AWS Marketplace: CMDB products that help track limits](https://aws.amazon.com/marketplace/search/results?searchTerms=CMDB) + [AWS Service Quotas (formerly referred to as service limits)](https://docs.aws.amazon.com/general/latest/gr/aws_service_limits.html) + [AWS Trusted Advisor Best Practice Checks (see the Service Limits section)](https://aws.amazon.com/premiumsupport/technology/trusted-advisor/best-practice-checklist/) + [AWS limit monitor on AWS answers](https://aws.amazon.com/answers/account-management/limit-monitor/) + [Amazon EC2 Service Limits](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-resource-limits.html) + [What is Service Quotas?](https://docs.aws.amazon.com/servicequotas/latest/userguide/intro.html) **Related videos:** + [AWS Live re:Inforce 2019 - Service Quotas](https://youtu.be/O9R5dWgtrVo) # REL01-BP03 Accommodate fixed service quotas and constraints through architecture REL01-BP03 Accommodate fixed service quotas and constraints through architecture Be aware of unchangeable service quotas and physical resources, and architect to prevent these from impacting reliability. Examples include network bandwidth, AWS Lambda payload size, throttle burst rate for API Gateway, and concurrent user connections to an Amazon Redshift cluster. **Common anti-patterns:** + Performing benchmarking for too short of time, utilizing the burst limit, but then expecting the service to perform at that capacity for sustained periods. + Choosing a design that uses one resource of a service per user or customer, unaware that there are design constraints that will cause this design to fail as you scale. **Benefits of establishing this best practice:** Tracking fixed quotes in AWS services and constraints in other parts of your workload, such as connectivity constraints, IP address constraints, and constraints in third-party services, allows you to detect when you are trending toward a quota and gives you the ability to address the quota before it's exceeded. **Level of risk exposed if this best practice is not established:** Medium ## Implementation guidance Implementation guidance + Be aware of fixed service quotas Be aware of fixed service quotas and constraints and architect around these. + [AWS Service Quotas](https://docs.aws.amazon.com/general/latest/gr/aws_service_limits.html) ## Resources Resources **Related documents:** + [AWS Marketplace: CMDB products that help track limits](https://aws.amazon.com/marketplace/search/results?searchTerms=CMDB) + [AWS Service Quotas (formerly referred to as service limits)](https://docs.aws.amazon.com/general/latest/gr/aws_service_limits.html) + [AWS Trusted Advisor Best Practice Checks (see the Service Limits section)](https://aws.amazon.com/premiumsupport/technology/trusted-advisor/best-practice-checklist/) + [AWS limit monitor on AWS answers](https://aws.amazon.com/answers/account-management/limit-monitor/) + [Amazon EC2 Service Limits](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-resource-limits.html) + [What Is Service Quotas?](https://docs.aws.amazon.com/servicequotas/latest/userguide/intro.html) **Related videos:** + [AWS Live re:Inforce 2019 - Service Quotas](https://youtu.be/O9R5dWgtrVo) # REL01-BP04 Monitor and manage quotas REL01-BP04 Monitor and manage quotas Evaluate your potential usage and increase your quotas appropriately, allowing for planned growth in usage. For supported services, you can manage your quotas by configuring CloudWatch alarms to monitor usage and alert you to approaching quotas. These alarms can be triggered from Service Quotas or from Trusted Advisor. You can also use metric filters on CloudWatch Logs to search and extract patterns in logs to determine if usage is approaching quota thresholds. **Common anti-patterns:** + Configuring alarms for when Service Quotas are being approached, but having no process on how to respond to an alert. + Only configuring alarms for services supported by Service Quotas and not monitoring other services. **Benefits of establishing this best practice:** Automatic tracking of the AWS service quotas and monitoring your usage against those quotas will allow you to see when you are approaching a quota limit. You can also use this monitoring data to assess when you might lower quotas to save costs. **Level of risk exposed if this best practice is not established:** Medium ## Implementation guidance Implementation guidance + Monitor and manage your quotas Evaluate your potential usage on AWS, increase your regional service quotas appropriately, and allow planned growth in usage. + Capture current resource consumption (for example, buckets, instances). Use service API operations, such as the Amazon EC2 DescribeInstances API, to collect current resource consumption. + Capture your current quotas Use AWS Service Quotas, AWS Trusted Advisor, and AWS documentation. + Use AWS Service Quotas, an AWS service that helps you manage your quotas for over 100 AWS services from one location. + Use Trusted Advisor service limits to determine your current service limits. + Use service API operations to determine current service quotas where supported. + Keep a record of quota increases that have been requested, and their status After a quota increase has been approved, ensure that you update your records to reflect the change to the quota. ## Resources Resources **Related documents:** + [AWS Marketplace: CMDB products that help track limits](https://aws.amazon.com/marketplace/search/results?searchTerms=CMDB) + [AWS Service Quotas (formerly referred to as service limits)](https://docs.aws.amazon.com/general/latest/gr/aws_service_limits.html) + [AWS Trusted Advisor Best Practice Checks for Service Limits](https://docs.aws.amazon.com/awssupport/latest/user/service-limits.html) + [AWS limit monitor on AWS answers](https://aws.amazon.com/answers/account-management/limit-monitor/) + [Amazon EC2 Service Limits](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-resource-limits.html) + [What Is Service Quotas?](https://docs.aws.amazon.com/servicequotas/latest/userguide/intro.html) + [Monitor Service Quotas using Amazon CloudWatch alarms](https://docs.aws.amazon.com/servicequotas/latest/userguide/configure-cloudwatch.html) **Related videos:** + [AWS Live re:Inforce 2019 - Service Quotas](https://youtu.be/O9R5dWgtrVo) # REL01-BP05 Automate quota management REL01-BP05 Automate quota management Implement tools to alert you when thresholds are being approached. You can automate quota increase requests by using AWS Service Quotas APIs. If you integrate your Configuration Management Database (CMDB) or ticketing system with Service Quotas, you can automate the tracking of quota increase requests and current quotas. In addition to the AWS SDK, Service Quotas offers automation using the AWS Command Line Interface (AWS CLI). **Common anti-patterns:** + Tracking the quotas and usage in spreadsheets. + Running reports on usage daily, weekly, or monthly, and then comparing usage to the quotas. **Benefits of establishing this best practice:** Automated tracking of the AWS service quotas and monitoring of your usage against that quota allows you to see when you are approaching a quota. You can set up automation to assist you in requesting a quota increase when needed. You might want to consider lowering some quotas when your usage trends in the opposite direction to realize the benefits of lowered risk (in case of compromised credentials) and cost savings. **Level of risk exposed if this best practice is not established:** Medium ## Implementation guidance Implementation guidance + Set up automated monitoring Implement tools using SDKs to alert you when thresholds are being approached. + Use Service Quotas and augment the service with an automated quota monitoring solution, such as AWS Limit Monitor or an offering from AWS Marketplace. + [What is Service Quotas?](https://docs.aws.amazon.com/servicequotas/latest/userguide/intro.html) + [Quota Monitor on AWS - AWS Solution](https://aws.amazon.com/answers/account-management/limit-monitor/) + Set up triggered responses based on quota thresholds, using Amazon SNS and AWS Service Quotas APIs. + Test automation. + Configure limit thresholds. + Integrate with change events from AWS Config, deployment pipelines, Amazon EventBridge, or third parties. + Artificially set low quota thresholds to test responses. + Set up triggers to take appropriate action on notifications and contact AWS Support when necessary. + Manually trigger change events. + Run a game day to test the quota increase change process. ## Resources Resources **Related documents:** + [APN Partner: partners that can help with configuration management](https://aws.amazon.com/partners/find/results/?keyword=Configuration+Management) + [AWS Marketplace: CMDB products that help track limits](https://aws.amazon.com/marketplace/search/results?searchTerms=CMDB) + [AWS Service Quotas (formerly referred to as service limits)](https://docs.aws.amazon.com/general/latest/gr/aws_service_limits.html) + [AWS Trusted Advisor Best Practice Checks (see the Service Limits section)](https://aws.amazon.com/premiumsupport/technology/trusted-advisor/best-practice-checklist/) + [Quota Monitor on AWS - AWS Solution](https://aws.amazon.com/answers/account-management/limit-monitor/) + [Amazon EC2 Service Limits](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-resource-limits.html) + [What is Service Quotas?](https://docs.aws.amazon.com/servicequotas/latest/userguide/intro.html) **Related videos:** + [AWS Live re:Inforce 2019 - Service Quotas](https://youtu.be/O9R5dWgtrVo) # REL01-BP06 Ensure that a sufficient gap exists between the current quotas and the maximum usage to accommodate failover REL01-BP06 Ensure that a sufficient gap exists between the current quotas and the maximum usage to accommodate failover When a resource fails, it might still be counted against quotas until it’s successfully terminated. Ensure that your quotas cover the overlap of all failed resources with replacements before the failed resources are terminated. You should consider an Availability Zone failure when calculating this gap. **Common anti-patterns:** + Setting service quotas based on current needs without accounting for failover scenarios. **Benefits of establishing this best practice:** When events potentially impact availability, the cloud allows you to implement strategies to mitigate or recover from these events. Such strategies often include creating additional resources to replace failed ones. Your quota strategy must accommodate these additional resources. **Level of risk exposed if this best practice is not established:** Medium ## Implementation guidance Implementation guidance + Ensure that there is enough gap between your service quota and your maximum usage to accommodate for a failover. + Determine your service quotas, accounting for your deployment patterns, availability requirements, and consumption growth. + Request quota increases if necessary. Plan for necessary time for quota increase requests to be fulfilled. + Determine your reliability requirements (also known as your number of 9's). + Establish your fault scenarios (for example, loss of a component, an Availability Zone, or a Region). + Establish your deployment methodology (for example, canary, blue/green, red/black, or rolling). + Include an appropriate buffer (for example, 15%) to the current limit. + Plan consumption growth (for example, monitor your trends in consumption). ## Resources Resources **Related documents:** + [AWS Marketplace: CMDB products that help track limits](https://aws.amazon.com/marketplace/search/results?searchTerms=CMDB) + [AWS Service Quotas (formerly referred to as service limits)](https://docs.aws.amazon.com/general/latest/gr/aws_service_limits.html) + [AWS Trusted Advisor Best Practice Checks (see the Service Limits section)](https://aws.amazon.com/premiumsupport/technology/trusted-advisor/best-practice-checklist/) + [Amazon EC2 Service Limits](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-resource-limits.html) + [What Is Service Quotas?](https://docs.aws.amazon.com/servicequotas/latest/userguide/intro.html) **Related videos:** + [AWS Live re:Inforce 2019 - Service Quotas](https://youtu.be/O9R5dWgtrVo)