# Indicators for continuous-delivery Ensure that your code base remains in a releasable state at all times and deploy more often and with more confidence using automated pipelines. **Topics** + [ # [DL.CD.1] Deploy changes to production frequently ](dl.cd.1-deploy-changes-to-production-frequently.md) + [ # [DL.CD.2] Deploy exclusively from trusted artifact repositories ](dl.cd.2-deploy-exclusively-from-trusted-artifact-repositories.md) + [ # [DL.CD.3] Integrate quality assurance into deployments ](dl.cd.3-integrate-quality-assurance-into-deployments.md) + [ # [DL.CD.4] Automate the entire deployment process ](dl.cd.4-automate-the-entire-deployment-process.md) + [ # [DL.CD.5] Ensure on-demand deployment capabilities ](dl.cd.5-ensure-on-demand-deployment-capabilities.md) + [ # [DL.CD.6] Refine delivery pipelines using metrics for continuous improvement ](dl.cd.6-refine-delivery-pipelines-using-metrics-for-continuous-improvement.md) + [ # [DL.CD.7] Remove manual approvals to practice continuous deployment ](dl.cd.7-remove-manual-approvals-to-practice-continuous-deployment.md) # [DL.CD.1] Deploy changes to production frequently **Category:** FOUNDATIONAL Frequent deployments to production encourages small, rapid, and iterative changes to the code base. Deploying small and validated changes regularly helps mitigate the risk associated with each deployment. Frequent deployments not only streamlines the testing and validation process, but also expedites the feedback loop, leading to quicker resolution of issues. Use a pipeline to automate the deployment of validated changes across various environments, including production. This pipeline should be automatically triggered, such as by the completion of continuous integration or an updated artifact in an artifact repository. Once invoked, the pipeline should automatically begin to deploy changes to non-production environments for further testing and validation. Upon successful validation, changes can be deployed to the production environment. When working in a DevOps environment, it is important to distinguish between *deploying* and *releasing*. Even after deploying changes to production, these changes might not necessarily be visible or accessible to all users. By using advanced deployment strategies and employing [feature flags](https://aws.amazon.com/systems-manager/features/appconfig/?whats-new-cards.sort-by=item.additionalFields.postDateTime&whats-new-cards.sort-order=desc&blog-posts-cards.sort-by=item.additionalFields.createdDate&blog-posts-cards.sort-order=desc#Feature_flags), teams can deploy code to production and decide when to release or rollback specific features in real time, offering more granular control over releasing new features to end users. Teams should focus on deploying small changes rather than bundling multiple changes into a single, large batch deployment. Accumulating changes complicates testing and validation, and it becomes challenging to ensure that all components interact correctly. The practice of deploying small changes demands discipline and commitment, but it improves deployment frequency, security, and enhanced collaboration while ensuring that the code base remains up-to-date and releasable at all times. # [DL.CD.2] Deploy exclusively from trusted artifact repositories **Category:** FOUNDATIONAL All artifacts involved in the delivery process should originate from a trusted artifact repository. These repositories contain validated, tested, and integrated artifacts that have been deemed safe for deployment. By using trusted artifact repositories, teams can ensure the security of deployed workloads, maintain quality and security standards, and promote trust in the delivery pipeline. The delivery pipeline should be restricted to using only trusted artifact repositories, which could be enforced through mechanisms such as allow lists, IP restrictions, or authentication controls. Additionally, we recommend using cryptographic signing to validate artifacts and including a validation stage in the pipeline to verify that the artifacts meet the necessary standards before deployment. In this way, the integrity and security of the deployed workloads are maintained consistently. **Related information:** + [Artifact Repository - AWS CodeArtifact](https://aws.amazon.com/codeartifact/) + [Fully Managed Container Registry - Amazon Elastic Container Registry](https://aws.amazon.com/ecr/) + [Code Repositories and Artifact Management \$1 AWS Marketplace](https://aws.amazon.com/marketplace/solutions/devops/code-repositories-and-artifact-management?aws-marketplace-cards.sort-by=item.additionalFields.headline&aws-marketplace-cards.sort-order=asc&awsf.aws-marketplace-devops-store-use-cases=*all) # [DL.CD.3] Integrate quality assurance into deployments **Category:** FOUNDATIONAL Integrating quality assurance (QA) processes into continuous delivery pipelines tests that the whole system is ready for release. This differs from previous quality checks in the development lifecycle as these tests validate that the software changes behave as expected when deployed into real-world environments. This provides the ability to test integration with other live systems, check for configuration errors, and test in environments that more closely mirror production. Incorporate QA stages into your delivery pipeline to automatically conduct required functional, non-functional, security, and data tests after deployments occur. Deployments to environments is the ideal enforcement point for quality assurance, with QA requirements being scoped to the environment being deployed to. If a test fails for one environment, it is a signal that deployment to subsequent environments might carry the same risk. Provide immediate feedback to the development team upon any test failures, so they can rectify issues quickly and maintain the integrity of the deployment pipeline. **Related information:** + [AWS Well-Architected Reliability Pillar: REL08-BP02 Integrate functional testing as part of your deployment](https://docs.aws.amazon.com/wellarchitected/latest/reliability-pillar/rel_tracking_change_management_functional_testing.html) + [AWS Well-Architected Reliability Pillar: REL08-BP03 Integrate resiliency testing as part of your deployment](https://docs.aws.amazon.com/wellarchitected/latest/reliability-pillar/rel_tracking_change_management_resiliency_testing.html) + [AWS Well-Architected Security Pillar: SEC11-BP02 Automate testing throughout the development and release lifecycle](https://docs.aws.amazon.com/wellarchitected/latest/framework/sec_appsec_automate_testing_throughout_lifecycle.html) + [Testing stages in continuous integration and continuous delivery](https://docs.aws.amazon.com/whitepapers/latest/practicing-continuous-integration-continuous-delivery/testing-stages-in-continuous-integration-and-continuous-delivery.html) + [Amazon's approach to high-availability deployment: Release guidance lifecycle](https://youtu.be/bCgD2bX1LI4?t=855) + [Testing software and systems at Amazon: Continuous integration and deployment](https://youtu.be/o1sc3cK9bMU?t=1206) + [The Amazon Software Development Process: Automated Testing](https://youtu.be/52SC80SFPOw?t=1340) # [DL.CD.4] Automate the entire deployment process **Category:** FOUNDATIONAL Automate as many stages of the delivery process as possible. Exceptions for continuous delivery might include optional manual approval gates. Automation reduces the risk of human error, brings consistency to deployments, and accelerates the delivery process. Use the delivery pipeline to automate every stage of deploying changes, from copying the build artifact to setting up any required configurations. While optional manual approval gates can exist, all other stages should be automated, maintaining the integrity of the artifact and reducing the likelihood of errors. Humans should not have access to the target environments or have the ability to inject code, parameters, configuration, or interfere with the integrity of the artifact in any way. Some organizations might still require manual oversight at certain stages as they evolve their DevOps capabilities. If the organization is early in its DevOps adoption or operates in a highly regulated environment, there might be a need for manual interventions or approvals at certain stages. These could be due to governance or regulatory requirements or simply the need for a human decision at a critical point in the deployment process. Over time, even for these organizations, the goal should be to have no manual deployment stages in the deployment of changes. **Related information:** + [AWS Well-Architected Reliability Pillar: REL08-BP05 Deploy changes with automation](https://docs.aws.amazon.com/wellarchitected/latest/reliability-pillar/rel_tracking_change_management_automated_changemgmt.html) + [AWS Well-Architected Security Pillar: SEC11-BP06 Deploy software programmatically](https://docs.aws.amazon.com/wellarchitected/latest/framework/sec_appsec_deploy_software_programmatically.html) + [What is Continuous Delivery?](https://aws.amazon.com/devops/continuous-delivery/) + [Amazon CodeCatalyst](https://codecatalyst.aws/explore) + [Building the pipeline](https://docs.aws.amazon.com/whitepapers/latest/practicing-continuous-integration-continuous-delivery/building-the-pipeline.html) + [Going faster with continuous delivery](https://aws.amazon.com/builders-library/going-faster-with-continuous-delivery/) + [AWS Deployment Pipeline Reference Architecture](https://aws-samples.github.io/aws-deployment-pipeline-reference-architecture) + [Deploy container applications in a multicloud environment using Amazon CodeCatalyst](https://aws.amazon.com/blogs/devops/deploy-container-applications-in-a-multicloud-environment-using-amazon-codecatalyst/) + [Amazon's approach to high-availability deployment: Release guidance lifecycle](https://youtu.be/bCgD2bX1LI4?t=855) + [Testing software and systems at Amazon: Continuous integration and deployment](https://youtu.be/o1sc3cK9bMU?t=1206) + [The Amazon Software Development Process: Continuous Delivery](https://youtu.be/52SC80SFPOw?t=814) # [DL.CD.5] Ensure on-demand deployment capabilities **Category:** FOUNDATIONAL Continuous delivery relies on the ability to ensure that every change is considered deliverable and can be deployed to production environments at any time. While the actual decision to deploy to production may still be manual, deployments should be able to occur on-demand as needed. Deployments should be able to occur during normal working hours without causing significant downtime or disruption to the business. Changes should not require synchronization with other systems and deployments should be able to occur regardless of the interdependence of other systems. By decoupling deployments from other systems and being able to perform them during normal business hours, teams can receive fast feedback and respond to any issues that arise, leading to quicker fixes and less disruption to users. To enable on-demand deployments, teams should employ advanced deployment strategies, such as blue/green deployments, canary releases, feature flags, or rolling updates. The ability to gradually roll out changes, coupled with modern application architectures and integrated QA processes, enables iterative delivery. Iterative delivery reduces the impact of potential issues throughout the deployment and allows for quick rollback if necessary. By using the right tools and strategies, deployments can be automated and run seamlessly, allowing for faster and more efficient delivery of applications and services. **Related information:** + [AWS Deployment Pipeline Reference Architecture](https://aws-samples.github.io/aws-deployment-pipeline-reference-architecture) # [DL.CD.6] Refine delivery pipelines using metrics for continuous improvement [DL.CD.6] Refine delivery pipelines using metrics for continuous improvement **Category:** RECOMMENDED Use key metrics—whether sourced from this guidance, established frameworks like [DORA](https://dora.dev/) or [SPACE](https://queue.acm.org/detail.cfm?id=3454124), or custom to your organization—to continually optimize the development lifecycle. Metrics such as deployment frequency, change lead time, failure rate, and time to recover serve as outcome-based lagging indicators. These indicators span many DevOps capabilities to provide insights into the efficiency and reliability of the full delivery process. While individual metrics offer granular insights to optimize specific continuous delivery capabilities, these aggregated metrics present a holistic overview of the end-to-end development lifecycle. Both granular and holistic metrics are important for continuous improvement. Use observability practices to continuously monitor the development lifecycle, including incorporating monitoring and logging into your delivery pipelines. Use logs to generate metrics, and use these metrics to identify areas for improvement. Make these metrics visible to all team members and use them to drive your continuous improvement efforts. Putting an emphasis on continually optimizing pipelines using metrics is recommended. When getting started with DevOps adoption, initial efforts should prioritize the establishment of a stable and effective delivery pipeline, with subsequent enhancements to the pipeline being driven by metrics. **Related information:** + [Deployment Pipeline Reference Architecture](https://pipelines.devops.aws.dev/application-pipeline/) + [AWS Observability Best Practices: Key Performance Indicators](https://aws-observability.github.io/observability-best-practices/guides/operational/business/key-performance-indicators/) + [DevOps Research and Assessment (DORA)](https://dora.dev/) + [SPACE](https://queue.acm.org/detail.cfm?id=3454124) # [DL.CD.7] Remove manual approvals to practice continuous deployment [DL.CD.7] Remove manual approvals to practice continuous deployment **Category:** OPTIONAL Fully automate all stages of the deployment process, allowing developers to push new code into the production environment using fully automated delivery pipelines—with no manual approval stages required. This is referred to as continuous deployment. Removing all manual deployment steps reduces potential errors and increases deployment speed. It allows developers to focus more on coding and less on deployment logistics, improving efficiency and productivity. Create fully automated pipelines which perform continuous integration and continuous deployment. A pipeline should trigger upon code changes being merged into the main release branch. This pipeline should perform all necessary quality assurance tests, build the application, and deploy the new version to the production environment. Automated governance capabilities ensure that guardrails are being followed, while observability functions such as alerts and logs provide visibility. This level of automation is a hallmark of mature DevOps practices. However, it is an optional capability as it is not always achievable or desired, especially in heavily regulated industries or in organizations with strict governance controls. **Related information:** + [Continuous Delivery vs. Continuous Deployment](https://aws.amazon.com/devops/continuous-delivery/) + [Practicing Continuous Integration and Continuous Delivery on AWS](https://docs.aws.amazon.com/whitepapers/latest/practicing-continuous-integration-continuous-delivery/implementing-continuous-integration-and-continuous-delivery.html)