# CNF continuous delivery This step consists of a sequence of steps that are carried out repeatedly to deploy changes that are part of container/configuration changes resulting in upgrades. The CNF continuous delivery is automated via pipelines and is specific to individual applications. AWS uses standard Helm charts to update specific CNFs. The code pipeline has pre- and post-checks for the application update status. The updated CI/CD pipeline is also integrated with a test automation framework to run automated testing. This abstraction allows a clean deployment of network functions. CNF continuous delivery and deployment can be broadly classified into the following categories: + **Application upgrades** — Most application upgrades are changes within the Kuberbetes application PODs. These updates can be applied automatically through the code pipeline. Most CNFs support in-place upgrades by providing multiple instances of application PODs. Multiple instances allow a rolling upgrade approach. Not all application POD changes support Helm upgrade. Pipelines take these variations into account and use Helm install/delete as required. + **Major upgrades** — Major upgrades are primarily database schema changes. This change cannot be applied without causing some downtime. The standard approach to these changes is to delete the application and re-create the relevant pods. During the process the application may not be available. The following tools are used for upgrades: + [**AWS CloudFormation**](https://aws.amazon.com/cloudformation/) enables customers to describe and provision all the infrastructure resources in JSON or YAML templates. CloudFormation provides a powerful extension mechanism through Lambda-backed custom resources. Customers can extend AWS CloudFormation beyond AWS resources, and provision the required resource in other environments; for example, on-premises resources in hybrid environments. AWS CDK offers developers the ability to build out code using higher level familiar programming languages such as Python, TypeScript, JavaScript, Java, and C\#, and then compile the code into a lower-level CloudFormation JSON format, which can then be deployed. + **BlueGreen deployment** — AWS supports and recommends blue/green and canary-based deployments in test as well as in production environments. [ Blue/green deployments ](https://aws.amazon.com/quickstart/architecture/blue-green-deployment/) enable customers to test a new application version in a contained environment. They provide an easy and graceful method to switch over production traffic. [ Canary-based deployments ](https://wa.aws.amazon.com/wat.concept.canary-deployment.en.html) extend this concept by enabling the non-production green environment to be tested with a small proportion of production traffic to uncover any issues caused by production traffic. The new application version is tested against internal simulated test traffic as well as small amounts of production traffic, which gives the user confidence before they switch over the production traffic. The production traffic is gradually increased until switchover is complete. Implementation involves weighted DNS and weighted ELB target groups. + **Automation** can be achieved by configuring AWS CodePipeline with the blue/green and canary-based deployment stages. The approval stage may be manually driven initially during provisioning, but later it should be fully automated. In the test environments, it is good practice to always test with a rollback action to validate forward and backward compatibility, before deploying into production. The blue/green deployment on clusters with service mesh depend on the support provided by the end-application and the routing gateway for the service mesh to accomplish a graceful transition. + [** AWS Systems Manager **](https://aws.amazon.com/systems-manager/) provides a unified user interface so you can view operational data from multiple AWS Services used by network functions deployed by CI/CD. Systems Manager enables you to automate operational tasks across your AWS resources. ![A diagram depicting Canary deployment.](https://docs.aws.amazon.com/whitepapers/latest/cicd_for_5g_networks_on_aws/images/cicd_5g10.png) *Canary deployment*