Upgrade from Amazon Linux 2 to Amazon Linux 2023

AL2023 introduces a new node initialization process nodeadm that uses a YAML configuration schema. If you’re using self-managed node groups or an AMI with a launch template, you’ll now need to provide additional cluster metadata explicitly when creating a new node group. An example of the minimum required parameters is as follows, where apiServerEndpoint, certificateAuthority, and service cidr are now required:

---
apiVersion: node.eks.aws/v1alpha1
kind: NodeConfig
spec:
  cluster:
    name: my-cluster
    apiServerEndpoint: https://example.com
    certificateAuthority: Y2VydGlmaWNhdGVBdXRob3JpdHk=
    cidr: 10.100.0.0/16

In AL2, the metadata from these parameters was discovered from the Amazon EKS DescribeCluster API call. With AL2023, this behavior has changed since the additional API call risks throttling during large node scale ups. This change doesn’t affect you if you’re using managed node groups without a launch template or if you’re using Karpenter. For more information on certificateAuthority and service cidr, see DescribeCluster in the Amazon EKS API Reference.

For AL2023, nodeadm also changes the format to apply parameters to the kubelet for each node using NodeConfigSpec. In AL2, this was done with the --kubelet-extra-args parameter. This is commonly used to add labels and taints to nodes. An example below shows applying maxPods and --node-labels to the node.

---
apiVersion: node.eks.aws/v1alpha1
kind: NodeConfig
spec:
  cluster:
    name: test-cluster
    apiServerEndpoint: https://example.com
    certificateAuthority: Y2VydGlmaWNhdGVBdXRob3JpdHk=
    cidr: 10.100.0.0/16
  kubelet:
    config:
      maxPods: 110
    flags:
      - --node-labels=karpenter.sh/capacity-type=on-demand,karpenter.sh/nodepool=test

Amazon VPC CNI version 1.16.2 or greater is required for AL2023.

AL2023 requires IMDSv2 by default. IMDSv2 has several benefits that help improve security posture. It uses a session-oriented authentication method that requires the creation of a secret token in a simple HTTP PUT request to start the session. A session’s token can be valid for anywhere between 1 second and 6 hours. For more information on how to transition from IMDSv1 to IMDSv2, see Transition to using Instance Metadata Service Version 2 and Get the full benefits of IMDSv2 and disable IMDSv1 across your AWS infrastructure. If you would like to use IMDSv1, you can still do so by manually overriding the settings using instance metadata option launch properties.

AL2023 features the next generation of unified control group hierarchy (cgroupv2). cgroupv2 is used to implement a container runtime, and by systemd. While AL2023 still includes code that can make the system run using cgroupv1, this isn’t a recommended or supported configuration. This configuration will be completely removed in a future major release of Amazon Linux.

eksctl version 0.176.0 or greater is required for eksctl to support AL2023.

If you’re using a custom AMI with a managed node group, you can perform an in-place upgrade by swapping the AMI ID in the launch template. You should ensure that your applications and any user data transfer over to AL2023 first before performing this upgrade strategy.

If you’re using managed node groups with either the standard launch template or with a custom launch template that doesn’t specify the AMI ID, you’re required to upgrade using a blue/green strategy. A blue/green upgrade is typically more complex and involves creating an entirely new node group where you would specify AL2023 as the AMI type. The new node group will need to then be carefully configured to ensure that all custom data from the AL2 node group is compatible with the new OS. Once the new node group has been tested and validated with your applications, Pods can be migrated from the old node group to the new node group. Once the migration is completed, you can delete the old node group.