Help improve this page

To contribute to this user guide, choose the Edit this page on GitHub link that is located in the right pane of every page.

Use EKS-optimized accelerated AMIs for GPU instances

Amazon EKS supports EKS-optimized Amazon Linux and Bottlerocket AMIs for GPU instances. The EKS-optimized accelerated AMIs simplify running AI and ML workloads in EKS clusters by providing pre-built, validated operating system images for the accelerated Kubernetes stack. In addition to the core Kubernetes components that are included in the standard EKS-optimized AMIs, the EKS-optimized accelerated AMIs include the kernel modules and drivers required to run the NVIDIA GPU G and P EC2 instances, and the AWS GPU Inferentia and Trainium EC2 instances in EKS clusters.

The table below shows the supported GPU instance types for each EKS-optimized accelerated AMI variant. See the EKS-optimized AL2023 releases and Bottlerocket releases on GitHub for the latest updates to the AMI variants.

EKS-optimized NVIDIA AMIs

By using the EKS-optimized NVIDIA AMIs, you agree to NVIDIA’s Cloud End User License Agreement (EULA).

To find the latest EKS-optimized NVIDIA AMIs, see Retrieve recommended Amazon Linux AMI IDs and Retrieve recommended Bottlerocket AMI IDs.

When using Amazon Elastic Fabric Adaptor (EFA) with the EKS-optimized AL2023 or Bottlerocket NVIDIA AMIs, you must install the EFA device plugin separately. For more information, see Run machine learning training on Amazon EKS with Elastic Fabric Adapter.

EKS AL2023 NVIDIA AMIs

When using the NVIDIA GPU operator with the EKS-optimized AL2023 NVIDIA AMIs, you must disable the operator installation of the driver and toolkit, as these are already included in the EKS AMIs. The EKS-optimized AL2023 NVIDIA AMIs do not include the NVIDIA Kubernetes device plugin or the NVIDIA DRA driver, and these must be installed separately. For more information, see Install NVIDIA Kubernetes device plugin.

In addition to the standard EKS AMI components, the EKS-optimized AL2023 NVIDIA AMIs include the following components.

For details on the NVIDIA CUDA user mode driver and the CUDA runtime/libraries used within application containers, see the NVIDIA documentation. The CUDA version shown from nvidia-smi is the version of the NVIDIA CUDA user mode driver installed on the host, which must be compatible with the CUDA runtime/libraries used in application containers.

To track the status of the EKS-optimized NVIDIA AMIs upgrade to NVIDIA driver 580 version, see GitHub issue #2470. The NVIDIA 580 driver is required to use CUDA 13+.

See the EKS AL2023 NVIDIA AMI installation script and kernel loading script for details on how the EKS AMIs configure the NVIDIA dependencies. See the EKS-optimized AL2023 releases on GitHub to see the component versions included in the AMIs. You can find the list of installed packages and their versions on a running EC2 instance with the dnf list installed command.

When building custom AMIs with the EKS-optimized AMIs as the base, it is not recommended or supported to run an operating system upgrade (ie. dnf upgrade) or upgrade any of the Kubernetes or GPU packages that are included in the EKS-optimized AMIs, as this risks breaking component compatibility. If you do upgrade the operating system or packages that are included in the EKS-optimized AMIs, it is recommended to thoroughly test in a development or staging environment before deploying to production.

When building custom AMIs for GPU instances, it is recommended to build separate custom AMIs for each instance type generation and family that you will run. The EKS-optimized accelerated AMIs selectively install drivers and packages at runtime based on the underlying instance type generation and family. For more information, see the EKS AMI scripts for installation and runtime.

EKS Bottlerocket NVIDIA AMIs

When using the NVIDIA GPU operator with the EKS-optimized Bottlerocket NVIDIA AMIs, you must disable the operator installation of the driver, toolkit, and device plugin as these are already included in the EKS AMIs.

In addition to the standard EKS AMI components, the EKS-optimized Bottlerocket NVIDIA AMIs include the following components. The minimal dependencies for EFA (kernel module and rdma-core) are installed in all Bottlerocket variants.

For details on the NVIDIA CUDA user mode driver and the CUDA runtime/libraries used within application containers, see the NVIDIA documentation. The CUDA version shown from nvidia-smi is the version of the NVIDIA CUDA user mode driver installed on the host, which must be compatible with the CUDA runtime/libraries used in application containers.

See the Bottlerocket Version Information in the Bottlerocket documentation for details on the installed packages and their versions. The EKS-optimized Bottlerocket NVIDIA AMIs support kernel 6.12 and NVIDIA driver 580 version for Kubernetes versions 1.33 and above. The NVIDIA 580 driver is required to use CUDA 13+.

EKS-optimized Neuron AMIs

For details on how to run training and inference workloads using Neuron with Amazon EKS, see the following references:

To find the latest EKS-optimized Neuron AMIs, see Retrieve recommended Amazon Linux AMI IDs and Retrieve recommended Bottlerocket AMI IDs.

When using Amazon Elastic Fabric Adaptor (EFA) with the EKS-optimized AL2023 or Bottlerocket Neuron AMIs, you must install the EFA device plugin separately. For more information, see Run machine learning training on Amazon EKS with Elastic Fabric Adapter.

EKS AL2023 Neuron AMIs

The EKS-optimized AL2023 Neuron AMIs do not include the Neuron Kubernetes device plugin or the Neuron Kubernetes scheduler extension, and these must be installed separately. For more information, see Install Neuron Kubernetes device plugin.

In addition to the standard EKS AMI components, the EKS-optimized AL2023 Neuron AMIs include the following components.

See the EKS AL2023 Neuron AMI installation script for details on how the EKS AMIs configure the Neuron dependencies. See the EKS-optimized AL2023 releases on GitHub to see the component versions included in the AMIs. You can find the list of installed packages and their versions on a running EC2 instance with the dnf list installed command.

EKS Bottlerocket Neuron AMIs

The standard Bottlerocket variants (aws-k8s) include the Neuron dependencies that are automatically detected and loaded when running on AWS Inferentia or Trainium EC2 instances.

The EKS-optimized Bottlerocket AMIs do not include the Neuron Kubernetes device plugin or the Neuron Kubernetes scheduler extension, and these must be installed separately. For more information, see Install Neuron Kubernetes device plugin.

In addition to the standard EKS AMI components, the EKS-optimized Bottlerocket Neuron AMIs include the following components.

When using the EKS-optimized Bottlerocket AMIs with Neuron instances, the following must be configured in the Bottlerocket user-data. This setting allows the container to take ownership of the mounted Neuron device based on the runAsUser and runAsGroup values provided in the workload specification. For more information on Neuron support in Bottlerocket, see the Quickstart on EKS readme on GitHub.

[settings]
[settings.kubernetes]
device-ownership-from-security-context = true

See the Bottlerocket kernel kit changelog for information on the Neuron driver version included in the EKS-optimized Bottlerocket AMIs.