# Enable enhanced networking with ENA on your EC2 instances
Elastic Network Adapter (ENA)
Amazon EC2 provides enhanced networking capabilities through the Elastic Network Adapter (ENA). To use enhanced networking, you must use an AMI that includes the required ENA driver or manually install it. Then you can enable ENA support on your instance.
To review release notes or install instructions for an ENA driver, see the tab that matches your instance operating system platform.
------
#### [ Linux ]
You can review the following documentation on GitHub:
+ Review [ENA Linux kernel driver release notes](https://github.com/amzn/amzn-drivers/blob/master/kernel/linux/ena/RELEASENOTES.md) on GitHub.
+ For an overview of the ENA Linux kernel driver that includes install instructions see [Linux kernel driver for Elastic Network Adapter (ENA) family](https://github.com/amzn/amzn-drivers/blob/master/kernel/linux/ena/README.rst) on GitHub.
------
#### [ Windows ]
You can review the following documentation from the **Manage device drivers** section of this guide:
+ [Track ENA Windows driver version releases](ena-driver-releases-windows.md).
+ [Install the ENA driver on EC2 Windows instances](ena-adapter-driver-install-upgrade-win.md).
------
For Nitro-based instances, enhanced networking capabilities vary by the Nitro version that the instance type implements.
To review network specifications for your instance, choose the instance family link for your instance type. If you're not sure which instance family applies, see [Naming conventions](https://docs.aws.amazon.com/ec2/latest/instancetypes/instance-type-names.html) in the *Amazon EC2 Instance Types* guide.
+ [Network specifications for accelerated computing instances](https://docs.aws.amazon.com/ec2/latest/instancetypes/ac.html#ac_network)
+ [Network specifications for compute optimized instances](https://docs.aws.amazon.com/ec2/latest/instancetypes/co.html#co_network)
+ [Network specifications for general purpose instances](https://docs.aws.amazon.com/ec2/latest/instancetypes/gp.html#gp_network)
+ [Network specifications for high-performance computing instances](https://docs.aws.amazon.com/ec2/latest/instancetypes/hpc.html#hpc_network)
+ [Network specifications for memory optimized instances](https://docs.aws.amazon.com/ec2/latest/instancetypes/mo.html#mo_network)
+ [Network specifications for storage optimized instances](https://docs.aws.amazon.com/ec2/latest/instancetypes/so.html#so_network)
**Topics**
+ [
## Prerequisites for enhanced networking with ENA
](#ena-requirements)
+ [
# Test whether enhanced networking is enabled
](test-enhanced-networking-ena.md)
+ [
# Enable enhanced networking on your instance
](enabling_enhanced_networking.md)
+ [
# ENA queues
](ena-queues.md)
+ [
# Troubleshoot the ENA kernel driver on Linux
](troubleshooting-ena.md)
+ [
# Troubleshoot the Elastic Network Adapter Windows driver
](troubleshoot-ena-driver.md)
## Prerequisites for enhanced networking with ENA
To prepare for enhanced networking using the ENA, set up your instance as follows:
+ Launch a [Nitro-based instance](instance-types.md#instance-hypervisor-type).
+ Ensure that the instance has internet connectivity.
+ If you have important data on the instance that you want to preserve, you should back that data up now by creating an AMI from your instance. Updating the ENA kernel driver and enabling the `enaSupport` attribute might render incompatible instances or operating systems unreachable. If you have a recent backup, your data will still be retained if this happens.
+ **Linux instances** – Launch the instance using a supported version of the Linux kernel and a supported distribution, so that ENA enhanced networking is enabled for your instance automatically. For more information, see [ENA Linux Kernel Driver Release Notes](https://github.com/amzn/amzn-drivers/blob/master/kernel/linux/ena/RELEASENOTES.md).
+ **Windows instances** – If the instance is running Windows Server 2008 R2 SP1, ensure that is has the [SHA-2 code signing support update](https://support.microsoft.com/en-us/help/4474419/sha-2-code-signing-support-update).
+ Use [AWS CloudShell](https://console.aws.amazon.com/cloudshell) from the AWS Management Console, or install and configure the [AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-getting-started.html) or the [AWS Tools for Windows PowerShell](https://docs.aws.amazon.com/powershell/latest/userguide/) on any computer you choose, preferably your local desktop or laptop. For more information, see [Access Amazon EC2](concepts.md#access-ec2) or the [AWS CloudShell User Guide](https://docs.aws.amazon.com/cloudshell/latest/userguide/welcome.html). Enhanced networking cannot be managed from the Amazon EC2 console.
# Test whether enhanced networking is enabled
Check whether ENA is enabled
You can test whether enhanced networking is enabled in your instances or your AMIs.
**Instance attribute**
Check the value of the `enaSupport` instance attribute.
------
#### [ AWS CLI ]
Use the [https://docs.aws.amazon.com/cli/latest/reference/ec2/describe-instances.html](https://docs.aws.amazon.com/cli/latest/reference/ec2/describe-instances.html) command.
```
aws ec2 describe-instances \
--instance-ids i-1234567890abcdef0 \
--query "Reservations[].Instances[].EnaSupport"
```
If enhanced networking is enabled, the output is as follows.
```
[
true
]
```
------
#### [ PowerShell ]
Use the [https://docs.aws.amazon.com/powershell/latest/reference/items/Get-EC2Instance.html](https://docs.aws.amazon.com/powershell/latest/reference/items/Get-EC2Instance.html) cmdlet.
```
(Get-EC2Instance -InstanceId i-1234567890abcdef0).Instances.EnaSupport
```
If enhanced networking is enabled, the output is as follows.
```
True
```
------
**Image attribute**
Check the value of the `enaSupport` image attribute.
------
#### [ AWS CLI ]
Use the [https://docs.aws.amazon.com/cli/latest/reference/ec2/describe-images.html](https://docs.aws.amazon.com/cli/latest/reference/ec2/describe-images.html) command.
```
aws ec2 describe-images \
--image-id ami-0abcdef1234567890 \
--query "Images[].EnaSupport"
```
If enhanced networking is enabled, the output is as follows.
```
[
true
]
```
------
#### [ PowerShell ]
Use the [https://docs.aws.amazon.com/powershell/latest/reference/items/Get-EC2Image.html](https://docs.aws.amazon.com/powershell/latest/reference/items/Get-EC2Image.html) cmdlet.
```
(Get-EC2Image -ImageId ami-0abcdef1234567890).EnaSupport
```
If enhanced networking is enabled, the output is as follows.
```
True
```
------
**Linux network interface driver**
Use the following command to verify that the `ena` kernel driver is being used on a particular interface, substituting the interface name that you want to check. If you are using a single interface (default), this is `eth0`. If your Linux distribution supports predictable network names, this could be a name like `ens5`. For more information, expand the section for RHEL, SUSE, and CentOS in [Enable enhanced networking on your instance](enabling_enhanced_networking.md).
In the following example, the `ena` kernel driver is not loaded, because the listed driver is `vif`.
```
[ec2-user ~]$ ethtool -i eth0
driver: vif
version:
firmware-version:
bus-info: vif-0
supports-statistics: yes
supports-test: no
supports-eeprom-access: no
supports-register-dump: no
supports-priv-flags: no
```
In this example, the `ena` kernel driver is loaded and at the minimum recommended version. This instance has enhanced networking properly configured.
```
[ec2-user ~]$ ethtool -i eth0
driver: ena
version: 1.5.0g
firmware-version:
expansion-rom-version:
bus-info: 0000:00:05.0
supports-statistics: yes
supports-test: no
supports-eeprom-access: no
supports-register-dump: no
supports-priv-flags: no
```
# Enable enhanced networking on your instance
Enable ENA for an instance
The procedure that you use depends on the operating system of the instance.
## Amazon Linux
The AMIs for Amazon Linux include the kernel driver required for enhanced networking with ENA installed and have ENA support enabled. Therefore, if you launch an instance with an HVM version of Amazon Linux on a supported instance type, enhanced networking is already enabled for your instance. For more information, see [Test whether enhanced networking is enabled](test-enhanced-networking-ena.md).
## Ubuntu
The latest Ubuntu HVM AMIs include the kernel driver required for enhanced networking with ENA installed and have ENA support enabled. Therefore, if you launch an instance with the latest Ubuntu HVM AMI on a supported instance type, enhanced networking is already enabled for your instance. For more information, see [Test whether enhanced networking is enabled](test-enhanced-networking-ena.md).
If you launched your instance using an older AMI and it does not have enhanced networking enabled already, you can install the `linux-aws` kernel package to get the latest enhanced networking drivers and update the required attribute.
**To install the `linux-aws` kernel package (Ubuntu 16.04 or later)**
Ubuntu 16.04 and 18.04 ship with the Ubuntu custom kernel (`linux-aws` kernel package). To use a different kernel, contact [Support](https://console.aws.amazon.com/support).
**To install the `linux-aws` kernel package (Ubuntu Trusty 14.04)**
1. Connect to your instance.
1. Update the package cache and packages.
```
ubuntu:~$ sudo apt-get update && sudo apt-get upgrade -y linux-aws
```
**Important**
If during the update process you are prompted to install `grub`, use `/dev/xvda` to install `grub` onto, and then choose to keep the current version of `/boot/grub/menu.lst`.
1. [EBS-backed instance] From your local computer, stop the instance using the Amazon EC2 console or one of the following commands: [https://docs.aws.amazon.com/cli/latest/reference/ec2/stop-instances.html](https://docs.aws.amazon.com/cli/latest/reference/ec2/stop-instances.html) (AWS CLI) or [https://docs.aws.amazon.com/powershell/latest/reference/items/Stop-EC2Instance.html](https://docs.aws.amazon.com/powershell/latest/reference/items/Stop-EC2Instance.html) (AWS Tools for Windows PowerShell).
[Instance store-backed instance] You can't stop the instance to modify the attribute. Instead, proceed to this procedure: [To enable enhanced networking on Ubuntu (instance store-backed instances)](#enhanced-networking-ena-instance-store-ubuntu).
1. From your local computer, enable the enhanced networking attribute using one of the following commands:
+ [https://docs.aws.amazon.com/cli/latest/reference/ec2/modify-instance-attribute.html](https://docs.aws.amazon.com/cli/latest/reference/ec2/modify-instance-attribute.html) (AWS CLI)
```
aws ec2 modify-instance-attribute --instance-id i-1234567890abcdef0 --ena-support
```
+ [https://docs.aws.amazon.com/powershell/latest/reference/items/Edit-EC2InstanceAttribute.html](https://docs.aws.amazon.com/powershell/latest/reference/items/Edit-EC2InstanceAttribute.html) (Tools for Windows PowerShell)
```
Edit-EC2InstanceAttribute -InstanceId i-1234567890abcdef0 -EnaSupport $true
```
1. (Optional) Create an AMI from the instance, as described in [Create an Amazon EBS-backed AMI](creating-an-ami-ebs.md). The AMI inherits the enhanced networking `enaSupport` attribute from the instance. Therefore, you can use this AMI to launch another instance with enhanced networking enabled by default.
1. From your local computer, start the instance using the Amazon EC2 console or one of the following commands: [https://docs.aws.amazon.com/cli/latest/reference/ec2/start-instances.html](https://docs.aws.amazon.com/cli/latest/reference/ec2/start-instances.html) (AWS CLI) or [https://docs.aws.amazon.com/powershell/latest/reference/items/Start-EC2Instance.html](https://docs.aws.amazon.com/powershell/latest/reference/items/Start-EC2Instance.html) (AWS Tools for Windows PowerShell).
**To enable enhanced networking on Ubuntu (instance store-backed instances)**
Follow the previous procedure until the step where you stop the instance. Create a new AMI as described in [Create an Amazon S3-backed AMI](creating-an-ami-instance-store.md), making sure to enable the enhanced networking attribute when you register the AMI.
+ [https://docs.aws.amazon.com/cli/latest/reference/ec2/register-image.html](https://docs.aws.amazon.com/cli/latest/reference/ec2/register-image.html) (AWS CLI)
```
aws ec2 register-image --ena-support ...
```
+ [https://docs.aws.amazon.com/powershell/latest/reference/items/Register-EC2Image.html](https://docs.aws.amazon.com/powershell/latest/reference/items/Register-EC2Image.html) (AWS Tools for Windows PowerShell)
```
Register-EC2Image -EnaSupport $true ...
```
## RHEL, SUSE, CentOS
The latest AMIs for Red Hat Enterprise Linux, SUSE Linux Enterprise Server, and CentOS include the kernel driver required for enhanced networking with ENA and have ENA support enabled. Therefore, if you launch an instance with the latest AMI on a supported instance type, enhanced networking is already enabled for your instance. For more information, see [Test whether enhanced networking is enabled](test-enhanced-networking-ena.md).
The following procedure provides the general steps for enabling enhanced networking on a Linux distribution other than Amazon Linux AMI or Ubuntu. For more information, such as detailed syntax for commands, file locations, or package and tool support, see the documentation for your Linux distribution.
**To enable enhanced networking on Linux**
1. Connect to your instance.
1. Clone the source code for the `ena` kernel driver on your instance from GitHub at [https://github.com/amzn/amzn-drivers](https://github.com/amzn/amzn-drivers). (SUSE Linux Enterprise Server 12 SP2 and later include ENA 2.02 by default, so you are not required to download and compile the ENA driver. For SUSE Linux Enterprise Server 12 SP2 and later, you should file a request to add the driver version you want to the stock kernel).
```
git clone https://github.com/amzn/amzn-drivers
```
1. Compile and install the `ena` kernel driver on your instance. These steps depend on the Linux distribution. For more information about compiling the kernel driver on Red Hat Enterprise Linux, see [How do I install the latest ENS driver for enhanced network support on an Amazon EC2 instance that runs RHEL?](https://repost.aws/knowledge-center/install-ena-driver-rhel-ec2)
1. Run the **sudo depmod** command to update kernel driver dependencies.
1. Update `initramfs` on your instance to ensure that the new kernel driver loads at boot time. For example, if your distribution supports **dracut**, you can use the following command.
```
dracut -f -v
```
1. Determine if your system uses predictable network interface names by default. Systems that use **systemd** or **udev** versions 197 or greater can rename Ethernet devices and they do not guarantee that a single network interface will be named `eth0`. This behavior can cause problems connecting to your instance. For more information and to see other configuration options, see [Predictable Network Interface Names](https://www.freedesktop.org/wiki/Software/systemd/PredictableNetworkInterfaceNames/) on the freedesktop.org website.
1. You can check the **systemd** or **udev** versions on RPM-based systems with the following command.
```
rpm -qa | grep -e '^systemd-[0-9]\+\|^udev-[0-9]\+'
systemd-208-11.el7_0.2.x86_64
```
In the above Red Hat Enterprise Linux 7 example, the **systemd** version is 208, so predictable network interface names must be disabled.
1. Disable predictable network interface names by adding the `net.ifnames=0` option to the `GRUB_CMDLINE_LINUX` line in `/etc/default/grub`.
```
sudo sed -i '/^GRUB\_CMDLINE\_LINUX/s/\"$/\ net\.ifnames\=0\"/' /etc/default/grub
```
1. Rebuild the grub configuration file.
```
sudo grub2-mkconfig -o /boot/grub2/grub.cfg
```
1. [EBS-backed instance] From your local computer, stop the instance using the Amazon EC2 console or one of the following commands: [https://docs.aws.amazon.com/cli/latest/reference/ec2/stop-instances.html](https://docs.aws.amazon.com/cli/latest/reference/ec2/stop-instances.html) (AWS CLI), [https://docs.aws.amazon.com/powershell/latest/reference/items/Stop-EC2Instance.html](https://docs.aws.amazon.com/powershell/latest/reference/items/Stop-EC2Instance.html) (AWS Tools for Windows PowerShell).
[Instance store-backed instance] You can't stop the instance to modify the attribute. Instead, proceed to this procedure: [To enable enhanced networking on Linux (instance store–backed instances)](#other-linux-enhanced-networking-ena-instance-store).
1. From your local computer, enable the enhanced networking `enaSupport` attribute using one of the following commands:
+ [https://docs.aws.amazon.com/cli/latest/reference/ec2/modify-instance-attribute.html](https://docs.aws.amazon.com/cli/latest/reference/ec2/modify-instance-attribute.html) (AWS CLI)
```
aws ec2 modify-instance-attribute --instance-id i-1234567890abcdef0 --ena-support
```
+ [https://docs.aws.amazon.com/powershell/latest/reference/items/Edit-EC2InstanceAttribute.html](https://docs.aws.amazon.com/powershell/latest/reference/items/Edit-EC2InstanceAttribute.html) (Tools for Windows PowerShell)
```
Edit-EC2InstanceAttribute -InstanceId i-1234567890abcdef0 -EnaSupport $true
```
1. (Optional) Create an AMI from the instance, as described in [Create an Amazon EBS-backed AMI](creating-an-ami-ebs.md). The AMI inherits the enhanced networking `enaSupport` attribute from the instance. Therefore, you can use this AMI to launch another instance with enhanced networking enabled by default.
If your instance operating system contains an `/etc/udev/rules.d/70-persistent-net.rules` file, you must delete it before creating the AMI. This file contains the MAC address for the Ethernet adapter of the original instance. If another instance boots with this file, the operating system will be unable to find the device and `eth0` might fail, causing boot issues. This file is regenerated at the next boot cycle, and any instances launched from the AMI create their own version of the file.
1. From your local computer, start the instance using the Amazon EC2 console or one of the following commands: [https://docs.aws.amazon.com/cli/latest/reference/ec2/start-instances.html](https://docs.aws.amazon.com/cli/latest/reference/ec2/start-instances.html) (AWS CLI) or [https://docs.aws.amazon.com/powershell/latest/reference/items/Start-EC2Instance.html](https://docs.aws.amazon.com/powershell/latest/reference/items/Start-EC2Instance.html) (AWS Tools for Windows PowerShell).
1. (Optional) Connect to your instance and verify that the kernel driver is installed.
If you are unable to connect to your instance after enabling enhanced networking, see [Troubleshoot the ENA kernel driver on Linux](troubleshooting-ena.md).
**To enable enhanced networking on Linux (instance store–backed instances)**
Follow the previous procedure until the step where you stop the instance. Create a new AMI as described in [Create an Amazon S3-backed AMI](creating-an-ami-instance-store.md), making sure to enable the enhanced networking attribute when you register the AMI.
+ [https://docs.aws.amazon.com/cli/latest/reference/ec2/register-image.html](https://docs.aws.amazon.com/cli/latest/reference/ec2/register-image.html) (AWS CLI)
```
aws ec2 register-image --ena-support ...
```
+ [https://docs.aws.amazon.com/powershell/latest/reference/items/Register-EC2Image.html](https://docs.aws.amazon.com/powershell/latest/reference/items/Register-EC2Image.html) (AWS Tools for Windows PowerShell)
```
Register-EC2Image -EnaSupport ...
```
## Ubuntu with DKMS
This method is for testing and feedback purposes only. It is not intended for use with production deployments. For production deployments, see [Ubuntu](#enhanced-networking-ena-ubuntu).
**Important**
Using DKMS voids the support agreement for your subscription. It should not be used for production deployments.
**To enable enhanced networking with ENA on Ubuntu (EBS-backed instances)**
1. Follow steps 1 and 2 in [Ubuntu](#enhanced-networking-ena-ubuntu).
1. Install the `build-essential` packages to compile the kernel driver and the `dkms` package so that your `ena` kernel driver is rebuilt every time your kernel is updated.
```
ubuntu:~$ sudo apt-get install -y build-essential dkms
```
1. Clone the source for the `ena` kernel driver on your instance from GitHub at [https://github.com/amzn/amzn-drivers](https://github.com/amzn/amzn-drivers).
```
ubuntu:~$ git clone https://github.com/amzn/amzn-drivers
```
1. Move the `amzn-drivers` package to the `/usr/src/` directory so DKMS can find it and build it for each kernel update. Append the version number (you can find the current version number in the release notes) of the source code to the directory name. For example, version `1.0.0` is shown in the following example.
```
ubuntu:~$ sudo mv amzn-drivers /usr/src/amzn-drivers-1.0.0
```
1. Create the DKMS configuration file with the following values, substituting your version of `ena`.
Create the file.
```
ubuntu:~$ sudo touch /usr/src/amzn-drivers-1.0.0/dkms.conf
```
Edit the file and add the following values.
```
ubuntu:~$ sudo vim /usr/src/amzn-drivers-1.0.0/dkms.conf
PACKAGE_NAME="ena"
PACKAGE_VERSION="1.0.0"
CLEAN="make -C kernel/linux/ena clean"
MAKE="make -C kernel/linux/ena/ BUILD_KERNEL=${kernelver}"
BUILT_MODULE_NAME[0]="ena"
BUILT_MODULE_LOCATION="kernel/linux/ena"
DEST_MODULE_LOCATION[0]="/updates"
DEST_MODULE_NAME[0]="ena"
AUTOINSTALL="yes"
```
1. Add, build, and install the `ena` kernel driver on your instance using DKMS.
Add the kernel driver to DKMS.
```
ubuntu:~$ sudo dkms add -m amzn-drivers -v 1.0.0
```
Build the kernel driver using the **dkms** command.
```
ubuntu:~$ sudo dkms build -m amzn-drivers -v 1.0.0
```
Install the kernel driver using **dkms**.
```
ubuntu:~$ sudo dkms install -m amzn-drivers -v 1.0.0
```
1. Rebuild `initramfs` so the correct kernel driver is loaded at boot time.
```
ubuntu:~$ sudo update-initramfs -u -k all
```
1. Verify that the `ena` kernel driver is installed using the modinfo ena command from [Test whether enhanced networking is enabled](test-enhanced-networking-ena.md).
```
ubuntu:~$ modinfo ena
filename: /lib/modules/3.13.0-74-generic/updates/dkms/ena.ko
version: 1.0.0
license: GPL
description: Elastic Network Adapter (ENA)
author: Amazon.com, Inc. or its affiliates
srcversion: 9693C876C54CA64AE48F0CA
alias: pci:v00001D0Fd0000EC21sv*sd*bc*sc*i*
alias: pci:v00001D0Fd0000EC20sv*sd*bc*sc*i*
alias: pci:v00001D0Fd00001EC2sv*sd*bc*sc*i*
alias: pci:v00001D0Fd00000EC2sv*sd*bc*sc*i*
depends:
vermagic: 3.13.0-74-generic SMP mod_unload modversions
parm: debug:Debug level (0=none,...,16=all) (int)
parm: push_mode:Descriptor / header push mode (0=automatic,1=disable,3=enable)
0 - Automatically choose according to device capability (default)
1 - Don't push anything to device memory
3 - Push descriptors and header buffer to device memory (int)
parm: enable_wd:Enable keepalive watchdog (0=disable,1=enable,default=1) (int)
parm: enable_missing_tx_detection:Enable missing Tx completions. (default=1) (int)
parm: numa_node_override_array:Numa node override map
(array of int)
parm: numa_node_override:Enable/Disable numa node override (0=disable)
(int)
```
1. Continue with Step 3 in [Ubuntu](#enhanced-networking-ena-ubuntu).
## Enable enhanced networking on Windows
If you launched your instance and it does not have enhanced networking enabled already, you must download and install the required network adapter driver on your instance, and then set the `enaSupport` instance attribute to activate enhanced networking.
**To enable enhanced networking**
1. Connect to your instance and log in as the local administrator.
1. [Windows Server 2016 and 2019 only] Run the following EC2Launch PowerShell script to configure the instance after the driver is installed.
```
PS C:\> C:\ProgramData\Amazon\EC2-Windows\Launch\Scripts\InitializeInstance.ps1 -Schedule
```
1. From the instance, install the driver as follows:
1. [Download](https://s3.amazonaws.com/ec2-windows-drivers-downloads/ENA/Latest/AwsEnaNetworkDriver.zip) the latest driver to the instance.
1. Extract the zip archive.
1. Install the driver by running the `install.ps1` PowerShell script.
**Note**
If you get an execution policy error, set the policy to `Unrestricted` (by default it is set to `Restricted` or `RemoteSigned`). In a command line, run `Set-ExecutionPolicy -ExecutionPolicy Unrestricted`, and then run the `install.ps1` PowerShell script again.
1. From your local computer, stop the instance using the Amazon EC2 console or one of the following commands: [https://docs.aws.amazon.com/cli/latest/reference/ec2/stop-instances.html](https://docs.aws.amazon.com/cli/latest/reference/ec2/stop-instances.html) (AWS CLI) or [https://docs.aws.amazon.com/powershell/latest/reference/items/Stop-EC2Instance.html](https://docs.aws.amazon.com/powershell/latest/reference/items/Stop-EC2Instance.html) (AWS Tools for Windows PowerShell).
1. Enable ENA support on your instance as follows:
1. From your local computer, check the EC2 instance ENA support attribute on your instance by running one of the following commands. If the attribute is not enabled, the output will be "[]" or blank. `EnaSupport` is set to `false` by default.
+ [https://docs.aws.amazon.com/cli/latest/reference/ec2/describe-instances.html](https://docs.aws.amazon.com/cli/latest/reference/ec2/describe-instances.html) (AWS CLI)
```
aws ec2 describe-instances --instance-ids i-1234567890abcdef0 --query "Reservations[].Instances[].EnaSupport"
```
+ [https://docs.aws.amazon.com/powershell/latest/reference/items/Get-EC2Instance.html](https://docs.aws.amazon.com/powershell/latest/reference/items/Get-EC2Instance.html) (Tools for Windows PowerShell)
```
(Get-EC2Instance -InstanceId i-1234567890abcdef0).Instances.EnaSupport
```
1. To enable ENA support, run one of the following commands:
+ [https://docs.aws.amazon.com/cli/latest/reference/ec2/modify-instance-attribute.html](https://docs.aws.amazon.com/cli/latest/reference/ec2/modify-instance-attribute.html) (AWS CLI)
```
aws ec2 modify-instance-attribute --instance-id i-1234567890abcdef0 --ena-support
```
+ [https://docs.aws.amazon.com/powershell/latest/reference/items/Edit-EC2InstanceAttribute.html](https://docs.aws.amazon.com/powershell/latest/reference/items/Edit-EC2InstanceAttribute.html) (AWS Tools for Windows PowerShell)
```
Edit-EC2InstanceAttribute -InstanceId i-1234567890abcdef0 -EnaSupport $true
```
If you encounter problems when you restart the instance, you can also disable ENA support using one of the following commands:
+ [https://docs.aws.amazon.com/cli/latest/reference/ec2/modify-instance-attribute.html](https://docs.aws.amazon.com/cli/latest/reference/ec2/modify-instance-attribute.html) (AWS CLI)
```
aws ec2 modify-instance-attribute --instance-id i-1234567890abcdef0 --no-ena-support
```
+ [https://docs.aws.amazon.com/powershell/latest/reference/items/Edit-EC2InstanceAttribute.html](https://docs.aws.amazon.com/powershell/latest/reference/items/Edit-EC2InstanceAttribute.html) (AWS Tools for Windows PowerShell)
```
Edit-EC2InstanceAttribute -InstanceId i-1234567890abcdef0 -EnaSupport $false
```
1. Verify that the attribute has been set to `true` using **describe-instances** or **Get-EC2Instance** as shown previously. You should now see the following output:
```
[
true
]
```
1. From your local computer, start the instance using the Amazon EC2 console or one of the following commands: [https://docs.aws.amazon.com/cli/latest/reference/ec2/start-instances.html](https://docs.aws.amazon.com/cli/latest/reference/ec2/start-instances.html) (AWS CLI) or [https://docs.aws.amazon.com/powershell/latest/reference/items/Start-EC2Instance.html](https://docs.aws.amazon.com/powershell/latest/reference/items/Start-EC2Instance.html) (AWS Tools for Windows PowerShell).
1. On the instance, validate that the ENA driver is installed and enabled as follows:
1. Right-click the network icon and choose **Open Network and Sharing Center**.
1. Choose the Ethernet adapter (for example, **Ethernet 2**).
1. Choose **Details**. For **Network Connection Details**, check that **Description** is **Amazon Elastic Network Adapter**.
1. (Optional) Create an AMI from the instance. The AMI inherits the `enaSupport` attribute from the instance. Therefore, you can use this AMI to launch another instance with ENA enabled by default.
# ENA queues
ENA queues are allocated to network interfaces with default static limits based on the instance type and size. On supported instance types, you can dynamically allocate these queues across Elastic Network Interfaces (ENIs). While the total queue count per instance depends on its type and size, you can configure multiple ENIs with ENA queues until you meet the maximum queue count for the ENI and the instance.
Flexible ENA queue allocation optimizes resource distribution, enabling maximum vCPU utilization. High network performance workloads typically require multiple ENA queues. You can fine-tune network performance and packets per second (PPS) by adjusting queue counts according to your specific workload needs. For example, network-intensive applications may require more queues compared to CPU-intensive applications.
**Topics**
+ [
## Supported instances
](#supported-instances)
+ [
## Modify the number of queues
](#modify)
## Supported instances
The following instances support dynamic allocation of multiple ENA queues.
### General purpose
| Instance type | Default ENA queues per interface | Maximum ENA queues per interface | Maximum ENA queues per instance |
| --- | --- | --- | --- |
| M6i |
| m6i.large | 2 | 2 | 6 |
| m6i.xlarge | 4 | 4 | 16 |
| m6i.2xlarge | 8 | 8 | 32 |
| m6i.4xlarge | 8 | 16 | 64 |
| m6i.8xlarge | 8 | 32 | 64 |
| m6i.12xlarge | 8 | 32 | 64 |
| m6i.16xlarge | 8 | 32 | 120 |
| m6i.24xlarge | 8 | 32 | 120 |
| m6i.32xlarge | 8 | 32 | 120 |
| M6id |
| m6id.large | 2 | 2 | 6 |
| m6id.xlarge | 4 | 4 | 16 |
| m6id.2xlarge | 8 | 8 | 32 |
| m6id.4xlarge | 8 | 16 | 64 |
| m6id.8xlarge | 8 | 32 | 64 |
| m6id.12xlarge | 8 | 32 | 64 |
| m6id.16xlarge | 8 | 32 | 120 |
| m6id.24xlarge | 8 | 32 | 120 |
| m6id.32xlarge | 8 | 32 | 120 |
| M6idn |
| m6idn.large | 2 | 2 | 6 |
| m6idn.xlarge | 4 | 4 | 16 |
| m6idn.2xlarge | 8 | 8 | 32 |
| m6idn.4xlarge | 8 | 16 | 64 |
| m6idn.8xlarge | 16 | 32 | 128 |
| m6idn.12xlarge | 16 | 32 | 128 |
| m6idn.16xlarge | 16 | 32 | 240 |
| m6idn.24xlarge | 32 | 32 | 480 |
| m6idn.32xlarge | 32 | 32 | 512 \$1 |
| M6in |
| m6in.large | 2 | 2 | 6 |
| m6in.xlarge | 4 | 4 | 16 |
| m6in.2xlarge | 8 | 8 | 32 |
| m6in.4xlarge | 8 | 16 | 64 |
| m6in.8xlarge | 16 | 32 | 128 |
| m6in.12xlarge | 16 | 32 | 128 |
| m6in.16xlarge | 16 | 32 | 240 |
| m6in.24xlarge | 32 | 32 | 480 |
| m6in.32xlarge | 32 | 32 | 512 \$1 |
| M8a |
| m8a.medium | 1 | 1 | 3 |
| m8a.large | 2 | 2 | 6 |
| m8a.xlarge | 4 | 4 | 16 |
| m8a.2xlarge | 8 | 8 | 32 |
| m8a.4xlarge | 8 | 16 | 64 |
| m8a.8xlarge | 8 | 32 | 128 |
| m8a.12xlarge | 16 | 64 | 192 |
| m8a.16xlarge | 16 | 64 | 256 |
| m8a.24xlarge | 16 | 128 | 384 |
| m8a.48xlarge | 32 | 128 | 768 |
| m8a.metal-24xl | 16 | 128 | 384 |
| m8a.metal-48xl | 32 | 128 | 768 |
| M8azn |
| m8azn.medium | 1 | 1 | 3 |
| m8azn.large | 2 | 2 | 8 |
| m8azn.xlarge | 4 | 4 | 16 |
| m8azn.3xlarge | 4 | 16 | 48 |
| m8azn.6xlarge | 8 | 32 | 96 |
| m8azn.12xlarge | 8 | 64 | 192 |
| m8azn.24xlarge | 16 | 128 | 384 |
| m8azn.metal-12xl | 8 | 64 | 192 |
| m8azn.metal-24xl | 16 | 128 | 384 |
| M8gb |
| m8gb.medium | 1 | 1 | 2 |
| m8gb.large | 2 | 2 | 6 |
| m8gb.xlarge | 4 | 4 | 16 |
| m8gb.2xlarge | 8 | 8 | 32 |
| m8gb.4xlarge | 8 | 16 | 64 |
| m8gb.8xlarge | 8 | 32 | 128 |
| m8gb.12xlarge | 16 | 64 | 192 |
| m8gb.16xlarge | 16 | 64 | 256 |
| m8gb.24xlarge | 16 | 128 | 384 |
| m8gb.48xlarge | 32 | 128 | 768 \$1 |
| m8gb.metal-24xl | 32 | 128 | 768 |
| m8gb.metal-48xl | 32 | 128 | 768 \$1 |
| M8gn |
| m8gn.medium | 1 | 1 | 2 |
| m8gn.large | 2 | 2 | 6 |
| m8gn.xlarge | 4 | 4 | 16 |
| m8gn.2xlarge | 8 | 8 | 32 |
| m8gn.4xlarge | 8 | 16 | 64 |
| m8gn.8xlarge | 8 | 32 | 128 |
| m8gn.12xlarge | 16 | 64 | 192 |
| m8gn.16xlarge | 16 | 64 | 256 |
| m8gn.24xlarge | 16 | 128 | 384 |
| m8gn.48xlarge | 32 | 128 | 768 \$1 |
| m8gn.metal-24xl | 32 | 128 | 768 |
| m8gn.metal-48xl | 32 | 128 | 768 \$1 |
| M8i |
| m8i.large | 2 | 2 | 6 |
| m8i.xlarge | 4 | 4 | 16 |
| m8i.2xlarge | 8 | 8 | 32 |
| m8i.4xlarge | 8 | 16 | 64 |
| m8i.8xlarge | 8 | 32 | 128 |
| m8i.12xlarge | 16 | 64 | 192 |
| m8i.16xlarge | 16 | 64 | 256 |
| m8i.24xlarge | 16 | 128 | 384 |
| m8i.32xlarge | 16 | 128 | 512 |
| m8i.48xlarge | 32 | 128 | 768 |
| m8i.96xlarge | 32 | 128 | 1536 |
| m8i.metal-48xl | 32 | 128 | 768 |
| m8i.metal-96xl | 32 | 128 | 1536 |
| M8id |
| m8id.large | 2 | 2 | 6 |
| m8id.xlarge | 4 | 4 | 16 |
| m8id.2xlarge | 8 | 8 | 32 |
| m8id.4xlarge | 8 | 16 | 64 |
| m8id.8xlarge | 8 | 32 | 128 |
| m8id.12xlarge | 16 | 64 | 192 |
| m8id.16xlarge | 16 | 64 | 256 |
| m8id.24xlarge | 16 | 128 | 384 |
| m8id.32xlarge | 16 | 128 | 512 |
| m8id.48xlarge | 32 | 128 | 768 |
| m8id.96xlarge | 32 | 128 | 1536 |
| m8id.metal-48xl | 32 | 128 | 768 |
| m8id.metal-96xl | 32 | 128 | 1536 |
| M8i-flex |
| m8i-flex.large | 1 | 1 | 3 |
| m8i-flex.xlarge | 2 | 2 | 8 |
| m8i-flex.2xlarge | 4 | 4 | 16 |
| m8i-flex.4xlarge | 4 | 8 | 32 |
| m8i-flex.8xlarge | 4 | 16 | 64 |
| m8i-flex.12xlarge | 8 | 32 | 96 |
| m8i-flex.16xlarge | 8 | 32 | 128 |
**Note**
\$1 These instance types feature multiple network cards. Other instance types feature a single network card. For more information, see [Network cards](using-eni.md#network-cards).
### Compute optimized
| Instance type | Default ENA queues per interface | Maximum ENA queues per interface | Maximum ENA queues per instance |
| --- | --- | --- | --- |
| C6i |
| c6i.large | 2 | 2 | 6 |
| c6i.xlarge | 4 | 4 | 16 |
| c6i.2xlarge | 8 | 8 | 32 |
| c6i.4xlarge | 8 | 16 | 64 |
| c6i.8xlarge | 8 | 32 | 64 |
| c6i.12xlarge | 8 | 32 | 64 |
| c6i.16xlarge | 8 | 32 | 120 |
| c6i.24xlarge | 8 | 32 | 120 |
| c6i.32xlarge | 8 | 32 | 120 |
| C6id |
| c6id.large | 2 | 2 | 6 |
| c6id.xlarge | 4 | 4 | 16 |
| c6id.2xlarge | 8 | 8 | 32 |
| c6id.4xlarge | 8 | 16 | 64 |
| c6id.8xlarge | 8 | 32 | 64 |
| c6id.12xlarge | 8 | 32 | 64 |
| c6id.16xlarge | 8 | 32 | 120 |
| c6id.24xlarge | 8 | 32 | 120 |
| c6id.32xlarge | 8 | 32 | 120 |
| C6in |
| c6in.large | 2 | 2 | 6 |
| c6in.xlarge | 4 | 4 | 16 |
| c6in.2xlarge | 8 | 8 | 32 |
| c6in.4xlarge | 8 | 16 | 64 |
| c6in.8xlarge | 16 | 32 | 128 |
| c6in.12xlarge | 16 | 32 | 128 |
| c6in.16xlarge | 16 | 32 | 240 |
| c6in.24xlarge | 32 | 32 | 480 |
| c6in.32xlarge | 32 | 32 | 512 \$1 |
| C8a |
| c8a.medium | 1 | 1 | 3 |
| c8a.large | 2 | 2 | 6 |
| c8a.xlarge | 4 | 4 | 16 |
| c8a.2xlarge | 8 | 8 | 32 |
| c8a.4xlarge | 8 | 16 | 64 |
| c8a.8xlarge | 8 | 32 | 128 |
| c8a.12xlarge | 16 | 64 | 192 |
| c8a.16xlarge | 16 | 64 | 256 |
| c8a.24xlarge | 16 | 128 | 384 |
| c8a.48xlarge | 32 | 128 | 768 |
| c8a.metal-24xl | 16 | 128 | 384 |
| c8a.metal-48xl | 32 | 128 | 768 |
| C8gb |
| c8gb.medium | 1 | 1 | 2 |
| c8gb.large | 2 | 2 | 6 |
| c8gb.xlarge | 4 | 4 | 16 |
| c8gb.2xlarge | 8 | 8 | 32 |
| c8gb.4xlarge | 8 | 16 | 64 |
| c8gb.8xlarge | 8 | 32 | 128 |
| c8gb.12xlarge | 16 | 64 | 192 |
| c8gb.16xlarge | 16 | 64 | 256 |
| c8gb.24xlarge | 16 | 128 | 384 |
| c8gb.48xlarge | 32 | 128 | 768 \$1 |
| c8gb.metal-24xl | 32 | 128 | 768 |
| c8gb.metal-48xl | 32 | 128 | 768 \$1 |
| C8gn |
| c8gn.medium | 1 | 1 | 2 |
| c8gn.large | 2 | 2 | 6 |
| c8gn.xlarge | 4 | 4 | 16 |
| c8gn.2xlarge | 8 | 8 | 32 |
| c8gn.4xlarge | 8 | 16 | 64 |
| c8gn.8xlarge | 8 | 32 | 128 |
| c8gn.12xlarge | 16 | 64 | 192 |
| c8gn.16xlarge | 16 | 64 | 256 |
| c8gn.24xlarge | 16 | 128 | 384 |
| c8gn.48xlarge | 32 | 128 | 768 \$1 |
| c8gn.metal-24xl | 32 | 128 | 768 |
| c8gn.metal-48xl | 32 | 128 | 768 \$1 |
| C8i |
| c8i.large | 2 | 2 | 6 |
| c8i.xlarge | 4 | 4 | 16 |
| c8i.2xlarge | 8 | 8 | 32 |
| c8i.4xlarge | 8 | 16 | 64 |
| c8i.8xlarge | 8 | 32 | 128 |
| c8i.12xlarge | 16 | 64 | 192 |
| c8i.16xlarge | 16 | 64 | 256 |
| c8i.24xlarge | 16 | 128 | 384 |
| c8i.32xlarge | 16 | 128 | 512 |
| c8i.48xlarge | 32 | 128 | 768 |
| c8i.96xlarge | 32 | 128 | 1536 |
| c8i.metal-48xl | 32 | 128 | 768 |
| c8i.metal-96xl | 32 | 128 | 1536 |
| C8id |
| c8id.large | 2 | 2 | 6 |
| c8id.xlarge | 4 | 4 | 16 |
| c8id.2xlarge | 8 | 8 | 32 |
| c8id.4xlarge | 8 | 16 | 64 |
| c8id.8xlarge | 8 | 32 | 128 |
| c8id.12xlarge | 16 | 64 | 192 |
| c8id.16xlarge | 16 | 64 | 256 |
| c8id.24xlarge | 16 | 128 | 384 |
| c8id.32xlarge | 16 | 128 | 512 |
| c8id.48xlarge | 32 | 128 | 768 |
| c8id.96xlarge | 32 | 128 | 1536 |
| c8id.metal-48xl | 32 | 128 | 768 |
| c8id.metal-96xl | 32 | 128 | 1536 |
| C8i-flex |
| c8i-flex.large | 1 | 1 | 3 |
| c8i-flex.xlarge | 2 | 2 | 8 |
| c8i-flex.2xlarge | 4 | 4 | 16 |
| c8i-flex.4xlarge | 4 | 8 | 32 |
| c8i-flex.8xlarge | 4 | 16 | 64 |
| c8i-flex.12xlarge | 8 | 32 | 96 |
| c8i-flex.16xlarge | 8 | 32 | 128 |
**Note**
\$1 These instance types feature multiple network cards. Other instance types feature a single network card. For more information, see [Network cards](using-eni.md#network-cards).
### Memory optimized
| Instance type | Default ENA queues per interface | Maximum ENA queues per interface | Maximum ENA queues per instance |
| --- | --- | --- | --- |
| R6i |
| r6i.large | 2 | 2 | 6 |
| r6i.xlarge | 4 | 4 | 16 |
| r6i.2xlarge | 8 | 8 | 32 |
| r6i.4xlarge | 8 | 16 | 64 |
| r6i.8xlarge | 8 | 32 | 64 |
| r6i.12xlarge | 8 | 32 | 64 |
| r6i.16xlarge | 8 | 32 | 120 |
| r6i.24xlarge | 8 | 32 | 120 |
| r6i.32xlarge | 8 | 32 | 120 |
| R6id |
| r6id.large | 2 | 2 | 6 |
| r6id.xlarge | 4 | 4 | 16 |
| r6id.2xlarge | 8 | 8 | 32 |
| r6id.4xlarge | 8 | 16 | 64 |
| r6id.8xlarge | 8 | 32 | 64 |
| r6id.12xlarge | 8 | 32 | 64 |
| r6id.16xlarge | 8 | 32 | 120 |
| r6id.24xlarge | 8 | 32 | 120 |
| r6id.32xlarge | 8 | 32 | 120 |
| R6idn |
| r6idn.large | 2 | 2 | 6 |
| r6idn.xlarge | 4 | 4 | 16 |
| r6idn.2xlarge | 8 | 8 | 32 |
| r6idn.4xlarge | 8 | 16 | 64 |
| r6idn.8xlarge | 16 | 32 | 128 |
| r6idn.12xlarge | 16 | 32 | 128 |
| r6idn.16xlarge | 16 | 32 | 240 |
| r6idn.24xlarge | 32 | 32 | 480 |
| r6idn.32xlarge | 32 | 32 | 512 \$1 |
| R6in |
| r6in.large | 2 | 2 | 6 |
| r6in.xlarge | 4 | 4 | 16 |
| r6in.2xlarge | 8 | 8 | 32 |
| r6in.4xlarge | 8 | 16 | 64 |
| r6in.8xlarge | 16 | 32 | 128 |
| r6in.12xlarge | 16 | 32 | 128 |
| r6in.16xlarge | 16 | 32 | 240 |
| r6in.24xlarge | 32 | 32 | 480 |
| r6in.32xlarge | 32 | 32 | 512 \$1 |
| R8a |
| r8a.medium | 1 | 1 | 3 |
| r8a.large | 2 | 2 | 6 |
| r8a.xlarge | 4 | 4 | 16 |
| r8a.2xlarge | 8 | 8 | 32 |
| r8a.4xlarge | 8 | 16 | 64 |
| r8a.8xlarge | 8 | 32 | 128 |
| r8a.12xlarge | 16 | 64 | 192 |
| r8a.16xlarge | 16 | 64 | 256 |
| r8a.24xlarge | 16 | 128 | 384 |
| r8a.48xlarge | 32 | 128 | 768 |
| r8a.metal-24xl | 16 | 128 | 384 |
| r8a.metal-48xl | 32 | 128 | 768 |
| R8gb |
| r8gb.medium | 1 | 1 | 2 |
| r8gb.large | 2 | 2 | 6 |
| r8gb.xlarge | 4 | 4 | 16 |
| r8gb.2xlarge | 8 | 8 | 32 |
| r8gb.4xlarge | 8 | 16 | 64 |
| r8gb.8xlarge | 8 | 32 | 128 |
| r8gb.12xlarge | 16 | 64 | 192 |
| r8gb.16xlarge | 16 | 64 | 256 |
| r8gb.24xlarge | 16 | 128 | 384 |
| r8gb.48xlarge | 32 | 128 | 768 \$1 |
| r8gb.metal-24xl | 32 | 128 | 768 |
| r8gb.metal-48xl | 32 | 128 | 768 \$1 |
| R8gn |
| r8gn.medium | 1 | 1 | 2 |
| r8gn.large | 2 | 2 | 6 |
| r8gn.xlarge | 4 | 4 | 16 |
| r8gn.2xlarge | 8 | 8 | 32 |
| r8gn.4xlarge | 8 | 16 | 64 |
| r8gn.8xlarge | 8 | 32 | 128 |
| r8gn.12xlarge | 16 | 64 | 192 |
| r8gn.16xlarge | 16 | 64 | 256 |
| r8gn.24xlarge | 16 | 128 | 384 |
| r8gn.48xlarge | 32 | 128 | 768 \$1 |
| r8gn.metal-24xl | 32 | 128 | 768 |
| r8gn.metal-48xl | 32 | 128 | 768 \$1 |
| R8i |
| r8i.large | 2 | 2 | 6 |
| r8i.xlarge | 4 | 4 | 16 |
| r8i.2xlarge | 8 | 8 | 32 |
| r8i.4xlarge | 8 | 16 | 64 |
| r8i.8xlarge | 8 | 32 | 128 |
| r8i.12xlarge | 16 | 64 | 192 |
| r8i.16xlarge | 16 | 64 | 256 |
| r8i.24xlarge | 16 | 128 | 384 |
| r8i.32xlarge | 16 | 128 | 512 |
| r8i.48xlarge | 32 | 128 | 768 |
| r8i.96xlarge | 32 | 128 | 1536 |
| r8i.metal-48xl | 32 | 128 | 768 |
| r8i.metal-96xl | 32 | 128 | 1536 |
| R8id |
| r8id.large | 2 | 2 | 6 |
| r8id.xlarge | 4 | 4 | 16 |
| r8id.2xlarge | 8 | 8 | 32 |
| r8id.4xlarge | 8 | 16 | 64 |
| r8id.8xlarge | 8 | 32 | 128 |
| r8id.12xlarge | 16 | 64 | 192 |
| r8id.16xlarge | 16 | 64 | 256 |
| r8id.24xlarge | 16 | 128 | 384 |
| r8id.32xlarge | 16 | 128 | 512 |
| r8id.48xlarge | 32 | 128 | 768 |
| r8id.96xlarge | 32 | 128 | 1536 |
| r8id.metal-48xl | 32 | 128 | 768 |
| r8id.metal-96xl | 32 | 128 | 1536 |
| R8i-flex |
| r8i-flex.large | 1 | 1 | 3 |
| r8i-flex.xlarge | 2 | 2 | 8 |
| r8i-flex.2xlarge | 4 | 4 | 16 |
| r8i-flex.4xlarge | 4 | 8 | 32 |
| r8i-flex.8xlarge | 4 | 16 | 64 |
| r8i-flex.12xlarge | 8 | 32 | 96 |
| r8i-flex.16xlarge | 8 | 32 | 128 |
| X8aedz |
| x8aedz.large | 2 | 2 | 8 |
| x8aedz.xlarge | 4 | 4 | 16 |
| x8aedz.3xlarge | 4 | 16 | 48 |
| x8aedz.6xlarge | 8 | 32 | 96 |
| x8aedz.12xlarge | 8 | 64 | 192 |
| x8aedz.24xlarge | 16 | 128 | 384 |
| x8aedz.metal-12xl | 8 | 64 | 192 |
| x8aedz.metal-24xl | 16 | 128 | 384 |
| X8i |
| x8i.large | 2 | 2 | 6 |
| x8i.xlarge | 4 | 4 | 16 |
| x8i.2xlarge | 8 | 8 | 32 |
| x8i.4xlarge | 8 | 16 | 64 |
| x8i.8xlarge | 8 | 32 | 128 |
| x8i.12xlarge | 16 | 64 | 192 |
| x8i.16xlarge | 16 | 64 | 256 |
| x8i.24xlarge | 16 | 128 | 384 |
| x8i.32xlarge | 16 | 128 | 512 |
| x8i.48xlarge | 32 | 128 | 768 |
| x8i.64xlarge | 32 | 128 | 1024 |
| x8i.96xlarge | 32 | 128 | 1536 |
| x8i.metal-48xl | 32 | 128 | 768 |
| x8i.metal-96xl | 32 | 128 | 1536 |
**Note**
\$1 These instance types feature multiple network cards. Other instance types feature a single network card. For more information, see [Network cards](using-eni.md#network-cards).
## Modify the number of queues
You can modify the number of ENA queues using AWS Management Console or AWS CLI. In the AWS Management Console, the ENA queues configuration is available under each **Network interface** setting.
To modify the number of ENA queues using the AWS CLI, use either one of the following commands. Before modifying the queue count, use the following command to check your current queue count.
```
aws ec2 describe-instances --instance-id i-1234567890abcdef0
```
**Note**
Your instance must be stopped before modifying the number of ENA queues.
The value for ENA queues must be a power of 2, such as, 1, 2, 4, 8, 16, 32, etc.
The number of queues allocated to any single ENI cannot exceed the number of vCPUs available on your instance.
`[attach-network-interface](https://docs.aws.amazon.com/cli/latest/reference/ec2/attach-network-interface.html)`
In the following example, 32 ENA queues are configured on an ENI.
```
aws ec2 attach-network-interface \
--network-interface-id eni-001aa1bb223cdd4e4 \
--instance-id i-1234567890abcdef0 \
--device-index 1 \
--ena-queue-count 32
```
`[run-instances](https://docs.aws.amazon.com/cli/latest/reference/ec2/run-instances.html)`
In the following example, 2 ENA queues each are configured on 3 ENIs.
```
aws ec2 run-instances \
--image-id ami-12ab3c30 \
--instance-type c6i.large \
--min-count 1 \
--max-count 1 \
--network-interfaces \
"[{\"DeviceIndex\":0,\"SubnetId\":\"subnet-123456789012a345a\",\"EnaQueueCount\":2},
{\"DeviceIndex\":1,\"SubnetId\":\"subnet-123456789012a345a\",\"EnaQueueCount\":2},
{\"DeviceIndex\":2,\"SubnetId\":\"subnet-123456789012a345a\",\"EnaQueueCount\":2}]"
```
`[modify-network-interface-attribute](https://docs.aws.amazon.com/cli/latest/reference/ec2/modify-network-interface-attribute.html)`
In the following example, 32 ENA queues are configured on an ENI.
```
aws ec2 modify-network-interface-attribute \
--network-interface-id eni-1234567890abcdef0 \
--attachment AttachmentId=eni-attach-12345678,EnaQueueCount=32
```
In the following example, the ENA count is reset to the default value.
```
aws ec2 modify-network-interface-attribute \
--network-interface-id eni-1234567890abcdef0 \
--attachment AttachmentId=eni-attach-12345678,DefaultEnaQueueCount=true
```
# Troubleshoot the ENA kernel driver on Linux
Troubleshoot ENA on Linux
The Elastic Network Adapter (ENA) is designed to improve operating system health and reduce the chances of long-term disruption because of unexpected hardware behavior and or failures. The ENA architecture keeps device or driver failures as transparent to the system as possible. This topic provides troubleshooting information for ENA.
If you are unable to connect to your instance, start with the [Troubleshoot connectivity issues](#ena-connectivity-issues) section.
If you experience performance degradation after migrating to a sixth generation instance type, see the article [What do I need to do before I migrate my EC2 instance to a sixth generation instance to make sure that I get maximum network performance?](https://repost.aws/knowledge-center/migrate-to-gen6-ec2-instance)
If you are able to connect to your instance, you can gather diagnostic information by using the failure detection and recovery mechanisms that are covered in the later sections of this topic.
**Topics**
+ [
## Troubleshoot connectivity issues
](#ena-connectivity-issues)
+ [
## Keep-alive mechanism
](#ena-keep-alive)
+ [
## Register read timeout
](#register-read-timeout-ena)
+ [
## Statistics
](#statistics-ena)
+ [
## Driver error logs in syslog
](#driver-error-logs-ena)
+ [
## Sub-optimal configuration notifications
](#ts-ena-sub-opt-config-notification)
## Troubleshoot connectivity issues
If you lose connectivity while enabling enhanced networking, the `ena` module might be incompatible with your instance's current running kernel. This can happen if you install the module for a specific kernel version (without **dkms**, or with an improperly configured **dkms.conf** file) and then your instance kernel is updated. If the instance kernel that is loaded at boot time does not have the `ena` module properly installed, your instance will not recognize the network adapter and your instance becomes unreachable.
If you enable enhanced networking for a PV instance or AMI, this can also make your instance unreachable.
If your instance becomes unreachable after enabling enhanced networking with ENA, you can disable the `enaSupport` attribute for your instance and it will fall back to the stock network adapter.
**To disable enhanced networking with ENA (EBS-backed instances)**
1. From your local computer, stop the instance by using the Amazon EC2 console, the [stop-instances](https://docs.aws.amazon.com/cli/latest/reference/ec2/stop-instances.html) command (AWS CLI), or the [Stop-EC2Instance](https://docs.aws.amazon.com/powershell/latest/reference/items/Stop-EC2Instance.html) cmdlet (AWS Tools for PowerShell).
1. From your local computer, disable the enhanced networking attribute by using the [modify-instance-attribute](https://docs.aws.amazon.com/cli/latest/reference/ec2/modify-instance-attribute.html) command with the `--no-ena-support` option or the [Edit-EC2InstanceAttribute](https://docs.aws.amazon.com/powershell/latest/reference/items/Edit-EC2InstanceAttribute.html) cmdlet with the `-EnaSupport $false` parameter.
1. From your local computer, start the instance using the Amazon EC2 console, the [start-instances](https://docs.aws.amazon.com/cli/latest/reference/ec2/start-instances.html) command, or the [Start-EC2Instance](https://docs.aws.amazon.com/powershell/latest/reference/items/Start-EC2Instance.html) cmdlet.
1. (Optional) Connect to your instance and try reinstalling the `ena` module with your current kernel version by following the steps in [Enable enhanced networking with ENA on your EC2 instances](enhanced-networking-ena.md).
**To disable enhanced networking with ENA (instance store-backed instances)**
1. Create a new AMI as described in [Create an Amazon S3-backed AMI](creating-an-ami-instance-store.md).
1. When you register the AMI, be sure to include the `--no-ena-support` option in the [stop-instances](https://docs.aws.amazon.com/cli/latest/reference/ec2/stop-instances.html) command (AWS CLI) or the `-EnaSupport $false` parameter in the [Register-EC2Image](https://docs.aws.amazon.com/powershell/latest/reference/items/Register-EC2Image.html) cmdlet.
## Keep-alive mechanism
The ENA device posts keep-alive events at a fixed rate (usually once every second). The ENA driver implements a watchdog mechanism, which checks for the presence of these keep-alive messages. If a message or messages are present, the watchdog is rearmed, otherwise the driver concludes that the device experienced a failure and then does the following:
+ Dumps its current statistics to syslog
+ Resets the ENA device
+ Resets the ENA driver state
The above reset procedure may result in some traffic loss for a short period of time (TCP connections should be able to recover), but should not otherwise affect the user.
The ENA device may also indirectly request a device reset procedure, by not sending a keep-alive notification, for example, if the ENA device reaches an unknown state after loading an irrecoverable configuration.
The following is an example of the reset procedure:
```
[18509.800135] ena 0000:00:07.0 eth1: Keep alive watchdog timeout. // The watchdog process initiates a reset
[18509.815244] ena 0000:00:07.0 eth1: Trigger reset is on
[18509.825589] ena 0000:00:07.0 eth1: tx_timeout: 0 // The driver logs the current statistics
[18509.834253] ena 0000:00:07.0 eth1: io_suspend: 0
[18509.842674] ena 0000:00:07.0 eth1: io_resume: 0
[18509.850275] ena 0000:00:07.0 eth1: wd_expired: 1
[18509.857855] ena 0000:00:07.0 eth1: interface_up: 1
[18509.865415] ena 0000:00:07.0 eth1: interface_down: 0
[18509.873468] ena 0000:00:07.0 eth1: admin_q_pause: 0
[18509.881075] ena 0000:00:07.0 eth1: queue_0_tx_cnt: 0
[18509.888629] ena 0000:00:07.0 eth1: queue_0_tx_bytes: 0
[18509.895286] ena 0000:00:07.0 eth1: queue_0_tx_queue_stop: 0
.......
........
[18511.280972] ena 0000:00:07.0 eth1: free uncompleted tx skb qid 3 idx 0x7 // At the end of the down process, the driver discards incomplete packets.
[18511.420112] [ENA_COM: ena_com_validate_version] ena device version: 0.10 //The driver begins its up process
[18511.420119] [ENA_COM: ena_com_validate_version] ena controller version: 0.0.1 implementation version 1
[18511.420127] [ENA_COM: ena_com_admin_init] ena_defs : Version:[b9692e8] Build date [Wed Apr 6 09:54:21 IDT 2016]
[18512.252108] ena 0000:00:07.0: Device watchdog is Enabled
[18512.674877] ena 0000:00:07.0: irq 46 for MSI/MSI-X
[18512.674933] ena 0000:00:07.0: irq 47 for MSI/MSI-X
[18512.674990] ena 0000:00:07.0: irq 48 for MSI/MSI-X
[18512.675037] ena 0000:00:07.0: irq 49 for MSI/MSI-X
[18512.675085] ena 0000:00:07.0: irq 50 for MSI/MSI-X
[18512.675141] ena 0000:00:07.0: irq 51 for MSI/MSI-X
[18512.675188] ena 0000:00:07.0: irq 52 for MSI/MSI-X
[18512.675233] ena 0000:00:07.0: irq 53 for MSI/MSI-X
[18512.675279] ena 0000:00:07.0: irq 54 for MSI/MSI-X
[18512.772641] [ENA_COM: ena_com_set_hash_function] Feature 10 isn't supported
[18512.772647] [ENA_COM: ena_com_set_hash_ctrl] Feature 18 isn't supported
[18512.775945] ena 0000:00:07.0: Device reset completed successfully // The reset process is complete
```
## Register read timeout
The ENA architecture suggests a limited usage of memory mapped I/O (MMIO) read operations. MMIO registers are accessed by the ENA device driver only during its initialization procedure.
If the driver logs (available in **dmesg** output) indicate failures of read operations, this may be caused by an incompatible or incorrectly compiled driver, a busy hardware device, or hardware failure.
Intermittent log entries that indicate failures on read operations should not be considered an issue; the driver will retry them in this case. However, a sequence of log entries containing read failures indicate a driver or hardware problem.
Below is an example of driver log entry indicating a read operation failure due to a timeout:
```
[ 47.113698] [ENA_COM: ena_com_reg_bar_read32] reading reg failed for timeout. expected: req id[1] offset[88] actual: req id[57006] offset[0]
[ 47.333715] [ENA_COM: ena_com_reg_bar_read32] reading reg failed for timeout. expected: req id[2] offset[8] actual: req id[57007] offset[0]
[ 47.346221] [ENA_COM: ena_com_dev_reset] Reg read32 timeout occurred
```
## Statistics
If you experience insufficient network performance or latency issues, you should retrieve the device statistics and examine them. These statistics can be obtained using **ethtool** as follows.
```
[ec2-user ~]$ ethtool -S ethN
NIC statistics:
tx_timeout: 0
suspend: 0
resume: 0
wd_expired: 0
interface_up: 1
interface_down: 0
admin_q_pause: 0
bw_in_allowance_exceeded: 0
bw_out_allowance_exceeded: 0
pps_allowance_exceeded: 0
conntrack_allowance_available: 450878
conntrack_allowance_exceeded: 0
linklocal_allowance_exceeded: 0
queue_0_tx_cnt: 4329
queue_0_tx_bytes: 1075749
queue_0_tx_queue_stop: 0
...
```
The following command output parameters are described below:
`tx_timeout`: *N*
The number of times that the Netdev watchdog was activated.
`suspend`: *N*
The number of times the driver performed a suspend operation.
`resume`: *N*
The number of times the driver performed a resume operation.
`wd_expired`: *N*
The number of times that the driver did not receive the keep-alive event in the preceding three seconds.
`interface_up`: *N*
The number of times that the ENA interface was brought up.
`interface_down`: *N*
The number of times that the ENA interface was brought down.
`admin_q_pause`: *N*
The number of times the admin queue was not found in a running state.
`bw_in_allowance_exceeded`: *N*
The number of packets queued or dropped because the inbound aggregate bandwidth exceeded the maximum for the instance.
`bw_out_allowance_exceeded`: *N*
The number of packets queued or dropped because the outbound aggregate bandwidth exceeded the maximum for the instance.
`pps_allowance_exceeded`: *N*
The number of packets queued or dropped because the bidirectional PPS exceeded the maximum for the instance. \$1
`conntrack_allowance_available`: *N*
The number of tracked connections that can be established by the instance before hitting the Connections Tracked allowance of that instance type. Only available for Nitro-based instances. Not supported with FreeBSD instances or DPDK environments.
`conntrack_allowance_exceeded`: *N*
The number of packets dropped because connection tracking exceeded the maximum for the instance and new connections could not be established. This can result in packet loss for traffic to or from the instance.
`linklocal_allowance_exceeded`: *N*
The number of packets dropped because the PPS of the traffic to local proxy services exceeded the maximum for the network interface. This impacts traffic to the Amazon DNS service, the Instance Metadata Service, and the Amazon Time Sync Service, but does not impact traffic to custom DNS resolvers.
`queue_N_tx_cnt`: *N*
The number of transmitted packets for this queue.
`queue_N_tx_bytes`: *N*
The number of transmitted bytes for this queue.
`queue_N_tx_queue_stop`: *N*
The number of times that queue *N* was full and stopped.
`queue_N_tx_queue_wakeup`: *N*
The number of times that queue *N* resumed after being stopped.
`queue_N_tx_dma_mapping_err`: *N*
Direct memory access error count. If this value is not 0, it indicates low system resources.
`queue_N_tx_linearize`: *N*
The number of times SKB linearization was attempted for this queue.
`queue_N_tx_linearize_failed`: *N*
The number of times SKB linearization failed for this queue.
`queue_N_tx_napi_comp`: *N*
The number of times the `napi` handler called `napi_complete` for this queue.
`queue_N_tx_tx_poll`: *N*
The number of times the `napi` handler was scheduled for this queue.
`queue_N_tx_doorbells`: *N*
The number of transmission doorbells for this queue.
`queue_N_tx_prepare_ctx_err`: *N*
The number of times `ena_com_prepare_tx` failed for this queue.
`queue_N_tx_bad_req_id`: *N*
Invalid `req_id` for this queue. The valid `req_id` is zero, minus the `queue_size`, minus 1.
`queue_N_tx_llq_buffer_copy`: *N*
The number of packets whose headers size are larger than llq entry for this queue.
`queue_N_tx_missed_tx`: *N*
The number of packets that were left uncompleted for this queue.
`queue_N_tx_unmask_interrupt`: *N*
The number of times the tx interrupt was unmasked for this queue.
`queue_N_rx_cnt`: *N*
The number of received packets for this queue.
`queue_N_rx_bytes`: *N*
The number of received bytes for this queue.
`queue_N_rx_rx_copybreak_pkt`: *N*
The number of times the rx queue received a packet that is less than the rx\$1copybreak packet size for this queue.
`queue_N_rx_csum_good`: *N*
The number of times the rx queue received a packet where the checksum was checked and was correct for this queue.
`queue_N_rx_refil_partial`: *N*
The number of times the driver did not succeed in refilling the empty portion of the rx queue with the buffers for this queue. If this value is not zero, it indicates low memory resources.
`queue_N_rx_bad_csum`: *N*
The number of times the `rx` queue had a bad checksum for this queue (only if rx checksum offload is supported).
`queue_N_rx_page_alloc_fail`: *N*
The number of time that page allocation failed for this queue. If this value is not zero, it indicates low memory resources.
`queue_N_rx_skb_alloc_fail`: *N*
The number of time that SKB allocation failed for this queue. If this value is not zero, it indicates low system resources.
`queue_N_rx_dma_mapping_err`: *N*
Direct memory access error count. If this value is not 0, it indicates low system resources.
`queue_N_rx_bad_desc_num`: *N*
Too many buffers per packet. If this value is not 0, it indicates the use of very small buffers.
`queue_N_rx_bad_req_id`: *N*
The req\$1id for this queue is not valid. The valid req\$1id is from [0, queue\$1size - 1 ].
`queue_N_rx_empty_rx_ring`: *N*
The number of times the rx queue was empty for this queue.
`queue_N_rx_csum_unchecked`: *N*
The number of times the rx queue received a packet whose checksum wasn't checked for this queue.
`queue_N_rx_xdp_aborted`: *N*
The number of times that an XDP packet was classified as XDP\$1ABORT.
`queue_N_rx_xdp_drop`: *N*
The number of times that an XDP packet was classified as XDP\$1DROP.
`queue_N_rx_xdp_pass`: *N*
The number of times that an XDP packet was classified as XDP\$1PASS.
`queue_N_rx_xdp_tx`: *N*
The number of times that an XDP packet was classified as XDP\$1TX.
`queue_N_rx_xdp_invalid`: *N*
The number of times that the XDP return code for the packet was not valid.
`queue_N_rx_xdp_redirect`: *N*
The number of times that an XDP packet was classified as XDP\$1REDIRECT.
`queue_N_xdp_tx_cnt`: *N*
The number of transmitted packets for this queue.
`queue_N_xdp_tx_bytes`: *N*
The number of transmitted bytes for this queue.
`queue_N_xdp_tx_queue_stop`: *N*
The number of times that this queue was full and stopped.
`queue_N_xdp_tx_queue_wakeup`: *N*
The number of times that this queue resumed after being stopped.
`queue_N_xdp_tx_dma_mapping_err`: *N*
Direct memory access error count. If this value is not 0, it indicates low system resources.
`queue_N_xdp_tx_linearize`: *N*
The number of times XDP buffer linearization was attempted for this queue.
`queue_N_xdp_tx_linearize_failed`: *N*
The number of times XDP buffer linearization failed for this queue.
`queue_N_xdp_tx_napi_comp`: *N*
The number of times the napi handler called napi\$1complete for this queue.
`queue_N_xdp_tx_tx_poll`: *N*
The number of times the napi handler was scheduled for this queue.
`queue_N_xdp_tx_doorbells`: *N*
The number of transmission doorbells for this queue.
`queue_N_xdp_tx_prepare_ctx_err`: *N*
The number of times ena\$1com\$1prepare\$1tx failed for this queue. This value should always be zero; if not, see the driver logs.
`queue_N_xdp_tx_bad_req_id`: *N*
The req\$1id for this queue is not valid. The valid req\$1id is from [0, queue\$1size - 1 ].
`queue_N_xdp_tx_llq_buffer_copy`: *N*
The number of packets that had their headers copied using llq buffer copy for this queue.
`queue_N_xdp_tx_missed_tx`: *N*
The number of times a tx queue entry missed a completion timeout for this queue.
`queue_N_xdp_tx_unmask_interrupt`: *N*
The number of times the tx interrupt was unmasked for this queue.
`ena_admin_q_aborted_cmd`: *N*
The number of admin commands that were aborted. This usually happens during the auto-recovery procedure.
`ena_admin_q_submitted_cmd`: *N*
The number of admin queue doorbells.
`ena_admin_q_completed_cmd`: *N*
The number of admin queue completions.
`ena_admin_q_out_of_space`: *N*
The number of times that the driver tried to submit new admin command, but the queue was full.
`ena_admin_q_no_completion`: *N*
The number of times that the driver did not get an admin completion for a command.
## Driver error logs in syslog
The ENA driver writes log messages to **syslog** during system boot. You can examine these logs to look for errors if you are experiencing issues. Below is an example of information logged by the ENA driver in **syslog** during system boot, along with some annotations for select messages.
```
Jun 3 22:37:46 ip-172-31-2-186 kernel: [ 478.416939] [ENA_COM: ena_com_validate_version] ena device version: 0.10
Jun 3 22:37:46 ip-172-31-2-186 kernel: [ 478.420915] [ENA_COM: ena_com_validate_version] ena controller version: 0.0.1 implementation version 1
Jun 3 22:37:46 ip-172-31-2-186 kernel: [ 479.256831] ena 0000:00:03.0: Device watchdog is Enabled
Jun 3 22:37:46 ip-172-31-2-186 kernel: [ 479.672947] ena 0000:00:03.0: creating 8 io queues. queue size: 1024
Jun 3 22:37:46 ip-172-31-2-186 kernel: [ 479.680885] [ENA_COM: ena_com_init_interrupt_moderation] Feature 20 isn't supported // Interrupt moderation is not supported by the device
Jun 3 22:37:46 ip-172-31-2-186 kernel: [ 479.691609] [ENA_COM: ena_com_get_feature_ex] Feature 10 isn't supported // RSS HASH function configuration is not supported by the device
Jun 3 22:37:46 ip-172-31-2-186 kernel: [ 479.694583] [ENA_COM: ena_com_get_feature_ex] Feature 18 isn't supported //RSS HASH input source configuration is not supported by the device
Jun 3 22:37:46 ip-172-31-2-186 kernel: [ 479.697433] [ENA_COM: ena_com_set_host_attributes] Set host attribute isn't supported
Jun 3 22:37:46 ip-172-31-2-186 kernel: [ 479.701064] ena 0000:00:03.0 (unnamed net_device) (uninitialized): Cannot set host attributes
Jun 3 22:37:46 ip-172-31-2-186 kernel: [ 479.704917] ena 0000:00:03.0: Elastic Network Adapter (ENA) found at mem f3000000, mac addr 02:8a:3c:1e:13:b5 Queues 8
Jun 3 22:37:46 ip-172-31-2-186 kernel: [ 480.805037] EXT4-fs (xvda1): re-mounted. Opts: (null)
Jun 3 22:37:46 ip-172-31-2-186 kernel: [ 481.025842] NET: Registered protocol family 10
```
**Which errors can I ignore?**
The following warnings that may appear in your system's error logs can be ignored for the Elastic Network Adapter:
Set host attribute isn't supported
Host attributes are not supported for this device.
failed to alloc buffer for rx queue
This is a recoverable error, and it indicates that there may have been a memory pressure issue when the error was thrown.
Feature *X* isn't supported
The referenced feature is not supported by the Elastic Network Adapter. Possible values for *X* include:
+ 10: RSS Hash function configuration is not supported for this device.
+ 12: RSS Indirection table configuration is not supported for this device.
+ 18: RSS Hash Input configuration is not supported for this device.
+ 20: Interrupt moderation is not supported for this device.
+ 27: The Elastic Network Adapter driver does not support polling the Ethernet capabilities from snmpd.
Failed to config AENQ
The Elastic Network Adapter does not support AENQ configuration.
Trying to set unsupported AENQ events
This error indicates an attempt to set an AENQ events group that is not supported by the Elastic Network Adapter.
## Sub-optimal configuration notifications
The ENA device detects sub-optimal configuration settings in the driver that you can change. The device notifies the ENA driver and logs a warning to the console. The following example shows the format of the warning message.
```
Sub-optimal configuration notification code: 1. Refer to AWS ENA documentation for additional details and mitigation options.
```
The following list shows notification code details and recommended actions for sub-optimal configuration findings.
+ **Code 1: ENA Express with wide LLQ configuration is not recommended**
ENA Express ENI is configured with wide LLQ. This configuration is sub-optimal and could impact performance for ENA Express. We recommend that you disable wide LLQ settings when you use ENA Express ENIs as follows.
```
sudo rmmod ena && sudo modprobe ena force_large_llq_header=0
```
For more information about optimal configuration for ENA Express, see [Improve network performance between EC2 instances with ENA Express](ena-express.md).
+ **Code 2: ENA Express ENI with sub-optimal Tx queue depth is not recommended**
ENA Express ENI is configured with sub-optimal Tx queue depth. This configuration could impact performance for ENA Express. We recommend that you enlarge all Tx queues to the maximum value for the network interface when you use ENA Express ENIs as follows.
You can run the following **ethtool** commands to adjust LLQ size. To learn more about how to control, query, and enable wide-LLQ, see the [Large Low-Latency Queue (Large LLQ)](https://github.com/amzn/amzn-drivers/tree/master/kernel/linux/ena#large-low-latency-queue-large-llq) topic of the Linux kernel driver for ENA documentation in the *Amazon Drivers GitHub repository*.
```
ethtool -g interface
```
Set your Tx queues to the maximum depth:
```
ethtool -G interface tx depth
```
For more information about optimal configuration for ENA Express, see [Improve network performance between EC2 instances with ENA Express](ena-express.md).
+ **Code 3: ENA with regular LLQ size and Tx packet traffic exceeds the maximum header supported size**
By default, ENA LLQ supports Tx packet header size up to 96 bytes. If the packet header size is larger than 96 bytes, the packet is dropped. To mitigate this issue, we recommend that you enable wide-LLQ, which increases the supported Tx packet header size to a maximum of 224 bytes.
However, when you enable wide-LLQ, the maximum Tx ring size is reduced from 1000 to 512 entries. Wide-LLQ is enabled by default for all Nitro v4 and later instance types.
+ Nitro v4 instance types have a default maximum wide-LLQ Tx ring size of 512 entries, which can't be changed.
+ Nitro v5 instance types have a default wide-LLQ Tx ring size of 512 entries, which you can increase up to 1000 entries.
You can run the following **ethtool** commands to adjust LLQ size. To learn more about how to control, query, and enable wide-LLQ, see the [Large Low-Latency Queue (Large LLQ)](https://github.com/amzn/amzn-drivers/tree/master/kernel/linux/ena#large-low-latency-queue-large-llq) topic of the Linux kernel driver for ENA documentation in the *Amazon Drivers GitHub repository*.
Find the maximum depth for your Tx queues:
```
ethtool -g interface
```
Set your Tx queues to the maximum depth:
```
ethtool -G interface tx depth
```
# Troubleshoot the Elastic Network Adapter Windows driver
Troubleshoot ENA on Windows
The Elastic Network Adapter (ENA) is designed to improve operating system health and to reduce unexpected hardware behavior or failures that can disrupt the operation of your Windows instance. The ENA architecture keeps device or driver failures as transparent to the operating system as possible.
## Collect diagnostic information on the instance
Collect diagnostic information
The steps to open Windows operating system (OS) tools vary, depending on what version of the OS is installed on your instance. In the following sections, we use the **Run** dialog to open the tools, which works the same across all OS versions. However, you can access these tools using any method that you prefer.
**Access the Run dialog**
+ Using the Windows logo key combination: `Windows` \$1 `R`
+ Using the search bar:
+ Enter `run` in the search bar.
+ Select the **Run** application from the search results.
Some steps require the context menu to access properties or context-sensitive actions. There are several ways to do this, depending on your OS version and hardware.
**Access the context menu**
+ Using your mouse: right-click an item to bring up its context menu.
+ Using your keyboard:
+ Depending on your OS version, use `Shift` \$1 `F10`, or `Ctrl` \$1 `Shift` \$1 `F10`.
+ If you have the context key on your keyboard (three horizontal lines in a box), select the item you want and then press the context key.
If you can connect to your instance, use the following techniques to gather diagnostic information for troubleshooting.
### Check ENA device status
To check the status of your ENA Windows driver using the Windows Device Manager, follow these steps:
1. Open the **Run** dialog using one of the methods described in the preceding section.
1. To open the Windows Device Manager, enter `devmgmt.msc` in the **Run** box.
1. Choose **OK**. This opens the Device Manager window.
1. Select the arrow to the left of **Network adapters** to expand the list.
1. Choose the name, or open the context menu for the **Amazon Elastic Network Adapter**, and then choose **Properties**. This opens the **Amazon Elastic Network Adapter Properties** dialog.
1. Verify that the message in the **General** tab says "This device is working properly."
### Investigate driver event messages
To review ENA Windows driver event logs using the Windows Event Viewer, follow these steps:
1. Open the **Run** dialog using one of the methods described in the preceding section.
1. To open the Windows Event Viewer, enter `eventvwr.msc` in the **Run** box.
1. Choose **OK**. This opens the Event Viewer window.
1. Expand the **Windows Logs** menu, and then choose **System**.
1. Under **Actions**, in the top-right panel, choose **Filter Current Log**. This displays the filtering dialog.
1. In the **Event sources** box, enter `ena`. This limits results to events that were generated by the ENA Windows driver.
1. Choose **OK**. This shows filtered event log results in the detail sections of the window.
1. To drill down into the details, select an event message from the list.
The following example shows an ENA driver event in the Windows Event Viewer system events list:
![\[Example: ENA driver event shown in the Windows Event Viewer system messages list.\]](http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/images/ena-event-viewer-example.png)
#### Event message summary
The following table shows event messages that the ENA Windows driver generates.
**Input**
| Event ID | ENA driver event description | Type |
| --- | --- | --- |
| 5001 | Hardware is out of resources | Error |
| 5002 | Adapter has detected a hardware error | Error |
| 5005 | Adapter has timed out on NDIS operation that did not complete in a timely manner | Error |
| 5032 | Adapter has failed to reset the device | Error |
| 5200 | Adapter has been initialized | Informational |
| 5201 | Adapter has been halted | Informational |
| 5202 | Adapter has been paused | Informational |
| 5203 | Adapter has been restarted | Informational |
| 5204 | Adapter has been shut down | Informational |
| 5205 | Adapter has been reset | Error |
| 5206 | Adapter has been surprise removed | Error |
| 5208 | Adapter initialization routine has failed | Error |
| 5210 | Adapter has encountered and successfully recovered an internal issue | Error |
### Review performance metrics
The ENA Windows driver publishes network performance metrics from the instances where metrics are enabled. You can view and enable metrics on the instance using the native Performance Monitor application. For more information about the metrics that the ENA Windows driver produces, see [Monitor network performance for ENA settings on your EC2 instance](monitoring-network-performance-ena.md).
On instances where ENA metrics are enabled, and the Amazon CloudWatch agent is installed, CloudWatch collects the metrics that are associated with the counters in Windows Performance Monitor, as well as some advanced metrics for ENA. These metrics are collected in addition to the metrics enabled by default on EC2 instances. For more information about the metrics, see [Metrics collected by the CloudWatch agent](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/metrics-collected-by-CloudWatch-agent.html) in the *Amazon CloudWatch User Guide*.
**Note**
Performance metrics are available for ENA driver versions 2.4.0 and later (also for version 2.2.3). ENA driver version 2.2.4 was rolled back due to potential performance degradation on the sixth generation EC2 instances. We recommend that you upgrade to the current version of the driver to ensure that you have the latest updates.
Some of the ways that you can use performance metrics include:
+ Troubleshoot instance performance issues.
+ Choose the right instance size for a workload.
+ Proactively plan scaling activities.
+ Benchmark applications to determine if they maximize the performance available on an instance.
**Refresh rate**
By default, the driver refreshes metrics using a 1-second interval. However, the application that retrieves the metrics might use a different interval for polling. You can change the refresh interval in Device Manager, using the advanced properties for the driver.
To change the metrics refresh interval for the ENA Windows driver, follow these steps:
1. Open the **Run** dialog using one of the methods described in the preceding section.
1. To open the Windows Device Manager, enter `devmgmt.msc` in the **Run** box.
1. Choose **OK**. This opens the Device Manager window.
1. Select the arrow to the left of **Network adapters** to expand the list.
1. Choose the name, or open the context menu for the **Amazon Elastic Network Adapter**, and then choose **Properties**. This opens the **Amazon Elastic Network Adapter Properties** dialog.
1. Open the **Advanced** tab in the pop-up window.
1. From the **Property** list, choose **Metrics Refresh Interval** to change the value.
1. When you are done, choose **OK**.
## Investigate sub-optimal configuration notifications
The ENA device detects sub-optimal configuration settings in the driver that you can change. The device notifies the ENA driver and logs an event notification. To review sub-optimal events in the Windows Event Viewer
1. Open the **Run** dialog using one of the methods described in the preceding section.
1. To open the Windows Event Viewer, enter `eventvwr.msc` in the **Run** box.
1. Choose **OK**. This opens the Event Viewer window.
1. Expand the **Windows Logs** menu, and then choose **System**.
1. Under **Actions**, in the top-right panel, choose **Filter Current Log**. This displays the filtering dialog.
1. In the **Event sources** box, enter `ena`. This limits results to events that were generated by the ENA Windows driver.
1. Choose **OK**. This shows filtered event log results in the detail sections of the window.
Events with ID `59000` notify you of sub-optimal configuration findings. Right-click an event and choose **Event Properties** to open a detailed view, or select **Preview Pane** from the **View** menu to see the same detail.
![\[Example: System event ID 59000 shown in the Windows Event Viewer preview pane.\]](http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/images/ena-sub-opt-event-general.png)
Open the **Details** tab to see the event code. In the **Binary Data: In words** section, the last word is the code.
![\[Example: The last word in the Binary Data section is shown highlighted.\]](http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/images/ena-sub-opt-event-detail.png)
The following list shows notification code details and recommended actions for sub-optimal configuration findings.
+ **Code `1`: ENA Express with wide LLQ configuration is not recommended**
ENA Express ENI is configured with wide LLQ. This configuration is sub-optimal and could impact performance for ENA Express. We recommend that you disable wide LLQ settings when you use ENA Express ENIs as follows.
1. To open the Windows Device Manager, enter `devmgmt.msc` in the **Run** box.
1. Choose **OK**. This opens the Device Manager window.
1. Select the arrow to the left of **Network adapters** to expand the list.
1. Open the device properties for the `Amazon Elastic Network Adapter`.
1. From there, open the **Advanced** tab to make your changes.
1. Select the **LLQ Header Size Policy** property, and set its value to `Normal (128 Bytes)`.
1. Choose **OK** to save your changes.
+ **Code `2`: ENA Express ENI with sub-optimal Tx queue depth is not recommended**
ENA Express ENI is configured with sub-optimal Tx queue depth. This configuration could impact performance for ENA Express. We recommend that you enlarge all Tx queues to the maximum value for the network interface when you use ENA Express ENIs as follows.
Follow these steps to enlarge the Tx queues to the maximum depth:
1. To open the Windows Device Manager, enter `devmgmt.msc` in the **Run** box.
1. Choose **OK**. This opens the Device Manager window.
1. Select the arrow to the left of **Network adapters** to expand the list.
1. Open the device properties for the `Amazon Elastic Network Adapter`.
1. From there, open the **Advanced** tab to make your changes.
1. Select the **Transmit Buffers** property, and set its value to the maximum supported.
1. Choose **OK** to save your changes.
## ENA adapter reset
The reset process starts when the ENA Windows driver detects an error on an adapter, and marks the adapter as unhealthy. The driver cannot reset itself, so it depends on the operating system to check the adapter health status, and call the reset handle for the ENA Windows driver. The reset process might result in a brief period of time where traffic loss occurs. However, TCP connections should be able to recover.
The ENA adapter might also indirectly request a device reset procedure, by failing to send a keep-alive notification. For example, if the ENA adapter reaches an unknown state after loading an irrecoverable configuration, it might stop sending keep-alive notifications.
**Common causes for ENA adapter reset**
+ Keep-alive messages are missing
The ENA adapter posts keep-alive events at a fixed rate (usually once every second). The ENA Windows driver implements a watchdog mechanism, which periodically checks for the presence of these keep-alive messages. If it detects one or more new messages since the last time it checked, it records a successful outcome. Otherwise, the driver concludes that the device experienced a failure, and initiates a reset sequence.
+ Packets are stuck in transmit queues
The ENA adapter verifies that packets are flowing through the transmit queues as expected. The ENA Windows driver detects if packets are getting stuck, and initiates a reset sequence if they are.
+ Read timeout for Memory Mapped I/O (MMIO) registers
To limit memory mapped I/O (MMIO) read operations, the ENA Windows driver accesses MMIO registers only during initialization and reset processes. If the driver detects a timeout, it takes one of the following actions, depending on what process was running:
+ If a timeout is detected during initialization, it fails the flow, which results in the driver displaying a yellow exclamation mark by the ENA adapter in Windows Device Manager.
+ If a timeout is detected during reset, it fails the flow. The OS then initiates a surprise removal of the ENA adapter, and recovers it by stopping and starting the adapter that was removed. For more information about surprise removal of a network interface card (NIC), see [Handling the Surprise Removal of a NIC](https://learn.microsoft.com/en-us/windows-hardware/drivers/network/handling-the-surprise-removal-of-a-nic) in the *Microsoft Windows Hardware Developer* documentation.
## Troubleshooting scenarios
The following scenarios can help you troubleshoot issues that you might experience with the ENA Windows driver. We recommend that you start with upgrading your ENA driver, if you don't have the latest version. To find the latest driver for your Windows OS version, see [Track ENA Windows driver version releases](ena-driver-releases-windows.md).
### Unexpected ENA driver version installed
#### Description
After you go through the steps to install a specific version of the ENA driver, the Windows Device Manager shows that Windows installed a different version of the ENA driver.
#### Cause
When you run the install for a driver package, Windows ranks all of the driver packages that are valid for the given device in the local [Driver Store](https://learn.microsoft.com/en-us/windows-hardware/drivers/install/driver-store) before it begins. Then it selects the package with the lowest rank value as the best match. This can be different from the package that you intended to install. For more information about the device driver package selection process, see [How Windows selects a driver package for a device](https://learn.microsoft.com/en-us/windows-hardware/drivers/install/how-windows-selects-a-driver-for-a-device) on the *Microsoft documentation website*.
#### Solution
To ensure that Windows installs your chosen driver package version, you can remove lower ranked driver packages from the Driver Store with the [PnPUtil](https://learn.microsoft.com/en-us/windows-hardware/drivers/devtest/pnputil) command line tool.
Follow these steps to update the ENA driver:
1. Connect to your instance and log in as the local administrator.
1. Open the Device Manager properties window, as described in the [Check ENA device status](#ts-ena-diagnostics-device-mgr) section. This opens the **General** tab of the **Amazon Elastic Network Adapter Properties** window.
1. Open the **Driver** tab.
1. Choose **Update Driver**. This opens the **Update Driver Software – Amazon Elastic Network Adapter** dialog box.
1. On the **How do you want to search for driver software?** page, choose **Browse my computer for driver software**.
1. On the **Browse for driver software on your computer** page, choose **Let me pick from a list of device drivers on my computer**, located below the search bar.
1. On the **Select the device driver you want to install for this hardware** page, choose **Have Disk...**.
1. In the **Install from Disk** window, choose **Browse...**, next to the file location from the dropdown list.
1. Navigate to the location where you downloaded the target ENA driver package. Select the file named `ena.inf` and choose **Open**.
1. To start the install, choose **OK**, and then choose **Next**.
1. If the installer doesn’t automatically reboot your instance, run the **Restart-Computer** PowerShell cmdlet.
```
PS C:\> Restart-Computer
```
### Device warning for ENA driver
#### Description
The ENA adapter icon in the Device Manager **Network adapters** section displays a warning sign (a yellow triangle with an exclamation mark inside).
The following example shows an ENA adapter with the warning icon in Windows Device Manager:
![\[Example: ENA adapter with warning icon shown in the Windows Device Manager.\]](http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/images/ena-adapter-device-mgr-warn.png)
#### Cause
This device warning is commonly caused by environment issues, which might require more research, and often require a process of elimination to determine the underlying cause. For a full list of device errors, see [Device Manager Error Messages](https://learn.microsoft.com/en-us/windows-hardware/drivers/install/device-manager-error-messages) in the Microsoft documentation.
#### Solution
The solution for this device warning depends on the root cause. The process of elimination described here includes a few basic steps to help identify and resolve the most common issues that might have a simple solution. Additional root cause analysis is required when these steps do not resolve the issue.
Follow these steps to help identify and resolve common issues:
1.
**Stop and start the device**
Open the Device Manager properties window, as described in the [Check ENA device status](#ts-ena-diagnostics-device-mgr) section. This opens the **General** tab of the **Amazon Elastic Network Adapter Properties** window, where the **Device status** displays the error code and a short message.
1. Open the **Driver** tab.
1. Choose **Disable Device**, and respond **Yes** to the warning message that displays.
1. Choose **Enable Device**.
1.
**Stop and start the EC2 instance**
If the adapter still shows the warning icon in Device Manager, the next step is to stop and start the EC2 instance. This relaunches the instance on different hardware in most cases.
1.
**Investigate possible instance resource issue**
If you have stopped and started your EC2 instance, and the problem persists, this might indicate a resource issue on your instance, such as insufficient memory.
### Connection timeout with adapter reset (error codes 5007, 5205)
#### Description
The Windows Event Viewer shows adapter timeout and reset events occurring in combination for ENA adapters. Messages resemble the following examples:
+ **Event ID 5007**: Amazon Elastic Network Adapter : Timed out during an operation.
+ **Event ID 5205**: Amazon Elastic Network Adapter : Adapter reset has been started.
Adapter resets cause minimal traffic disruption. Even when there are multiple resets, it would be unusual for them to cause any severe network disruption.
#### Cause
This sequence of events indicates that the ENA Windows driver initiated a reset for an ENA adapter that was unresponsive. However, the mechanism that the device driver uses to detect this issue is subject to false positives resulting from CPU 0 starvation.
#### Solution
If this combination of errors happens frequently, check your resource allocations to see where adjustments might be helpful.
1. Open the **Run** dialog using one of the methods described in the preceding section.
1. To open the Windows Resource Monitor, enter `resmon` in the **Run** box.
1. Choose **OK**. This opens the Resource Monitor window.
1. Open the **CPU** tab. Per-CPU usage graphs are shown along the right side of the Resource Monitor window.
1. Check the usage levels for CPU 0 to see if they are too high.
We recommend that you configure RSS to exclude CPU 0 for the ENA adapter on larger instance types (more than 16 vCPU). For smaller instance types, configuring RSS might improve the experience, but due to the lower number of available cores, testing is necessary to ensure that constraining CPU cores does not negatively impact performance.
Use the **Set-NetAdapterRss** command to configure RSS for your ENA adapter, as shown in the following example.
```
Set-NetAdapterRss -name (Get-NetAdapter | Where-Object {$_.InterfaceDescription -like "*Elastic*"}).Name -Baseprocessorgroup 0 -BaseProcessorNumber 1
```
### Migrating to a sixth generation instance infrastructure impacts performance or attachment
#### Description
If you migrate to a sixth generation EC2 instance, you might experience reduced performance or ENA attachment failures if you haven't updated your ENA Windows driver version.
#### Cause
The sixth generation EC2 instance types require the following minimum version of the ENA Windows driver, based on the instance operating system (OS).
**Minimum version**
| Windows Server version | ENA driver version |
| --- | --- |
| Windows Server 2008 R2 | 2.2.3 or 2.4.0 |
| Windows Server 2012 and later | 2.2.3 and later |
| Windows Workstation | 2.2.3 and later |
#### Solution
Before you upgrade to a sixth generation EC2 instance, make sure that the AMI you launch from has compatible drivers based on the instance OS as shown in the previous table. For more information, see [What do I need to do before migrating my EC2 instance to a sixth generation instance to make sure that I get maximum network performance?](https://repost.aws/knowledge-center/migrate-to-gen6-ec2-instance) in the *AWS re:Post Knowledge Center*.
### Suboptimal performance for the elastic network interface
#### Description
The ENA interface is not performing as expected.
#### Cause
Root cause analysis for performance issues is a process of elimination. There are too many variables involved to name a common cause.
#### Solution
The first step in your root cause analysis is to review the diagnostic information for the instance that is not performing as expected, to determine if there are errors that might be causing the issue. For more information, see the [Collect diagnostic information on the instance](#ts-ena-drv-collect-diagnostics) section.
You might need to modify the default operating system configuration to achieve maximum network performance on instances with enhanced networking. Some optimizations, such as turning on checksum offloading and enabling RSS, are configured by default in official Windows AMIs. For other optimizations that you can apply to the ENA adapter, see the performance adjustments shown in [ENA adapter performance adjustments](#ts-ena-drv-perf-adj).
We recommend that you proceed with caution, and limit device property adjustments to those that are listed in this section, or to specific changes that are recommended by the AWS support team.
To change ENA adapter properties, follow these steps:
1. Open the **Run** dialog using one of the methods described in the preceding section.
1. To open the Windows Device Manager, enter `devmgmt.msc` in the **Run** box.
1. Choose **OK**. This opens the Device Manager window.
1. Select the arrow to the left of **Network adapters** to expand the list.
1. Choose the name, or open the context menu for the **Amazon Elastic Network Adapter**, and then choose **Properties**. This opens the **Amazon Elastic Network Adapter Properties** dialog.
1. To make your changes, open the **Advanced** tab.
1. When you're done, choose **OK** to save your changes.
The following example shows an ENA adapter property in the Windows Device Manager:
![\[Example: ENA adapter property shown in the Windows Device Manager.\]](http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/images/ena-adapter-device-mgr-property.png)
##### ENA adapter performance adjustments
The following table includes properties that can be adjusted to improve performance for the ENA interface.
**Input**
| Property | Description | Default value | Adjustment |
| --- | --- | --- | --- |
| Receive Buffers | Controls the number of entries in the software receive queues. | 1024 | Can be increased up to a maximum of 8192. |
| Receive Side Scaling (RSS) | Enables the efficient distribution of network receive processing across multiple CPUs in multiprocessor systems. | Enabled | You can spread the load across multiple processors. To learn more, see [Optimize network performance on EC2 Windows instances](enhanced-networking-os.md). |
| Maximum Number of RSS Queues | Sets the maximum number of RSS queues allowed when `RSS` is enabled. | 32 | The number of RSS queues is determined during driver initialization, and includes the following limitations (among others): [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/troubleshoot-ena-driver.html) You can set the value from 1-32, depending on your instance and hardware generation limits. To learn more, see [Optimize network performance on EC2 Windows instances](enhanced-networking-os.md). |
| Jumbo packet | Enables the use of jumbo ethernet frames (more than 1500 bytes of payload). | Disabled (this limits payload to 1500 bytes or less) | Value can be set up to `9015`, which translates to 9001 bytes of payload. This is the maximum payload for jumbo ethernet frames. See [Considerations for using jumbo ethernet frames](#ts-ena-drv-jumbo-frames). |
##### Considerations for using jumbo ethernet frames
Jumbo frames allow more than 1500 bytes of data by increasing the payload size per packet, which increases the percentage of the packet that is not packet overhead. Fewer packets are needed to send the same amount of usable data. However, traffic is limited to a maximum MTU of 1500 in the following cases:
+ Traffic outside of a given AWS Region for EC2 Classic.
+ Traffic outside of a single VPC.
+ Traffic over an inter-Region VPC peering connection.
+ Traffic over VPN connections.
+ Traffic over an internet gateway.
**Note**
Packets over 1500 bytes are fragmented. If you have the `Don't Fragment` flag set in the IP header, these packets are dropped.
Jumbo frames should be used with caution for internet-bound traffic, or any traffic that leaves a VPC. Packets are fragmented by intermediate systems, which slows down this traffic. To use jumbo frames inside of a VPC without impacting outbound traffic that's leaving the VPC, try one of the following options:
Configure the MTU size by route.
Use multiple network interfaces with different MTU sizes and different routes.
**Recommended use cases for jumbo frames**
Jumbo frames can be useful for traffic inside of and between VPCs. We recommend using jumbo frames for the following use cases:
+ For instances that are collocated inside of a cluster placement group, jumbo frames help to achieve the maximum network throughput possible. For more information, see [Placement groups for your Amazon EC2 instances](placement-groups.md).
+ You can use jumbo frames for traffic between your VPCs and your on-premises networks over Direct Connect. For more information about using Direct Connect, and verifying jumbo frame capability, see [MTU for private virtual interfaces or transit virtual interfaces](https://docs.aws.amazon.com/directconnect/latest/UserGuide/WorkingWithVirtualInterfaces.html#set-jumbo-frames-vif.html) in the *Direct Connect User Guide*.
+ For more information about supported MTU sizes for transit gateways, see [Quotas for your transit gateways](https://docs.aws.amazon.com/vpc/latest/tgw/transit-gateway-quotas.html#mtu-quota) in the *Amazon VPC Transit Gateways*.