# Using Amazon Linux 2023 outside of Amazon EC2 The Amazon Linux 2023 container images can be run in compatible container runtime environments. For more information on how to use Amazon Linux 2023 inside a container, see [AL2023 in containers](container.md). Amazon Linux 2023 (AL2023) can also be run as a virtualized guest outside of directly being run on Amazon EC2. There are currently KVM (`qcow2`), VMware (`OVA`), and Hyper-V (`vhdx`) images available. **Note** The configuration of Amazon Linux 2023 images differs from Amazon Linux 2. If you are coming from [ Running Amazon Linux 2 as a virtual machine on premises ](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/amazon-linux-2-virtual-machine.html) you will need to adapt your configuration to be compatible with AL2023. # Download Amazon Linux 2023 images for use with KVM, VMware, and Hyper-V Amazon Linux 2023 disk images for use with KVM, VMware, and Hyper-V can be downloaded from [cdn.amazonlinux.com](https://cdn.amazonlinux.com/al2023/os-images/latest/). # Supported configurations of Amazon Linux 2023 for use in non-Amazon EC2 virtualized environments This section covers the requirements for running Amazon Linux 2023 in non-Amazon EC2 virtualized environments such as on KVM, VMware, or and Hyper-V. The base [AL2023 system requirements](system-requirements.md) apply to all non-Amazon EC2 virtualized environments. A list of supported device models is detailed for each hypervisor environment in the following topics. KVM, VMware, and Hyper-V provide many configuration options, and care needs to be taken in order to configure them for your security, performance, and reliability needs. For more information, check the documentation provided by your hypervisor. **Topics** + [Requirements for running AL2023 on KVM](kvm-supported-configurations.md) + [Requirements for running AL2023 on VMware](vmware-supported-configurations.md) + [Requirements for running Amazon Linux 2023 on Hyper-V](hyperv-supported-configurations.md) # Requirements for running AL2023 on KVM This section describes the requirements for running AL2023 on KVM. The KVM images of AL2023 are available for both `aarch64` and `x86-64` architectures. These requirements are in addition to the base [AL2023 system requirements](system-requirements.md) for the KVM images. **Topics** + [KVM host requirements for running AL2023 on KVM](#kvm-host-requirements) + [Device support for AL2023 on KVM](#kvm-devices) + [Boot mode (UEFI and BIOS) support for AL2023 on KVM](#kvm-boot-modes) + [Limitations running AL2023 on KVM](#kvm-limitations) ## KVM host requirements for running AL2023 on KVM The KVM images are currently qualified on a host running Ubuntu 22.04.3 LTS with `qemu` version 6.2\$1dfsg-2ubuntu6.15, provided by this Ubuntu version, using a `q35` machine type for `x86-64` and a `virt` machine type for `aarch64`. ## Device support for AL2023 on KVM **The `qemu` device models tested for use with AL2023 KVM images (both `aarch64` and `x86-64`) are:** + `virtio-blk` (`virtio` block device) + `virtio-scsi` (`virtio` SCSI controller with disk device) + `virtio-net` (`virtio` network device) + `ahci` (for use with the virtual CD-ROM drive) + `usb-storage` (over `xhci`) **Additional `qemu` device models enabled in AL2023 KVM image qualification, but not heavily exercised are:** + `VGA` (`qemu` VGA) on `x86-64` only + `virtio-rng` (virtual random number generator) + legacy AT keyboard and PS/2 mouse devices + legacy serial device ## Boot mode (UEFI and BIOS) support for AL2023 on KVM The `x86-64` image is tested with both legacy BIOS and UEFI boot modes. The `aarch64` images are tested with UEFI boot mode. **Note** By default, when using UEFI boot mode, some virtual machine managers will provision the VM with Microsoft Secure Boot keys which enables Secure Boot. This configuration will not boot AL2023. Because the AL2023 boot loader isn’t signed by Microsoft, the VM must be provisioned either without UEFI keys, or with the AL2023 keys for Secure Boot. **Important** Secure Boot support for KVMimages has not been validated yet. ## Limitations running AL2023 on KVM There are some known limitations in running AL2023 on KVM. **Note** Code implementing some of the listed unsupported functionality might exist in AL2023 and function correctly. The list of unsupported functionality exists so that you can make informed decisions about what to rely upon working today, and what the Amazon Linux team will qualify as working as part of future updates. **Known Limitations with running AL2023 on KVM** + The KVM guest agent is not currently packaged or supported. + Hot plugging and unplugging CPU, memory, or any other device type is not supported. + VM hibernation is not supported. + VM migration is not supported. + Passthrough of any device such as through PCI Passthrough, or USB Passthrough is not supported. # Requirements for running AL2023 on VMware This section describes the requirements for running AL2023 on VMware. The VMware images of AL2023 are available for only the `x86-64` architecture. VMware images for `aarch64` are not available or supported. These requirements are in addition to the base [AL2023 system requirements](system-requirements.md) for the VMware images. **Topics** + [VMware host requirements for running AL2023 on VMware](#vmware-host-requirements) + [Device support for AL2023 on VMware](#vmware-devices) + [Boot mode (UEFI and BIOS) support for AL2023 on VMware](#vmware-boot-modes) + [Limitations running AL2023 on VMware](#vmware-limitations) ## VMware host requirements for running AL2023 on VMware **The AL2023 VMware OVA images are currently qualified on the following:** + VMware Workstation 17.5.0 running on hosts using an Intel(R) Xeon(R) Platinum 8124M processor + VMware vSphere 8.0 using an Intel(R) Xeon(R) Platinum 8275CL processor The AL2023 VMware OVA images specify a *Machine Hardware Version* of 13. **VMware Machine Hardware Version 13 is supported by:** + ESXi 6.5 or later + VMware Workstation 14 or later ## Device support for AL2023 on VMware **The following VMware device models were tested for use with AL2023 VMware OVA images (`x86-64` only):** + `vmw_pvscsi` (VMware paravirtualized SCSI controller) + `vmxnet3` (VMware paravirtualized network device) + `ata_piix` (legacy IDE for use with the virtual CD-ROM drive only) **Additional VMware device models enabled in AL2023 VMware image qualification, but not heavily exercised:** + `vmw_vmci` and related `vsock` interface (virtual socket transport for the VMware guest agent) + `vmw_balloon` memory balloon device + VMware `SVGA` controller + legacy AT keyboard and PS/2 mouse devices The VMware guest agent package (`open-vm-tools`) is available and installed by default in the AL2023 VMware OVA images. ## Boot mode (UEFI and BIOS) support for AL2023 on VMware As of the 2023.3.20231211 release, the AL2023 VMware OVA image has been validated in both legacy BIOS and UEFI boot modes. The OVA default configuration is still legacy BIOS but can be changed by the user. **Important** Secure Boot support requires UEFI, which has not been validated for AL2023 running on VMware. ## Limitations running AL2023 on VMware There are some known limitations in running AL2023 on VMware. **Note** Code implementing some of the listed unsupported functionality may exist in AL2023 and function correctly. The list of unsupported functionality exists so that customers can make informed decisions about what to rely upon working today, and what the Amazon Linux team will qualify as working as part of future updates. **Known limitations with running AL2023 on VMware** + UEFI Secure Boot is not currently validated with AL2023 on VMware. + Hot plugging and unplugging CPU, memory, or any other device type is not supported. + VM hibernation is not supported. + VM migration is not supported. + Passthrough of any device such as through PCI Passthrough, or USB Passthrough is not supported. # Requirements for running Amazon Linux 2023 on Hyper-V This section covers the requirements for running Amazon Linux 2023 on Hyper-V. The Hyper-V images of AL2023 are available only for the `x86-64` architecture. Hyper-V images for `aarch64` are not available or supported at this time. This section covers additional requirements on top of the base [AL2023 system requirements](system-requirements.md) for the Hyper-V images. **Topics** + [Hyper-V host requirements for running Amazon Linux 2023 on Hyper-V](#hyperv-host-requirements) + [Device support for Amazon Linux 2023 on Hyper-V](#hyperv-devices) + [Limitations running Amazon Linux 2023 on Hyper-V](#hyperv-limitations) ## Hyper-V host requirements for running Amazon Linux 2023 on Hyper-V The main qualification of Amazon Linux 2023 on Hyper-V happens on Windows Server 2022 running on an EC2 `c5.metal` instance. ## Device support for Amazon Linux 2023 on Hyper-V Amazon Linux 2023 is tested on both *Generation 1* and *Generation 2* Hyper-V virtual machines with the following set of virtualized hardware: + Generation 1 (legacy BIOS boot) VM + Generation 2 (UEFI boot - No secure boot) VM + The following device models are tested for use with AL2023 Hyper-V images: + Hyper-V virtual storage `hv_storvsc` for the root disk and the emulated CD-ROM drive on *Generation 2* VMs + Emulated PIIX IDE `ata_piix` for the virtual CD-ROM drive on *Generation 1* VMs + Hyper-V virtual ethernet `hv_netvsc` + The following device models are enabled but lightly tested: + Legacy VGA text mode on *Generation 1* VMs + UEFI Firmware based framebuffer `simpledrmfb` on *Generation 2* VMs + Hyper-V Balloon `hv_balloon` + Hyper-V HID/Mouse `hid_hyperv` + The following device modes are *not* enabled in AL2023 at this time: + Hyper-V PCI pass-through + Hyper-V DRM Graphics **Important** For *Generation 2* virtual machines, Secure Boot is not supported and must be disabled prior to launching the virtual machine for a successful boot of Amazon Linux 2023. Hyper-V currently only supports Secure Boot with software components signed by Microsoft's own keys while the Amazon Linux bootloader is signed by an Amazon private key. Hyper-V doesn’t support importing 3rd party keys at this point. ## Limitations running Amazon Linux 2023 on Hyper-V The following are some known limitations in running Amazon Linux 2023 on Hyper-V: **Note** Code implementing some of the listed unsupported functionality may exist in AL2023 and function correctly. The list of unsupported functionality exists so that customers can make informed decisions about what to rely upon working today, and what the Amazon Linux team will qualify as working as part of future updates. **Known Limitations with running AL2023 on Hyper-V** + UEFI Secure Boot mode is not currently supported nor functional with AL2023 on Hyper-V + Hot plugging and unplugging CPU, memory, or any other device type is not supported. + Virtual Machine (VM) hibernation is not supported. + Virtual Machine (VM) migration is not supported. + Passthrough of any device such as through PCI Passthrough, or USB Passthrough is not supported. # Amazon Linux 2023 Set up and `cloud-init` configuration when used outside Amazon EC2 This section covers how to set up and configure a Amazon Linux 2023 virtual machine when not run directly on Amazon EC2, such as when on KVM, VMware, or Hyper-V. By default, an Amazon Linux 2023 virtual machine images don’t come provisioned with any user password or ssh key and will obtain its network configuration via DHCP on the first discovered network interface. This means that by default, without additional configuration, there is no way to connect to the resulting virtual machine. Thus, some form of configuration needs to be provided to the virtual machine. The standard mechanism to do this for Amazon Linux is via `cloud-init` data sources. Amazon Linux 2023 has been qualified with the following data sources: ** NoCloud ** This is the traditional method of configuring on-premises images via a virtual CD-ROM containing a seed ISO9660 image with `cloud-init` configuration files. ** VMware ** Amazon Linux 2023 additionally supports configuring VMware images running on vSphere via the VMware specific data source via VMware `guestinfo.userdata` and `guestinfo.metadata`. **Note** The configuration of the data sources can differ from Amazon Linux 2. More specifically, Amazon Linux 2023 uses `systemd-networkd` for its configuration and requires the use of `cloud-init` "Networking Config Version 2" as documented in [the `cloud-init` network configuration documentation](https://cloudinit.readthedocs.io/en/22.2/topics/network-config.html). The complete documentation for `cloud-init` configuration mechanisms for the version of `cloud-init` packaged in Amazon Linux 2023 can be found in the [upstream `cloud-init` documentation](https://cloudinit.readthedocs.io/en/22.2/). # NoCloud (`seed.iso`) `cloud-init` configuration for Amazon Linux 2023 on KVM and VMware This section covers how to create and use a `seed.iso` image to configure Amazon Linux 2023 running on KVM or VMware. Because KVM and VMware environments do not have [Amazon EC2 Instance Meta Data Service (IMDS)](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/configuring-instance-metadata-service), an alternate method of configuring Amazon Linux 2023 is required, and providing a `seed.iso` image is one of those methods. The `seed.iso` boot image includes the initial configuration information that is needed to boot and configure your new virtual machine, such as the network configuration, host name, and user data. **Note** The `seed.iso` image includes only the configuration information required to boot the VM. It does not include the Amazon Linux 2023 operating system files. To generate the `seed.iso` image, you need at least two configuration files, sometimes three: ** `meta-data` ** This file typically includes the hostname for the virtual machine. ** `user-data` ** This file typically configures user accounts, their passwords, ssh key pairs, and/or access mechanisms. By default, the Amazon Linux 2023 KVM and VMware images create an `ec2-user` user account. You can use the `user-data` configuration file to set the password and/or ssh keys for this default user account. ** `network-config` (optional) ** This file typically provides a network configuration for the virtual machine which will override the default one. The default configuration is to use DHCP on the first available network interface. **Create the `seed.iso` disk image** 1. On a Linux or macOS computer, create a new folder named `seedconfig` and navigate into it. **Note** Using Windows or another Operating System to complete these steps is possible, but you will have to find the equivalent tool to `mkisofs` to complete creating the `seed.iso` image. 1. Create the `meta-data` configuration file. 1. Create a new file named `meta-data`. 1. Open the `meta-data` file using your preferred editor and add the following, replacing *vm-hostname* with the host name for the VM: ``` #cloud-config local-hostname: vm-hostname ``` 1. Save and close the `meta-data` configuration file. 1. Create the `user-data` configuration file. 1. Create a new file named `user-data`. 1. Open the `user-data` file using your preferred editor and add the following, making substitutions as needed: ``` #cloud-config #vim:syntax=yaml users: # A user by the name 'ec2-user' is created in the image by default. - default - name: ec2-user ssh_authorized_keys: - ssh-rsa ssh-key # In the above line, replace ssh key with the content of your ssh public key. ``` 1. You can optionally add more user accounts to the `user-data` configuration file. You can specify additional user accounts, their access mechanisms, passwords, and key pairs. For more information about the supported directives, see the [upstream `cloud-init` documentation](https://cloudinit.readthedocs.io/en/22.2/topics/format.html). 1. Save and close the `user-data` configuration file. 1. (Optional) Create the `network-config` configuration file. 1. Create a new file named `network-config`. 1. Open the `network-config` file using your preferred editor and add the following, replacing the various IP addresses with the appropriate ones for your setup. ``` #cloud-config version: 2 ethernets: enp1s0: addresses: - 192.168.122.161/24 gateway4: 192.168.122.1 nameservers: addresses: 192.168.122.1 ``` **Note** `cloud-init` network configuration provides mechanisms to match against the MAC address of the interface instead of specifying the interface name which can change depending on the VM configuration. This (and more) `cloud-init` features for network configuration are described in more detail in the [upstream `cloud-init` Network Config Version 2 documentation](https://cloudinit.readthedocs.io/en/22.2/topics/network-config-format-v2.html). 1. Save and close the `network-config` configuration file. 1. Create the `seed.iso` disk image using the `meta-data`, `user-data`, and optional `network-config` configuration files created in the previous steps. Do one of the following, depending on the OS you are creating the `seed.iso` disk image on. + On Linux systems, use a tool such as **mkisofs** or **genisoimage** to create the completed `seed.iso` file. Navigate into the `seedconfig` folder, and run the following command: ``` $ mkisofs -output seed.iso -volid cidata -joliet -rock user-data meta-data ``` + If you use a `network-config`, include it in the invocation of **mkisofs**: ``` $ mkisofs -output seed.iso -volid cidata -joliet -rock user-data meta-data network-config ``` + On macOS systems, you can use a tool such as **hdiutil** to generate the finished `seed.iso` file. Since **hdiutil** takes a pathname rather than a list of files, the same invocation can be used regardless of if a `network-config` configuration file has been created or not. ``` $ hdiutil makehybrid -o seed.iso -hfs -joliet -iso -default-volume-name cidata seedconfig/ ``` 1. The resulting `seed.iso` file can now be attached to your new Amazon Linux 2023 Virtual Machine using a virtual CD-ROM drive for `cloud-init` to find on first boot and apply the configuration to the system. # VMware guestinfo `cloud-init` configuration for AL2023 on VMware VMware environments do not have the [Amazon EC2 Instance Meta Data Service (IMDS)](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/configuring-instance-metadata-service), so an alternate method of configuring AL2023 is required. This section describes how to use an alternative configuration mechanism to the `seed.iso` virtual CD-ROM drive that is available in VMware vSphere. This method of configuration uses the VMware `extraconfig` mechanism to provide configuration data to `cloud-init`. For each of the following keys, a corresponding ***keyname*.encoding** property must be provided. The following keys can be provided to the VMware `extraconfig` mechanism. ** `guestinfo.metadata` ** JSON or YAML containing `cloud-init` meta-data ** `guestinfo.userdata` ** A YAML document containing `cloud-init` user-data in the `cloud-config` format. ** `guestinfo.vendordata` (optional) ** YAML containing `cloud-init` vendor-data The corresponding encoding properties ( `guestinfo.metadata.encoding`, `guestinfo.userdata.encoding`, and `guestinfo.vendordata.encoding`) can contain: ** `base64` ** The content of the property is `base64` encoded. ** `gzip+base64` ** The content of the property is compressed with `gzip` after `base64` encoding. **Note** The `seed.iso` method supports a separate (optional) `network-config` configuration file. VMware `guestinfo` differs in how the networking configuration is provided. Additional information is provided in the following section. If an explicit network configuration is desired, it should be embedded in the `metadata` in the form of two YAML or JSON properties: ** `network` ** Contains the encoded network configuration in JSON or YAML form. ** `network.encoding` ** Contains the encoding of the above network configuration data. The `cloud-init` supported encodings are the same as for the `guestinfo` data: `base64` and `gzip+base64`. **Example Using the VMware vSphere **govc** CLI tool to pass configuration with `guestinfo`** 1. Prepare the `meta-data`, `user-data`, and optional `network-config` configuration files as described in [NoCloud (`seed.iso`) `cloud-init` configuration for Amazon Linux 2023 on KVM and VMware](seed-iso.md). 1. Convert the configuration files into formats usable by VMware `guestinfo`. ``` # 'meta-data', `user-data` and `network-config` are the configuration # files in the same format that would be used by a NoCloud (seed.iso) # data source, read-them and convert them to VMware guestinfo # # The VM_NAME variable is assumed to be set to the name of the VM # It is assumed that the necessary govc environment (credentials etc...) are already set metadata=$(cat "meta-data") userdata=$(cat "user-data") if [ -e "network-config" ] ; then # We need to embed the network config inside the meta-data netconf=$(base64 -w0 "network-config") metadata=$(printf "%s\nnetwork: %s\nnetwork.encoding: base64" "$metadata" "$netconf") fi metadata=$(base64 -w0 <<< "$metadata") govc vm.change -vm "$VM_NAME" \ -e guestinfo.metadata="$metadata" \ -e guestinfo.metadata.encoding="base64" userdata=$(base64 -w0 <<< "$userdata") govc vm.change -vm "$VM_NAME" \ -e guestinfo.userdata="$userdata" \ -e guestinfo.userdata.encoding="base64" ``` # Comparing packages installed on Amazon Linux 2023 standard AMI with the AL2023 KVM Image A comparison of the RPMs present on the AL2023 standard AMI compared with the RPMs present on the AL2023 KVM image. [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/linux/al2023/ug/al2023-ami-kvm-image.html) # Comparing packages installed on Amazon Linux 2023 standard AMI with the AL2023 VMware OVA Image A comparison of the RPMs present on the AL2023 standard AMI compared with the RPMs present on the AL2023 VMware OVA image. [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/linux/al2023/ug/al2023-ami-vmware-image.html) # Comparing packages installed on Amazon Linux 2023 standard AMI with the AL2023 Hyper-V image A comparison of the RPMs present on the AL2023 standard AMI compared with the RPMs present on the AL2023 Hyper-V image. [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/linux/al2023/ug/al2023-ami-hyperv-image.html)