# Hosts for custom test environments Device Farm supports a set of operating systems with pre-configured software through the use of a test host environment. During test execution, Device Farm utilizes Amazon-managed instances (hosts) that dynamically connect to the selected device under test. This instance is fully cleaned up and not re-used between runs, and is terminated with its generated artifacts after the test run concludes. **Topics** + [Available test hosts for custom test environments](#custom-test-environments-hosts-available) + [Selecting a test host for custom test environments](#test-host-selection) + [Supported software within custom test environments](custom-test-environments-hosts-software.md) + [Test environment for Android devices](custom-test-environments-hosts-android.md) + [Test environment for iOS devices](custom-test-environments-hosts-ios.md) ## Available test hosts for custom test environments The test hosts are fully managed by Device Farm. The following table lists the currently available and supported Device Farm test hosts for custom test environments. | Device Platform | Test Host | Operating System | Architecture(s) | Supported Devices | | --- | --- | --- | --- | --- | | Android | amazon\$1linux\$12 | Amazon Linux 2 | x86\$164 | Android 6 and above | | iOS | macos\$1sequoia | macOS Sequoia (version 15) | arm64 | iOS 15 to 26 | **Note** Periodically, Device Farm adds new test hosts for a device platform to support newer device OS versions and its dependencies. When this occurs, older test hosts for the respective device platform are subject to end of support. ### Operating system version Each available test host uses a specific version of the operating system supported on Device Farm at the time. Although we try to be on the latest OS version, this may might not be the latest publicly distributed version available. Device Farm will periodically update the operating system with minor version updates and security patches. To know the specific version (including the minor version) of the operating system in use during your test run, you can add the following snippet of code to any of your test spec file's phases. **Example** ``` phases: install: commands: # The following example prints the instance's operating system version details - |- if [[ "Darwin" == "$(uname)" ]]; then echo "$(sw_vers --productName) $(sw_vers --productVersion) ($(sw_vers --buildVersion))"; else echo "$(. /etc/os-release && echo $PRETTY_NAME) ($(uname -r))"; fi ``` ## Selecting a test host for custom test environments You can specify the Android and iOS test host in the appropriate `android_test_host` and `ios_test_host` variables of your [test spec file](custom-test-environment-test-spec.md#custom-test-environment-test-spec-syntax). If you do not specify a test host selection for the given device platform, tests will run on the test host that Device Farm has set as the default for the specified device and test configuration. **Important** When testing on iOS 18 and below, a legacy test host will be used when a host is not selected. For more information, see the topic on the [Legacy iOS test host](custom-test-environments-hosts-ios.md#legacy-ios-host). As an example, review the following code snippet: **Example** ``` version: 0.1 android_test_host: amazon_linux_2 ios_test_host: macos_sequoia phases: # ... ``` # Supported software within custom test environments Device Farm uses host machines that are pre-installed with many of the necessary software libraries to run test frameworks supported on our service, providing a ready testing environment on launch. Device Farm supports multiple languages through the use of our software selection mechanism, and will periodically update the versions of the languages included in the environment. For any other required software, you can modify the test spec file to install from your test package, download from the internet, or access private sources within your VPC (see [VPC ENI](https://docs.aws.amazon.com//devicefarm/latest/developerguide/vpc-eni.html) for more information). For more information, see [Test spec examples](custom-test-environment-test-spec.md#custom-test-environment-test-spec-example). ## Pre-configured software In order to facilitate device testing on each platform, the following tooling is provided on the test host: | Tools | Device Platform(s) | | --- | --- | | Android SDK Build-Tools | Android | | Android SDK Platform-Tools (includes `adb`) | Android | | Xcode | iOS | ## Selectable software In addition to the pre-configured software on the host, Device Farm offers a way to select certain versions of supported software via the `devicefarm-cli` tooling. The following table contains the selectable software and the test hosts that contain them. | Software / Tool | Hosts that support this software | Command to use in your test spec | | --- | --- | --- | | Java 17 | amazon\$1linux\$12 macos\$1sequoia | `devicefarm-cli use java 17` | | Java 11 | amazon\$1linux\$12 macos\$1sequoia | `devicefarm-cli use java 11` | | Java 8 | amazon\$1linux\$12 macos\$1sequoia | `devicefarm-cli use java 8` | | Node.js 22 | amazon\$1linux\$12 macos\$1sequoia | `devicefarm-cli use node 22` | | Node.js 20 | amazon\$1linux\$12 macos\$1sequoia | `devicefarm-cli use node 20` | | Node.js 18 | amazon\$1linux\$12 macos\$1sequoia | `devicefarm-cli use node 18` | | Node.js 16 | amazon\$1linux\$12 | `devicefarm-cli use node 16` | | Python 3.12 | amazon\$1linux\$12 macos\$1sequoia | `devicefarm-cli use python 3.12` | | Python 3.11 | amazon\$1linux\$12 macos\$1sequoia | `devicefarm-cli use python 3.11` | | Python 3.10 | amazon\$1linux\$12 macos\$1sequoia | `devicefarm-cli use python 3.10` | | Python 3.9 | amazon\$1linux\$12 macos\$1sequoia | `devicefarm-cli use python 3.9` | | Python 3.8 | amazon\$1linux\$12 | `devicefarm-cli use python 3.8` | | Ruby 3.2 | amazon\$1linux\$12 macos\$1sequoia | `devicefarm-cli use ruby 3.2` | | Ruby 2.7 | amazon\$1linux\$12 | `devicefarm-cli use ruby 2.7` | | Appium 3 | amazon\$1linux\$12 macos\$1sequoia | `devicefarm-cli use appium 3` | | Appium 2 | amazon\$1linux\$12 macos\$1sequoia | `devicefarm-cli use appium 2` | | Appium 1 | amazon\$1linux\$12 | `devicefarm-cli use appium 1` | | Xcode 26 | macos\$1sequoia | `devicefarm-cli use xcode 26` | | Xcode 16 | macos\$1sequoia | `devicefarm-cli use xcode 16` | The test host also includes commonly used supporting tools for each software version, such as the `pip` and `npm` package managers (included with Python and Node.js respectively) and dependencies (such as the Appium UIAutomator2 Driver) for tools like Appium. This ensures you have the tools needed to work with the supported test frameworks. # Using the devicefarm-cli tool in custom test environments The test host uses a standardized version management tool called ` devicefarm-cli` to select software versions. This tool is separate from the AWS CLI and only available on the Device Farm test host. With `devicefarm-cli`, you can switch to any pre-installed software version on the test host. This provides a straightforward way to maintain your Device Farm test spec file over time and gives you a predictable mechanism to upgrade software versions in the future. **Important** This command line tool is not available on legacy iOS hosts. For more information, see the topic on the [Legacy iOS test host](custom-test-environments-hosts-ios.md#legacy-ios-host). The snippet below shows the `help` page of `devicefarm-cli`: ``` $ devicefarm-cli help Usage: devicefarm-cli COMMAND [ARGS] Commands: help Prints this usage message. list Lists all versions of software configurable via this CLI. use Configures the software for usage within the current shell's environment. ``` Let's review a couple of examples using `devicefarm-cli`. To use the tool to change the Python version from *3.10* to *3.9* in your test spec file, run the following commands: ``` $ python --version Python 3.10.12 $ devicefarm-cli use python 3.9 $ python --version Python 3.9.17 ``` To change the Appium version from *1* to *2* : ``` $ appium --version 1.22.3 $ devicefarm-cli use appium 2 $ appium --version 2.1.2 ``` **Tip** Note that when you select a software version, `devicefarm-cli` also switches the supporting tools for those languages, such as `pip` for Python and `npm` for NodeJS. For more information about the preinstalled software on the test host, see [Supported software within custom test environments](custom-test-environments-hosts-software.md). # Test environment for Android devices AWS Device Farm utilizes Amazon Elastic Compute Cloud (EC2) host machines running Amazon Linux 2 to execute Android tests. When you schedule a test run, Device Farm allocates a dedicated host for each device to independently run tests. The host machines terminate after the test run along with any generated artifacts. The Amazon Linux 2 host provides several advantages: + **Faster, more reliable testing**: Compared to the legacy host, the new test host significantly improves test speed, especially reducing test start times. The Amazon Linux 2 host also demonstrates greater stability and reliability during testing. + **Enhanced Remote Access for manual testing**: Upgrades to the latest test host and improvements lead to lower latency and better video performance for Android manual testing. + **Standard software version selection**: Device Farm now standardizes major programming language support on the test host as well as Appium framework versions. For supported languages (currently Java, Python, Node.js, and Ruby) and Appium, the new test host provides long-term stable releases soon after launch. Centralized version management through the `devicefarm-cli` tool enables test spec file development with a consistent experience across frameworks. **Topics** + [Supported IP ranges for the Amazon Linux 2 test environment in Device Farm](amazon-linux-2-ip-ranges.md) # Supported IP ranges for the Amazon Linux 2 test environment in Device Farm Customers often need to know the IP range from which Device Farm's traffic originates, particularly for configuring their firewalls and security settings. For Amazon EC2 test hosts, the IP range encompasses the entire `us-west-2` region. For Amazon Linux 2 test hosts, which is the default option for new Android runs, the ranges have been restricted. The traffic now originates from a specific set of NAT gateways, restricting the IP range to the following addresses: **** | IP Ranges | | --- | | **44.236.137.143** | | **52.13.151.244** | | **52.35.189.191** | | **54.201.250.26** | For more information about Android test environments in Device Farm, see [Test environment for Android devices](custom-test-environments-hosts-android.md). # Test environment for iOS devices Device Farm utilizes Amazon-managed macOS instances (hosts) that dynamically connect to the iOS device during the test run. Each host is pre-configured with software that enables device testing on various popular test platforms, such as XCTestUI and Appium. The current iteration of the iOS test host has improved upon the testing experience when compared to previous versions, including: + ** Consistent host OS and tooling experience for iOS 15 to iOS 26 ** Before, the test host was determined by the device in use, leading to a fragmented software environment when executing on multiple iOS versions. The current experience allows simple host selection to enable a consistent environment across devices. This will enable the same macOS version and tooling (such as Xcode) to be available across each iOS device. + ** Performance improvements for iOS 15 and 16 tests ** Using updated infrastructure, setup time has improved substantially for iOS 15 and 16 tests. + ** Standardized selectable software versions for supported dependencies ** We now have the `devicefarm-cli` software selection system on both iOS and Android test hosts, enabling you to select your preferred version of our supported dependencies. For supported dependencies (such as Java, Python, Node.js, Ruby, and Appium), versions will be selectable via the test spec. For an idea of how this feature works, please see the topic on [Supported software within custom test environments](custom-test-environments-hosts-software.md). **Important** If executing on iOS 18 and below, your tests will execute on legacy test hosts by default. See the topic below on how to migrate away from legacy hosts. ## Legacy iOS test host For existing tests on iOS 18 and below, the legacy test hosts are selected by default for custom test environments. The following table contains the test host version that is executed with by the iOS device version. | Operating System | Architecture(s) | Default for Devices | | --- | --- | --- | | macOS Sonoma (version 14) | arm64 | iOS 18 | | macOS Ventura (version 13) | arm64 | iOS 17 | | macOS Monterey (version 12) | x86\$164 | iOS 16 and below | In order to select the newer test hosts, see the topic regarding [Migrating your custom test environments to the new iOS test hosts](ios-host-migration.md). ## Supported software for iOS devices In order to support iOS device testing, Device Farm test hosts for iOS devices come pre-configured with Xcode and its associated command line tooling. For other available software, please review the topic regarding [Supported software within custom test environments](custom-test-environments-hosts-software.md). # Migrating your custom test environments to the new iOS test hosts To migrate existing tests from the legacy host to the new macOS test host, you will need to develop new test spec files based on your pre-existing ones. The recommended approach is to start with the example test spec file for your desired test types, then migrate relevant commands from your old test spec file to the new one. This lets you leverage new features and optimizations of the example test spec for the new host while reusing snippets your existing code. **Topics** + [Tutorial: Migrating iOS test spec files with the console](#ios-host-migration-console-tutorial) + [Differences between the new and legacy test hosts](#ios-host-migration-differences) ## Tutorial: Migrating iOS test spec files with the console In this example, the Device Farm console will be used to onboard an existing iOS device test spec to use the new test host. ### Step 1: Creating a new test spec files with the console 1. Sign in to the [AWS Device Farm console](https://console.aws.amazon.com/devicefarm). 1. Navigate to the Device Farm project containing your automation tests. 1. Download a copy of the existing test spec your wish to onboard with. 1. Click the "Project Settings" option and navigate to the **Uploads** tab. 1. Navigate to the test spec file that you wish to onboard with. 1. Click the **Download** button to make a local copy of this file. 1. Navigate back to the Project page and click **Create run**. 1. Fill in the options on the wizard as if you were to start a new run, but stop at the **Select test spec** option. 1. Using the iOS test spec selected by default, click the **Create a test spec** button. 1. Modify the test specification that was selected by *default* in the text editor. 1. If not already present, modify the test spec file to select the new host using: ``` ios_test_host: macos_sequoia ``` 1. From the copy of your test spec downloaded in a prior step, review each ` phase`. 1. Copy commands from the old test spec's phases into each respective phase in the new test spec, ignoring commands related to installing or selecting Java, Python, Node.js, Ruby, Appium, or Xcode. 1. Enter a new file name in the **Save as** text box. 1. Click the **Save as new** button to save your changes. For an example of a test spec file you can use as a reference, see the example provided in [Test spec examples](custom-test-environment-test-spec.md#custom-test-environment-test-spec-example). ### Step 2: Selecting software pre-installed software In the new test host, pre-installed software versions are selected using a new standardized version management tool called `devicefarm-cli`. This tooling is now the recommended approach for using the various software we provide on the test hosts. As an example, you would add the following line to use a different JDK 17 your test environment: ``` - devicefarm-cli use java 17 ``` For more information on the software supported available, please review: [Supported software within custom test environments](custom-test-environments-hosts-software.md). ### Step 3: Using Appium and its dependencies via the software selection tooling The new test host only supports Appium 2.x and above. Please explicitly select the Appium version using the `devicefarm-cli`, while removing legacy tooling such as ` avm`. For example: ``` # This line using 'avm' should be removed # - avm 2.3.1 # And the following lines should be added - devicefarm-cli use appium 2 # Selects the version - appium --version # Prints the version ``` The Appium version selected with `devicefarm-cli` comes preinstalled with a compatible version of the XCUITest driver for iOS. Additionally, you will need to update your test spec to use ` DEVICEFARM_APPIUM_WDA_DERIVED_DATA_PATH_V9` instead of ` DEVICEFARM_WDA_DERIVED_DATA_PATH`. The new environment variable points to a pre-built version of WebDriverAgent 9.x, which is the latest supported version for Appium 2 tests. For more information, review [Selecting a WebDriverAgent version for iOS tests](test-types-appium.md#test-types-appium-select-wda) and [Environment variables for Appium tests](custom-test-environment-variables.md#custom-test-environment-variables-appium). ## Differences between the new and legacy test hosts When you're editing your test spec file to use the new iOS test host and transitioning your tests from the legacy test host, be aware of these key environment differences: + ** Xcode versions: ** In the legacy test host environment, the Xcode version available was based on the iOS version of the device used for testing. For example, tests on iOS 18 devices used Xcode 16 in the legacy host, whereas tests on iOS 17 used Xcode 15. In the new host environment, all devices can access the same versions of Xcode, allowing a consistent environment for tests on devices with different versions. For a list of currently available Xcode versions, see [Supported software](custom-test-environments-hosts-software.md). + ** Selecting software versions: ** In many instances, the default software versions have changed, so if you weren't explicitly selecting your software version in the legacy test host before, you may want to specify it now in the new test host using [`devicefarm-cli`](custom-test-environments-hosts-software-cli.md). In the vast majority of use cases, we recommend that customers explicitly select the versions of software they use. By selecting a software version with `devicefarm-cli` you'll have a predictable and consistent experience with it and receive ample amounts of warnings if Device Farm plans to remove that version from the test host. Moreover, software selection tools like `nvm`, `pyenv`, ` avm`, and `rvm` have been removed in favor of the new ` devicefarm-cli` software selection system. + ** Available software versions: ** Many versions of previously pre-installed software have been removed, and many new versions have been added. So, ensure that when using the `devicefarm-cli` to select your software versions, you select versions which are in the [supported version list](custom-test-environments-hosts-software.md). + ** The `libimobiledevice` suite of tools have been removed ** in favor of newer / first party tooling to track current iOS device testing and industry standards. For iOS 17 and above, you can migrate most commands to use similar Xcode tooling, called `devicectl`. For information on `devicectl`, you can run `xcrun devicectl help` from a machine with Xcode installed. + ** File paths that are hard-coded ** in your legacy host test spec file as absolute paths will most likely not work as expected in the new test host, and they're generally not recommended for test spec file use. We recommend that you use relative paths and environment variables for all test spec file code. For more information, review the topic on [Best practices for custom test environment execution](custom-test-environments-best-practices.md). + ** Operating system version and architecture: ** The legacy test hosts were using a variety of macOS versions and CPU architectures based on the assigned device. As a result, users may notice some differences in the available system libraries available in the environment. For more information on the previous host OS version, review [Legacy iOS test host](custom-test-environments-hosts-ios.md#legacy-ios-host). + **For Appium** users, the way to select the WebDriverAgent has changed to a use environment variable prefix ` DEVICEFARM_APPIUM_WDA_DERIVED_DATA_PATH_V` instead of the old ` DEVICEFARM_WDA_DERIVED_DATA_PATH_V` prefix. For more information on the updated variable, review [Environment variables for Appium tests](custom-test-environment-variables.md#custom-test-environment-variables-appium). + **For Appium Java** users, the new test host does not contain any pre-installed JAR files in its class path, whereas the previous host contained one for the TestNG framework (via an environment variable `$DEVICEFARM_TESTNG_JAR`). We recommend that customers package the necessary JAR files for their test frameworks inside their test package and remove instances of the `$DEVICEFARM_TESTNG_JAR` variable from their test spec files. We recommend reaching out to the service team through a support case if you have any feedback or questions about the differences between the test hosts from a software perspective.