# About Amazon EMR Releases An Amazon EMR release is a set of open-source applications from the big-data ecosystem. Each release comprises different big-data applications, components, and features that you select to have Amazon EMR install and configure when you create a cluster. Applications are packaged using a system based on [Apache BigTop](http://bigtop.apache.org/), which is an open-source project associated with the Hadoop ecosystem. This guide provides information for applications included in Amazon EMR releases. For more information about getting started and working with Amazon EMR, see the [Amazon EMR Management Guide](https://docs.aws.amazon.com/emr/latest/ManagementGuide/). When you launch a cluster, you can choose from multiple releases of Amazon EMR. This allows you to test and use application versions that fit your compatibility requirements. You specify the release number with the *release label*. Release labels are in the form `emr-x.x.x`. For example, `emr-7.12.0`. Beginning with Amazon EMR 5.18.0, you can use the Amazon EMR artifact repository to build your job code against the exact versions of libraries and dependencies that are available with specific Amazon EMR releases. For more information, see [Checking dependencies using the Amazon EMR artifact repository](emr-artifact-repository.md). To get updates when a new Amazon EMR release is available, subscribe to the [RSS feed for Amazon EMR release notes](https://docs.aws.amazon.com/emr/latest/ReleaseGuide/amazon-emr-release-notes.rss). **Latest release** details, including application versions, release notes, components, and configuration classifications of Amazon EMR 7.x, 6.x, and 5.x series: + [Amazon EMR Release 7.12.0](emr-7120-release.md) + [Amazon EMR Release 6.15.0](emr-6150-release.md) + [Amazon EMR Release 5.36.2](emr-5362-release.md) **Note** New Amazon EMR releases are made available in different Regions over a period of several days, beginning with the first Region on the initial release date. The latest release version may not be available in your Region during this period. **Release notes** for the latest Amazon EMR releases and a history of all releases: + [What's new?](emr-whatsnew.md) + [Amazon EMR archive of release notes](emr-whatsnew-history.md) **A comprehensive history of application versions** in each Amazon EMR release: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Details for each Amazon EMR release** and differences between release series, where applicable: + [Amazon EMR 7.x release versions](emr-release-7x.md) + [Amazon EMR 6.x release versions](emr-release-6x.md) + [Amazon EMR 5.x release versions](emr-release-5x.md) + [Amazon EMR 4.x release versions](emr-release-4x.md) + [Amazon EMR 2.x and 3.x AMI versions](emr-release-3x.md) # Amazon EMR standard support ## Understanding Amazon EMR releases The *Amazon EMR release* is the vehicle that delivers the necessary software to run your open-source applications on EC2, EKS, or Serverless platforms. An Amazon EMR release is composed of three pieces: the *Runtime Environment*, *Core Engines*, and *Extras*. + **Runtime environment** – The runtime environment includes the operating system that’s included as part of the Amazon Machine Image (AMI) or other container image when you launch your cluster or container (for example, Amazon Linux 2023). It also includes the language runtimes such as the Amazon Corretto JDK, along with other related tools that the image installs. + **Core engines** – These include the core open-source software such as Apache Hive, and open table formats such as Apache Iceberg. + **Extras** – These include convenience libraries and Python packages such as `mariadb-connector-java` and open-source software such as Apache Pig. The Amazon EMR release version that you select bundles all these components into a new version of Amazon Linux (AL) based Amazon Machine Images (AMI) or container images. Amazon EMR maintains the same AL version for all nodes of a cluster, EKS container, or serverless application. Amazon EMR aims to get the latest *Runtime Environment* within 90 days of their release by Amazon Linux, and new open-source versions of *Core Engines* within 90 days from the upstream release. *Extras* are released as needed. You can find the list of the software in release notes for each Amazon EMR release. ## Release versioning To help you understand the scope of changes in each release, Amazon EMR uses semantic versioning. Semantic versioning follows a format of `..`. As an example, let’s consider the 7.0.0 release. The first digit denotes the major version release, or **Major Release**. Major Releases typically bring substantial changes, improvements, and new features to Runtime Environment or Core Engine that are not backward compatible. For example, the 7.x Major Release uses Amazon Linux 2023 with Amazon Corretto 17 JDK as default. As a result, the release contains several breaking changes compared to 6.x Major Release, which ships on Amazon Linux 2 and with Amazon Corretto 8 JDK as default. The second digit denotes the minor version release, or **Minor Release**. Minor Releases are non-backward compatible releases that contain incremental changes, improvements, and features to Core Engines and Extras, and new functionality. For example, 6.15 uses Apache Spark 3.4.1, compared to 6.11.1, which ships with Apache Spark 3.3.2. The third digit denotes a patch version release, or **Patch Release**. Patch Releases are intended to be backward compatible releases that contain fixes and patch updates to *Core Engine* and *Extras*, but no new functionality or OSS version upgrades. For example, 6.11.1 and 6.11.0 both contain the same Apache Spark 3.3.2 OSS version. As of 6.6.x, the latest Patch Release offers the latest *Runtime Environment* patch available. *Runtime Environment* does not affect the Patch Release of Amazon EMR. For example, 6.11.1 launched with AL version 2.0.20240109.0 from December 19, 2023 to January 10, 2024, but with AL version 2.0.20240124.0 from January 11, 2024. Some Amazon EMR on EKS releases allow you to add tags or suffixes to the sematic versions to provide you with alternate functionalities. For example, the `emr-6.15.0-latest` release of Amazon EMR on EKS launches an Amazon Corretto 8 JDK enabled release version, whereas the `emr-6.15.0-java17-latest` release of Amazon EMR on EKS allows you to launch an Amazon Corretto 17 JDK enabled release version. ## Support policy **Introduction** Amazon EMR aims to issue new Minor Releases at a minimum of once every 90 days, and to provide support for Minor Releases for a period of 24 months starting from the release date. This support covers *Runtime Environment* and *Core Engines* with their associated dependencies, and doesn’t cover *Extras* or their associated dependencies. *Runtime Environment* also includes proprietary platform components and APIs that are needed to start, stop, and operate the clusters and applications. This consistent release schedule ensures a predictable cycle, making it more convenient for you to plan, test, and transition to a version that provides support and security. **What to expect with Standard Support** Standard Support provides fixes on technical support tickets for issues that you encounter for *Runtime Environment* and *Core Engines* components under recommended configurations. All fixes are subject to availability. The recommended configuration is the use of Amazon EMR without any modification, additions, or changes to the binaries and configurations present in an Amazon EMR release, except those found in the [Amazon EMR Documentation](https://docs.aws.amazon.com/emr/). Amazon EMR deploys fixes to the latest patch, minor, or major version of the Amazon EMR release within 90 days of us verifying the fix. Amazon EMR automatically applies fixes when you launch a new EMR on EC2 cluster, launch a new Amazon EMR on EKS container, or trigger a new EMR Serverless job. Extra components are provided as a convenience and Amazon EMR does **not** provide fixes related to *Extras*. **Components** Standard Support covers *Runtime Environment* and *Core Engines* components – for example, operating system, language runtimes, and core open-source software like Apache Hive and Apache Iceberg. You can find the full list of supported components for each release in [About Amazon EMR Releases](emr-release-components.md) and [Amazon Linux FAQs](https://aws.amazon.com/linux/amazon-linux-2023/faqs/). The following list describes the support that we provide for different component types under Standard Support: + **Runtime Environment components**: *Runtime Environment* components will receive fixes on technical support tickets. Fixes are categorized as (a) critical bugs, (b) critical data corruption issues, and (c) critical security issues. When eligible, Amazon EMR will backport fixes for *Runtime Environment* to older versions. For operational compatibility with the open-source Core Engine components, certain *Runtime Environment* components must remain at specific versions to prevent breaking customer applications. For these components, Amazon EMR is dependent on upstream open-source for availability of fixes. When fixes are available in open source, we will provide the latest stable version within 90 days of them being verified by Amazon EMR. + **Core Engines components**: *Core Engines* provides the latest versions for many open-source projects, each having hundreds of transitive dependency libraries. While the open-source communities managing these projects attempt to address issues and known Common Vulnerabilities and Exposures (CVE) on a frequent basis, the latest versions might still contain known bugs and CVEs. Amazon EMR is dependent on upstream open-source for availability of fixes and will provide the latest stable version as part of the *Core Engine* components within 90 days of them being verified by Amazon EMR. In certain cases, Amazon EMR might provide a fix for a CVE in one of the *Core Engine* components that needs to be addressed ahead of upstream open source. Amazon EMR also provides you with technical support and fixes on features that are added on top of open source under recommended configurations. We don’t backport fixes for *Core Engine* components to older Patch or Minor versions. + **Extras components**: Amazon EMR does **not** support *Extras* components. Extra components are open-source projects provided as convenience and Amazon EMR does not provide fixes for issues encountered with them. Any support requests or fixes can be addressed through the open-source community supporting these components. **Standard Support lifecycle** The following describes the milestones in the Standard Support lifecycle: + **Standard Support**: Amazon EMR releases are eligible for Standard Support 24 months from the date of release. You can create technical support cases and expect updates for issues that you encounter with these releases. + **End of Support**: After Standard Support ends, Amazon EMR releases enter End of Support (EoS) stage for 12 months. EoS releases are not eligible for technical support and you won't be able to create any support cases for clusters, containers, or jobs running on these releases. EoS releases won't receive any fixes, patches or updates. EoS releases will be removed from the console, but will continue to be available through the API and AWS CLI. You can always continue to run workloads on EoS releases. We strongly recommend that you migrate to the latest Amazon EMR release so that you continue to receive security patches, remain eligible for technical support, and can create support cases when needed. + **End of Life**: After the End of Support (EoS) term, releases are considered End of Life (EoL). Although you can continue to run EoL clusters, Amazon EMR reserves the right to remove EoL releases from the API and SDK on a case-by-case basis due to security and operational concerns. We strongly recommend that you migrate to the latest Amazon EMR, release because EoL versions can be removed from the API and SDK in exceptional cases. **Bridge Support** Amazon EMR announced this new support policy on July 25, 2024. Under this policy, versions of Amazon EMR released on or before July 24th, 2022 are now designated as End of Support. However, to provide you with additional time to plan and migrate to newer versions, Amazon EMR will offer Bridge Support, equivalent to Standard Support, for these older versions that released within the two-year period before this announcement. After July 25, 2024, you can refer to the current status and support timelines in the release notes. The following table shows the support status for all existing Amazon EMR releases at time of announcement of the policy, July 25, 2024: **Releases and supported periods** | Amazon EMR release version | Initial release date | Standard support end date | End of support start date | End of life start date | | --- | --- | --- | --- | --- | | [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-standard-support.html) | January 1, 2013 to July 25, 2024 | Bridge support until July 31, 2026 | August 1, 2026 | August 1, 2027 | **Note** Dates for bridge support have been extended to July 31, 2026, for all eligible releases. You can use Apache Spark Upgrade Agent to upgrade your Apache Spark existing applications on EMR on EC2 and EMR Serverless from older EMR versions to latest EMR version. To learn more, see [What is Apache Spark Upgrade Agent for Amazon EMR](spark-upgrades.md). ## Considerations Standard Support is available for all Amazon EMR deployment models (EMR on EC2, Amazon EMR on EKS, and EMR Serverless), in all Regions where Amazon EMR is available, at no additional cost. Clusters running with recommended configurations are automatically eligible for support as described in the policy, so you don’t need to take any additional actions to activate support. + Standard Support only supports components required for Amazon EMR clusters. Amazon EMR can’t guarantee security patching and fix availability in the case when *Core Engine* open-source components reach EoL upstream, or when security updates are no longer available for dependencies. While you can opt-in to install *Extras*, Amazon EMR won't support them or their dependencies. For example, you can install third-party applications in your custom AMI to harden the security of your cluster, install additional components or copy objects using bootstrap action scripts, or SSH into your cluster and upgrade the default package versions. Amazon EMR does not support these components. Standard Support doesn’t cover customer provided bootstrap actions, packages, libraries, your custom code and bring-your-own custom applications that you can configure Amazon EMR to install for your convenience. + Your existing clusters won't be impacted, regardless of which Amazon EMR release they're running. You can continue to run existing clusters without disruption. You can also continue to launch new clusters and run jobs on any of the existing releases and new releases. All existing releases and new releases at time of the policy becoming effective are covered by Standard Support for 24 months from initial date of release of the Amazon EMR release version. Amazon EMR will provide you with bridge support during the initial announcement of the policy. To receive uninterrupted support, we advise testing your applications and promptly upgrading to the most recent Amazon EMR release. + Amazon EMR won't change Standard Support components on existing releases or clusters. However, Amazon EMR reserves the right to honor the upstream End of Life on a case-by-case basis, and remove such components in the new releases, or move the existing release to End of Support (EoS), or End of Life (EoL) status in exceptional cases. We'll notify you of any removal through available channels. + When a new Region is launched, Amazon EMR will only support Amazon EMR release versions under Standard Support, released in six months prior to the date of of when new Region becomes generally available. + Amazon EMR won't automatically update your existing clusters to latest versions. However, you can choose to update new clusters to the latest patch versions if desired. + The cost of your unsupported Amazon EMR usage will count toward your AWS bill. Even if you are using Amazon EMR in an unsupported way, the costs associated with that usage are still part of your overall AWS consumption and will be included in the calculation of your support fees. For more information, contact [AWS Developer Support](https://aws.amazon.com/premiumsupport/). # What is Apache Spark Upgrade Agent for Amazon EMR ## Introduction The Apache Spark Upgrade Agent for Amazon EMR is a conversational AI capability that accelerates Apache Spark version upgrades for your EMR applications. Traditional Spark upgrades require months of engineering effort to analyze API changes, resolve dependency conflicts, and validate functional correctness. The agent simplifies the upgrade process through natural language prompts, automated code transformation, and data quality validation. You can use the agent to upgrade PySpark and Scala applications running on Amazon EMR on EC2 and Amazon EMR Serverless. The agent analyzes your code, identifies required changes, and performs automated transformations while maintaining your approval control over all modifications. **Note** The Apache Spark Upgrade Agent is available at no additional cost as part of Amazon EMR. You only pay for the underlying Amazon EMR resources used during validation job execution. ## Architecture Overview The upgrade agent has three main components: any MCP-compatible AI Assistant in your development environment for interaction, the [ MCP Proxy for AWS](https://github.com/aws/mcp-proxy-for-aws) that handles secure communication between your client and the MCP server, and the Amazon SageMaker Unified Studio Managed MCP Server (in preview) that provides specialized Spark upgrade tools for Amazon EMR. This diagram illustrates how you interact with the Amazon SageMaker Unified Studio Managed MCP Server through your AI Assistant. ![\[Apache Spark Upgrade Agent\]](http://docs.aws.amazon.com/emr/latest/ReleaseGuide/images/SparkUpgradeIntroduction.png) The AI assistant will orchestrate the upgrade using specialized tools provided by the MCP server following these steps: 1. **Planning**: The agent analyzes your project structure and generates or revises an upgrade plan that guides the end-to-end Spark upgrade process. 1. **Compile and Build**: The agent updates the build environment and dependencies, compiles the project, and iteratively fixes build and test failures. 1. **Spark code edit tools**: The agent applies targeted code updates to resolve Spark version incompatibilities, fixing both build-time and runtime errors. 1. **Execute & Validation**: The agent submits remote validation jobs to EMR, monitors execution and logs, and iteratively fixes runtime and data-quality issues. 1. **Observability**: The agent tracks upgrade progress using EMR observability tools and allows users to view upgrade analyses and status at any time. Please refer to [Using Spark Upgrade Tools](emr-spark-upgrade-agent-tools.md) for a list of major tools for each steps. **Topics** + [Introduction](#emr-spark-upgrade-agent-intro) + [Architecture Overview](#emr-spark-upgrade-agent-architecture) + [Setup for Upgrade Agent](emr-spark-upgrade-agent-setup.md) + [Using the Upgrade Agent](emr-spark-upgrade-agent-using.md) + [Features and Capabilities](emr-spark-upgrade-agent-features.md) + [Troubleshooting and Q&A](emr-spark-upgrade-agent-troubleshooting.md) + [Spark Upgrade Agent Workflow In Details](emr-spark-upgrade-agent-workflow-details.md) + [Enable Data Quality Validation](emr-spark-upgrade-agent-data-quality-validation.md) + [Prompt Examples for the Spark Upgrade Agent](emr-spark-upgrade-agent-prompt-examples.md) + [Creating target EMR Cluster/EMR-S application from existing ones](emr-spark-upgrade-agent-target-cluster.md) + [IAM Role Setup](emr-spark-upgrade-agent-iam-role.md) + [Configuring Interface VPC Endpoints for Amazon SageMaker Unified Studio MCP](spark-upgrade-agent-vpc-endpoints.md) + [Using Spark Upgrade Tools](emr-spark-upgrade-agent-tools.md) + [Cross-region processing for the Apache Spark Upgrade Agent](emr-spark-upgrade-agent-cross-region.md) + [Logging Amazon SageMaker Unified Studio MCP calls using AWS CloudTrail](spark-upgrade-cloudtrail-integration.md) + [Service improvements for Apache Spark Agents](emr-spark-agent-service-improvements.md) # Setup for Upgrade Agent **Note** The Apache Spark Upgrade Agent uses cross-region inference to process natural language requests and generate responses. For more details please refer to this page on [Cross-region processing for the Apache Spark Upgrade Agent](emr-spark-upgrade-agent-cross-region.md). The Amazon SageMaker Unified Studio MCP server is in preview and is subject to change. ## Prerequisites Before we begin our setup process for integration with Kiro CLI, make sure you have the following installed on your workstation: + [ Install AWS CLI ](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html) + [ Install Python 3.10\$1 ](https://www.python.org/downloads/release/python-3100/) + [ Install the `uv` package manager ](https://docs.astral.sh/uv/getting-started/installation/) for [ MCP Proxy for AWS](https://github.com/aws/mcp-proxy-for-aws?tab=readme-ov-file) + [ Install Kiro CLI ](https://kiro.dev/docs/cli/) + AWS local credentials configured (via [AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-configure.html), environment variables, or IAM roles) - for local operations such uploading upgraded job artifacts for EMR validation job execution. ## Setup Resources You can use an CloudFormation template to setup the resource for the MCP server. These templates are samples that you should modify to meet your requirements. The template creates the following resources for the upgrade process: + IAM role which has permissions to call MCP Server and required permissions for upgrade process for the underlying EMR platform. + Amazon S3 staging bucket used to upload upgrade artifacts and optional KMS key for Amazon S3 encryption. Choose one of the **Launch Stack** buttons in the following table. This launches the stack on the CloudFormation console in the respective region. | Region | Launch | | --- | --- | | US East (N. Virginia) | [https://console.aws.amazon.com/cloudformation/home?region=us-east-1#/stacks/new?stackName=spark-upgrade-mcp-setup&templateURL=https%3A%2F%2Fsmus-mcp-cfn-template-prod-us-east-1.s3.us-east-1.amazonaws.com/cloudformation/spark-upgrade-mcp-setup.yaml](https://console.aws.amazon.com/cloudformation/home?region=us-east-1#/stacks/new?stackName=spark-upgrade-mcp-setup&templateURL=https%3A%2F%2Fsmus-mcp-cfn-template-prod-us-east-1.s3.us-east-1.amazonaws.com/cloudformation/spark-upgrade-mcp-setup.yaml) | | US East (Ohio) | [https://console.aws.amazon.com/cloudformation/home?region=us-east-2#/stacks/new?stackName=spark-upgrade-mcp-setup&templateURL=https%3A%2F%2Fsmus-mcp-cfn-template-prod-us-east-2.s3.us-east-2.amazonaws.com/cloudformation/spark-upgrade-mcp-setup.yaml](https://console.aws.amazon.com/cloudformation/home?region=us-east-2#/stacks/new?stackName=spark-upgrade-mcp-setup&templateURL=https%3A%2F%2Fsmus-mcp-cfn-template-prod-us-east-2.s3.us-east-2.amazonaws.com/cloudformation/spark-upgrade-mcp-setup.yaml) | | US West (Oregon) | [https://console.aws.amazon.com/cloudformation/home?region=us-west-2#/stacks/new?stackName=spark-upgrade-mcp-setup&templateURL=https%3A%2F%2Fsmus-mcp-cfn-template-prod-us-west-2.s3.us-west-2.amazonaws.com/cloudformation/spark-upgrade-mcp-setup.yaml](https://console.aws.amazon.com/cloudformation/home?region=us-west-2#/stacks/new?stackName=spark-upgrade-mcp-setup&templateURL=https%3A%2F%2Fsmus-mcp-cfn-template-prod-us-west-2.s3.us-west-2.amazonaws.com/cloudformation/spark-upgrade-mcp-setup.yaml) | | Europe (Ireland) | [https://console.aws.amazon.com/cloudformation/home?region=eu-west-1#/stacks/new?stackName=spark-upgrade-mcp-setup&templateURL=https%3A%2F%2Fsmus-mcp-cfn-template-prod-eu-west-1.s3.eu-west-1.amazonaws.com/cloudformation/spark-upgrade-mcp-setup.yaml](https://console.aws.amazon.com/cloudformation/home?region=eu-west-1#/stacks/new?stackName=spark-upgrade-mcp-setup&templateURL=https%3A%2F%2Fsmus-mcp-cfn-template-prod-eu-west-1.s3.eu-west-1.amazonaws.com/cloudformation/spark-upgrade-mcp-setup.yaml) | | Asia Pacific (Tokyo) | [https://console.aws.amazon.com/cloudformation/home?region=ap-northeast-1#/stacks/new?stackName=spark-upgrade-mcp-setup&templateURL=https%3A%2F%2Fsmus-mcp-cfn-template-prod-ap-northeast-1.s3.ap-northeast-1.amazonaws.com/cloudformation/spark-upgrade-mcp-setup.yaml](https://console.aws.amazon.com/cloudformation/home?region=ap-northeast-1#/stacks/new?stackName=spark-upgrade-mcp-setup&templateURL=https%3A%2F%2Fsmus-mcp-cfn-template-prod-ap-northeast-1.s3.ap-northeast-1.amazonaws.com/cloudformation/spark-upgrade-mcp-setup.yaml) | | Asia Pacific (Singapore) | [https://console.aws.amazon.com/cloudformation/home?region=ap-southeast-1#/stacks/new?stackName=spark-upgrade-mcp-setup&templateURL=https%3A%2F%2Fsmus-mcp-cfn-template-prod-ap-southeast-1.s3.ap-southeast-1.amazonaws.com/cloudformation/spark-upgrade-mcp-setup.yaml](https://console.aws.amazon.com/cloudformation/home?region=ap-southeast-1#/stacks/new?stackName=spark-upgrade-mcp-setup&templateURL=https%3A%2F%2Fsmus-mcp-cfn-template-prod-ap-southeast-1.s3.ap-southeast-1.amazonaws.com/cloudformation/spark-upgrade-mcp-setup.yaml) | | Asia Pacific (Sydney) | [https://console.aws.amazon.com/cloudformation/home?region=ap-southeast-2#/stacks/new?stackName=spark-upgrade-mcp-setup&templateURL=https%3A%2F%2Fsmus-mcp-cfn-template-prod-ap-southeast-2.s3.ap-southeast-2.amazonaws.com/cloudformation/spark-upgrade-mcp-setup.yaml](https://console.aws.amazon.com/cloudformation/home?region=ap-southeast-2#/stacks/new?stackName=spark-upgrade-mcp-setup&templateURL=https%3A%2F%2Fsmus-mcp-cfn-template-prod-ap-southeast-2.s3.ap-southeast-2.amazonaws.com/cloudformation/spark-upgrade-mcp-setup.yaml) | | Canada (Central) | [https://console.aws.amazon.com/cloudformation/home?region=ca-central-1#/stacks/new?stackName=spark-upgrade-mcp-setup&templateURL=https%3A%2F%2Fsmus-mcp-cfn-template-prod-ca-central-1.s3.ca-central-1.amazonaws.com/cloudformation/spark-upgrade-mcp-setup.yaml](https://console.aws.amazon.com/cloudformation/home?region=ca-central-1#/stacks/new?stackName=spark-upgrade-mcp-setup&templateURL=https%3A%2F%2Fsmus-mcp-cfn-template-prod-ca-central-1.s3.ca-central-1.amazonaws.com/cloudformation/spark-upgrade-mcp-setup.yaml) | | South America (São Paulo) | [https://console.aws.amazon.com/cloudformation/home?region=sa-east-1#/stacks/new?stackName=spark-upgrade-mcp-setup&templateURL=https%3A%2F%2Fsmus-mcp-cfn-template-prod-sa-east-1.s3.sa-east-1.amazonaws.com/cloudformation/spark-upgrade-mcp-setup.yaml](https://console.aws.amazon.com/cloudformation/home?region=sa-east-1#/stacks/new?stackName=spark-upgrade-mcp-setup&templateURL=https%3A%2F%2Fsmus-mcp-cfn-template-prod-sa-east-1.s3.sa-east-1.amazonaws.com/cloudformation/spark-upgrade-mcp-setup.yaml) | | Europe (Frankfurt) | [https://console.aws.amazon.com/cloudformation/home?region=eu-central-1#/stacks/new?stackName=spark-upgrade-mcp-setup&templateURL=https%3A%2F%2Fsmus-mcp-cfn-template-prod-eu-central-1.s3.eu-central-1.amazonaws.com/cloudformation/spark-upgrade-mcp-setup.yaml](https://console.aws.amazon.com/cloudformation/home?region=eu-central-1#/stacks/new?stackName=spark-upgrade-mcp-setup&templateURL=https%3A%2F%2Fsmus-mcp-cfn-template-prod-eu-central-1.s3.eu-central-1.amazonaws.com/cloudformation/spark-upgrade-mcp-setup.yaml) | | Europe (Stockholm) | [https://console.aws.amazon.com/cloudformation/home?region=eu-north-1#/stacks/new?stackName=spark-upgrade-mcp-setup&templateURL=https%3A%2F%2Fsmus-mcp-cfn-template-prod-eu-north-1.s3.eu-north-1.amazonaws.com/cloudformation/spark-upgrade-mcp-setup.yaml](https://console.aws.amazon.com/cloudformation/home?region=eu-north-1#/stacks/new?stackName=spark-upgrade-mcp-setup&templateURL=https%3A%2F%2Fsmus-mcp-cfn-template-prod-eu-north-1.s3.eu-north-1.amazonaws.com/cloudformation/spark-upgrade-mcp-setup.yaml) | | Europe (London) | [https://console.aws.amazon.com/cloudformation/home?region=eu-west-2#/stacks/new?stackName=spark-upgrade-mcp-setup&templateURL=https%3A%2F%2Fsmus-mcp-cfn-template-prod-eu-west-2.s3.eu-west-2.amazonaws.com/cloudformation/spark-upgrade-mcp-setup.yaml](https://console.aws.amazon.com/cloudformation/home?region=eu-west-2#/stacks/new?stackName=spark-upgrade-mcp-setup&templateURL=https%3A%2F%2Fsmus-mcp-cfn-template-prod-eu-west-2.s3.eu-west-2.amazonaws.com/cloudformation/spark-upgrade-mcp-setup.yaml) | | Europe (Paris) | [https://console.aws.amazon.com/cloudformation/home?region=eu-west-3#/stacks/new?stackName=spark-upgrade-mcp-setup&templateURL=https%3A%2F%2Fsmus-mcp-cfn-template-prod-eu-west-3.s3.eu-west-3.amazonaws.com/cloudformation/spark-upgrade-mcp-setup.yaml](https://console.aws.amazon.com/cloudformation/home?region=eu-west-3#/stacks/new?stackName=spark-upgrade-mcp-setup&templateURL=https%3A%2F%2Fsmus-mcp-cfn-template-prod-eu-west-3.s3.eu-west-3.amazonaws.com/cloudformation/spark-upgrade-mcp-setup.yaml) | | Asia Pacific (Seoul) | [https://console.aws.amazon.com/cloudformation/home?region=ap-northeast-2#/stacks/new?stackName=spark-upgrade-mcp-setup&templateURL=https%3A%2F%2Fsmus-mcp-cfn-template-prod-ap-northeast-2.s3.ap-northeast-2.amazonaws.com/cloudformation/spark-upgrade-mcp-setup.yaml](https://console.aws.amazon.com/cloudformation/home?region=ap-northeast-2#/stacks/new?stackName=spark-upgrade-mcp-setup&templateURL=https%3A%2F%2Fsmus-mcp-cfn-template-prod-ap-northeast-2.s3.ap-northeast-2.amazonaws.com/cloudformation/spark-upgrade-mcp-setup.yaml) | | Asia Pacific (Mumbai) | [https://console.aws.amazon.com/cloudformation/home?region=ap-south-1#/stacks/new?stackName=spark-upgrade-mcp-setup&templateURL=https%3A%2F%2Fsmus-mcp-cfn-template-prod-ap-south-1.s3.ap-south-1.amazonaws.com/cloudformation/spark-upgrade-mcp-setup.yaml](https://console.aws.amazon.com/cloudformation/home?region=ap-south-1#/stacks/new?stackName=spark-upgrade-mcp-setup&templateURL=https%3A%2F%2Fsmus-mcp-cfn-template-prod-ap-south-1.s3.ap-south-1.amazonaws.com/cloudformation/spark-upgrade-mcp-setup.yaml) | Proceed to the **Specify stack details** page, enter the **Stack name**. Enter additional information under **Parameters**. Provide the following information and proceed to create the stack. + **SparkUpgradeIAMRoleName** - Name of the IAM role to use for Spark upgrade process + **EnableEMREC2** - Enable EMR-EC2 upgrade permissions (default: true) + **EnableEMRServerless** - Enable EMR-Serverless upgrade permissions (default: true) + **StagingBucketPath** - Amazon S3 path for staging artifacts (e.g., s3://my-bucket/spark-upgrade or my-bucket/spark-upgrade). Leave empty to auto-generate a new bucket + **UseS3Encryption** - Enable KMS encryption for Amazon S3 staging bucket (default: false, set to true to use KMS encryption instead of default S3 encryption) + **S3KmsKeyArn** - (Optional) ARN of existing KMS key for Amazon S3 bucket encryption. Only used if UseS3Encryption is true and you have an existing bucket with a KMS key + **CloudWatchKmsKeyArn** - (Optional) ARN of existing KMS key for CloudWatch Logs encryption (EMR Serverless only, leave empty for default encryption) + **EMRServerlessS3LogPath** - (Optional) S3 path where EMR-Serverless application logs are stored (e.g., s3://my-bucket/emr-serverless-logs or my-bucket/emr-serverless-logs). When provided, grants the IAM role read access to these logs for analysis. Only used when EnableEMRServerless is true + **ExecutionRoleToGrantS3Access** - (Optional) IAM Role Name or ARN of existing EMR-EC2/EMR-Serverless execution role to grant Amazon S3 staging bucket access. Only applies when a new staging bucket is created. Useful for granting EMR job execution roles access to the staging bucket. Supports both simple role names and ARNs with paths. You may also download and review [the CloudFormation template](https://github.com/aws-samples/aws-emr-utilities/blob/03c20fece616de23ec0ea5389f0113a5bc65fc3a/utilities/apache-spark-agents/spark-upgrade-agent-cloudformation/spark-upgrade-mcp-setup.yaml), specify the options above and launch the template by yourself with CloudFormation CLI commands, see below for an example: ``` # deploy the stack with CloudFormation CLI commands aws cloudformation deploy \ --template-file spark-upgrade-mcp-setup.yaml \ --stack-name spark-mcp-setup \ --region \ --capabilities CAPABILITY_NAMED_IAM \ --parameter-overrides \ ExecutionRoleToGrantS3Access= # retrieve the 1-line instruction to set the local environment variables, which will be used for the following MCP server configuration aws cloudformation describe-stacks \ --stack-name spark-mcp-setup \ --region \ --query "Stacks[0].Outputs[?OutputKey=='ExportCommand'].OutputValue" \ --output text ``` Open the Outputs tab (or retrieve from the CloudFormation describe-stacks CLI command above) and copy the 1-line instruction `ExportCommand`, then execute it in your local environment. Example 1-line instruction: ``` export SMUS_MCP_REGION= && export IAM_ROLE=arn:aws:iam::111122223333:role/spark-upgrade-role-xxxxxx && export STAGING_BUCKET_PATH= ``` Then run the following command locally to setup the IAM profile and MCP server configuration ``` # Step 1: Configure AWS CLI Profile aws configure set profile.spark-upgrade-profile.role_arn ${IAM_ROLE} aws configure set profile.spark-upgrade-profile.source_profile aws configure set profile.spark-upgrade-profile.region ${SMUS_MCP_REGION} # Step 2: if you are using Kiro CLI, use the following command to add the MCP configuration kiro-cli-chat mcp add \ --name "spark-upgrade" \ --command "uvx" \ --args "[\"mcp-proxy-for-aws@latest\",\"https://sagemaker-unified-studio-mcp.${SMUS_MCP_REGION}.api.aws/spark-upgrade/mcp\", \"--service\", \"sagemaker-unified-studio-mcp\", \"--profile\", \"spark-upgrade-profile\", \"--region\", \"${SMUS_MCP_REGION}\", \"--read-timeout\", \"180\"]" \ --timeout 180000\ --scope global ``` This should update `~/.kiro/settings/mcp.json` to include the MCP server configuration as below. ``` { "mcpServers": { "spark-upgrade": { "type": "stdio", "command": "uvx", "args": [ "mcp-proxy-for-aws@latest", "https://sagemaker-unified-studio-mcp.us-east-1.api.aws/spark-upgrade/mcp", "--service", "sagemaker-unified-studio-mcp", "--profile", "spark-upgrade-profile", "--region", "us-east-1", "--read-timeout", "180" ], "timeout": 180000, "disabled": false } } } ``` See [Using the Upgrade Agent](emr-spark-upgrade-agent-using.md) for the configuration guidance for different MCP clients like Kiro, Cline and GitHub CoPilot. # Using the Upgrade Agent ## Supported Deployment Modes Apache Spark Upgrade Agent for Amazon EMR support the following two deployment mode for the end-to-end Spark application upgrade experience including build file upgrade, script/dependency upgrade, local test and validation with target EMR cluster or EMR Serverless application, and data quality validation. + EMR on EC2 + EMR Serverless Please refer to [Features and Capabilities](emr-spark-upgrade-agent-features.md) to understand the detailed features, capacities and limitations. ## Supported Interfaces ### Integration With Amazon SageMaker Unified Studio VS Code Editor Spaces On Amazon SageMaker Unified Studio VS Code Editor Spaces, you can configure the IAM profile and MCP config as described in the Setup for Upgrade Agent by simply following the screenshot below: ![\[Apache Spark Upgrade Agent\]](http://docs.aws.amazon.com/emr/latest/ReleaseGuide/images/SparkUpgradeVSCodeEditor.png) A demonstration of the EMR on EC2 upgrade experience with SMUS VS code editor. It starts with a simple prompt to ask to the Agent start the Spark Upgrade process. ``` Upgrade my Spark application from EMR version 6.0.0 to 7.12.0. Use EMR-EC2 Cluster to run the validation and s3 paths s3:// to store updated application artifacts. Use spark-upgrade-profile for AWS CLI operations. ``` [![AWS Videos](http://img.youtube.com/vi/https://www.youtube.com/embed/Hk4_GPvYeqI/0.jpg)](http://www.youtube.com/watch?v=https://www.youtube.com/embed/Hk4_GPvYeqI) ### Integration With Kiro CLI (QCLI) Start Kiro CLI or your AI Assistant and verify the loaded tools for the upgrade agent. ``` ... spark-upgrade (MCP): - check_and_update_build_environment * not trusted - check_and_update_python_environment * not trusted - check_job_status * not trusted - compile_and_build_project * not trusted ... ``` A demonstration of the EMR Serverless upgrade experience with Kiro CLI. You can simply start the Upgrade process with the following prompt: ``` Upgrade my Spark application from EMR version 6.0.0 to 7.12.0. Use EMR-Serverless Applicaion and execution role to run the validation and s3 paths s3:// to store updated application artifacts. ``` [![AWS Videos](http://img.youtube.com/vi/https://www.youtube.com/embed/Kw8E2wWv4uQ/0.jpg)](http://www.youtube.com/watch?v=https://www.youtube.com/embed/Kw8E2wWv4uQ) ### Integration with Other IDEs The [configuration](emr-spark-upgrade-agent-setup.md#spark-upgrade-agent-setup-resources) can also be used in other IDEs to connect to the Managed MCP server: + **Integration With Cline** - To use the MCP Server with Cline, modify the `cline_mcp_settings.json` and add the configuration above. Consult [Cline's documentation](https://docs.cline.bot/mcp/configuring-mcp-servers) for more information on how to manage MCP configuration. + **Integration With Claude Code** To use the MCP Server with Claude Code, modify the configuration file to include the MCP configuration. The file path varies depending on your operating system. Refer to [https://code.claude.com/docs/en/mcp](https://code.claude.com/docs/en/mcp) for detailed setup. + **Integration With GitHub Copilot** - To use the MCP server with GitHub Copilot, follow the instruction in [ https://docs.github.com/en/copilot/how-tos/provide-context/use-mcp/extend-copilot-chat-with-mcp](https://docs.github.com/en/copilot/how-tos/provide-context/use-mcp/extend-copilot-chat-with-mcp) to modify the corresponding configuration file and follow the instructions per each IDE to activate the setup. ## Setup EMR Cluster or EMR Serverless Application for the Target Version Create the EMR cluster or EMR Serverless application with the expected Spark version that you plan to use for the upgraded application. The target EMR Cluster or EMR-S Application will be used to submit the validation job runs after the Spark application artifacts are upgraded to verify successful upgrade or fix additional errors encountered during the validation. If you already have a target EMR cluster or EMR Serverless application, you can refer to the existing one and skip this step. Use non-production developer accounts and select sample mock datasets that represent your production data but are smaller in size for validation with Spark Upgrades. Please refer to this page for the guidance to create a target EMR cluster or EMR Serverless application from existing ones: [Creating target EMR Cluster/EMR-S application from existing ones](emr-spark-upgrade-agent-target-cluster.md). # Features and Capabilities ## Supported Technologies + **Languages**: Python and Scala applications + **Build Systems**: Maven and SBT for Scala projects; requirements.txt, Pipfile, and Setuptools for Python projects + **Target Platforms**: Amazon EMR and EMR Serverless + **Supported Versions**: We support Apache Spark upgrades from version 2.4 to 3.5. The corresponding deployment mode mappings are as follows + **For EMR-EC2** + **Source Version:** EMR 5.20.0 and later + **Target Version:** EMR 7.12.0 and earlier, should be newer than EMR 5.20.0 + **For EMR Serverless** + **Source Version:** EMR Serverless 6.6.0 and later + **Target Version:** EMR Serverless 7.12.0 and earlier ## What We Upgrade The upgrade agent provide comprehensive Spark application upgrades: + **Build Configuration**: Automatically updates dependency management files (pom.xml, requirements.txt, etc.) + **Source Code**: Fixes API compatibility issues and deprecated method usage + **Test Code**: Ensures unit and integration tests work with the target Spark version + **Dependencies**: Upgrades packaged dependencies to versions compatible with target EMR version + **Validation**: Compiles and validates applications on target EMR clusters + **Data Quality Analysis**: Detects schema differences, value-level statistical drifts (min/max/mean), and aggregate row-count mismatches, with detailed impact reporting. ## Available Regions The Spark Upgrade Agent is available in the following regions: + **Asia Pacific**: Tokyo (ap-northeast-1), Seoul (ap-northeast-2), Singapore (ap-southeast-1), Sydney (ap-southeast-2), and Mumbai (ap-south-1) + **North America**: Canada (ca-central-1) + **Europe**: Stockholm (eu-north-1), Ireland (eu-west-1), London (eu-west-2), Paris (eu-west-3), and Frankfurt (eu-central-1) + **South America**: São Paulo (sa-east-1) + **United States**: North Virginia (us-east-1), Ohio (us-east-2), and Oregon (us-west-2) ## Scope of Upgrades and User Requirements + **Cluster Management**: Spark Upgrade Agent focuses on application code upgrades. Target EMR clusters for new versions must be created and managed by users. + **Bootstrap Actions**: Spark Upgrade Agent does not upgrade custom bootstrap scripts outside of Spark application code. They need to be upgraded by the user. + **Upgrade for Build and Tests**: The upgrade agent will perform build and run your unit and integration tests on your development environment locally to validate that applications compile successfully with the target Spark version. If you have restrictions (security policies, resource limitations, network restrictions, or corporate guidelines) for Spark application code for local execution, consider using [ Amazon SageMaker Unified Studio VSCode IDE Spaces](https://docs.aws.amazon.com/sagemaker-unified-studio/latest/userguide/create-space.html) or EC2 to run the upgrade agent. The upgrade agent uses your target EMR-EC2 cluster or EMR-S applications to validate and upgrade end-to-end. + **Error-Driven Approach**: The upgrade agent uses an error-driven methodology, making one fix at a time based on compilation or runtime errors rather than multiple fixes at once. This iterative approach ensures each issue is properly addressed before proceeding to the next. + **Private Dependencies**: Dependencies installed from private artifact repositories cannot be automatically upgraded as part of this process. They must be upgraded by the user. + **Regional resources**: Spark upgrade agent is regional and uses the underlying EMR resources in that region for the upgrade process. Cross-region upgrades are not supported. # Troubleshooting and Q&A ## Troubleshooting The error message from the upgrade process is available in different ways for different MCP clients. In this page, we list some general guidance for common issues you may see using Apache Spark Upgrade Agent for Amazon EMR. **Topics** + [Error: SMUS Managed MCP server failed to load](#spark-upgrade-agent-mcp-server-failed) + [Observation : Slow Tool Loading](#spark-upgrade-agent-slow-tool-loading) + [Error: Tool Invocation Failed with Throttling Error](#spark-upgrade-agent-throttling-error) + [Error: Tool Responds With User Error](#spark-upgrade-agent-user-error) + [Error: Tool Responds With Internal Error](#spark-upgrade-agent-internal-error) ### Error: SMUS Managed MCP server failed to load + Verify your MCP configuration are properly configured. + **Validate JSON syntax**: + Ensure your JSON is valid with no syntax errors + Check for missing commas, quotes, or brackets + Verify your local AWS credentials and verify the policy for the MCP IAM role are properly configured. + Run /mcp to verify MCP server availability for `Kiro-CLI` case ### Observation : Slow Tool Loading + Tools may take a few seconds to be loaded on first attempt of launching the server. + If tools don't appear, try restarting the chat, + Run `/tools` command to verify tool availability. + Run `/mcp` if the server is launched without error. ### Error: Tool Invocation Failed with Throttling Error If you hit your service limit, please wait for a few seconds to issue a tool invocation if you see the throttling exception. ### Error: Tool Responds With User Error + AccessDeniedException - check the error message and fix the permission issue. + InvalidInputException - check the error message and correct the tool input parameters. + ResourceNotFoundException - check the error message and fix the input parameter for resource reference. ### Error: Tool Responds With Internal Error + If you see `The service is handling high-volume requests` please retry the tool invocation in a few seconds. + If you see `INTERNAL SERVICE EXCEPTION` please document the analysis id, tool name, any error message available from the mcp log or tool response and optional sanitized conversation history and reach out to the AWS support. ## Q&A ### 1. Should I enable "trust" setting for the tools by default? Do not turn on "trust" setting by default for all tool calls initially and operate on git-versioned build environment. Review each tool execution to understand what changes are being made, for example review the upgrade plan before providing consent. Never trust the Kiro or your Agent's `fs_write` tool - this tool modifies your project files and you should always review changes before they are saved/committed. In general, examine all proposed file changes before allowing them to be written. Use git to version changes and create checkpoints throughout the upgrade process. Build confidence gradually - If you are repeatedly using the system for multiple upgrades and have gained confidence in the tools, you can selectively trust certain tools. ### 2. What are the common example prompts to leverage the upgrade tools? Please refer to [Prompt Examples for the Spark Upgrade Agent](emr-spark-upgrade-agent-prompt-examples.md) for the prompt examples about leveraging upgrade tools. # Spark Upgrade Agent Workflow In Details To initiate the upgrade process, you will need the Spark application code cloned to your developer environment (locally or EC2 or [ Amazon SageMaker Unified Studio IDE Spaces](https://docs.aws.amazon.com/sagemaker-unified-studio/latest/userguide/create-space.html)), preferably with Git version control initialized. Additionally, an EMR cluster running the target Spark version must be provisioned and accessible. Finally, a designated Amazon S3 bucket path should be configured to store deployment artifacts and upgrade summary throughout the upgrade process. Once these requirements are established, you can submit a prompt like the following to kick off the upgrade workflow: ``` Upgrade my Spark application from EMR version 6.0.0 to 7.12.0. Use EMR-EC2 Cluster to run the validation and s3 paths s3:// to store updated application artifacts. ``` At this point, the agent will orchestrate the upgrade using specialized tools (for more [details](emr-spark-upgrade-agent-tools.md)). The workflow follows these steps: 1. **Generate Plan**: The agent will analyze your project structure and generate an upgrade plan. Review the plan and provide your consent to proceed. 1. **Plan Review and Customization**: When prompted to review the plan, you have several options: 1. **Proceed as-is**: Accept the plan and continue with execution 1. **Provide feedback**: Customize the plan by: 1. Removing unnecessary steps - Example: Remove any integration test execution. Only compile/build locally, then proceed to EMR validation. 1. Adding additional steps - Example: Add a step to run test file `tests/test_jobs/test_etl_job_x.py` before EMR validation. 1. Modifying the upgrade approach - Example: Enforce Python 3.10 and Java 17 during the build and validation steps. 1. The agent will regenerate the plan based on your feedback and ask for consent again. This process continues until you approve the final plan 1. **Compile and Build**: The agent will make iterative changes to fix build errors until the application compiles and builds successfully. 1. **Run unit and Integration Tests**: If the project has tests, the agent will run the tests after a successful build. If any tests fail, the agent will modify the source code iteratively until the tests pass before proceeding to EMR validation. 1. **Runtime Fixes and Validation**: The agent will validate the application on the target EMR cluster and iteratively fix any runtime errors until validation is successful. Upon completion, you'll see a summary of all changes made for compatibility. 1. **Summary for the upgrade**: Once the upgrade is complete, the agent will provide a summary of all code and configuration changes, dependency version updates, and any detected data quality mismatches for your review. # Enable Data Quality Validation You can enable data quality checks by providing both source and target cluster IDs in your prompt. The system runs your existing application on the source cluster to collect baseline metadata for comparison. **Note: Only Spark write operations can be tracked for data quality.** ``` Upgrade my pyspark application /pyspark-example-24/ from EMR version 6.0.0 to 7.12.0. Use EMR-EC2 Cluster for source version run and for target version run. Use s3 path s3:// to store updated application artifacts and s3:///metadata for storing metadata. Enable data quality checks. ``` ## Data Quality Workflow Differences The workflow follows the same steps as the standard upgrade process with these additional steps in the order above: + **[After Step 3: Plan Review and Customization]** + **Build with Current Configuration**: Build the application with current configuration for source cluster submission. + **Validate on Source EMR Cluster**: Run original application on source Spark version and collect output metadata for baseline comparison. + **[After Step 7: Summary for the upgrade] Data Quality Summary**: Data quality comparison report between versions and analysis. ### Data quality mismatch capability currently includes: + **Schema Checks**: Detects changes in column structure: missing or newly added columns, data type differences, and nullability changes. + **Value Checks** *(numeric and string columns only)* + Compares min, max, and mean (mean only for numeric columns). + For strings, min and max are based on lexicographical order. + **Aggregated Statistical Checks**: Compares total row counts between source and target outputs. ## Data Quality Validation: Scope and Limitations Data Quality Validation supports EMR-EC2 step using spark-submit command with Spark version >= 3.0 and the EMR cluster cannot have StepConcurrencyLevel > 1. The Data Quality Validation evaluates statistics at the Spark query plan's data sink nodes (Data source/Transforms's metadata are not captured) and covers common Spark write operations including file writes, database inserts, table creation, and various data source outputs. # Prompt Examples for the Spark Upgrade Agent We list a few different prompt examples that can be used in the upgrade process. ## 1. Provide existing EMR Step information for the agent for upgrading a Spark application ``` Upgrade my Spark application from EMR version 6.0.0 to 7.12.0. Use EMR-EC2 Cluster to run the validation and s3 paths s3:// to store updated application artifacts. The Spark application is already executed in the source EMR-EC2 Cluster with step id . ``` ## 2. List analysis (used for Upgrades) and check the status of a specific analysis ``` Can you list all the spark upgrade analysis. ``` ``` Describe the analysis with analysis id . ``` ## 3. Modify an upgrade plan after it's generated ``` Help me to update the generated upgrade plan to skip the local test step. ``` ## 4. Override the default configuration for a upgrade validation job execution ``` For submitting my EMR Severless validaiton job. I would like to use S3 logging instead of CW logging, and use this S3 path . ``` # Creating target EMR Cluster/EMR-S application from existing ones If you already have an EMR-EC2 cluster running the source Spark version, you can clone it to create a new cluster with the same configuration but an updated EMR release version to run the validation steps during the Upgrade process. ## Steps: 1. Sign in to the AWS Management Console and open the Amazon EMR console. 1. In the left navigation pane, choose Clusters under EMR on EC2. 1. From the cluster list: + Use search or filters if needed to find your cluster. + Select the check box next to the cluster you want to clone. + The Clone option will appear at the top of the list. Choose Clone. + If the cluster has steps configured, choose Include steps and then Continue to clone the steps along with the other cluster settings. 1. Review the settings for the new cluster that have been copied from the original cluster. 1. Update the Amazon EMR release version to the target version. 1. As a best practice for cost efficiency, consider enabling autoscaling when configuring the cluster to automatically adjust capacity based on workload demands. 1. When you're satisfied with the configuration, select Create cluster to launch the new cluster. 1. Wait for the cluster to reach the Running status and note the cluster ID. Provide this ID to the agent when prompted to validate the updated application. 1. If you prefer to create a new cluster from scratch, please refer to the EMR documentation: [ https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-gs.html ](https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-gs.html). ## Cloning an EMR Serverless application to the target Release Version There is no direct way to clone an EMR Serverless application to a new one with a different EMR release version. Instead, you can use the EMR Serverless SDK/CLI or the console to create a new application and reuse the configuration from an existing one. Here are some helper CLI commands to get started: ``` aws emr-serverless get-application --application-id XXX > old-config.json cat old-config.json | jq '{ name: (.application.name + ""), releaseLabel: "", type: .application.type, initialCapacity: .application.initialCapacity, maximumCapacity: .application.maximumCapacity, autoStartConfiguration: .application.autoStartConfiguration, autoStopConfiguration: .application.autoStopConfiguration, tags: .application.tags, architecture: .application.architecture, runtimeConfiguration: .application.runtimeConfiguration, monitoringConfiguration: .application.monitoringConfiguration }' > new-config.json aws emr-serverless create-application --cli-input-json file://new-config.json ``` **Note:** Ensure that the job execution role for the new application allows EMR Serverless to assume the role. Review the role's trust policy. If you are reusing the same job execution role from the old application, update the trust policy to include the new application as shown below: ``` #Replace the old application id and new application id in the policy { "Version": "2012-10-17", "Statement": [ { "Sid": "ServerlessTrustPolicy", "Effect": "Allow", "Principal": { "Service": "emr-serverless.amazonaws.com" }, "Action": "sts:AssumeRole", "Condition": { "StringLike": { "aws:SourceArn": [ "arn:aws:emr-serverless:us-east-1::/applications/", "arn:aws:emr-serverless:us-east-1::/applications/" ] } } } ] } ``` # IAM Role Setup The CloudFormation stack in Setup Instructions automates the IAM role setup for you. If you want to manually execute it, please follow the instructions below: ## IAM Role Setup for MCP server To access the SMUS Managed MCP server, an IAM role is required with the following inline policy: ``` { "Version": "2012-10-17", "Statement": [ { "Sid": "AllowUseSagemakerUnifiedStudioMcpServer", "Effect": "Allow", "Action": [ "sagemaker-unified-studio-mcp:InvokeMcp", "sagemaker-unified-studio-mcp:CallReadOnlyTool", "sagemaker-unified-studio-mcp:CallPrivilegedTool" ], "Resource": [ "*" ] } ] } ``` In the next steps, we will create a profile for this role. Whichever account assumes this role to obtain the credentials should be added to the assume role policy. ``` { "Version": "2012-10-17", "Statement": [ { "Sid": "AllowAccountToAssumeRole", "Effect": "Allow", "Principal": { "AWS": "arn:aws:iam:::root" }, "Action": "sts:AssumeRole" } ] } ``` ## Additional Permissions by Deployment Mode (EMR-EC2/EMR-S) ### EMR-EC2 Applications Replace Amazon S3 staging bucket in the policy with the Amazon S3 bucket where you want the upgraded artifacts to be stored ``` { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "elasticmapreduce:DescribeCluster", "elasticmapreduce:DescribeStep", "elasticmapreduce:ListSteps", "elasticmapreduce:ListClusters", "elasticmapreduce:DescribeJobFlows", "elasticmapreduce:AddJobFlowSteps", "elasticmapreduce:CreatePersistentAppUI", "elasticmapreduce:DescribePersistentAppUI", "elasticmapreduce:GetPersistentAppUIPresignedURL" ], "Resource": "*" }, { "Effect": "Allow", "Action": [ "s3:GetBucket*", "s3:GetObject*", "s3:List*", "s3:Put*" ], "Resource": [ "arn:aws:s3:::", "arn:aws:s3:::/*" ] } ] } ``` #### KMS permissions - Staging Bucket If the staging bucket is encrypted with a CMK, add the following policy. The service will automatically use the CMK configured on the bucket when uploading data. ``` { "Effect": "Allow", "Action": [ "kms:GenerateDataKey", "kms:Encrypt" ], "Resource": "arn:aws:kms:::key/" } ``` ### EMR Serverless Applications Replace Amazon S3 staging bucket in the policy with the Amazon S3 bucket where you want the upgraded artifacts to be stored ``` { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "emr-serverless:StartJobRun", "emr-serverless:GetJobRun", "emr-serverless:GetApplication", "emr-serverless:ListApplications", "emr-serverless:GetDashboardForJobRun" ], "Resource": "*" }, { "Effect": "Allow", "Action": "iam:PassRole", "Resource": "*", "Condition": { "StringLike": { "iam:PassedToService": "emr-serverless.amazonaws.com" } } }, { "Effect": "Allow", "Action": [ "logs:GetLogEvents", "logs:DescribeLogStreams" ], "Resource": "arn:aws:logs:*:*:log-group:*" }, { "Effect": "Allow", "Action": [ "s3:GetBucket*", "s3:GetObject*", "s3:List*", "s3:Put*" ], "Resource": [ "arn:aws:s3:::", "arn:aws:s3:::/*" ] } ] } ``` #### KMS permissions - Staging Bucket If the staging bucket is encrypted with a CMK, add the following policy. The service will automatically use the CMK configured on the bucket when uploading data ``` { "Effect": "Allow", "Action": [ "kms:GenerateDataKey", "kms:Encrypt" ], "Resource": "arn:aws:kms:::key/" } ``` #### KMS permissions - CloudWatch Logs If the CloudWatch Logs are encrypted with a CMK, add the following policy so the service can read the EMR-Serverless application logs. ``` { "Effect": "Allow", "Action": [ "kms:Decrypt", "kms:DescribeKey" ], "Resource": "arn:aws:kms:::key/" } ``` # Configuring Interface VPC Endpoints for Amazon SageMaker Unified Studio MCP You can establish a private connection between your VPC and Amazon SageMaker Unified Studio MCP service by creating an *interface VPC endpoint*. Interface endpoints are powered by [Amazon VPC](https://aws.amazon.com/vpc/), which enables you to privately access the MCP server in your VPC without an internet gateway, NAT device, VPN connection, or connection. Instances in your VPC don't need public IP addresses to communicate with the MCP service and traffic between your VPC and MCP service does not leave the Amazon network. Each interface endpoint is represented by one or more [Elastic Network Interfaces](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-eni.html) in your VPC subnets. For more information, see [Interface VPC endpoints](https://docs.aws.amazon.com/vpc/latest/userguide/vpce-interface.html) in the *Amazon VPC User Guide*. ## Step 1: Creating an interface VPC endpoint for Amazon SageMaker Unified Studio MCP You can create a VPC endpoint for the Amazon SageMaker Unified Studio MCP service using either the Amazon VPC console or the AWS CLI. For more information, see [Creating an interface endpoint](https://docs.aws.amazon.com/vpc/latest/userguide/vpce-interface.html#create-interface-endpoint) in the *Amazon VPC User Guide*. Create a VPC endpoint for Amazon SageMaker Unified Studio MCP using the following service name: + com.amazonaws.**.sagemaker-unified-studio-mcp If you enable private DNS for the endpoint, you can make API requests to Amazon SageMaker Unified Studio MCP using its default DNS name for the Region, for example, `sagemaker-unified-studio-mcp.us-east-1.api.aws` For more information, see [Accessing a service through an interface endpoint](https://docs.aws.amazon.com/vpc/latest/userguide/vpce-interface.html#access-service-though-endpoint) in the *Amazon VPC User Guide*. ## Step 2: Creating a VPC endpoint Policy for Amazon SageMaker Unified Studio MCP You can attach an endpoint policy to your VPC endpoint that controls access to Amazon SageMaker Unified Studio MCP. The policy specifies the following information: + The principal that can perform actions. + The actions that can be performed. + The resources on which actions can be performed. For more information, see [Controlling access to services with VPC endpoints](https://docs.aws.amazon.com/vpc/latest/userguide/vpc-endpoints-access.html) in the *Amazon VPC User Guide*. ### Example: VPC endpoint policy to allow MCP access to a specific IAM role The following is an example of an endpoint policy for Amazon SageMaker Unified Studio MCP access. When attached to an endpoint, this policy grants access to the listed Amazon SageMaker Unified Studio MCP actions for a specific IAM role principal on all resources. ``` { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": { "AWS": "arn:aws:iam::ACCOUNT-ID:role/YourRoleName" }, "Action": [ "sagemaker-unified-studio-mcp:InvokeMcp", "sagemaker-unified-studio-mcp:CallReadOnlyTool", "sagemaker-unified-studio-mcp:CallPrivilegedTool" ], "Resource": "*" } ] } ``` ## Step 3: Testing your VPC The `curl` command validates end-to-end network connectivity from your VPC network (EC2) to the VPC endpoint by making an HTTP/HTTPS request. A curl response receiving a Message back from the MCP server confirms that the complete network path is functional. ### Method 1: With Private DNS Enabled (Recommended) ``` curl https://sagemaker-unified-studio-mcp.us-east-1.api.aws/spark-upgrade/mcp ``` ### Method 2: Without Private DNS Enabled ``` curl -k https://vpce-0069xxxx-ejwh6xxx.sagemaker-unified-studio-mcp.us-east-1.vpce.amazonaws.com/spark-upgrade/mcp ``` **Note** The `-k` flag bypasses SSL certificate verification due to hostname mismatch between the VPC endpoint DNS name and the certificate's Common Name (CN). In both cases, the curl command returns a response: `{"Message":"...."}`. Returning with a Message verifies the successful network path connectivity to the VPC endpoint of the MCP service. ## Step 4: Start using the MCP VPC endpoint Once you have verified the connection, you can follow the steps to configure the MCP in [Setup for Upgrade Agent](emr-spark-upgrade-agent-setup.md). Simply use the private VPC endpoint in your MCP configuration. # Using Spark Upgrade Tools ## Available Upgrade Tools The MCP service provides several tools to assist with Spark upgrades. The major tools are the following: | | Tool name | Tool Category | Description | | --- | --- | --- | --- | | 1 | generate\$1spark\$1upgrade\$1plan | Planner | Generate a EMR-EC2/EMR-S upgrade plan | | 2 | reuse\$1existing\$1spark\$1upgrade\$1plan | Planner | Reuse existing local upgrade plan | | 3 | update\$1build\$1configuration | Build | Upgrade build configuration files | | 4 | check\$1and\$1update\$1build\$1environment | Build | Check/update Java environment for Spark upgrade | | 5 | compile\$1and\$1build\$1project | Build | Provide guidance for compiling/building project | | 6 | run\$1validation\$1job | Test | Submit Spark app to EMR-EC2/EMR-S | | 7 | check\$1job\$1status | Test | Check status of EMR-EC2 step or EMR-S job run | | 8 | check\$1and\$1update\$1python\$1environment | Spark code edit | Check/update Python env for Spark upgrade | | 9 | fix\$1upgrade\$1failure | Spark code edit | Analyze a failure and suggest fixes | | 10 | get\$1data\$1quality\$1summary | Observablity | Retrieve the data quality summary after the application is upgraded | | 11 | describe\$1upgrade\$1analysis | Observablity | describe the given analysis | | 12 | list\$1upgrade\$1analyses | Observablity | list all analyses created by current account | For a full list of all the upgrade tools provided by the SMUS Managed MCP server, you may list all tools from the server. # Cross-region processing for the Apache Spark Upgrade Agent The Apache Spark Upgrade Agent uses cross-region inference to process natural language requests and generate responses. With cross-region inference, the agent will automatically route your inference request to optimize performance, maximizing available compute resources and model availability, and providing the best customer experience. The type of cross-region inference used depends on the region where you run the Apache Spark Upgrade Agent. In most Regions the agent will select the optimal region within your geography to process your inference requests. However, in some Regions an inference request made by the Agent will be securely routed to all available compute resources across all global commercial AWS Regions. ## Cross-Region Inference The Apache Spark Upgrade agent is powered by and uses cross-region inference to distribute traffic across different AWS Regions to enhance large language model (LLM) inference performance and reliability. Although cross-region inference does not change where your Spark application or your upgrade experience is hosted or your data is stored, your input prompts and output results may be transmitted to different Regions for inference processing. All data will be transmitted encrypted across Amazon's secure network. There is no additional cost for using cross-region inference. ## Supported regions for cross-region inference ### Regions Using Geographic cross-region inference For most Regions, cross-region inference requests are kept within AWS Regions that are part of the same geography where you run the Apache Spark Upgrade Agent. For example, a request made from the agent in the US East (N. Virginia) Region is routed only to AWS Regions within the United States geography. The following table describes what Regions your requests may be routed to depending on the geography where the request originated: | | Supported geography | Inference regions | | --- | --- | --- | | 1 | United States | US East (N. Virginia) (us-east-1), US West (Oregon) (us-west-2), US East (Ohio) (us-east-2), US West (N. California) (us-west-2) | | 2 | Europe | Europe (Frankfurt) (eu-central-1), Europe (Ireland) (eu-west-1), Europe (Paris) (eu-west-3), Europe (Stockholm) (eu-north-1), Europe (London) (eu-west-2) | | 3 | Asia Pacific | Asia Pacific (Tokyo) (ap-northeast-1), Asia Pacific (Seoul) (ap-northeast-2), Asia Pacific (Mumbai) (ap-south-1) | ### Regions Using Global Cross-Region Inference **Important** The following AWS Regions use global cross-region inference. When you use the Apache Spark Upgrade Agent in these Regions, your requests may be transmitted globally to other AWS Regions for inference processing to optimize performance and availability: + South America (Sao Paulo) (sa-east-1) + Asia Pacific (Singapore) (ap-southeast-1) + Asia Pacific (Sydney) (ap-southeast-2) + Canada (Central) (ca-central-1) # Logging Amazon SageMaker Unified Studio MCP calls using AWS CloudTrail Amazon SageMaker Unified Studio MCP Server is integrated with AWS CloudTrail, a service that provides a record of actions taken by a user, role, or an AWS service in Amazon SageMaker Unified Studio MCP Server. CloudTrail captures all API calls for Amazon SageMaker Unified Studio MCP Server as events. The calls captured include calls to Amazon SageMaker Unified Studio MCP Server and code calls to other AWS operations during the tool executions from the SageMaker Unified Studio MCP Server. If you create a trail, you can enable continuous delivery of CloudTrail events to an Amazon S3 bucket, including events for Amazon SageMaker Unified Studio MCP Server. If you don't configure a trail, you can still view the most recent events in the CloudTrail console in **Event history**. Using the information collected by CloudTrail, you can determine the request that was made to Amazon SageMaker Unified Studio MCP Server, the IP address from which the request was made, who made the request, when it was made, and additional details. To learn more about CloudTrail, see the [AWS CloudTrail User Guide](https://docs.aws.amazon.com/awscloudtrail/latest/userguide/). ## Amazon SageMaker Unified Studio MCP Server information in CloudTrail CloudTrail is enabled on your AWS account when you create the account. When activity occurs in Amazon SageMaker Unified Studio MCP Server, that activity is recorded in a CloudTrail event along with other AWS service events in **Event history**. You can view, search, and download recent events in your AWS account. For more information, see [Viewing events with CloudTrail Event history](https://docs.aws.amazon.com/awscloudtrail/latest/userguide/view-cloudtrail-events.html). For an ongoing record of events in your AWS account, including events for SageMaker Unified Studio MCP Server, create a trail. A trail enables CloudTrail to deliver log files to an Amazon S3 bucket. By default, when you create a trail in the console, the trail applies to all AWS Regions. The trail logs events from all Regions in the AWS partition and delivers the log files to the Amazon S3 bucket that you specify. Additionally, you can configure other AWS services to further analyze and act upon the event data collected in CloudTrail logs. For more information, see the following: + [Overview for creating a trail](https://docs.aws.amazon.com/awscloudtrail/latest/userguide/cloudtrail-create-and-update-a-trail.html) + [CloudTrail supported services and integrations](https://docs.aws.amazon.com/awscloudtrail/latest/userguide/cloudtrail-aws-service-specific-topics.html) + [Configuring Amazon SNS notifications for CloudTrail](https://docs.aws.amazon.com/awscloudtrail/latest/userguide/configure-sns-notifications-for-cloudtrail.html) + [Receiving CloudTrail log files from multiple regions](https://docs.aws.amazon.com/awscloudtrail/latest/userguide/receive-cloudtrail-log-files-from-multiple-regions.html) and [Receiving CloudTrail log files from multiple accounts](https://docs.aws.amazon.com/awscloudtrail/latest/userguide/cloudtrail-receive-logs-from-multiple-accounts.html) All SageMaker Unified Studio MCP Server tool invocations and API calls to AWS services during the tool executions are logged by CloudTrail. For example, calls to the different tools and AWS service calls made from the tools generate entries in the CloudTrail log files. Every event or log entry contains information about who generated the request. The identity information helps you determine the following: + Whether the request was made with root or IAM user credentials. + Whether the request was made with temporary security credentials for a role or federated user. + Whether the request was made by another AWS service. For more information, see the [CloudTrail userIdentity element](https://docs.aws.amazon.com/awscloudtrail/latest/userguide/cloudtrail-event-reference-user-identity.html). ## Understanding Amazon SageMaker Unified Studio MCP Server log file entries A trail is a configuration that enables delivery of events as log files to an Amazon S3 bucket that you specify. CloudTrail log files contain one or more log entries. An event represents a single request from any source and includes information about the requested action, the date and time of the action, request parameters, and so on. CloudTrail log files aren't an ordered stack trace of the public API calls, so they don't appear in any specific order. The following example shows a CloudTrail log entry that demonstrates the `CallTool` action. ``` { "eventVersion": "1.09", "userIdentity": { ... }, "eventTime": "...", "eventSource": "sagemaker-unified-studio-mcp.amazonaws.com", "eventName": "CallPrivilegedTool", "awsRegion": "us-east-1", "sourceIPAddress": "...", "userAgent": "...", "requestParameters": { "id": 1, "method": "tools/call", "params": { "name": "generate_spark_upgrade_plan", "arguments": "***", "_meta": { "progressToken": 1 } }, "jsonrpc": "2.0" }, "responseElements": { "result": { "content": "***", "structuredContent": "***", "isError": false }, "id": 1, "jsonrpc": "2.0" }, "requestID": "12345678-1234-1234-1234-123456789012", "eventID": "87654321-4321-4321-4321-210987654321", "readOnly": false, "eventType": "AwsMcpEvent", "managementEvent": true, "recipientAccountId": "123456789012", "eventCategory": "Management" } ``` The following example shows a CloudTrail log entry that demonstrates the `AddJobFlowSteps` action from Amazon SageMaker Unified Studio MCP during an upgrade tool invocation. ``` { "eventVersion": "1.11", "userIdentity": { "type": "AssumedRole", "principalId": "...", "arn": "...", "accountId": "123456789012", "accessKeyId": "...", "sessionContext": { ... }, "invokedBy": "sagemaker-unified-studio-mcp.amazonaws.com" }, "eventTime": "...", "eventSource": "elasticmapreduce.amazonaws.com", "eventName": "AddJobFlowSteps", "awsRegion": "us-east-1", "sourceIPAddress": "sagemaker-unified-studio-mcp.amazonaws.com", "userAgent": "sagemaker-unified-studio-mcp.amazonaws.com", "requestParameters": { "jobFlowId": "j-2PY4KXXXXXX63", "steps": [ ... ] }, "responseElements": { "stepIds": [ ... ] }, "requestID": "12345678-1234-1234-1234-123456789013", "eventID": "87654321-4321-4321-4321-210987654322", "readOnly": false, "eventType": "AwsApiCall", "managementEvent": true, "recipientAccountId": "123456789012", "sharedEventID": "12345678-1234-1234-1234-123456789012", "vpcEndpointId": "sagemaker-unified-studio-mcp.amazonaws.com", "vpcEndpointAccountId": "sagemaker-unified-studio-mcp.amazonaws.com", "eventCategory": "Management" } ``` # Service improvements for Apache Spark Agents The Apache Spark agent for Amazon EMR may use content, for example, to help the Agent provide better responses to common questions, fix operational issues, or for debugging. ## Content that AWS may use for service improvement + Your natural language prompts and generated responses from the Apache Spark agents for Amazon EMR, and Amazon SageMaker Notebooks ## Content that AWS does not use for service improvement + Code that you write yourself for the Spark applications + SageMaker Notebook context and metadata + Data from your Data Catalog or other data sources Only Amazon employees will have access to the data. Your trust, privacy, and the security of your Customer Content are our highest priority and ensure that our use complies with our commitments to you. For more information, see Data Privacy FAQ. ## How to opt out To opt out of data collection for the Apache Spark Agents configure an AI services opt-out policy in for Amazon SageMaker Unified Studio MCP Service. For more information, see [ AI services opt-out policies ](https://docs.aws.amazon.com/organizations/latest/userguide/orgs_manage_policies_ai-opt-out.html) in the *AWS Organizations User Guide*. When you configure an AI services opt-out policy, it has the following effects: + AWS will delete the data that it collected and stored for service improvement prior to your opt out (if any). + After you opt out, AWS will no longer collect or store this data. + AWS will no longer use your content for service improvement. # Amazon EMR 7.x release versions This section contains application versions, release notes, component versions, and configuration classifications available in each Amazon EMR 7.x release version. When you launch a cluster, you can choose from multiple releases of Amazon EMR. This allows you to test and use application versions that fit your compatibility requirements. You specify the release number with the *release label*. Release labels are in the form `emr-x.x.x`. For example, `emr-7.12.0`. New Amazon EMR releases are made available in different Regions over a period of several days, beginning with the first Region on the initial release date. The latest release version may not be available in your Region during this period. For a comprehensive table of application versions in every Amazon EMR 7.x release, see [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md). **Topics** + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Amazon EMR release 7.12.0](emr-7120-release.md) + [Amazon EMR release 7.11.0](emr-7110-release.md) + [Amazon EMR release 7.10.0](emr-7100-release.md) + [Amazon EMR release 7.9.0](emr-790-release.md) + [Amazon EMR release 7.8.0](emr-780-release.md) + [Amazon EMR release 7.7.0](emr-770-release.md) + [Amazon EMR release 7.6.0](emr-760-release.md) + [Amazon EMR release 7.5.0](emr-750-release.md) + [Amazon EMR release 7.4.0](emr-740-release.md) + [Amazon EMR release 7.3.0](emr-730-release.md) + [Amazon EMR release 7.2.0](emr-720-release.md) + [Amazon EMR release 7.1.0](emr-710-release.md) + [Amazon EMR release 7.0.0](emr-700-release.md) # Application versions in Amazon EMR 7.x releases The following table lists the application versions that are available in each Amazon EMR 7.x release. **Application version information** | | emr-7.12.0 | emr-7.11.0 | emr-7.10.0 | emr-7.9.0 | emr-7.8.0 | emr-7.7.0 | emr-7.6.0 | emr-7.5.0 | emr-7.4.0 | emr-7.3.0 | emr-7.2.0 | emr-7.1.0 | emr-7.0.0 | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | AWS SDK for Java | 2.35.5, 1.12.792 | 2.35.5, 1.12.792 | 2.31.48, 1.12.782 | 2.31.16, 1.12.782 | 2.29.52, 1.12.780 | 2.29.44, 1.12.780 | 2.29.25, 1.12.779 | 2.28.8, 1.12.772 | 2.28.8, 1.12.772 | 2.25.70, 1.12.747 | 2.23.18, 1.12.705 | 2.23.18, 1.12.656 | 2.20.160-amzn-0, 1.12.569 | | Python | 3.9, 3.11 | 3.9, 3.11 | 3.9, 3.11 | 3.9, 3.11 | 3.9, 3.11 | 3.9, 3.11 | 3.9, 3.11 | 3.9, 3.11 | 3.9, 3.11 | 3.9, 3.11 | 3.9, 3.11 | 3.9, 3.11 | 3.9 | | Scala | 2.12.18 | 2.12.18 | 2.12.18 | 2.12.18 | 2.12.18 | 2.12.18 | 2.12.18 | 2.12.18 | 2.12.18 | 2.12.18 | 2.12.17 | 2.12.17 | 2.12.17 | | AmazonCloudWatchAgent | 1.300032.2-amzn-0 | 1.300032.2-amzn-0 | 1.300032.2-amzn-0 | 1.300032.2-amzn-0 | 1.300032.2-amzn-0 | 1.300032.2-amzn-0 | 1.300032.2-amzn-0 | 1.300032.2-amzn-0 | 1.300032.2-amzn-0 | 1.300032.2-amzn-0 | 1.300032.2-amzn-0 | 1.300032.2-amzn-0 | 1.300031.1-amzn-0 | | Delta | 3.3.2-amzn-1 | 3.3.2-amzn-0 | 3.3.0-amzn-2 | 3.3.0-amzn-1 | 3.3.0-amzn-0 | 3.2.1-amzn-2 | 3.2.1-amzn-1 | 3.2.1-amzn-0 | 3.2.0-amzn-1 | 3.2.0-amzn-0 | 3.1.0 | 3.0.0 | 3.0.0 | | Flink | 1.20.0-amzn-6 | 1.20.0-amzn-5 | 1.20.0-amzn-4 | 1.20.0-amzn-3 | 1.20.0-amzn-2 | 1.20.0-amzn-1 | 1.20.0-amzn-0 | 1.19.1-amzn-1 | 1.19.1-amzn-0 | 1.18.1-amzn-2 | 1.18.1-amzn-1 | 1.18.1-amzn-0 | 1.18.0-amzn-0 | | Ganglia | - | - | - | - | - | - | - | - | - | - | - | - | - | | HBase | 2.6.2-amzn-3 | 2.6.2-amzn-2 | 2.6.2-amzn-1 | 2.6.2-amzn-0 | 2.6.1-amzn-2 | 2.6.1-amzn-1 | 2.6.1-amzn-0 | 2.5.10-amzn-0 | 2.5.5-amzn-0 | 2.4.17-amzn-7 | 2.4.17-amzn-6 | 2.4.17-amzn-5 | 2.4.17-amzn-4 | | HCatalog | 3.1.3-amzn-21 | 3.1.3-amzn-20 | 3.1.3-amzn-19 | 3.1.3-amzn-18 | 3.1.3-amzn-17 | 3.1.3-amzn-16 | 3.1.3-amzn-15 | 3.1.3-amzn-14 | 3.1.3-amzn-13 | 3.1.3-amzn-12 | 3.1.3-amzn-11 | 3.1.3-amzn-10 | 3.1.3-amzn-9 | | Hadoop | 3.4.1-amzn-4 | 3.4.1-amzn-3 | 3.4.1-amzn-2 | 3.4.1-amzn-1 | 3.4.1-amzn-0 | 3.4.0-amzn-3 | 3.4.0-amzn-2 | 3.4.0-amzn-1 | 3.4.0-amzn-0 | 3.3.6-amzn-5 | 3.3.6-amzn-4 | 3.3.6-amzn-3 | 3.3.6-amzn-2 | | Hive | 3.1.3-amzn-21 | 3.1.3-amzn-20 | 3.1.3-amzn-19 | 3.1.3-amzn-18 | 3.1.3-amzn-17 | 3.1.3-amzn-16 | 3.1.3-amzn-15 | 3.1.3-amzn-14 | 3.1.3-amzn-13 | 3.1.3-amzn-12 | 3.1.3-amzn-11 | 3.1.3-amzn-10 | 3.1.3-amzn-9 | | Hudi | 1.0.2-amzn-1 | 1.0.2-amzn-0 | 0.15.0-amzn-7 | 0.15.0-amzn-6 | 0.15.0-amzn-5 | 0.15.0-amzn-4 | 0.15.0-amzn-3 | 0.15.0-amzn-2 | 0.15.0-amzn-1 | 0.15.0-amzn-0 | 0.14.1-amzn-1 | 0.14.1-amzn-0 | 0.14.0-amzn-1 | | Hue | 4.11.0 | 4.11.0 | 4.11.0 | 4.11.0 | 4.11.0 | 4.11.0 | 4.11.0 | 4.11.0 | 4.11.0 | 4.11.0 | 4.11.0 | 4.11.0 | 4.11.0 | | Iceberg | 1.10.0-amzn-0 | 1.9.1-amzn-0 | 1.8.1-amzn-0 | 1.7.1-amzn-2 | 1.7.1-amzn-1 | 1.7.1-amzn-0 | 1.6.1-amzn-2 | 1.6.1-amzn-1 | 1.6.1-amzn-0 | 1.5.2-amzn-0 | 1.5.0-amzn-0 | 1.4.3-amzn-0 | 1.4.2-amzn-0 | | JupyterEnterpriseGateway | 2.6.0 | 2.6.0 | 2.6.0 | 2.6.0 | 2.6.0 | 2.6.0 | 2.6.0 | 2.6.0 | 2.6.0 | 2.6.0 | 2.6.0 | 2.6.0 | 2.6.0 | | JupyterHub | 1.5.0 | 1.5.0 | 1.5.0 | 1.5.0 | 1.5.0 | 1.5.0 | 1.5.0 | 1.5.0 | 1.5.0 | 1.5.0 | 1.5.0 | 1.5.0 | 1.5.0 | | Livy | 0.8.0-incubating | 0.8.0-incubating | 0.8.0-incubating | 0.8.0-incubating | 0.8.0-incubating | 0.8.0-incubating | 0.8.0-incubating | 0.8.0-incubating | 0.8.0-incubating | 0.8.0-incubating | 0.8.0-incubating | 0.8.0-incubating | 0.7.1-incubating | | MXNet | - | - | - | - | - | - | - | - | - | - | 1.9.1 | 1.9.1 | 1.9.1 | | Mahout | - | - | - | - | - | - | - | - | - | - | - | - | - | | Oozie | 5.2.1 | 5.2.1 | 5.2.1 | 5.2.1 | 5.2.1 | 5.2.1 | 5.2.1 | 5.2.1 | 5.2.1 | 5.2.1 | 5.2.1 | 5.2.1 | 5.2.1 | | Phoenix | 5.2.1 | 5.2.1 | 5.2.1 | 5.2.1 | 5.2.1 | 5.2.1 | 5.2.0 | 5.2.0 | 5.2.0 | 5.1.3 | 5.1.3 | 5.1.3 | 5.1.3 | | Pig | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | | Presto | 0.287-amzn-6 | 0.287-amzn-5 | 0.287-amzn-4 | 0.287-amzn-3 | 0.287-amzn-2 | 0.287-amzn-1 | 0.287-amzn-1 | 0.287-amzn-0 | 0.287-amzn-0 | 0.285-amzn-1 | 0.285-amzn-0 | 0.284-amzn-0 | 0.283-amzn-1 | | Spark | 3.5.6-amzn-1 | 3.5.6-amzn-0 | 3.5.5-amzn-1 | 3.5.5-amzn-0 | 3.5.4-amzn-0 | 3.5.3-amzn-1 | 3.5.3-amzn-0 | 3.5.2-amzn-1 | 3.5.2-amzn-0 | 3.5.1-amzn-1 | 3.5.1-amzn-0 | 3.5.0-amzn-1 | 3.5.0-amzn-0 | | Sqoop | - | - | - | - | - | - | - | - | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | | TensorFlow | 2.19.0 | 2.19.0 | 2.18.0 | 2.16.1 | 2.16.1 | 2.16.1 | 2.16.1 | 2.16.1 | 2.16.1 | 2.16.1 | 2.11.0 | 2.11.0 | 2.11.0 | | Tez | 0.10.2-amzn-19 | 0.10.2-amzn-18 | 0.10.2-amzn-17 | 0.10.2-amzn-16 | 0.10.2-amzn-15 | 0.10.2-amzn-14 | 0.10.2-amzn-13 | 0.10.2-amzn-12 | 0.10.2-amzn-11 | 0.10.2-amzn-10 | 0.10.2-amzn-9 | 0.10.2-amzn-8 | 0.10.2-amzn-7 | | Trino (PrestoSQL) | 476-amzn-1 | 475-amzn-0 | 470-amzn-0 | 467-amzn-2 | 467-amzn-1 | 467-amzn-0 | 457-amzn-0 | 446-amzn-0 | 446-amzn-0 | 442-amzn-0 | 436-amzn-0 | 435-amzn-0 | 426-amzn-1 | | Zeppelin | 0.11.1 | 0.11.1 | 0.11.1 | 0.11.1 | 0.11.1 | 0.11.1 | 0.11.1 | 0.11.1 | 0.11.1 | 0.11.1 | 0.10.1 | 0.10.1 | 0.10.1 | | ZooKeeper | 3.9.3-amzn-4 | 3.9.3-amzn-3 | 3.9.3-amzn-2 | 3.9.3-amzn-1 | 3.9.3-amzn-0 | 3.9.2-amzn-1 | 3.9.2-amzn-0 | 3.9.2-amzn-0 | 3.9.2-amzn-0 | 3.9.1-amzn-1 | 3.9.1-amzn-1 | 3.9.1-amzn-0 | 3.5.10 | # Amazon EMR release 7.12.0 ## 7.12.0 supported lifecycle The following table describes the supported lifecycle dates for Amazon EMR release 7.12.0. | Support phase | Date | | --- | --- | | Initial release date | November 21, 2025 | | Standard support until | November 21, 2027 | | End of support | November 22, 2027 | | End of life | November 21, 2028 | ## 7.12.0 application versions This release includes the following applications: [https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/Install-CloudWatch-Agent.html](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/Install-CloudWatch-Agent.html), [https://delta.io/](https://delta.io/), [https://flink.apache.org/](https://flink.apache.org/), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [https://hudi.apache.org](https://hudi.apache.org), [http://gethue.com/](http://gethue.com/), [https://iceberg.apache.org/](https://iceberg.apache.org/), [https://jupyter-enterprise-gateway.readthedocs.io/en/latest/](https://jupyter-enterprise-gateway.readthedocs.io/en/latest/), [https://jupyterhub.readthedocs.io/en/latest/#](https://jupyterhub.readthedocs.io/en/latest/#), [https://livy.incubator.apache.org/](https://livy.incubator.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [https://www.tensorflow.org/](https://www.tensorflow.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://trino.io/](https://trino.io/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-7.12.0 | emr-7.11.0 | emr-7.10.0 | emr-7.9.0 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 2.35.5, 1.12.792 | 2.35.5, 1.12.792 | 2.31.48, 1.12.782 | 2.31.16, 1.12.782 | | Python | 3.9, 3.11 | 3.9, 3.11 | 3.9, 3.11 | 3.9, 3.11 | | Scala | 2.12.18 | 2.12.18 | 2.12.18 | 2.12.18 | | AmazonCloudWatchAgent | 1.300032.2-amzn-0 | 1.300032.2-amzn-0 | 1.300032.2-amzn-0 | 1.300032.2-amzn-0 | | Delta | 3.3.2-amzn-1 | 3.3.2-amzn-0 | 3.3.0-amzn-2 | 3.3.0-amzn-1 | | Flink | 1.20.0-amzn-6 | 1.20.0-amzn-5 | 1.20.0-amzn-4 | 1.20.0-amzn-3 | | Ganglia | - | - | - | - | | HBase | 2.6.2-amzn-3 | 2.6.2-amzn-2 | 2.6.2-amzn-1 | 2.6.2-amzn-0 | | HCatalog | 3.1.3-amzn-21 | 3.1.3-amzn-20 | 3.1.3-amzn-19 | 3.1.3-amzn-18 | | Hadoop | 3.4.1-amzn-4 | 3.4.1-amzn-3 | 3.4.1-amzn-2 | 3.4.1-amzn-1 | | Hive | 3.1.3-amzn-21 | 3.1.3-amzn-20 | 3.1.3-amzn-19 | 3.1.3-amzn-18 | | Hudi | 1.0.2-amzn-1 | 1.0.2-amzn-0 | 0.15.0-amzn-7 | 0.15.0-amzn-6 | | Hue | 4.11.0 | 4.11.0 | 4.11.0 | 4.11.0 | | Iceberg | 1.10.0-amzn-0 | 1.9.1-amzn-0 | 1.8.1-amzn-0 | 1.7.1-amzn-2 | | JupyterEnterpriseGateway | 2.6.0 | 2.6.0 | 2.6.0 | 2.6.0 | | JupyterHub | 1.5.0 | 1.5.0 | 1.5.0 | 1.5.0 | | Livy | 0.8.0-incubating | 0.8.0-incubating | 0.8.0-incubating | 0.8.0-incubating | | MXNet | - | - | - | - | | Mahout | - | - | - | - | | Oozie | 5.2.1 | 5.2.1 | 5.2.1 | 5.2.1 | | Phoenix | 5.2.1 | 5.2.1 | 5.2.1 | 5.2.1 | | Pig | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | | Presto | 0.287-amzn-6 | 0.287-amzn-5 | 0.287-amzn-4 | 0.287-amzn-3 | | Spark | 3.5.6-amzn-1 | 3.5.6-amzn-0 | 3.5.5-amzn-1 | 3.5.5-amzn-0 | | Sqoop | - | - | - | - | | TensorFlow | 2.19.0 | 2.19.0 | 2.18.0 | 2.16.1 | | Tez | 0.10.2-amzn-19 | 0.10.2-amzn-18 | 0.10.2-amzn-17 | 0.10.2-amzn-16 | | Trino (PrestoSQL) | 476-amzn-1 | 475-amzn-0 | 470-amzn-0 | 467-amzn-2 | | Zeppelin | 0.11.1 | 0.11.1 | 0.11.1 | 0.11.1 | | ZooKeeper | 3.9.3-amzn-4 | 3.9.3-amzn-3 | 3.9.3-amzn-2 | 3.9.3-amzn-1 | # 7.12.0 common vulnerabilities and exposures The following table lists all CVEs that don't impact EMR clusters that run on recommended configurations of Amazon EMR 7.12.0. Amazon EMR is dependent on upstream open-source for availability of fixes and will provide the latest stable version as part of the Core Engine components within 90 days of Amazon EMR verifying the fixes. This table was updated on November 13, 2025. | CVE ID | Severity | CVE details URL | | --- | --- | --- | | CVE-2015-1832 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2015-1832](https://nvd.nist.gov/vuln/detail/CVE-2015-1832) | | CVE-2016-5018 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2016-5018](https://nvd.nist.gov/vuln/detail/CVE-2016-5018) | | CVE-2017-15095 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2017-15095](https://nvd.nist.gov/vuln/detail/CVE-2017-15095) | | CVE-2017-17485 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2017-17485](https://nvd.nist.gov/vuln/detail/CVE-2017-17485) | | CVE-2017-7525 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2017-7525](https://nvd.nist.gov/vuln/detail/CVE-2017-7525) | | CVE-2018-11307 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-11307](https://nvd.nist.gov/vuln/detail/CVE-2018-11307) | | CVE-2018-11307 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-11307](https://nvd.nist.gov/vuln/detail/CVE-2018-11307) | | CVE-2018-14718 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-14718](https://nvd.nist.gov/vuln/detail/CVE-2018-14718) | | CVE-2018-14718 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-14718](https://nvd.nist.gov/vuln/detail/CVE-2018-14718) | | CVE-2018-14719 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-14719](https://nvd.nist.gov/vuln/detail/CVE-2018-14719) | | CVE-2018-14719 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-14719](https://nvd.nist.gov/vuln/detail/CVE-2018-14719) | | CVE-2018-14720 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-14720](https://nvd.nist.gov/vuln/detail/CVE-2018-14720) | | CVE-2018-14720 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-14720](https://nvd.nist.gov/vuln/detail/CVE-2018-14720) | | CVE-2018-14721 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-14721](https://nvd.nist.gov/vuln/detail/CVE-2018-14721) | | CVE-2018-14721 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-14721](https://nvd.nist.gov/vuln/detail/CVE-2018-14721) | | CVE-2018-19360 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-19360](https://nvd.nist.gov/vuln/detail/CVE-2018-19360) | | CVE-2018-19360 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-19360](https://nvd.nist.gov/vuln/detail/CVE-2018-19360) | | CVE-2018-19361 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-19361](https://nvd.nist.gov/vuln/detail/CVE-2018-19361) | | CVE-2018-19361 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-19361](https://nvd.nist.gov/vuln/detail/CVE-2018-19361) | | CVE-2018-19362 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-19362](https://nvd.nist.gov/vuln/detail/CVE-2018-19362) | | CVE-2018-19362 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-19362](https://nvd.nist.gov/vuln/detail/CVE-2018-19362) | | CVE-2018-7489 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-7489](https://nvd.nist.gov/vuln/detail/CVE-2018-7489) | | CVE-2018-7489 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-7489](https://nvd.nist.gov/vuln/detail/CVE-2018-7489) | | CVE-2019-10202 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-10202](https://nvd.nist.gov/vuln/detail/CVE-2019-10202) | | CVE-2019-14379 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-14379](https://nvd.nist.gov/vuln/detail/CVE-2019-14379) | | CVE-2019-14379 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-14379](https://nvd.nist.gov/vuln/detail/CVE-2019-14379) | | CVE-2019-14540 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-14540](https://nvd.nist.gov/vuln/detail/CVE-2019-14540) | | CVE-2019-14540 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-14540](https://nvd.nist.gov/vuln/detail/CVE-2019-14540) | | CVE-2019-14892 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-14892](https://nvd.nist.gov/vuln/detail/CVE-2019-14892) | | CVE-2019-14892 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-14892](https://nvd.nist.gov/vuln/detail/CVE-2019-14892) | | CVE-2019-14893 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-14893](https://nvd.nist.gov/vuln/detail/CVE-2019-14893) | | CVE-2019-14893 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-14893](https://nvd.nist.gov/vuln/detail/CVE-2019-14893) | | CVE-2019-16335 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-16335](https://nvd.nist.gov/vuln/detail/CVE-2019-16335) | | CVE-2019-16335 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-16335](https://nvd.nist.gov/vuln/detail/CVE-2019-16335) | | CVE-2019-16942 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-16942](https://nvd.nist.gov/vuln/detail/CVE-2019-16942) | | CVE-2019-16942 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-16942](https://nvd.nist.gov/vuln/detail/CVE-2019-16942) | | CVE-2019-16943 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-16943](https://nvd.nist.gov/vuln/detail/CVE-2019-16943) | | CVE-2019-16943 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-16943](https://nvd.nist.gov/vuln/detail/CVE-2019-16943) | | CVE-2019-17267 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-17267](https://nvd.nist.gov/vuln/detail/CVE-2019-17267) | | CVE-2019-17267 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-17267](https://nvd.nist.gov/vuln/detail/CVE-2019-17267) | | CVE-2019-17531 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-17531](https://nvd.nist.gov/vuln/detail/CVE-2019-17531) | | CVE-2019-17531 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-17531](https://nvd.nist.gov/vuln/detail/CVE-2019-17531) | | CVE-2019-20330 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-20330](https://nvd.nist.gov/vuln/detail/CVE-2019-20330) | | CVE-2019-20330 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-20330](https://nvd.nist.gov/vuln/detail/CVE-2019-20330) | | CVE-2020-8840 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2020-8840](https://nvd.nist.gov/vuln/detail/CVE-2020-8840) | | CVE-2020-8840 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2020-8840](https://nvd.nist.gov/vuln/detail/CVE-2020-8840) | | CVE-2020-9546 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2020-9546](https://nvd.nist.gov/vuln/detail/CVE-2020-9546) | | CVE-2020-9546 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2020-9546](https://nvd.nist.gov/vuln/detail/CVE-2020-9546) | | CVE-2020-9547 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2020-9547](https://nvd.nist.gov/vuln/detail/CVE-2020-9547) | | CVE-2020-9547 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2020-9547](https://nvd.nist.gov/vuln/detail/CVE-2020-9547) | | CVE-2020-9548 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2020-9548](https://nvd.nist.gov/vuln/detail/CVE-2020-9548) | | CVE-2020-9548 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2020-9548](https://nvd.nist.gov/vuln/detail/CVE-2020-9548) | | CVE-2022-1471 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2022-1471](https://nvd.nist.gov/vuln/detail/CVE-2022-1471) | | CVE-2022-1471 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2022-1471](https://nvd.nist.gov/vuln/detail/CVE-2022-1471) | | CVE-2022-37865 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2022-37865](https://nvd.nist.gov/vuln/detail/CVE-2022-37865) | | CVE-2022-39135 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2022-39135](https://nvd.nist.gov/vuln/detail/CVE-2022-39135) | | CVE-2022-46337 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2022-46337](https://nvd.nist.gov/vuln/detail/CVE-2022-46337) | | CVE-2022-46337 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2022-46337](https://nvd.nist.gov/vuln/detail/CVE-2022-46337) | | CVE-2024-51504 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2024-51504](https://nvd.nist.gov/vuln/detail/CVE-2024-51504) | | CVE-2024-51504 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2024-51504](https://nvd.nist.gov/vuln/detail/CVE-2024-51504) | | CVE-2024-52046 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2024-52046](https://nvd.nist.gov/vuln/detail/CVE-2024-52046) | | CVE-2024-52046 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2024-52046](https://nvd.nist.gov/vuln/detail/CVE-2024-52046) | | CVE-2024-53990 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2024-53990](https://nvd.nist.gov/vuln/detail/CVE-2024-53990) | | CVE-2025-30065 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2025-30065](https://nvd.nist.gov/vuln/detail/CVE-2025-30065) | | CVE-2025-30065 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2025-30065](https://nvd.nist.gov/vuln/detail/CVE-2025-30065) | | CVE-2014-3643 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2014-3643](https://nvd.nist.gov/vuln/detail/CVE-2014-3643) | | CVE-2015-5237 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2015-5237](https://nvd.nist.gov/vuln/detail/CVE-2015-5237) | | CVE-2015-5237 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2015-5237](https://nvd.nist.gov/vuln/detail/CVE-2015-5237) | | CVE-2016-6796 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2016-6796](https://nvd.nist.gov/vuln/detail/CVE-2016-6796) | | CVE-2018-1000180 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2018-1000180](https://nvd.nist.gov/vuln/detail/CVE-2018-1000180) | | CVE-2018-1000180 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2018-1000180](https://nvd.nist.gov/vuln/detail/CVE-2018-1000180) | | CVE-2018-12022 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2018-12022](https://nvd.nist.gov/vuln/detail/CVE-2018-12022) | | CVE-2018-12022 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2018-12022](https://nvd.nist.gov/vuln/detail/CVE-2018-12022) | | CVE-2018-12023 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2018-12023](https://nvd.nist.gov/vuln/detail/CVE-2018-12023) | | CVE-2018-12023 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2018-12023](https://nvd.nist.gov/vuln/detail/CVE-2018-12023) | | CVE-2018-5968 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2018-5968](https://nvd.nist.gov/vuln/detail/CVE-2018-5968) | | CVE-2019-0205 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2019-0205](https://nvd.nist.gov/vuln/detail/CVE-2019-0205) | | CVE-2019-0205 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2019-0205](https://nvd.nist.gov/vuln/detail/CVE-2019-0205) | | CVE-2019-10172 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2019-10172](https://nvd.nist.gov/vuln/detail/CVE-2019-10172) | | CVE-2019-12086 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2019-12086](https://nvd.nist.gov/vuln/detail/CVE-2019-12086) | | CVE-2019-12086 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2019-12086](https://nvd.nist.gov/vuln/detail/CVE-2019-12086) | | CVE-2019-14439 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2019-14439](https://nvd.nist.gov/vuln/detail/CVE-2019-14439) | | CVE-2019-14439 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2019-14439](https://nvd.nist.gov/vuln/detail/CVE-2019-14439) | | CVE-2020-10650 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-10650](https://nvd.nist.gov/vuln/detail/CVE-2020-10650) | | CVE-2020-10650 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-10650](https://nvd.nist.gov/vuln/detail/CVE-2020-10650) | | CVE-2020-10672 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-10672](https://nvd.nist.gov/vuln/detail/CVE-2020-10672) | | CVE-2020-10672 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-10672](https://nvd.nist.gov/vuln/detail/CVE-2020-10672) | | CVE-2020-10673 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-10673](https://nvd.nist.gov/vuln/detail/CVE-2020-10673) | | CVE-2020-10673 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-10673](https://nvd.nist.gov/vuln/detail/CVE-2020-10673) | | CVE-2020-10968 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-10968](https://nvd.nist.gov/vuln/detail/CVE-2020-10968) | | CVE-2020-10968 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-10968](https://nvd.nist.gov/vuln/detail/CVE-2020-10968) | | CVE-2020-10969 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-10969](https://nvd.nist.gov/vuln/detail/CVE-2020-10969) | | CVE-2020-10969 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-10969](https://nvd.nist.gov/vuln/detail/CVE-2020-10969) | | CVE-2020-11111 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-11111](https://nvd.nist.gov/vuln/detail/CVE-2020-11111) | | CVE-2020-11111 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-11111](https://nvd.nist.gov/vuln/detail/CVE-2020-11111) | | CVE-2020-11112 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-11112](https://nvd.nist.gov/vuln/detail/CVE-2020-11112) | | CVE-2020-11112 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-11112](https://nvd.nist.gov/vuln/detail/CVE-2020-11112) | | CVE-2020-11113 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-11113](https://nvd.nist.gov/vuln/detail/CVE-2020-11113) | | CVE-2020-11113 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-11113](https://nvd.nist.gov/vuln/detail/CVE-2020-11113) | | CVE-2020-11619 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-11619](https://nvd.nist.gov/vuln/detail/CVE-2020-11619) | | CVE-2020-11619 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-11619](https://nvd.nist.gov/vuln/detail/CVE-2020-11619) | | CVE-2020-11620 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-11620](https://nvd.nist.gov/vuln/detail/CVE-2020-11620) | | CVE-2020-11620 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-11620](https://nvd.nist.gov/vuln/detail/CVE-2020-11620) | | CVE-2020-13949 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-13949](https://nvd.nist.gov/vuln/detail/CVE-2020-13949) | | CVE-2020-13949 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-13949](https://nvd.nist.gov/vuln/detail/CVE-2020-13949) | | CVE-2020-13949 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-13949](https://nvd.nist.gov/vuln/detail/CVE-2020-13949) | | CVE-2020-14060 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-14060](https://nvd.nist.gov/vuln/detail/CVE-2020-14060) | | CVE-2020-14060 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-14060](https://nvd.nist.gov/vuln/detail/CVE-2020-14060) | | CVE-2020-14061 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-14061](https://nvd.nist.gov/vuln/detail/CVE-2020-14061) | | CVE-2020-14061 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-14061](https://nvd.nist.gov/vuln/detail/CVE-2020-14061) | | CVE-2020-14062 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-14062](https://nvd.nist.gov/vuln/detail/CVE-2020-14062) | | CVE-2020-14062 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-14062](https://nvd.nist.gov/vuln/detail/CVE-2020-14062) | | CVE-2020-14195 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-14195](https://nvd.nist.gov/vuln/detail/CVE-2020-14195) | | CVE-2020-14195 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-14195](https://nvd.nist.gov/vuln/detail/CVE-2020-14195) | | CVE-2020-24616 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-24616](https://nvd.nist.gov/vuln/detail/CVE-2020-24616) | | CVE-2020-24616 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-24616](https://nvd.nist.gov/vuln/detail/CVE-2020-24616) | | CVE-2020-24750 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-24750](https://nvd.nist.gov/vuln/detail/CVE-2020-24750) | | CVE-2020-24750 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-24750](https://nvd.nist.gov/vuln/detail/CVE-2020-24750) | | CVE-2020-25649 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-25649](https://nvd.nist.gov/vuln/detail/CVE-2020-25649) | | CVE-2020-28052 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-28052](https://nvd.nist.gov/vuln/detail/CVE-2020-28052) | | CVE-2020-35490 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-35490](https://nvd.nist.gov/vuln/detail/CVE-2020-35490) | | CVE-2020-35490 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-35490](https://nvd.nist.gov/vuln/detail/CVE-2020-35490) | | CVE-2020-35491 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-35491](https://nvd.nist.gov/vuln/detail/CVE-2020-35491) | | CVE-2020-35491 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-35491](https://nvd.nist.gov/vuln/detail/CVE-2020-35491) | | CVE-2020-35728 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-35728](https://nvd.nist.gov/vuln/detail/CVE-2020-35728) | | CVE-2020-35728 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-35728](https://nvd.nist.gov/vuln/detail/CVE-2020-35728) | | CVE-2020-36179 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36179](https://nvd.nist.gov/vuln/detail/CVE-2020-36179) | | CVE-2020-36179 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36179](https://nvd.nist.gov/vuln/detail/CVE-2020-36179) | | CVE-2020-36180 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36180](https://nvd.nist.gov/vuln/detail/CVE-2020-36180) | | CVE-2020-36180 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36180](https://nvd.nist.gov/vuln/detail/CVE-2020-36180) | | CVE-2020-36181 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36181](https://nvd.nist.gov/vuln/detail/CVE-2020-36181) | | CVE-2020-36181 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36181](https://nvd.nist.gov/vuln/detail/CVE-2020-36181) | | CVE-2020-36182 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36182](https://nvd.nist.gov/vuln/detail/CVE-2020-36182) | | CVE-2020-36182 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36182](https://nvd.nist.gov/vuln/detail/CVE-2020-36182) | | CVE-2020-36183 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36183](https://nvd.nist.gov/vuln/detail/CVE-2020-36183) | | CVE-2020-36183 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36183](https://nvd.nist.gov/vuln/detail/CVE-2020-36183) | | CVE-2020-36184 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36184](https://nvd.nist.gov/vuln/detail/CVE-2020-36184) | | CVE-2020-36184 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36184](https://nvd.nist.gov/vuln/detail/CVE-2020-36184) | | CVE-2020-36185 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36185](https://nvd.nist.gov/vuln/detail/CVE-2020-36185) | | CVE-2020-36185 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36185](https://nvd.nist.gov/vuln/detail/CVE-2020-36185) | | CVE-2020-36186 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36186](https://nvd.nist.gov/vuln/detail/CVE-2020-36186) | | CVE-2020-36186 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36186](https://nvd.nist.gov/vuln/detail/CVE-2020-36186) | | CVE-2020-36187 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36187](https://nvd.nist.gov/vuln/detail/CVE-2020-36187) | | CVE-2020-36187 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36187](https://nvd.nist.gov/vuln/detail/CVE-2020-36187) | | CVE-2020-36188 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36188](https://nvd.nist.gov/vuln/detail/CVE-2020-36188) | | CVE-2020-36188 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36188](https://nvd.nist.gov/vuln/detail/CVE-2020-36188) | | CVE-2020-36189 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36189](https://nvd.nist.gov/vuln/detail/CVE-2020-36189) | | CVE-2020-36189 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36189](https://nvd.nist.gov/vuln/detail/CVE-2020-36189) | | CVE-2020-36518 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36518](https://nvd.nist.gov/vuln/detail/CVE-2020-36518) | | CVE-2020-36518 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36518](https://nvd.nist.gov/vuln/detail/CVE-2020-36518) | | CVE-2021-0341 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-0341](https://nvd.nist.gov/vuln/detail/CVE-2021-0341) | | CVE-2021-20190 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-20190](https://nvd.nist.gov/vuln/detail/CVE-2021-20190) | | CVE-2021-20190 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-20190](https://nvd.nist.gov/vuln/detail/CVE-2021-20190) | | CVE-2021-31684 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-31684](https://nvd.nist.gov/vuln/detail/CVE-2021-31684) | | CVE-2021-34538 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-34538](https://nvd.nist.gov/vuln/detail/CVE-2021-34538) | | CVE-2021-34538 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-34538](https://nvd.nist.gov/vuln/detail/CVE-2021-34538) | | CVE-2021-34538 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-34538](https://nvd.nist.gov/vuln/detail/CVE-2021-34538) | | CVE-2022-25647 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-25647](https://nvd.nist.gov/vuln/detail/CVE-2022-25647) | | CVE-2022-25647 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-25647](https://nvd.nist.gov/vuln/detail/CVE-2022-25647) | | CVE-2022-25857 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-25857](https://nvd.nist.gov/vuln/detail/CVE-2022-25857) | | CVE-2022-3171 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-3171](https://nvd.nist.gov/vuln/detail/CVE-2022-3171) | | CVE-2022-3171 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-3171](https://nvd.nist.gov/vuln/detail/CVE-2022-3171) | | CVE-2022-3509 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-3509](https://nvd.nist.gov/vuln/detail/CVE-2022-3509) | | CVE-2022-3509 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-3509](https://nvd.nist.gov/vuln/detail/CVE-2022-3509) | | CVE-2022-3510 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-3510](https://nvd.nist.gov/vuln/detail/CVE-2022-3510) | | CVE-2022-36364 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-36364](https://nvd.nist.gov/vuln/detail/CVE-2022-36364) | | CVE-2022-37866 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-37866](https://nvd.nist.gov/vuln/detail/CVE-2022-37866) | | CVE-2022-40149 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-40149](https://nvd.nist.gov/vuln/detail/CVE-2022-40149) | | CVE-2022-40149 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-40149](https://nvd.nist.gov/vuln/detail/CVE-2022-40149) | | CVE-2022-40150 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-40150](https://nvd.nist.gov/vuln/detail/CVE-2022-40150) | | CVE-2022-40150 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-40150](https://nvd.nist.gov/vuln/detail/CVE-2022-40150) | | CVE-2022-41404 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-41404](https://nvd.nist.gov/vuln/detail/CVE-2022-41404) | | CVE-2022-42003 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-42003](https://nvd.nist.gov/vuln/detail/CVE-2022-42003) | | CVE-2022-42003 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-42003](https://nvd.nist.gov/vuln/detail/CVE-2022-42003) | | CVE-2022-42003 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-42003](https://nvd.nist.gov/vuln/detail/CVE-2022-42003) | | CVE-2022-42003 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-42003](https://nvd.nist.gov/vuln/detail/CVE-2022-42003) | | CVE-2022-42004 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-42004](https://nvd.nist.gov/vuln/detail/CVE-2022-42004) | | CVE-2022-42004 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-42004](https://nvd.nist.gov/vuln/detail/CVE-2022-42004) | | CVE-2022-42004 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-42004](https://nvd.nist.gov/vuln/detail/CVE-2022-42004) | | CVE-2022-45685 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-45685](https://nvd.nist.gov/vuln/detail/CVE-2022-45685) | | CVE-2022-45685 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-45685](https://nvd.nist.gov/vuln/detail/CVE-2022-45685) | | CVE-2022-45693 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-45693](https://nvd.nist.gov/vuln/detail/CVE-2022-45693) | | CVE-2022-45693 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-45693](https://nvd.nist.gov/vuln/detail/CVE-2022-45693) | | CVE-2022-46751 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-46751](https://nvd.nist.gov/vuln/detail/CVE-2022-46751) | | CVE-2022-46751 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-46751](https://nvd.nist.gov/vuln/detail/CVE-2022-46751) | | CVE-2023-1370 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-1370](https://nvd.nist.gov/vuln/detail/CVE-2023-1370) | | CVE-2023-1436 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-1436](https://nvd.nist.gov/vuln/detail/CVE-2023-1436) | | CVE-2023-1436 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-1436](https://nvd.nist.gov/vuln/detail/CVE-2023-1436) | | CVE-2023-2976 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-2976](https://nvd.nist.gov/vuln/detail/CVE-2023-2976) | | CVE-2023-2976 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-2976](https://nvd.nist.gov/vuln/detail/CVE-2023-2976) | | CVE-2023-2976 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-2976](https://nvd.nist.gov/vuln/detail/CVE-2023-2976) | | CVE-2023-2976 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-2976](https://nvd.nist.gov/vuln/detail/CVE-2023-2976) | | CVE-2023-2976 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-2976](https://nvd.nist.gov/vuln/detail/CVE-2023-2976) | | CVE-2023-2976 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-2976](https://nvd.nist.gov/vuln/detail/CVE-2023-2976) | | CVE-2023-2976 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-2976](https://nvd.nist.gov/vuln/detail/CVE-2023-2976) | | CVE-2023-2976 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-2976](https://nvd.nist.gov/vuln/detail/CVE-2023-2976) | | CVE-2023-2976 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-2976](https://nvd.nist.gov/vuln/detail/CVE-2023-2976) | | CVE-2023-6378 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-6378](https://nvd.nist.gov/vuln/detail/CVE-2023-6378) | | CVE-2023-6378 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-6378](https://nvd.nist.gov/vuln/detail/CVE-2023-6378) | | CVE-2023-6481 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-6481](https://nvd.nist.gov/vuln/detail/CVE-2023-6481) | | CVE-2023-6481 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-6481](https://nvd.nist.gov/vuln/detail/CVE-2023-6481) | | CVE-2024-13009 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-13009](https://nvd.nist.gov/vuln/detail/CVE-2024-13009) | | CVE-2024-13009 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-13009](https://nvd.nist.gov/vuln/detail/CVE-2024-13009) | | CVE-2024-29857 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-29857](https://nvd.nist.gov/vuln/detail/CVE-2024-29857) | | CVE-2024-29857 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-29857](https://nvd.nist.gov/vuln/detail/CVE-2024-29857) | | CVE-2024-29857 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-29857](https://nvd.nist.gov/vuln/detail/CVE-2024-29857) | | CVE-2024-29857 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-29857](https://nvd.nist.gov/vuln/detail/CVE-2024-29857) | | CVE-2024-30172 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-30172](https://nvd.nist.gov/vuln/detail/CVE-2024-30172) | | CVE-2024-30172 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-30172](https://nvd.nist.gov/vuln/detail/CVE-2024-30172) | | CVE-2024-34447 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-34447](https://nvd.nist.gov/vuln/detail/CVE-2024-34447) | | CVE-2024-34447 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-34447](https://nvd.nist.gov/vuln/detail/CVE-2024-34447) | | CVE-2024-36114 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-36114](https://nvd.nist.gov/vuln/detail/CVE-2024-36114) | | CVE-2024-47561 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-47561](https://nvd.nist.gov/vuln/detail/CVE-2024-47561) | | CVE-2024-7254 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-7254](https://nvd.nist.gov/vuln/detail/CVE-2024-7254) | | CVE-2024-7254 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-7254](https://nvd.nist.gov/vuln/detail/CVE-2024-7254) | | CVE-2024-7254 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-7254](https://nvd.nist.gov/vuln/detail/CVE-2024-7254) | | CVE-2024-7254 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-7254](https://nvd.nist.gov/vuln/detail/CVE-2024-7254) | | CVE-2024-7254 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-7254](https://nvd.nist.gov/vuln/detail/CVE-2024-7254) | | CVE-2024-7254 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-7254](https://nvd.nist.gov/vuln/detail/CVE-2024-7254) | | CVE-2024-7254 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-7254](https://nvd.nist.gov/vuln/detail/CVE-2024-7254) | | CVE-2024-9823 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-9823](https://nvd.nist.gov/vuln/detail/CVE-2024-9823) | | CVE-2025-24970 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2025-24970](https://nvd.nist.gov/vuln/detail/CVE-2025-24970) | | CVE-2025-24970 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2025-24970](https://nvd.nist.gov/vuln/detail/CVE-2025-24970) | | CVE-2025-27553 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2025-27553](https://nvd.nist.gov/vuln/detail/CVE-2025-27553) | | CVE-2025-27817 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2025-27817](https://nvd.nist.gov/vuln/detail/CVE-2025-27817) | | CVE-2025-46762 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2025-46762](https://nvd.nist.gov/vuln/detail/CVE-2025-46762) | | CVE-2025-46762 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2025-46762](https://nvd.nist.gov/vuln/detail/CVE-2025-46762) | | CVE-2025-46762 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2025-46762](https://nvd.nist.gov/vuln/detail/CVE-2025-46762) | | CVE-2025-48734 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2025-48734](https://nvd.nist.gov/vuln/detail/CVE-2025-48734) | | CVE-2025-52434 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2025-52434](https://nvd.nist.gov/vuln/detail/CVE-2025-52434) | | CVE-2025-52999 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2025-52999](https://nvd.nist.gov/vuln/detail/CVE-2025-52999) | | CVE-2025-52999 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2025-52999](https://nvd.nist.gov/vuln/detail/CVE-2025-52999) | | CVE-2025-52999 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2025-52999](https://nvd.nist.gov/vuln/detail/CVE-2025-52999) | | CVE-2025-52999 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2025-52999](https://nvd.nist.gov/vuln/detail/CVE-2025-52999) | | CVE-2025-52999 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2025-52999](https://nvd.nist.gov/vuln/detail/CVE-2025-52999) | | CVE-2025-52999 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2025-52999](https://nvd.nist.gov/vuln/detail/CVE-2025-52999) | | CVE-2025-55163 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2025-55163](https://nvd.nist.gov/vuln/detail/CVE-2025-55163) | | CVE-2025-58056 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2025-58056](https://nvd.nist.gov/vuln/detail/CVE-2025-58056) | | CVE-2025-58057 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2025-58057](https://nvd.nist.gov/vuln/detail/CVE-2025-58057) | | CVE-2025-58057 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2025-58057](https://nvd.nist.gov/vuln/detail/CVE-2025-58057) | | CVE-2025-58057 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2025-58057](https://nvd.nist.gov/vuln/detail/CVE-2025-58057) | | CVE-2025-59250 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2025-59250](https://nvd.nist.gov/vuln/detail/CVE-2025-59250) | | CVE-2012-5783 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2012-5783](https://nvd.nist.gov/vuln/detail/CVE-2012-5783) | | CVE-2012-5783 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2012-5783](https://nvd.nist.gov/vuln/detail/CVE-2012-5783) | | CVE-2012-6153 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2012-6153](https://nvd.nist.gov/vuln/detail/CVE-2012-6153) | | CVE-2012-6153 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2012-6153](https://nvd.nist.gov/vuln/detail/CVE-2012-6153) | | CVE-2018-10237 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2018-10237](https://nvd.nist.gov/vuln/detail/CVE-2018-10237) | | CVE-2018-10237 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2018-10237](https://nvd.nist.gov/vuln/detail/CVE-2018-10237) | | CVE-2018-10237 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2018-10237](https://nvd.nist.gov/vuln/detail/CVE-2018-10237) | | CVE-2018-10237 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2018-10237](https://nvd.nist.gov/vuln/detail/CVE-2018-10237) | | CVE-2018-10237 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2018-10237](https://nvd.nist.gov/vuln/detail/CVE-2018-10237) | | CVE-2018-11798 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2018-11798](https://nvd.nist.gov/vuln/detail/CVE-2018-11798) | | CVE-2018-1313 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2018-1313](https://nvd.nist.gov/vuln/detail/CVE-2018-1313) | | CVE-2019-12384 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2019-12384](https://nvd.nist.gov/vuln/detail/CVE-2019-12384) | | CVE-2019-12384 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2019-12384](https://nvd.nist.gov/vuln/detail/CVE-2019-12384) | | CVE-2019-12814 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2019-12814](https://nvd.nist.gov/vuln/detail/CVE-2019-12814) | | CVE-2019-12814 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2019-12814](https://nvd.nist.gov/vuln/detail/CVE-2019-12814) | | CVE-2020-13955 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2020-13955](https://nvd.nist.gov/vuln/detail/CVE-2020-13955) | | CVE-2020-13956 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2020-13956](https://nvd.nist.gov/vuln/detail/CVE-2020-13956) | | CVE-2020-15250 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2020-15250](https://nvd.nist.gov/vuln/detail/CVE-2020-15250) | | CVE-2020-15250 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2020-15250](https://nvd.nist.gov/vuln/detail/CVE-2020-15250) | | CVE-2020-15522 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2020-15522](https://nvd.nist.gov/vuln/detail/CVE-2020-15522) | | CVE-2020-15522 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2020-15522](https://nvd.nist.gov/vuln/detail/CVE-2020-15522) | | CVE-2020-15522 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2020-15522](https://nvd.nist.gov/vuln/detail/CVE-2020-15522) | | CVE-2020-17521 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2020-17521](https://nvd.nist.gov/vuln/detail/CVE-2020-17521) | | CVE-2020-17521 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2020-17521](https://nvd.nist.gov/vuln/detail/CVE-2020-17521) | | CVE-2020-1945 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2020-1945](https://nvd.nist.gov/vuln/detail/CVE-2020-1945) | | CVE-2020-26939 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2020-26939](https://nvd.nist.gov/vuln/detail/CVE-2020-26939) | | CVE-2020-26939 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2020-26939](https://nvd.nist.gov/vuln/detail/CVE-2020-26939) | | CVE-2020-26939 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2020-26939](https://nvd.nist.gov/vuln/detail/CVE-2020-26939) | | CVE-2021-22569 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2021-22569](https://nvd.nist.gov/vuln/detail/CVE-2021-22569) | | CVE-2021-22569 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2021-22569](https://nvd.nist.gov/vuln/detail/CVE-2021-22569) | | CVE-2021-27568 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2021-27568](https://nvd.nist.gov/vuln/detail/CVE-2021-27568) | | CVE-2021-28170 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2021-28170](https://nvd.nist.gov/vuln/detail/CVE-2021-28170) | | CVE-2021-29425 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2021-29425](https://nvd.nist.gov/vuln/detail/CVE-2021-29425) | | CVE-2021-36373 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2021-36373](https://nvd.nist.gov/vuln/detail/CVE-2021-36373) | | CVE-2021-36374 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2021-36374](https://nvd.nist.gov/vuln/detail/CVE-2021-36374) | | CVE-2022-38749 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2022-38749](https://nvd.nist.gov/vuln/detail/CVE-2022-38749) | | CVE-2022-38750 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2022-38750](https://nvd.nist.gov/vuln/detail/CVE-2022-38750) | | CVE-2022-38751 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2022-38751](https://nvd.nist.gov/vuln/detail/CVE-2022-38751) | | CVE-2022-38752 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2022-38752](https://nvd.nist.gov/vuln/detail/CVE-2022-38752) | | CVE-2022-41854 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2022-41854](https://nvd.nist.gov/vuln/detail/CVE-2022-41854) | | CVE-2023-0833 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2023-0833](https://nvd.nist.gov/vuln/detail/CVE-2023-0833) | | CVE-2023-33201 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2023-33201](https://nvd.nist.gov/vuln/detail/CVE-2023-33201) | | CVE-2023-33201 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2023-33201](https://nvd.nist.gov/vuln/detail/CVE-2023-33201) | | CVE-2023-33201 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2023-33201](https://nvd.nist.gov/vuln/detail/CVE-2023-33201) | | CVE-2023-33201 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2023-33201](https://nvd.nist.gov/vuln/detail/CVE-2023-33201) | | CVE-2023-33202 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2023-33202](https://nvd.nist.gov/vuln/detail/CVE-2023-33202) | | CVE-2023-33202 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2023-33202](https://nvd.nist.gov/vuln/detail/CVE-2023-33202) | | CVE-2023-33202 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2023-33202](https://nvd.nist.gov/vuln/detail/CVE-2023-33202) | | CVE-2023-33202 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2023-33202](https://nvd.nist.gov/vuln/detail/CVE-2023-33202) | | CVE-2023-42503 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2023-42503](https://nvd.nist.gov/vuln/detail/CVE-2023-42503) | | CVE-2023-51074 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2023-51074](https://nvd.nist.gov/vuln/detail/CVE-2023-51074) | | CVE-2024-12798 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-12798](https://nvd.nist.gov/vuln/detail/CVE-2024-12798) | | CVE-2024-12798 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-12798](https://nvd.nist.gov/vuln/detail/CVE-2024-12798) | | CVE-2024-12798 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-12798](https://nvd.nist.gov/vuln/detail/CVE-2024-12798) | | CVE-2024-12798 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-12798](https://nvd.nist.gov/vuln/detail/CVE-2024-12798) | | CVE-2024-12798 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-12798](https://nvd.nist.gov/vuln/detail/CVE-2024-12798) | | CVE-2024-12798 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-12798](https://nvd.nist.gov/vuln/detail/CVE-2024-12798) | | CVE-2024-23944 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-23944](https://nvd.nist.gov/vuln/detail/CVE-2024-23944) | | CVE-2024-23945 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-23945](https://nvd.nist.gov/vuln/detail/CVE-2024-23945) | | CVE-2024-23945 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-23945](https://nvd.nist.gov/vuln/detail/CVE-2024-23945) | | CVE-2024-23953 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-23953](https://nvd.nist.gov/vuln/detail/CVE-2024-23953) | | CVE-2024-23953 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-23953](https://nvd.nist.gov/vuln/detail/CVE-2024-23953) | | CVE-2024-25710 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-25710](https://nvd.nist.gov/vuln/detail/CVE-2024-25710) | | CVE-2024-25710 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-25710](https://nvd.nist.gov/vuln/detail/CVE-2024-25710) | | CVE-2024-26308 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-26308](https://nvd.nist.gov/vuln/detail/CVE-2024-26308) | | CVE-2024-26308 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-26308](https://nvd.nist.gov/vuln/detail/CVE-2024-26308) | | CVE-2024-29025 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-29025](https://nvd.nist.gov/vuln/detail/CVE-2024-29025) | | CVE-2024-29869 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-29869](https://nvd.nist.gov/vuln/detail/CVE-2024-29869) | | CVE-2024-29869 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-29869](https://nvd.nist.gov/vuln/detail/CVE-2024-29869) | | CVE-2024-30171 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-30171](https://nvd.nist.gov/vuln/detail/CVE-2024-30171) | | CVE-2024-30171 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-30171](https://nvd.nist.gov/vuln/detail/CVE-2024-30171) | | CVE-2024-30171 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-30171](https://nvd.nist.gov/vuln/detail/CVE-2024-30171) | | CVE-2024-30171 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-30171](https://nvd.nist.gov/vuln/detail/CVE-2024-30171) | | CVE-2024-31141 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-31141](https://nvd.nist.gov/vuln/detail/CVE-2024-31141) | | CVE-2024-47535 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-47535](https://nvd.nist.gov/vuln/detail/CVE-2024-47535) | | CVE-2024-47535 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-47535](https://nvd.nist.gov/vuln/detail/CVE-2024-47535) | | CVE-2024-47554 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-47554](https://nvd.nist.gov/vuln/detail/CVE-2024-47554) | | CVE-2024-47554 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-47554](https://nvd.nist.gov/vuln/detail/CVE-2024-47554) | | CVE-2024-47554 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-47554](https://nvd.nist.gov/vuln/detail/CVE-2024-47554) | | CVE-2024-56128 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-56128](https://nvd.nist.gov/vuln/detail/CVE-2024-56128) | | CVE-2024-6763 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-6763](https://nvd.nist.gov/vuln/detail/CVE-2024-6763) | | CVE-2024-6763 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-6763](https://nvd.nist.gov/vuln/detail/CVE-2024-6763) | | CVE-2024-6763 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-6763](https://nvd.nist.gov/vuln/detail/CVE-2024-6763) | | CVE-2024-6763 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-6763](https://nvd.nist.gov/vuln/detail/CVE-2024-6763) | | CVE-2024-6763 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-6763](https://nvd.nist.gov/vuln/detail/CVE-2024-6763) | | CVE-2024-8184 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-8184](https://nvd.nist.gov/vuln/detail/CVE-2024-8184) | | CVE-2025-11226 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2025-11226](https://nvd.nist.gov/vuln/detail/CVE-2025-11226) | | CVE-2025-11226 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2025-11226](https://nvd.nist.gov/vuln/detail/CVE-2025-11226) | | CVE-2025-11226 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2025-11226](https://nvd.nist.gov/vuln/detail/CVE-2025-11226) | | CVE-2025-25193 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2025-25193](https://nvd.nist.gov/vuln/detail/CVE-2025-25193) | | CVE-2025-25193 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2025-25193](https://nvd.nist.gov/vuln/detail/CVE-2025-25193) | | CVE-2025-30474 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2025-30474](https://nvd.nist.gov/vuln/detail/CVE-2025-30474) | | CVE-2025-3588 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2025-3588](https://nvd.nist.gov/vuln/detail/CVE-2025-3588) | | CVE-2025-46392 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2025-46392](https://nvd.nist.gov/vuln/detail/CVE-2025-46392) | | CVE-2025-48924 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2025-48924](https://nvd.nist.gov/vuln/detail/CVE-2025-48924) | | CVE-2025-48924 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2025-48924](https://nvd.nist.gov/vuln/detail/CVE-2025-48924) | | CVE-2025-48924 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2025-48924](https://nvd.nist.gov/vuln/detail/CVE-2025-48924) | | CVE-2025-48924 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2025-48924](https://nvd.nist.gov/vuln/detail/CVE-2025-48924) | | CVE-2025-48924 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2025-48924](https://nvd.nist.gov/vuln/detail/CVE-2025-48924) | | CVE-2025-48924 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2025-48924](https://nvd.nist.gov/vuln/detail/CVE-2025-48924) | | CVE-2025-48924 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2025-48924](https://nvd.nist.gov/vuln/detail/CVE-2025-48924) | | CVE-2025-48924 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2025-48924](https://nvd.nist.gov/vuln/detail/CVE-2025-48924) | | CVE-2025-49128 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2025-49128](https://nvd.nist.gov/vuln/detail/CVE-2025-49128) | | CVE-2025-49128 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2025-49128](https://nvd.nist.gov/vuln/detail/CVE-2025-49128) | | CVE-2025-49128 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2025-49128](https://nvd.nist.gov/vuln/detail/CVE-2025-49128) | | CVE-2025-53864 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2025-53864](https://nvd.nist.gov/vuln/detail/CVE-2025-53864) | | CVE-2025-58457 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2025-58457](https://nvd.nist.gov/vuln/detail/CVE-2025-58457) | | CVE-2025-58457 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2025-58457](https://nvd.nist.gov/vuln/detail/CVE-2025-58457) | | CVE-2025-59419 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2025-59419](https://nvd.nist.gov/vuln/detail/CVE-2025-59419) | | CVE-2025-59419 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2025-59419](https://nvd.nist.gov/vuln/detail/CVE-2025-59419) | | CVE-2025-8885 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2025-8885](https://nvd.nist.gov/vuln/detail/CVE-2025-8885) | | CVE-2025-8885 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2025-8885](https://nvd.nist.gov/vuln/detail/CVE-2025-8885) | | CVE-2025-8916 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2025-8916](https://nvd.nist.gov/vuln/detail/CVE-2025-8916) | | CVE-2025-8916 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2025-8916](https://nvd.nist.gov/vuln/detail/CVE-2025-8916) | | CVE-2025-8916 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2025-8916](https://nvd.nist.gov/vuln/detail/CVE-2025-8916) | | CVE-2025-8916 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2025-8916](https://nvd.nist.gov/vuln/detail/CVE-2025-8916) | | CVE-2020-8908 | LOW | [https://nvd.nist.gov/vuln/detail/CVE-2020-8908](https://nvd.nist.gov/vuln/detail/CVE-2020-8908) | | CVE-2020-8908 | LOW | [https://nvd.nist.gov/vuln/detail/CVE-2020-8908](https://nvd.nist.gov/vuln/detail/CVE-2020-8908) | | CVE-2020-8908 | LOW | [https://nvd.nist.gov/vuln/detail/CVE-2020-8908](https://nvd.nist.gov/vuln/detail/CVE-2020-8908) | | CVE-2020-8908 | LOW | [https://nvd.nist.gov/vuln/detail/CVE-2020-8908](https://nvd.nist.gov/vuln/detail/CVE-2020-8908) | | CVE-2020-8908 | LOW | [https://nvd.nist.gov/vuln/detail/CVE-2020-8908](https://nvd.nist.gov/vuln/detail/CVE-2020-8908) | | CVE-2020-8908 | LOW | [https://nvd.nist.gov/vuln/detail/CVE-2020-8908](https://nvd.nist.gov/vuln/detail/CVE-2020-8908) | | CVE-2020-8908 | LOW | [https://nvd.nist.gov/vuln/detail/CVE-2020-8908](https://nvd.nist.gov/vuln/detail/CVE-2020-8908) | | CVE-2020-8908 | LOW | [https://nvd.nist.gov/vuln/detail/CVE-2020-8908](https://nvd.nist.gov/vuln/detail/CVE-2020-8908) | | CVE-2020-8908 | LOW | [https://nvd.nist.gov/vuln/detail/CVE-2020-8908](https://nvd.nist.gov/vuln/detail/CVE-2020-8908) | | CVE-2024-12801 | LOW | [https://nvd.nist.gov/vuln/detail/CVE-2024-12801](https://nvd.nist.gov/vuln/detail/CVE-2024-12801) | | CVE-2024-12801 | LOW | [https://nvd.nist.gov/vuln/detail/CVE-2024-12801](https://nvd.nist.gov/vuln/detail/CVE-2024-12801) | | CVE-2024-12801 | LOW | [https://nvd.nist.gov/vuln/detail/CVE-2024-12801](https://nvd.nist.gov/vuln/detail/CVE-2024-12801) | | CVE-2025-46551 | LOW | [https://nvd.nist.gov/vuln/detail/CVE-2025-46551](https://nvd.nist.gov/vuln/detail/CVE-2025-46551) | ## 7.12.0 release notes The following release notes include information for Amazon EMR release 7.12.0. + **New features** + **Serverless storage for EMR Serverless** – Amazon EMR serverless introduces serverless storage, with EMR release 7.12 and later, that eliminates local disk provisioning for Apache Spark workloads. EMR Serverless automatically handles intermediate data operation such as shuffle with no storage charges. Serverless storage decouples storage from compute, allowing Spark to release workers immediately when idle rather than keeping them active to preserve temporary data. To learn more, see [Serverless storage](https://docs.aws.amazon.com/emr/latest/EMR-Serverless-UserGuide/jobs-serverless-storage.html). + **Application upgrades** – Amazon EMR 7.12.0 application upgrades include Delta 3.3.2-amzn-1, Flink 1.20.0-amzn-6, HBase 2.6.2-amzn-3, HCatalog 3.1.3-amzn-21, Hadoop 3.4.1-amzn-4, Hive 3.1.3-amzn-21, Hudi 1.0.2-amzn-1, Iceberg 1.10.0-amzn-0, Presto 0.287-amzn-6, Spark 3.5.6-amzn-1, Tez 0.10.2-amzn-19, Trino 476-amzn-1, and ZooKeeper 3.9.3-amzn-4. + **Iceberg Materialized Views** - Starting Amazon EMR 7.12.0, Amazon EMR Spark supports creation and management of Iceberg Materialized Views (MV) + **Hudi Full Table Access** - Starting Amazon EMR 7.12.0, Amazon EMR now supports Full Table Access (FTA) control for Apache Hudi in Apache Spark based on your policies defined in Lake Formation. This feature enables read and write operations from your Amazon EMR Spark jobs on Lake Formation registered tables when the job role has full table access. + **Iceberg version upgrade** - Amazon EMR 7.12.0 supports Apache Iceberg version 1.10 + **Spark native FGAC support using Lake Formation** - You can perform DDL/DML operations (like CREATE, ALTER, DELETE, DROP) with fine grained access control for Apache Hive, Apache Iceberg and Delta Lake tables registered in Lake Formation. + **Audit context for Spark jobs** - Starting Amazon EMR 7.12.0, audit context for Amazon EMR Spark jobs with Lake Formation enabled clusters will be available for AWS Glue and Lake Formation API calls in CloudTrail logs. + Amazon EMR 7.12.0 improves the GetClusterSessionCredentials session credential authentication process, significantly reducing latency for Livy Interactive Sessions and on-cluster UI access on IAM Identity Center-enabled clusters compared to previous Amazon EMR releases. + Starting Amazon EMR 7.12.0, Amazon EMR HBase on S3 has added a read-replica prewarm feature to simplify Amazon EMR version upgrades. This feature can be used to upgrade existing HBase clusters to and from Amazon EMR version 7.12\$1 + **Known issues and limitations** + **Athena SQL compatibility** + Athena SQL cannot read Iceberg table type V3 created by Amazon EMR Spark due to the following error: GENERIC\$1INTERNAL\$1ERROR: Cannot read unsupported version 3. + Writing data using Amazon EMR Spark into iceberg tables created using Athena fails due to the following error: Property 'write.object-storage.path' has been deprecated and will be removed in 2.0, use 'write.data.path' instead. + **Iceberg Materialized Views (MV)** - Amazon EMR 7.12.0 has limited support for view SQL clause for creation of materialized views, query rewrite and incremental refresh. More details can be found in the Iceberg Materialized Views feature documentation page + **Hudi FTA writes** require using HoodieCredentialedHadoopStorage for credential vending during job execution. Set the following configuration when running Hudi jobs: + `hoodie.storage.class=org.apache.spark.sql.hudi.storage.HoodieCredentialedHadoopStorage` + Hudi FTA write support works only with the default Hudi configurations. Custom or non-default Hudi settings may not be fully supported and could result in unexpected behavior. Clustering for Hudi Merge-On-Read (MOR) tables is also not supported under FTA write mode. + **Missing step and container logs** - From Amazon EMR 7.9 to Amazon EMR 7.12, there is a bug where step logs and/or container logs may not get uploaded to S3 if the step or application is running longer for than 3 hours. Please upgrade to Amazon EMR 7.13 to resolve this issue. + **Changes, enhancements, and resolved issues** + Starting EMR 7.12.0, EMR supports KMS Key ARN for service-hosted spark logs through persistent spark history server. Previous EMR versions only supported KeyID. + Starting EMR 7.12.0, EMR has addressed an issue with Hue that prevented use of an SSL certificate when hosting Hue on EMR Clusters. ## 7.12.0 default Java versions Amazon EMR release 7.12.0 ships with Amazon Corretto 17 (built on OpenJDK) by default for applications that support Corretto 17 (JDK 17), with the exception of Apache Livy. The following table shows the default Java versions for applications in Amazon EMR 7.12.0. | Application | Java / Amazon Corretto version (default is bold) | | --- | --- | | Delta | 17, 11, 8 | | Flink | 17, 11, 8 | | HBase | 17, 11, 8 | | HCatalog | 17, 11, 8 | | Hadoop | 17, 11, 8 | | Hive | 17, 11, 8 | | Hudi | 17, 11, 8 | | Iceberg | 17, 11 | | Livy | 17, 11, 8 | | Oozie | 17, 11, 8 | | Phoenix | 17, 11, 8 | | PrestoDB | 8 | | Spark | 17, 11, 8 | | Spark RAPIDS | 17, 11, 8 | | Tez | 17, 11, 8 | | Trino | 21, 17 | | Zeppelin | 17, 11, 8 | | Pig | 17, 11, 8 | | Zookeeper | 17, 11, 8 | ## 7.12.0 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | adot-java-agent | 1.31.0 | A Java Agent that collects metrics from application daemons. | | delta | 3.3.2-amzn-1 | Delta lake is an open table format for huge analytic datasets | | emr-amazon-cloudwatch-agent | 1.300032.2-amzn-0 | An application that collects internal system-level metrics and custom application metrics from Amazon EC2 instances. | | emr-ddb | 5.7.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 3.21.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.25.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-notebook-env | 1.18.0 | Conda env for emr notebook which includes jupyter enterprise gateway | | emr-s3-dist-cp | 2.42.0 | Distributed copy application optimized for Amazon S3. | | emr-s3-select | 2.21.0 | EMR S3Select Connector | | emr-wal-cli | 1.3.6 | Cli used for emrwal list/deletion. | | emrfs | 2.73.0 | Amazon S3 connector for Hadoop ecosystem applications. | | flink-client | 1.20.0-amzn-6 | Apache Flink command line client scripts and applications. | | flink-jobmanager-config | 1.20.0-amzn-6 | Managing resources on EMR nodes for Apache Flink JobManager. | | hadoop-client | 3.4.1-amzn-4 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 3.4.1-amzn-4 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 3.4.1-amzn-4 | HDFS command-line client and library | | hadoop-hdfs-namenode | 3.4.1-amzn-4 | HDFS service for tracking file names and block locations. | | hadoop-hdfs-zkfc | 3.4.1-amzn-4 | ZKFC service for tracking namenodes for HA mode. | | hadoop-hdfs-journalnode | 3.4.1-amzn-4 | HDFS service for managing the Hadoop filesystem journal on HA clusters. | | hadoop-httpfs-server | 3.4.1-amzn-4 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 3.4.1-amzn-4 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 3.4.1-amzn-4 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 3.4.1-amzn-4 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 3.4.1-amzn-4 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 3.4.1-amzn-4 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 2.6.2-amzn-3 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 2.6.2-amzn-3 | Service for serving one or more HBase regions. | | hbase-client | 2.6.2-amzn-3 | HBase command-line client. | | hbase-rest-server | 2.6.2-amzn-3 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 2.6.2-amzn-3 | Service providing a Thrift endpoint to HBase. | | hbase-operator-tools | 2.6.2-amzn-3 | Repair tool for Apache HBase clusters. | | hcatalog-client | 3.1.3-amzn-21 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 3.1.3-amzn-21 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 3.1.3-amzn-21 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 3.1.3-amzn-21 | Hive command line client. | | hive-hbase | 3.1.3-amzn-21 | Hive-hbase client. | | hive-metastore-server | 3.1.3-amzn-21 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server2 | 3.1.3-amzn-21 | Service for accepting Hive queries as web requests. | | hudi | 1.0.2-amzn-1 | Incremental processing framework to power data pipeline at low latency and high efficiency. | | hudi-presto | 1.0.2-amzn-1 | Bundle library for running Presto with Hudi. | | hudi-trino | 1.0.2-amzn-1 | Bundle library for running Trino with Hudi. | | hudi-spark | 1.0.2-amzn-1 | Bundle library for running Spark with Hudi. | | hue-server | 4.11.0 | Web application for analyzing data using Hadoop ecosystem applications | | iceberg | 1.10.0-amzn-0 | Apache Iceberg is an open table format for huge analytic datasets | | jupyterhub | 1.5.0 | Multi-user server for Jupyter notebooks | | livy-server | 0.8.0-incubating | REST interface for interacting with Apache Spark | | nginx | 1.12.1 | nginx [engine x] is an HTTP and reverse proxy server | | mariadb-server | 5.5.68\$1 | MariaDB database server. | | nvidia-cuda | 12.5.0 | Nvidia drivers and Cuda toolkit | | oozie-client | 5.2.1 | Oozie command-line client. | | oozie-server | 5.2.1 | Service for accepting Oozie workflow requests. | | opencv | 4.7.0 | Open Source Computer Vision Library. | | phoenix-library | 5.2.1 | The phoenix libraries for server and client | | phoenix-connectors | 6.0.0 | Apache Phoenix-Connectors for Spark-3 | | phoenix-query-server | 6.0.0 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.287-amzn-6 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.287-amzn-6 | Service for executing pieces of a query. | | presto-client | 0.287-amzn-6 | Presto command-line client which is installed on an HA cluster's stand-by masters where Presto server is not started. | | trino-coordinator | 476-amzn-1 | Service for accepting queries and managing query execution among trino-workers. | | trino-worker | 476-amzn-1 | Service for executing pieces of a query. | | trino-client | 476-amzn-1 | Trino command-line client which is installed on an HA cluster's stand-by masters where Trino server is not started. | | pig-client | 0.17.0 | Pig command-line client. | | r | 4.3.2 | The R Project for Statistical Computing | | ranger-kms-server | 2.0.0 | Apache Ranger Key Management System | | spark-client | 3.5.6-amzn-1 | Spark command-line clients. | | spark-history-server | 3.5.6-amzn-1 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 3.5.6-amzn-1 | In-memory execution engine for YARN. | | spark-yarn-slave | 3.5.6-amzn-1 | Apache Spark libraries needed by YARN slaves. | | spark-rapids | 25.08.0-amzn-0 | Nvidia Spark RAPIDS plugin that accelerates Apache Spark with GPUs. | | tensorflow | 2.19.0 | TensorFlow open source software library for high performance numerical computation. | | tez-on-yarn | 0.10.2-amzn-19 | The tez YARN application and libraries. | | tez-on-worker | 0.10.2-amzn-19 | The tez YARN application and libraries for worker nodes. | | zeppelin-server | 0.11.1 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.9.3-amzn-4 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.9.3-amzn-4 | ZooKeeper command line client. | ## 7.12.0 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). Reconfiguration actions occur when you specify a configuration for instance groups in a running cluster. Amazon EMR only initiates reconfiguration actions for the classifications that you modify. For more information, see [Reconfigure an instance group in a running cluster](emr-configure-apps-running-cluster.md). **emr-7.12.0 classifications** | Classifications | Description | Reconfiguration Actions | | --- | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | Restarts the ResourceManager service. | | container-executor | Change values in Hadoop YARN's container-executor.cfg file. | Not available. | | container-log4j | Change values in Hadoop YARN's container-log4j.properties file. | Not available. | | core-site | Change values in Hadoop's core-site.xml file. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts Hadoop KMS, Ranger KMS, HiveServer2, Hive MetaStore, Hadoop Httpfs, and MapReduce-HistoryServer. | | docker-conf | Change docker related settings. | Not available. | | emrfs-site | Change EMRFS settings. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts HBaseRegionserver, HBaseMaster, HBaseThrift, HBaseRest, HiveServer2, Hive MetaStore, Hadoop Httpfs, and MapReduce-HistoryServer. | | flink-conf | Change flink-conf.yaml settings. | Restarts Flink history server. | | flink-log4j | Change Flink log4j.properties settings. | Restarts Flink history server. | | flink-log4j-session | Change Flink log4j-session.properties settings for Kubernetes/Yarn session. | Restarts Flink history server. | | flink-log4j-cli | Change Flink log4j-cli.properties settings. | Restarts Flink history server. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts PhoenixQueryserver, HiveServer2, Hive MetaStore, and MapReduce-HistoryServer. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | Restarts the Hadoop HDFS services SecondaryNamenode, Datanode, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts Hadoop KMS, Hadoop Httpfs, and MapReduce-HistoryServer. | | hadoop-ssl-server | Change hadoop ssl server configuration | Not available. | | hadoop-ssl-client | Change hadoop ssl client configuration | Not available. | | hbase | Amazon EMR-curated settings for Apache HBase. | Custom EMR specific property. Sets emrfs-site and hbase-site configs. See those for their associated restarts. | | hbase-env | Change values in HBase's environment. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. | | hbase-log4j2 | Change values in HBase's hbase-log4j2.properties file. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbase.properties file. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | Not available. | | hbase-site | Change values in HBase's hbase-site.xml file. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. Additionally restarts Phoenix QueryServer. | | hdfs-encryption-zones | Configure HDFS encryption zones. | This classification should not be reconfigured. | | hdfs-env | Change values in the HDFS environment. | Restarts Hadoop HDFS services Namenode, Datanode, and ZKFC. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Additionally restarts Hadoop Httpfs. | | hcatalog-env | Change values in HCatalog's environment. | Restarts Hive HCatalog Server. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | Restarts Hive HCatalog Server. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | Restarts Hive HCatalog Server. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | Restarts Hive WebHCat server. | | hcatalog-webhcat-log4j2 | Change values in HCatalog WebHCat's log4j2.properties. | Restarts Hive WebHCat server. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | Restarts Hive WebHCat server. | | hive | Amazon EMR-curated settings for Apache Hive. | Sets configurations to launch Hive LLAP service. | | hive-beeline-log4j2 | Change values in Hive's beeline-log4j2.properties file. | Not available. | | hive-parquet-logging | Change values in Hive's parquet-logging.properties file. | Not available. | | hive-env | Change values in the Hive environment. | Restarts HiveServer2, HiveMetastore, and Hive HCatalog-Server. Runs Hive schemaTool CLI commands to verify hive-metastore. | | hive-exec-log4j2 | Change values in Hive's hive-exec-log4j2.properties file. | Not available. | | hive-llap-daemon-log4j2 | Change values in Hive's llap-daemon-log4j2.properties file. | Not available. | | hive-log4j2 | Change values in Hive's hive-log4j2.properties file. | Not available. | | hive-site | Change values in Hive's hive-site.xml file | Restarts HiveServer2, HiveMetastore, and Hive HCatalog-Server. Runs Hive schemaTool CLI commands to verify hive-metastore. Also restarts Oozie and Zeppelin. | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | Not available. | | hue-ini | Change values in Hue's ini file | Restarts Hue. Also activates Hue config override CLI commands to pick up new configurations. | | httpfs-env | Change values in the HTTPFS environment. | Restarts Hadoop Httpfs service. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | Restarts Hadoop Httpfs service. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | Not available. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | Restarts Hadoop-KMS service. | | hadoop-kms-java-home | Change Hadoop's KMS java home | Not available. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | Not available. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | Restarts Hadoop-KMS and Ranger-KMS service. | | hudi-env | Change values in the Hudi environment. | Not available. | | hudi-defaults | Change values in Hudi's hudi-defaults.conf file. | Not available. | | iceberg-defaults | Change values in Iceberg's iceberg-defaults.conf file. | Not available. | | delta-defaults | Change values in Delta's delta-defaults.conf file. | Not available. | | jupyter-notebook-conf | Change values in Jupyter Notebook's jupyter\$1notebook\$1config.py file. | Not available. | | jupyter-hub-conf | Change values in JupyterHubs's jupyterhub\$1config.py file. | Not available. | | jupyter-s3-conf | Configure Jupyter Notebook S3 persistence. | Not available. | | jupyter-sparkmagic-conf | Change values in Sparkmagic's config.json file. | Not available. | | livy-conf | Change values in Livy's livy.conf file. | Restarts Livy Server. | | livy-env | Change values in the Livy environment. | Restarts Livy Server. | | livy-log4j2 | Change Livy log4j2.properties settings. | Restarts Livy Server. | | mapred-env | Change values in the MapReduce application's environment. | Restarts Hadoop MapReduce-HistoryServer. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | Restarts Hadoop MapReduce-HistoryServer. | | oozie-env | Change values in Oozie's environment. | Restarts Oozie. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | Restarts Oozie. | | oozie-site | Change values in Oozie's oozie-site.xml file. | Restarts Oozie. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | Not available. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | Not available. | | phoenix-log4j2 | Change values in Phoenix's log4j2.properties file. | Restarts Phoenix-QueryServer. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | Not available. | | pig-env | Change values in the Pig environment. | Not available. | | pig-properties | Change values in Pig's pig.properties file. | Restarts Oozie. | | pig-log4j | Change values in Pig's log4j.properties file. | Not available. | | presto-log | Change values in Presto's log.properties file. | Restarts Presto-Server (for PrestoDB) | | presto-config | Change values in Presto's config.properties file. | Restarts Presto-Server (for PrestoDB) | | presto-password-authenticator | Change values in Presto's password-authenticator.properties file. | Not available. | | presto-env | Change values in Presto's presto-env.sh file. | Restarts Presto-Server (for PrestoDB) | | presto-node | Change values in Presto's node.properties file. | Not available. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | Not available. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | Not available. | | presto-connector-hive | Change values in Presto's hive.properties file. | Restarts Presto-Server (for PrestoDB) | | presto-connector-jmx | Change values in Presto's jmx.properties file. | Not available. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | Not available. | | presto-connector-lakeformation | Change values in Presto's lakeformation.properties file. | Restarts Presto-Server (for PrestoDB) | | presto-connector-localfile | Change values in Presto's localfile.properties file. | Not available. | | presto-connector-memory | Change values in Presto's memory.properties file. | Not available. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | Not available. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | Not available. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | Not available. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | Not available. | | presto-connector-redis | Change values in Presto's redis.properties file. | Not available. | | presto-connector-redshift | Change values in Presto's redshift.properties file. | Not available. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | Not available. | | presto-connector-tpcds | Change values in Presto's tpcds.properties file. | Not available. | | trino-log | Change values in Trino's log.properties file. | Restarts Trino-Server (for Trino) | | trino-config | Change values in Trino's config.properties file. | Restarts Trino-Server (for Trino) | | trino-password-authenticator | Change values in Trino's password-authenticator.properties file. | Restarts Trino-Server (for Trino) | | trino-env | Change values in Trino's trino-env.sh file. | Restarts Trino-Server (for Trino) | | trino-node | Change values in Trino's node.properties file. | Not available. | | trino-connector-blackhole | Change values in Trino's blackhole.properties file. | Not available. | | trino-connector-cassandra | Change values in Trino's cassandra.properties file. | Not available. | | trino-connector-delta | Change values in Trino's delta.properties file. | Restarts Trino-Server (for Trino) | | trino-connector-hive | Change values in Trino's hive.properties file. | Restarts Trino-Server (for Trino) | | trino-exchange-manager | Change values in Trino's exchange-manager.properties file. | Restarts Trino-Server (for Trino) | | trino-connector-iceberg | Change values in Trino's iceberg.properties file. | Restarts Trino-Server (for Trino) | | trino-connector-hudi | Change values in Trino's hudi.properties file. | Restarts Trino-Server (for Trino) | | trino-connector-jmx | Change values in Trino's jmx.properties file. | Not available. | | trino-connector-kafka | Change values in Trino's kafka.properties file. | Not available. | | trino-connector-localfile | Change values in Trino's localfile.properties file. | Not available. | | trino-connector-memory | Change values in Trino's memory.properties file. | Not available. | | trino-connector-mongodb | Change values in Trino's mongodb.properties file. | Not available. | | trino-connector-mysql | Change values in Trino's mysql.properties file. | Not available. | | trino-connector-postgresql | Change values in Trino's postgresql.properties file. | Not available. | | trino-connector-raptor | Change values in Trino's raptor.properties file. | Not available. | | trino-connector-redis | Change values in Trino's redis.properties file. | Not available. | | trino-connector-redshift | Change values in Trino's redshift.properties file. | Not available. | | trino-connector-tpch | Change values in Trino's tpch.properties file. | Not available. | | trino-connector-tpcds | Change values in Trino's tpcds.properties file. | Not available. | | ranger-kms-dbks-site | Change values in dbks-site.xml file of Ranger KMS. | Restarts Ranger KMS Server. | | ranger-kms-site | Change values in ranger-kms-site.xml file of Ranger KMS. | Restarts Ranger KMS Server. | | ranger-kms-env | Change values in the Ranger KMS environment. | Restarts Ranger KMS Server. | | ranger-kms-logback | Change values in kms-logback.xml file of Ranger KMS. | Not available. | | ranger-kms-db-ca | Change values for CA file on S3 for MySQL SSL connection with Ranger KMS. | Not available. | | spark | Amazon EMR-curated settings for Apache Spark. | This property modifies spark-defaults. See actions there. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | Restarts Spark history server and Spark thrift server. | | spark-env | Change values in the Spark environment. | Restarts Spark history server and Spark thrift server. | | spark-hive-site | Change values in Spark's hive-site.xml file | Not available. | | spark-log4j2 | Change values in Spark's log4j2.properties file. | Restarts Spark history server and Spark thrift server. | | spark-metrics | Change values in Spark's metrics.properties file. | Restarts Spark history server and Spark thrift server. | | tez-site | Change values in Tez's tez-site.xml file. | Restart Oozie and HiveServer2. | | yarn-env | Change values in the YARN environment. | Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts MapReduce-HistoryServer. | | yarn-site | Change values in YARN's yarn-site.xml file. | Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts Livy Server and MapReduce-HistoryServer. | | zeppelin-env | Change values in the Zeppelin environment. | Restarts Zeppelin. | | zeppelin-site | Change configuration settings in zeppelin-site.xml. | Restarts Zeppelin. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | Restarts Zookeeper server. | | zookeeper-logback | Change values in ZooKeeper's logback.xml file. | Restarts Zookeeper server. | | cloudwatch-logs | Configure CloudWatch Logs integration for EMR cluster nodes. | Not available. | | emr-metrics | Change emr metric settings for this node. | Restarts the CloudWatchAgent service. | ## 7.12.0 change log **Change log for 7.12.0 release and release notes** | Date | Event | Description | | --- | --- | --- | | 2025-11-21 | Docs publication | Amazon EMR 7.12.0 release notes first published | # Amazon EMR release 7.11.0 ## 7.11.0 supported lifecycle The following table describes the supported lifecycle dates for Amazon EMR release 7.11.0. | Support phase | Date | | --- | --- | | Initial release date | November 03, 2025 | | Standard support until | November 03, 2027 | | End of support | November 04, 2027 | | End of life | November 03, 2028 | ## 7.11.0 application versions This release includes the following applications: [https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/Install-CloudWatch-Agent.html](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/Install-CloudWatch-Agent.html), [https://delta.io/](https://delta.io/), [https://flink.apache.org/](https://flink.apache.org/), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [https://hudi.apache.org](https://hudi.apache.org), [http://gethue.com/](http://gethue.com/), [https://iceberg.apache.org/](https://iceberg.apache.org/), [https://jupyter-enterprise-gateway.readthedocs.io/en/latest/](https://jupyter-enterprise-gateway.readthedocs.io/en/latest/), [https://jupyterhub.readthedocs.io/en/latest/#](https://jupyterhub.readthedocs.io/en/latest/#), [https://livy.incubator.apache.org/](https://livy.incubator.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [https://www.tensorflow.org/](https://www.tensorflow.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://trino.io/](https://trino.io/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-7.11.0 | emr-7.10.0 | emr-7.9.0 | emr-7.8.0 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 2.35.5, 1.12.792 | 2.31.48, 1.12.782 | 2.31.16, 1.12.782 | 2.29.52, 1.12.780 | | Python | 3.9, 3.11 | 3.9, 3.11 | 3.9, 3.11 | 3.9, 3.11 | | Scala | 2.12.18 | 2.12.18 | 2.12.18 | 2.12.18 | | AmazonCloudWatchAgent | 1.300032.2-amzn-0 | 1.300032.2-amzn-0 | 1.300032.2-amzn-0 | 1.300032.2-amzn-0 | | Delta | 3.3.2-amzn-0 | 3.3.0-amzn-2 | 3.3.0-amzn-1 | 3.3.0-amzn-0 | | Flink | 1.20.0-amzn-5 | 1.20.0-amzn-4 | 1.20.0-amzn-3 | 1.20.0-amzn-2 | | Ganglia | - | - | - | - | | HBase | 2.6.2-amzn-2 | 2.6.2-amzn-1 | 2.6.2-amzn-0 | 2.6.1-amzn-2 | | HCatalog | 3.1.3-amzn-20 | 3.1.3-amzn-19 | 3.1.3-amzn-18 | 3.1.3-amzn-17 | | Hadoop | 3.4.1-amzn-3 | 3.4.1-amzn-2 | 3.4.1-amzn-1 | 3.4.1-amzn-0 | | Hive | 3.1.3-amzn-20 | 3.1.3-amzn-19 | 3.1.3-amzn-18 | 3.1.3-amzn-17 | | Hudi | 1.0.2-amzn-0 | 0.15.0-amzn-7 | 0.15.0-amzn-6 | 0.15.0-amzn-5 | | Hue | 4.11.0 | 4.11.0 | 4.11.0 | 4.11.0 | | Iceberg | 1.9.1-amzn-0 | 1.8.1-amzn-0 | 1.7.1-amzn-2 | 1.7.1-amzn-1 | | JupyterEnterpriseGateway | 2.6.0 | 2.6.0 | 2.6.0 | 2.6.0 | | JupyterHub | 1.5.0 | 1.5.0 | 1.5.0 | 1.5.0 | | Livy | 0.8.0-incubating | 0.8.0-incubating | 0.8.0-incubating | 0.8.0-incubating | | MXNet | - | - | - | - | | Mahout | - | - | - | - | | Oozie | 5.2.1 | 5.2.1 | 5.2.1 | 5.2.1 | | Phoenix | 5.2.1 | 5.2.1 | 5.2.1 | 5.2.1 | | Pig | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | | Presto | 0.287-amzn-5 | 0.287-amzn-4 | 0.287-amzn-3 | 0.287-amzn-2 | | Spark | 3.5.6-amzn-0 | 3.5.5-amzn-1 | 3.5.5-amzn-0 | 3.5.4-amzn-0 | | Sqoop | - | - | - | - | | TensorFlow | 2.19.0 | 2.18.0 | 2.16.1 | 2.16.1 | | Tez | 0.10.2-amzn-18 | 0.10.2-amzn-17 | 0.10.2-amzn-16 | 0.10.2-amzn-15 | | Trino (PrestoSQL) | 475-amzn-0 | 470-amzn-0 | 467-amzn-2 | 467-amzn-1 | | Zeppelin | 0.11.1 | 0.11.1 | 0.11.1 | 0.11.1 | | ZooKeeper | 3.9.3-amzn-3 | 3.9.3-amzn-2 | 3.9.3-amzn-1 | 3.9.3-amzn-0 | # 7.11.0 common vulnerabilities and exposures The following table lists all CVEs that don't impact EMR clusters that run on recommended configurations of Amazon EMR 7.11.0. Amazon EMR is dependent on upstream open-source for availability of fixes and will provide the latest stable version as part of the Core Engine components within 90 days of Amazon EMR verifying the fixes. This table was updated on November 18, 2025. | CVE ID | Severity | CVE details URL | | --- | --- | --- | | CVE-2015-1832 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2015-1832](https://nvd.nist.gov/vuln/detail/CVE-2015-1832) | | CVE-2016-5018 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2016-5018](https://nvd.nist.gov/vuln/detail/CVE-2016-5018) | | CVE-2017-15095 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2017-15095](https://nvd.nist.gov/vuln/detail/CVE-2017-15095) | | CVE-2017-17485 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2017-17485](https://nvd.nist.gov/vuln/detail/CVE-2017-17485) | | CVE-2017-7525 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2017-7525](https://nvd.nist.gov/vuln/detail/CVE-2017-7525) | | CVE-2018-11307 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-11307](https://nvd.nist.gov/vuln/detail/CVE-2018-11307) | | CVE-2018-11307 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-11307](https://nvd.nist.gov/vuln/detail/CVE-2018-11307) | | CVE-2018-14718 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-14718](https://nvd.nist.gov/vuln/detail/CVE-2018-14718) | | CVE-2018-14718 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-14718](https://nvd.nist.gov/vuln/detail/CVE-2018-14718) | | CVE-2018-14719 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-14719](https://nvd.nist.gov/vuln/detail/CVE-2018-14719) | | CVE-2018-14719 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-14719](https://nvd.nist.gov/vuln/detail/CVE-2018-14719) | | CVE-2018-14720 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-14720](https://nvd.nist.gov/vuln/detail/CVE-2018-14720) | | CVE-2018-14720 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-14720](https://nvd.nist.gov/vuln/detail/CVE-2018-14720) | | CVE-2018-14721 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-14721](https://nvd.nist.gov/vuln/detail/CVE-2018-14721) | | CVE-2018-14721 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-14721](https://nvd.nist.gov/vuln/detail/CVE-2018-14721) | | CVE-2018-19360 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-19360](https://nvd.nist.gov/vuln/detail/CVE-2018-19360) | | CVE-2018-19360 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-19360](https://nvd.nist.gov/vuln/detail/CVE-2018-19360) | | CVE-2018-19361 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-19361](https://nvd.nist.gov/vuln/detail/CVE-2018-19361) | | CVE-2018-19361 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-19361](https://nvd.nist.gov/vuln/detail/CVE-2018-19361) | | CVE-2018-19362 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-19362](https://nvd.nist.gov/vuln/detail/CVE-2018-19362) | | CVE-2018-19362 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-19362](https://nvd.nist.gov/vuln/detail/CVE-2018-19362) | | CVE-2018-7489 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-7489](https://nvd.nist.gov/vuln/detail/CVE-2018-7489) | | CVE-2018-7489 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-7489](https://nvd.nist.gov/vuln/detail/CVE-2018-7489) | | CVE-2019-10202 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-10202](https://nvd.nist.gov/vuln/detail/CVE-2019-10202) | | CVE-2019-14379 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-14379](https://nvd.nist.gov/vuln/detail/CVE-2019-14379) | | CVE-2019-14379 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-14379](https://nvd.nist.gov/vuln/detail/CVE-2019-14379) | | CVE-2019-14540 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-14540](https://nvd.nist.gov/vuln/detail/CVE-2019-14540) | | CVE-2019-14540 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-14540](https://nvd.nist.gov/vuln/detail/CVE-2019-14540) | | CVE-2019-14892 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-14892](https://nvd.nist.gov/vuln/detail/CVE-2019-14892) | | CVE-2019-14892 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-14892](https://nvd.nist.gov/vuln/detail/CVE-2019-14892) | | CVE-2019-14893 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-14893](https://nvd.nist.gov/vuln/detail/CVE-2019-14893) | | CVE-2019-14893 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-14893](https://nvd.nist.gov/vuln/detail/CVE-2019-14893) | | CVE-2019-16335 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-16335](https://nvd.nist.gov/vuln/detail/CVE-2019-16335) | | CVE-2019-16335 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-16335](https://nvd.nist.gov/vuln/detail/CVE-2019-16335) | | CVE-2019-16942 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-16942](https://nvd.nist.gov/vuln/detail/CVE-2019-16942) | | CVE-2019-16942 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-16942](https://nvd.nist.gov/vuln/detail/CVE-2019-16942) | | CVE-2019-16943 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-16943](https://nvd.nist.gov/vuln/detail/CVE-2019-16943) | | CVE-2019-16943 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-16943](https://nvd.nist.gov/vuln/detail/CVE-2019-16943) | | CVE-2019-17267 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-17267](https://nvd.nist.gov/vuln/detail/CVE-2019-17267) | | CVE-2019-17267 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-17267](https://nvd.nist.gov/vuln/detail/CVE-2019-17267) | | CVE-2019-17531 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-17531](https://nvd.nist.gov/vuln/detail/CVE-2019-17531) | | CVE-2019-17531 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-17531](https://nvd.nist.gov/vuln/detail/CVE-2019-17531) | | CVE-2019-20330 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-20330](https://nvd.nist.gov/vuln/detail/CVE-2019-20330) | | CVE-2019-20330 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-20330](https://nvd.nist.gov/vuln/detail/CVE-2019-20330) | | CVE-2020-8840 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2020-8840](https://nvd.nist.gov/vuln/detail/CVE-2020-8840) | | CVE-2020-8840 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2020-8840](https://nvd.nist.gov/vuln/detail/CVE-2020-8840) | | CVE-2020-9546 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2020-9546](https://nvd.nist.gov/vuln/detail/CVE-2020-9546) | | CVE-2020-9546 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2020-9546](https://nvd.nist.gov/vuln/detail/CVE-2020-9546) | | CVE-2020-9547 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2020-9547](https://nvd.nist.gov/vuln/detail/CVE-2020-9547) | | CVE-2020-9547 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2020-9547](https://nvd.nist.gov/vuln/detail/CVE-2020-9547) | | CVE-2020-9548 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2020-9548](https://nvd.nist.gov/vuln/detail/CVE-2020-9548) | | CVE-2020-9548 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2020-9548](https://nvd.nist.gov/vuln/detail/CVE-2020-9548) | | CVE-2022-1471 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2022-1471](https://nvd.nist.gov/vuln/detail/CVE-2022-1471) | | CVE-2022-1471 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2022-1471](https://nvd.nist.gov/vuln/detail/CVE-2022-1471) | | CVE-2022-37865 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2022-37865](https://nvd.nist.gov/vuln/detail/CVE-2022-37865) | | CVE-2022-39135 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2022-39135](https://nvd.nist.gov/vuln/detail/CVE-2022-39135) | | CVE-2022-46337 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2022-46337](https://nvd.nist.gov/vuln/detail/CVE-2022-46337) | | CVE-2022-46337 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2022-46337](https://nvd.nist.gov/vuln/detail/CVE-2022-46337) | | CVE-2024-51504 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2024-51504](https://nvd.nist.gov/vuln/detail/CVE-2024-51504) | | CVE-2024-51504 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2024-51504](https://nvd.nist.gov/vuln/detail/CVE-2024-51504) | | CVE-2024-52046 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2024-52046](https://nvd.nist.gov/vuln/detail/CVE-2024-52046) | | CVE-2024-52046 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2024-52046](https://nvd.nist.gov/vuln/detail/CVE-2024-52046) | | CVE-2024-53990 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2024-53990](https://nvd.nist.gov/vuln/detail/CVE-2024-53990) | | CVE-2025-30065 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2025-30065](https://nvd.nist.gov/vuln/detail/CVE-2025-30065) | | CVE-2014-3643 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2014-3643](https://nvd.nist.gov/vuln/detail/CVE-2014-3643) | | CVE-2015-5237 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2015-5237](https://nvd.nist.gov/vuln/detail/CVE-2015-5237) | | CVE-2015-5237 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2015-5237](https://nvd.nist.gov/vuln/detail/CVE-2015-5237) | | CVE-2016-6796 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2016-6796](https://nvd.nist.gov/vuln/detail/CVE-2016-6796) | | CVE-2018-1000180 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2018-1000180](https://nvd.nist.gov/vuln/detail/CVE-2018-1000180) | | CVE-2018-1000180 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2018-1000180](https://nvd.nist.gov/vuln/detail/CVE-2018-1000180) | | CVE-2018-12022 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2018-12022](https://nvd.nist.gov/vuln/detail/CVE-2018-12022) | | CVE-2018-12022 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2018-12022](https://nvd.nist.gov/vuln/detail/CVE-2018-12022) | | CVE-2018-12023 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2018-12023](https://nvd.nist.gov/vuln/detail/CVE-2018-12023) | | CVE-2018-12023 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2018-12023](https://nvd.nist.gov/vuln/detail/CVE-2018-12023) | | CVE-2018-5968 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2018-5968](https://nvd.nist.gov/vuln/detail/CVE-2018-5968) | | CVE-2019-0205 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2019-0205](https://nvd.nist.gov/vuln/detail/CVE-2019-0205) | | CVE-2019-0205 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2019-0205](https://nvd.nist.gov/vuln/detail/CVE-2019-0205) | | CVE-2019-10172 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2019-10172](https://nvd.nist.gov/vuln/detail/CVE-2019-10172) | | CVE-2019-12086 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2019-12086](https://nvd.nist.gov/vuln/detail/CVE-2019-12086) | | CVE-2019-12086 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2019-12086](https://nvd.nist.gov/vuln/detail/CVE-2019-12086) | | CVE-2019-14439 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2019-14439](https://nvd.nist.gov/vuln/detail/CVE-2019-14439) | | CVE-2019-14439 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2019-14439](https://nvd.nist.gov/vuln/detail/CVE-2019-14439) | | CVE-2020-10650 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-10650](https://nvd.nist.gov/vuln/detail/CVE-2020-10650) | | CVE-2020-10650 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-10650](https://nvd.nist.gov/vuln/detail/CVE-2020-10650) | | CVE-2020-10672 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-10672](https://nvd.nist.gov/vuln/detail/CVE-2020-10672) | | CVE-2020-10672 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-10672](https://nvd.nist.gov/vuln/detail/CVE-2020-10672) | | CVE-2020-10673 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-10673](https://nvd.nist.gov/vuln/detail/CVE-2020-10673) | | CVE-2020-10673 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-10673](https://nvd.nist.gov/vuln/detail/CVE-2020-10673) | | CVE-2020-10968 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-10968](https://nvd.nist.gov/vuln/detail/CVE-2020-10968) | | CVE-2020-10968 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-10968](https://nvd.nist.gov/vuln/detail/CVE-2020-10968) | | CVE-2020-10969 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-10969](https://nvd.nist.gov/vuln/detail/CVE-2020-10969) | | CVE-2020-10969 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-10969](https://nvd.nist.gov/vuln/detail/CVE-2020-10969) | | CVE-2020-11111 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-11111](https://nvd.nist.gov/vuln/detail/CVE-2020-11111) | | CVE-2020-11111 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-11111](https://nvd.nist.gov/vuln/detail/CVE-2020-11111) | | CVE-2020-11112 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-11112](https://nvd.nist.gov/vuln/detail/CVE-2020-11112) | | CVE-2020-11112 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-11112](https://nvd.nist.gov/vuln/detail/CVE-2020-11112) | | CVE-2020-11113 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-11113](https://nvd.nist.gov/vuln/detail/CVE-2020-11113) | | CVE-2020-11113 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-11113](https://nvd.nist.gov/vuln/detail/CVE-2020-11113) | | CVE-2020-11619 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-11619](https://nvd.nist.gov/vuln/detail/CVE-2020-11619) | | CVE-2020-11619 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-11619](https://nvd.nist.gov/vuln/detail/CVE-2020-11619) | | CVE-2020-11620 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-11620](https://nvd.nist.gov/vuln/detail/CVE-2020-11620) | | CVE-2020-11620 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-11620](https://nvd.nist.gov/vuln/detail/CVE-2020-11620) | | CVE-2020-13949 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-13949](https://nvd.nist.gov/vuln/detail/CVE-2020-13949) | | CVE-2020-13949 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-13949](https://nvd.nist.gov/vuln/detail/CVE-2020-13949) | | CVE-2020-13949 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-13949](https://nvd.nist.gov/vuln/detail/CVE-2020-13949) | | CVE-2020-14060 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-14060](https://nvd.nist.gov/vuln/detail/CVE-2020-14060) | | CVE-2020-14060 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-14060](https://nvd.nist.gov/vuln/detail/CVE-2020-14060) | | CVE-2020-14061 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-14061](https://nvd.nist.gov/vuln/detail/CVE-2020-14061) | | CVE-2020-14061 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-14061](https://nvd.nist.gov/vuln/detail/CVE-2020-14061) | | CVE-2020-14062 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-14062](https://nvd.nist.gov/vuln/detail/CVE-2020-14062) | | CVE-2020-14062 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-14062](https://nvd.nist.gov/vuln/detail/CVE-2020-14062) | | CVE-2020-14195 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-14195](https://nvd.nist.gov/vuln/detail/CVE-2020-14195) | | CVE-2020-14195 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-14195](https://nvd.nist.gov/vuln/detail/CVE-2020-14195) | | CVE-2020-24616 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-24616](https://nvd.nist.gov/vuln/detail/CVE-2020-24616) | | CVE-2020-24616 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-24616](https://nvd.nist.gov/vuln/detail/CVE-2020-24616) | | CVE-2020-24750 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-24750](https://nvd.nist.gov/vuln/detail/CVE-2020-24750) | | CVE-2020-24750 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-24750](https://nvd.nist.gov/vuln/detail/CVE-2020-24750) | | CVE-2020-25649 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-25649](https://nvd.nist.gov/vuln/detail/CVE-2020-25649) | | CVE-2020-28052 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-28052](https://nvd.nist.gov/vuln/detail/CVE-2020-28052) | | CVE-2020-35490 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-35490](https://nvd.nist.gov/vuln/detail/CVE-2020-35490) | | CVE-2020-35490 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-35490](https://nvd.nist.gov/vuln/detail/CVE-2020-35490) | | CVE-2020-35491 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-35491](https://nvd.nist.gov/vuln/detail/CVE-2020-35491) | | CVE-2020-35491 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-35491](https://nvd.nist.gov/vuln/detail/CVE-2020-35491) | | CVE-2020-35728 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-35728](https://nvd.nist.gov/vuln/detail/CVE-2020-35728) | | CVE-2020-35728 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-35728](https://nvd.nist.gov/vuln/detail/CVE-2020-35728) | | CVE-2020-36179 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36179](https://nvd.nist.gov/vuln/detail/CVE-2020-36179) | | CVE-2020-36179 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36179](https://nvd.nist.gov/vuln/detail/CVE-2020-36179) | | CVE-2020-36180 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36180](https://nvd.nist.gov/vuln/detail/CVE-2020-36180) | | CVE-2020-36180 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36180](https://nvd.nist.gov/vuln/detail/CVE-2020-36180) | | CVE-2020-36181 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36181](https://nvd.nist.gov/vuln/detail/CVE-2020-36181) | | CVE-2020-36181 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36181](https://nvd.nist.gov/vuln/detail/CVE-2020-36181) | | CVE-2020-36182 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36182](https://nvd.nist.gov/vuln/detail/CVE-2020-36182) | | CVE-2020-36182 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36182](https://nvd.nist.gov/vuln/detail/CVE-2020-36182) | | CVE-2020-36183 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36183](https://nvd.nist.gov/vuln/detail/CVE-2020-36183) | | CVE-2020-36183 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36183](https://nvd.nist.gov/vuln/detail/CVE-2020-36183) | | CVE-2020-36184 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36184](https://nvd.nist.gov/vuln/detail/CVE-2020-36184) | | CVE-2020-36184 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36184](https://nvd.nist.gov/vuln/detail/CVE-2020-36184) | | CVE-2020-36185 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36185](https://nvd.nist.gov/vuln/detail/CVE-2020-36185) | | CVE-2020-36185 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36185](https://nvd.nist.gov/vuln/detail/CVE-2020-36185) | | CVE-2020-36186 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36186](https://nvd.nist.gov/vuln/detail/CVE-2020-36186) | | CVE-2020-36186 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36186](https://nvd.nist.gov/vuln/detail/CVE-2020-36186) | | CVE-2020-36187 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36187](https://nvd.nist.gov/vuln/detail/CVE-2020-36187) | | CVE-2020-36187 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36187](https://nvd.nist.gov/vuln/detail/CVE-2020-36187) | | CVE-2020-36188 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36188](https://nvd.nist.gov/vuln/detail/CVE-2020-36188) | | CVE-2020-36188 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36188](https://nvd.nist.gov/vuln/detail/CVE-2020-36188) | | CVE-2020-36189 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36189](https://nvd.nist.gov/vuln/detail/CVE-2020-36189) | | CVE-2020-36189 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36189](https://nvd.nist.gov/vuln/detail/CVE-2020-36189) | | CVE-2020-36518 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36518](https://nvd.nist.gov/vuln/detail/CVE-2020-36518) | | CVE-2020-36518 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36518](https://nvd.nist.gov/vuln/detail/CVE-2020-36518) | | CVE-2021-0341 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-0341](https://nvd.nist.gov/vuln/detail/CVE-2021-0341) | | CVE-2021-20190 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-20190](https://nvd.nist.gov/vuln/detail/CVE-2021-20190) | | CVE-2021-20190 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-20190](https://nvd.nist.gov/vuln/detail/CVE-2021-20190) | | CVE-2021-31684 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-31684](https://nvd.nist.gov/vuln/detail/CVE-2021-31684) | | CVE-2021-34538 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-34538](https://nvd.nist.gov/vuln/detail/CVE-2021-34538) | | CVE-2021-34538 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-34538](https://nvd.nist.gov/vuln/detail/CVE-2021-34538) | | CVE-2021-34538 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-34538](https://nvd.nist.gov/vuln/detail/CVE-2021-34538) | | CVE-2022-25647 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-25647](https://nvd.nist.gov/vuln/detail/CVE-2022-25647) | | CVE-2022-25647 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-25647](https://nvd.nist.gov/vuln/detail/CVE-2022-25647) | | CVE-2022-25857 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-25857](https://nvd.nist.gov/vuln/detail/CVE-2022-25857) | | CVE-2022-3171 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-3171](https://nvd.nist.gov/vuln/detail/CVE-2022-3171) | | CVE-2022-3171 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-3171](https://nvd.nist.gov/vuln/detail/CVE-2022-3171) | | CVE-2022-3509 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-3509](https://nvd.nist.gov/vuln/detail/CVE-2022-3509) | | CVE-2022-3509 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-3509](https://nvd.nist.gov/vuln/detail/CVE-2022-3509) | | CVE-2022-3510 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-3510](https://nvd.nist.gov/vuln/detail/CVE-2022-3510) | | CVE-2022-36364 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-36364](https://nvd.nist.gov/vuln/detail/CVE-2022-36364) | | CVE-2022-37866 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-37866](https://nvd.nist.gov/vuln/detail/CVE-2022-37866) | | CVE-2022-40149 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-40149](https://nvd.nist.gov/vuln/detail/CVE-2022-40149) | | CVE-2022-40149 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-40149](https://nvd.nist.gov/vuln/detail/CVE-2022-40149) | | CVE-2022-40150 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-40150](https://nvd.nist.gov/vuln/detail/CVE-2022-40150) | | CVE-2022-40150 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-40150](https://nvd.nist.gov/vuln/detail/CVE-2022-40150) | | CVE-2022-41404 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-41404](https://nvd.nist.gov/vuln/detail/CVE-2022-41404) | | CVE-2022-42003 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-42003](https://nvd.nist.gov/vuln/detail/CVE-2022-42003) | | CVE-2022-42003 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-42003](https://nvd.nist.gov/vuln/detail/CVE-2022-42003) | | CVE-2022-42003 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-42003](https://nvd.nist.gov/vuln/detail/CVE-2022-42003) | | CVE-2022-42003 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-42003](https://nvd.nist.gov/vuln/detail/CVE-2022-42003) | | CVE-2022-42003 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-42003](https://nvd.nist.gov/vuln/detail/CVE-2022-42003) | | CVE-2022-42004 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-42004](https://nvd.nist.gov/vuln/detail/CVE-2022-42004) | | CVE-2022-42004 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-42004](https://nvd.nist.gov/vuln/detail/CVE-2022-42004) | | CVE-2022-42004 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-42004](https://nvd.nist.gov/vuln/detail/CVE-2022-42004) | | CVE-2022-42004 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-42004](https://nvd.nist.gov/vuln/detail/CVE-2022-42004) | | CVE-2022-45685 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-45685](https://nvd.nist.gov/vuln/detail/CVE-2022-45685) | | CVE-2022-45685 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-45685](https://nvd.nist.gov/vuln/detail/CVE-2022-45685) | | CVE-2022-45693 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-45693](https://nvd.nist.gov/vuln/detail/CVE-2022-45693) | | CVE-2022-45693 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-45693](https://nvd.nist.gov/vuln/detail/CVE-2022-45693) | | CVE-2022-46751 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-46751](https://nvd.nist.gov/vuln/detail/CVE-2022-46751) | | CVE-2022-46751 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-46751](https://nvd.nist.gov/vuln/detail/CVE-2022-46751) | | CVE-2023-1370 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-1370](https://nvd.nist.gov/vuln/detail/CVE-2023-1370) | | CVE-2023-1436 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-1436](https://nvd.nist.gov/vuln/detail/CVE-2023-1436) | | CVE-2023-1436 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-1436](https://nvd.nist.gov/vuln/detail/CVE-2023-1436) | | CVE-2023-2976 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-2976](https://nvd.nist.gov/vuln/detail/CVE-2023-2976) | | CVE-2023-2976 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-2976](https://nvd.nist.gov/vuln/detail/CVE-2023-2976) | | CVE-2023-2976 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-2976](https://nvd.nist.gov/vuln/detail/CVE-2023-2976) | | CVE-2023-2976 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-2976](https://nvd.nist.gov/vuln/detail/CVE-2023-2976) | | CVE-2023-2976 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-2976](https://nvd.nist.gov/vuln/detail/CVE-2023-2976) | | CVE-2023-2976 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-2976](https://nvd.nist.gov/vuln/detail/CVE-2023-2976) | | CVE-2023-2976 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-2976](https://nvd.nist.gov/vuln/detail/CVE-2023-2976) | | CVE-2023-2976 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-2976](https://nvd.nist.gov/vuln/detail/CVE-2023-2976) | | CVE-2023-2976 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-2976](https://nvd.nist.gov/vuln/detail/CVE-2023-2976) | | CVE-2023-6378 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-6378](https://nvd.nist.gov/vuln/detail/CVE-2023-6378) | | CVE-2023-6378 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-6378](https://nvd.nist.gov/vuln/detail/CVE-2023-6378) | | CVE-2023-6481 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-6481](https://nvd.nist.gov/vuln/detail/CVE-2023-6481) | | CVE-2023-6481 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-6481](https://nvd.nist.gov/vuln/detail/CVE-2023-6481) | | CVE-2024-13009 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-13009](https://nvd.nist.gov/vuln/detail/CVE-2024-13009) | | CVE-2024-13009 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-13009](https://nvd.nist.gov/vuln/detail/CVE-2024-13009) | | CVE-2024-29857 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-29857](https://nvd.nist.gov/vuln/detail/CVE-2024-29857) | | CVE-2024-29857 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-29857](https://nvd.nist.gov/vuln/detail/CVE-2024-29857) | | CVE-2024-29857 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-29857](https://nvd.nist.gov/vuln/detail/CVE-2024-29857) | | CVE-2024-29857 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-29857](https://nvd.nist.gov/vuln/detail/CVE-2024-29857) | | CVE-2024-30172 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-30172](https://nvd.nist.gov/vuln/detail/CVE-2024-30172) | | CVE-2024-30172 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-30172](https://nvd.nist.gov/vuln/detail/CVE-2024-30172) | | CVE-2024-34447 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-34447](https://nvd.nist.gov/vuln/detail/CVE-2024-34447) | | CVE-2024-34447 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-34447](https://nvd.nist.gov/vuln/detail/CVE-2024-34447) | | CVE-2024-36114 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-36114](https://nvd.nist.gov/vuln/detail/CVE-2024-36114) | | CVE-2024-7254 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-7254](https://nvd.nist.gov/vuln/detail/CVE-2024-7254) | | CVE-2024-7254 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-7254](https://nvd.nist.gov/vuln/detail/CVE-2024-7254) | | CVE-2024-7254 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-7254](https://nvd.nist.gov/vuln/detail/CVE-2024-7254) | | CVE-2024-7254 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-7254](https://nvd.nist.gov/vuln/detail/CVE-2024-7254) | | CVE-2024-7254 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-7254](https://nvd.nist.gov/vuln/detail/CVE-2024-7254) | | CVE-2024-7254 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-7254](https://nvd.nist.gov/vuln/detail/CVE-2024-7254) | | CVE-2024-7254 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-7254](https://nvd.nist.gov/vuln/detail/CVE-2024-7254) | | CVE-2024-9823 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-9823](https://nvd.nist.gov/vuln/detail/CVE-2024-9823) | | CVE-2025-24970 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2025-24970](https://nvd.nist.gov/vuln/detail/CVE-2025-24970) | | CVE-2025-24970 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2025-24970](https://nvd.nist.gov/vuln/detail/CVE-2025-24970) | | CVE-2025-27553 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2025-27553](https://nvd.nist.gov/vuln/detail/CVE-2025-27553) | | CVE-2025-27817 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2025-27817](https://nvd.nist.gov/vuln/detail/CVE-2025-27817) | | CVE-2025-46762 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2025-46762](https://nvd.nist.gov/vuln/detail/CVE-2025-46762) | | CVE-2025-46762 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2025-46762](https://nvd.nist.gov/vuln/detail/CVE-2025-46762) | | CVE-2025-48734 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2025-48734](https://nvd.nist.gov/vuln/detail/CVE-2025-48734) | | CVE-2025-52434 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2025-52434](https://nvd.nist.gov/vuln/detail/CVE-2025-52434) | | CVE-2025-52999 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2025-52999](https://nvd.nist.gov/vuln/detail/CVE-2025-52999) | | CVE-2025-52999 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2025-52999](https://nvd.nist.gov/vuln/detail/CVE-2025-52999) | | CVE-2025-52999 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2025-52999](https://nvd.nist.gov/vuln/detail/CVE-2025-52999) | | CVE-2025-52999 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2025-52999](https://nvd.nist.gov/vuln/detail/CVE-2025-52999) | | CVE-2025-52999 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2025-52999](https://nvd.nist.gov/vuln/detail/CVE-2025-52999) | | CVE-2025-52999 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2025-52999](https://nvd.nist.gov/vuln/detail/CVE-2025-52999) | | CVE-2025-55163 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2025-55163](https://nvd.nist.gov/vuln/detail/CVE-2025-55163) | | CVE-2025-55163 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2025-55163](https://nvd.nist.gov/vuln/detail/CVE-2025-55163) | | CVE-2025-55163 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2025-55163](https://nvd.nist.gov/vuln/detail/CVE-2025-55163) | | CVE-2025-58056 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2025-58056](https://nvd.nist.gov/vuln/detail/CVE-2025-58056) | | CVE-2025-58056 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2025-58056](https://nvd.nist.gov/vuln/detail/CVE-2025-58056) | | CVE-2025-58056 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2025-58056](https://nvd.nist.gov/vuln/detail/CVE-2025-58056) | | CVE-2025-58056 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2025-58056](https://nvd.nist.gov/vuln/detail/CVE-2025-58056) | | CVE-2025-58057 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2025-58057](https://nvd.nist.gov/vuln/detail/CVE-2025-58057) | | CVE-2025-58057 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2025-58057](https://nvd.nist.gov/vuln/detail/CVE-2025-58057) | | CVE-2025-58057 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2025-58057](https://nvd.nist.gov/vuln/detail/CVE-2025-58057) | | CVE-2025-58057 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2025-58057](https://nvd.nist.gov/vuln/detail/CVE-2025-58057) | | CVE-2025-58057 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2025-58057](https://nvd.nist.gov/vuln/detail/CVE-2025-58057) | | CVE-2025-59250 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2025-59250](https://nvd.nist.gov/vuln/detail/CVE-2025-59250) | | CVE-2012-5783 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2012-5783](https://nvd.nist.gov/vuln/detail/CVE-2012-5783) | | CVE-2012-5783 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2012-5783](https://nvd.nist.gov/vuln/detail/CVE-2012-5783) | | CVE-2012-6153 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2012-6153](https://nvd.nist.gov/vuln/detail/CVE-2012-6153) | | CVE-2012-6153 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2012-6153](https://nvd.nist.gov/vuln/detail/CVE-2012-6153) | | CVE-2018-10237 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2018-10237](https://nvd.nist.gov/vuln/detail/CVE-2018-10237) | | CVE-2018-10237 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2018-10237](https://nvd.nist.gov/vuln/detail/CVE-2018-10237) | | CVE-2018-10237 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2018-10237](https://nvd.nist.gov/vuln/detail/CVE-2018-10237) | | CVE-2018-10237 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2018-10237](https://nvd.nist.gov/vuln/detail/CVE-2018-10237) | | CVE-2018-10237 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2018-10237](https://nvd.nist.gov/vuln/detail/CVE-2018-10237) | | CVE-2018-11798 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2018-11798](https://nvd.nist.gov/vuln/detail/CVE-2018-11798) | | CVE-2018-1313 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2018-1313](https://nvd.nist.gov/vuln/detail/CVE-2018-1313) | | CVE-2019-12384 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2019-12384](https://nvd.nist.gov/vuln/detail/CVE-2019-12384) | | CVE-2019-12384 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2019-12384](https://nvd.nist.gov/vuln/detail/CVE-2019-12384) | | CVE-2019-12814 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2019-12814](https://nvd.nist.gov/vuln/detail/CVE-2019-12814) | | CVE-2019-12814 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2019-12814](https://nvd.nist.gov/vuln/detail/CVE-2019-12814) | | CVE-2020-13955 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2020-13955](https://nvd.nist.gov/vuln/detail/CVE-2020-13955) | | CVE-2020-13956 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2020-13956](https://nvd.nist.gov/vuln/detail/CVE-2020-13956) | | CVE-2020-15250 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2020-15250](https://nvd.nist.gov/vuln/detail/CVE-2020-15250) | | CVE-2020-15250 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2020-15250](https://nvd.nist.gov/vuln/detail/CVE-2020-15250) | | CVE-2020-15522 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2020-15522](https://nvd.nist.gov/vuln/detail/CVE-2020-15522) | | CVE-2020-15522 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2020-15522](https://nvd.nist.gov/vuln/detail/CVE-2020-15522) | | CVE-2020-15522 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2020-15522](https://nvd.nist.gov/vuln/detail/CVE-2020-15522) | | CVE-2020-17521 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2020-17521](https://nvd.nist.gov/vuln/detail/CVE-2020-17521) | | CVE-2020-17521 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2020-17521](https://nvd.nist.gov/vuln/detail/CVE-2020-17521) | | CVE-2020-1945 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2020-1945](https://nvd.nist.gov/vuln/detail/CVE-2020-1945) | | CVE-2020-26939 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2020-26939](https://nvd.nist.gov/vuln/detail/CVE-2020-26939) | | CVE-2020-26939 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2020-26939](https://nvd.nist.gov/vuln/detail/CVE-2020-26939) | | CVE-2020-26939 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2020-26939](https://nvd.nist.gov/vuln/detail/CVE-2020-26939) | | CVE-2021-22569 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2021-22569](https://nvd.nist.gov/vuln/detail/CVE-2021-22569) | | CVE-2021-22569 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2021-22569](https://nvd.nist.gov/vuln/detail/CVE-2021-22569) | | CVE-2021-27568 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2021-27568](https://nvd.nist.gov/vuln/detail/CVE-2021-27568) | | CVE-2021-28170 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2021-28170](https://nvd.nist.gov/vuln/detail/CVE-2021-28170) | | CVE-2021-29425 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2021-29425](https://nvd.nist.gov/vuln/detail/CVE-2021-29425) | | CVE-2021-36373 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2021-36373](https://nvd.nist.gov/vuln/detail/CVE-2021-36373) | | CVE-2021-36374 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2021-36374](https://nvd.nist.gov/vuln/detail/CVE-2021-36374) | | CVE-2022-38749 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2022-38749](https://nvd.nist.gov/vuln/detail/CVE-2022-38749) | | CVE-2022-38750 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2022-38750](https://nvd.nist.gov/vuln/detail/CVE-2022-38750) | | CVE-2022-38751 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2022-38751](https://nvd.nist.gov/vuln/detail/CVE-2022-38751) | | CVE-2022-38752 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2022-38752](https://nvd.nist.gov/vuln/detail/CVE-2022-38752) | | CVE-2022-41854 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2022-41854](https://nvd.nist.gov/vuln/detail/CVE-2022-41854) | | CVE-2023-0833 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2023-0833](https://nvd.nist.gov/vuln/detail/CVE-2023-0833) | | CVE-2023-33201 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2023-33201](https://nvd.nist.gov/vuln/detail/CVE-2023-33201) | | CVE-2023-33201 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2023-33201](https://nvd.nist.gov/vuln/detail/CVE-2023-33201) | | CVE-2023-33201 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2023-33201](https://nvd.nist.gov/vuln/detail/CVE-2023-33201) | | CVE-2023-33201 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2023-33201](https://nvd.nist.gov/vuln/detail/CVE-2023-33201) | | CVE-2023-33202 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2023-33202](https://nvd.nist.gov/vuln/detail/CVE-2023-33202) | | CVE-2023-33202 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2023-33202](https://nvd.nist.gov/vuln/detail/CVE-2023-33202) | | CVE-2023-33202 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2023-33202](https://nvd.nist.gov/vuln/detail/CVE-2023-33202) | | CVE-2023-33202 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2023-33202](https://nvd.nist.gov/vuln/detail/CVE-2023-33202) | | CVE-2023-42503 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2023-42503](https://nvd.nist.gov/vuln/detail/CVE-2023-42503) | | CVE-2023-51074 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2023-51074](https://nvd.nist.gov/vuln/detail/CVE-2023-51074) | | CVE-2024-12798 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-12798](https://nvd.nist.gov/vuln/detail/CVE-2024-12798) | | CVE-2024-12798 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-12798](https://nvd.nist.gov/vuln/detail/CVE-2024-12798) | | CVE-2024-12798 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-12798](https://nvd.nist.gov/vuln/detail/CVE-2024-12798) | | CVE-2024-12798 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-12798](https://nvd.nist.gov/vuln/detail/CVE-2024-12798) | | CVE-2024-12798 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-12798](https://nvd.nist.gov/vuln/detail/CVE-2024-12798) | | CVE-2024-12798 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-12798](https://nvd.nist.gov/vuln/detail/CVE-2024-12798) | | CVE-2024-23944 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-23944](https://nvd.nist.gov/vuln/detail/CVE-2024-23944) | | CVE-2024-23945 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-23945](https://nvd.nist.gov/vuln/detail/CVE-2024-23945) | | CVE-2024-23945 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-23945](https://nvd.nist.gov/vuln/detail/CVE-2024-23945) | | CVE-2024-23953 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-23953](https://nvd.nist.gov/vuln/detail/CVE-2024-23953) | | CVE-2024-23953 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-23953](https://nvd.nist.gov/vuln/detail/CVE-2024-23953) | | CVE-2024-25710 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-25710](https://nvd.nist.gov/vuln/detail/CVE-2024-25710) | | CVE-2024-25710 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-25710](https://nvd.nist.gov/vuln/detail/CVE-2024-25710) | | CVE-2024-26308 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-26308](https://nvd.nist.gov/vuln/detail/CVE-2024-26308) | | CVE-2024-26308 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-26308](https://nvd.nist.gov/vuln/detail/CVE-2024-26308) | | CVE-2024-29025 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-29025](https://nvd.nist.gov/vuln/detail/CVE-2024-29025) | | CVE-2024-29869 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-29869](https://nvd.nist.gov/vuln/detail/CVE-2024-29869) | | CVE-2024-29869 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-29869](https://nvd.nist.gov/vuln/detail/CVE-2024-29869) | | CVE-2024-30171 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-30171](https://nvd.nist.gov/vuln/detail/CVE-2024-30171) | | CVE-2024-30171 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-30171](https://nvd.nist.gov/vuln/detail/CVE-2024-30171) | | CVE-2024-30171 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-30171](https://nvd.nist.gov/vuln/detail/CVE-2024-30171) | | CVE-2024-30171 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-30171](https://nvd.nist.gov/vuln/detail/CVE-2024-30171) | | CVE-2024-31141 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-31141](https://nvd.nist.gov/vuln/detail/CVE-2024-31141) | | CVE-2024-47535 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-47535](https://nvd.nist.gov/vuln/detail/CVE-2024-47535) | | CVE-2024-47535 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-47535](https://nvd.nist.gov/vuln/detail/CVE-2024-47535) | | CVE-2024-47554 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-47554](https://nvd.nist.gov/vuln/detail/CVE-2024-47554) | | CVE-2024-47554 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-47554](https://nvd.nist.gov/vuln/detail/CVE-2024-47554) | | CVE-2024-47554 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-47554](https://nvd.nist.gov/vuln/detail/CVE-2024-47554) | | CVE-2024-56128 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-56128](https://nvd.nist.gov/vuln/detail/CVE-2024-56128) | | CVE-2024-6763 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-6763](https://nvd.nist.gov/vuln/detail/CVE-2024-6763) | | CVE-2024-6763 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-6763](https://nvd.nist.gov/vuln/detail/CVE-2024-6763) | | CVE-2024-6763 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-6763](https://nvd.nist.gov/vuln/detail/CVE-2024-6763) | | CVE-2024-6763 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-6763](https://nvd.nist.gov/vuln/detail/CVE-2024-6763) | | CVE-2024-6763 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-6763](https://nvd.nist.gov/vuln/detail/CVE-2024-6763) | | CVE-2024-8184 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-8184](https://nvd.nist.gov/vuln/detail/CVE-2024-8184) | | CVE-2025-11226 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2025-11226](https://nvd.nist.gov/vuln/detail/CVE-2025-11226) | | CVE-2025-11226 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2025-11226](https://nvd.nist.gov/vuln/detail/CVE-2025-11226) | | CVE-2025-11226 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2025-11226](https://nvd.nist.gov/vuln/detail/CVE-2025-11226) | | CVE-2025-25193 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2025-25193](https://nvd.nist.gov/vuln/detail/CVE-2025-25193) | | CVE-2025-25193 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2025-25193](https://nvd.nist.gov/vuln/detail/CVE-2025-25193) | | CVE-2025-30474 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2025-30474](https://nvd.nist.gov/vuln/detail/CVE-2025-30474) | | CVE-2025-3588 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2025-3588](https://nvd.nist.gov/vuln/detail/CVE-2025-3588) | | CVE-2025-46392 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2025-46392](https://nvd.nist.gov/vuln/detail/CVE-2025-46392) | | CVE-2025-48924 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2025-48924](https://nvd.nist.gov/vuln/detail/CVE-2025-48924) | | CVE-2025-48924 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2025-48924](https://nvd.nist.gov/vuln/detail/CVE-2025-48924) | | CVE-2025-48924 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2025-48924](https://nvd.nist.gov/vuln/detail/CVE-2025-48924) | | CVE-2025-48924 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2025-48924](https://nvd.nist.gov/vuln/detail/CVE-2025-48924) | | CVE-2025-48924 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2025-48924](https://nvd.nist.gov/vuln/detail/CVE-2025-48924) | | CVE-2025-48924 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2025-48924](https://nvd.nist.gov/vuln/detail/CVE-2025-48924) | | CVE-2025-48924 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2025-48924](https://nvd.nist.gov/vuln/detail/CVE-2025-48924) | | CVE-2025-48924 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2025-48924](https://nvd.nist.gov/vuln/detail/CVE-2025-48924) | | CVE-2025-48924 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2025-48924](https://nvd.nist.gov/vuln/detail/CVE-2025-48924) | | CVE-2025-49128 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2025-49128](https://nvd.nist.gov/vuln/detail/CVE-2025-49128) | | CVE-2025-49128 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2025-49128](https://nvd.nist.gov/vuln/detail/CVE-2025-49128) | | CVE-2025-49128 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2025-49128](https://nvd.nist.gov/vuln/detail/CVE-2025-49128) | | CVE-2025-53864 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2025-53864](https://nvd.nist.gov/vuln/detail/CVE-2025-53864) | | CVE-2025-58457 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2025-58457](https://nvd.nist.gov/vuln/detail/CVE-2025-58457) | | CVE-2025-58457 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2025-58457](https://nvd.nist.gov/vuln/detail/CVE-2025-58457) | | CVE-2025-59419 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2025-59419](https://nvd.nist.gov/vuln/detail/CVE-2025-59419) | | CVE-2025-8885 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2025-8885](https://nvd.nist.gov/vuln/detail/CVE-2025-8885) | | CVE-2025-8885 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2025-8885](https://nvd.nist.gov/vuln/detail/CVE-2025-8885) | | CVE-2025-8916 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2025-8916](https://nvd.nist.gov/vuln/detail/CVE-2025-8916) | | CVE-2025-8916 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2025-8916](https://nvd.nist.gov/vuln/detail/CVE-2025-8916) | | CVE-2025-8916 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2025-8916](https://nvd.nist.gov/vuln/detail/CVE-2025-8916) | | CVE-2025-8916 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2025-8916](https://nvd.nist.gov/vuln/detail/CVE-2025-8916) | | CVE-2020-8908 | LOW | [https://nvd.nist.gov/vuln/detail/CVE-2020-8908](https://nvd.nist.gov/vuln/detail/CVE-2020-8908) | | CVE-2020-8908 | LOW | [https://nvd.nist.gov/vuln/detail/CVE-2020-8908](https://nvd.nist.gov/vuln/detail/CVE-2020-8908) | | CVE-2020-8908 | LOW | [https://nvd.nist.gov/vuln/detail/CVE-2020-8908](https://nvd.nist.gov/vuln/detail/CVE-2020-8908) | | CVE-2020-8908 | LOW | [https://nvd.nist.gov/vuln/detail/CVE-2020-8908](https://nvd.nist.gov/vuln/detail/CVE-2020-8908) | | CVE-2020-8908 | LOW | [https://nvd.nist.gov/vuln/detail/CVE-2020-8908](https://nvd.nist.gov/vuln/detail/CVE-2020-8908) | | CVE-2020-8908 | LOW | [https://nvd.nist.gov/vuln/detail/CVE-2020-8908](https://nvd.nist.gov/vuln/detail/CVE-2020-8908) | | CVE-2020-8908 | LOW | [https://nvd.nist.gov/vuln/detail/CVE-2020-8908](https://nvd.nist.gov/vuln/detail/CVE-2020-8908) | | CVE-2020-8908 | LOW | [https://nvd.nist.gov/vuln/detail/CVE-2020-8908](https://nvd.nist.gov/vuln/detail/CVE-2020-8908) | | CVE-2020-8908 | LOW | [https://nvd.nist.gov/vuln/detail/CVE-2020-8908](https://nvd.nist.gov/vuln/detail/CVE-2020-8908) | | CVE-2024-12801 | LOW | [https://nvd.nist.gov/vuln/detail/CVE-2024-12801](https://nvd.nist.gov/vuln/detail/CVE-2024-12801) | | CVE-2024-12801 | LOW | [https://nvd.nist.gov/vuln/detail/CVE-2024-12801](https://nvd.nist.gov/vuln/detail/CVE-2024-12801) | | CVE-2024-12801 | LOW | [https://nvd.nist.gov/vuln/detail/CVE-2024-12801](https://nvd.nist.gov/vuln/detail/CVE-2024-12801) | | CVE-2025-46551 | LOW | [https://nvd.nist.gov/vuln/detail/CVE-2025-46551](https://nvd.nist.gov/vuln/detail/CVE-2025-46551) | ## 7.11.0 release notes The following release notes include information for Amazon EMR release 7.11.0. + **New features** + **Application upgrades** – Amazon EMR 7.11.0 application upgrades include Delta 3.3.2-amzn-0, Flink 1.20.0-amzn-5, HBase 2.6.2-amzn-2, HCatalog 3.1.3-amzn-20, Hadoop 3.4.1-amzn-3, Hive 3.1.3-amzn-20, Hudi 1.0.2-amzn-0, Iceberg 1.9.1-amzn-0, Presto 0.287-amzn-5, Spark 3.5.6-amzn-0, TensorFlow 2.19.0, Tez 0.10.2-amzn-18, Trino 475-amzn-0, and ZooKeeper 3.9.3-amzn-3. + Amazon EMR on EC2 now supports IAM Identity Center User Background Sessions + **User Background Sessions**: Enables long-running Spark workloads to continue running even after users log off from SageMaker Unified Studio, supporting sessions up to 90 days + **Flexible Background Session Configuration**: Two-level configuration (IAM Identity Center instance and Amazon EMR-EC2 cluster) with customizable background session duration from 15 minutes to 90 days (default: 7 days) + **Trusted Identity Propagation**: Maintains secure identity context throughout the background session lifecycle using Amazon EMR's trusted identity propagation feature + **SageMaker Unified Studio Integration**: Background sessions initiated through Livy interactive sessions in SageMaker Unified Studio + **Long-running sessions with corporate identities** - Amazon SageMaker Unified Studio now supports long-running sessions with corporate identities through IAM Identity Center's Trusted Identity Propagation (TIP). Users can launch interactive notebooks and data processing sessions on Amazon EMR and AWS Glue that persist using corporate credentials, even when logged off or sessions expire. Sessions run for up to 90 days (default 7 days) while maintaining identity permissions and consistent security controls. + **Known issues and limitations** + **Missing step and container logs** - From Amazon EMR 7.9 to Amazon EMR 7.12, there is a bug where step logs and/or container logs may not get uploaded to S3 if the step or application is running for longer than 3 hours. Please upgrade to Amazon EMR 7.13 to resolve this issue. ## 7.11.0 default Java versions Amazon EMR release 7.11.0 ships with Amazon Corretto 17 (built on OpenJDK) by default for applications that support Corretto 17 (JDK 17), with the exception of Apache Livy. The following table shows the default Java versions for applications in Amazon EMR 7.11.0. | Application | Java / Amazon Corretto version (default is bold) | | --- | --- | | Delta | 17, 11, 8 | | Flink | 17, 11, 8 | | HBase | 17, 11, 8 | | HCatalog | 17, 11, 8 | | Hadoop | 17, 11, 8 | | Hive | 17, 11, 8 | | Hudi | 17, 11, 8 | | Iceberg | 17, 11 | | Livy | 17, 11, 8 | | Oozie | 17, 11, 8 | | Phoenix | 17, 11, 8 | | PrestoDB | 8 | | Spark | 17, 11, 8 | | Spark RAPIDS | 17, 11, 8 | | Tez | 17, 11, 8 | | Trino | 21, 17 | | Zeppelin | 17, 11, 8 | | Pig | 17, 11, 8 | | Zookeeper | 17, 11, 8 | ## 7.11.0 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | adot-java-agent | 1.31.0 | A Java Agent that collects metrics from application daemons. | | delta | 3.3.2-amzn-0 | Delta lake is an open table format for huge analytic datasets | | emr-amazon-cloudwatch-agent | 1.300032.2-amzn-0 | An application that collects internal system-level metrics and custom application metrics from Amazon EC2 instances. | | emr-ddb | 5.7.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 3.20.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.24.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-notebook-env | 1.18.0 | Conda env for emr notebook which includes jupyter enterprise gateway | | emr-s3-dist-cp | 2.41.0 | Distributed copy application optimized for Amazon S3. | | emr-s3-select | 2.20.0 | EMR S3Select Connector | | emr-wal-cli | 1.3.5 | Cli used for emrwal list/deletion. | | emrfs | 2.72.0 | Amazon S3 connector for Hadoop ecosystem applications. | | flink-client | 1.20.0-amzn-5 | Apache Flink command line client scripts and applications. | | flink-jobmanager-config | 1.20.0-amzn-5 | Managing resources on EMR nodes for Apache Flink JobManager. | | hadoop-client | 3.4.1-amzn-3 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 3.4.1-amzn-3 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 3.4.1-amzn-3 | HDFS command-line client and library | | hadoop-hdfs-namenode | 3.4.1-amzn-3 | HDFS service for tracking file names and block locations. | | hadoop-hdfs-zkfc | 3.4.1-amzn-3 | ZKFC service for tracking namenodes for HA mode. | | hadoop-hdfs-journalnode | 3.4.1-amzn-3 | HDFS service for managing the Hadoop filesystem journal on HA clusters. | | hadoop-httpfs-server | 3.4.1-amzn-3 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 3.4.1-amzn-3 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 3.4.1-amzn-3 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 3.4.1-amzn-3 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 3.4.1-amzn-3 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 3.4.1-amzn-3 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 2.6.2-amzn-2 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 2.6.2-amzn-2 | Service for serving one or more HBase regions. | | hbase-client | 2.6.2-amzn-2 | HBase command-line client. | | hbase-rest-server | 2.6.2-amzn-2 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 2.6.2-amzn-2 | Service providing a Thrift endpoint to HBase. | | hbase-operator-tools | 2.6.2-amzn-2 | Repair tool for Apache HBase clusters. | | hcatalog-client | 3.1.3-amzn-20 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 3.1.3-amzn-20 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 3.1.3-amzn-20 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 3.1.3-amzn-20 | Hive command line client. | | hive-hbase | 3.1.3-amzn-20 | Hive-hbase client. | | hive-metastore-server | 3.1.3-amzn-20 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server2 | 3.1.3-amzn-20 | Service for accepting Hive queries as web requests. | | hudi | 1.0.2-amzn-0 | Incremental processing framework to power data pipeline at low latency and high efficiency. | | hudi-presto | 1.0.2-amzn-0 | Bundle library for running Presto with Hudi. | | hudi-trino | 1.0.2-amzn-0 | Bundle library for running Trino with Hudi. | | hudi-spark | 1.0.2-amzn-0 | Bundle library for running Spark with Hudi. | | hue-server | 4.11.0 | Web application for analyzing data using Hadoop ecosystem applications | | iceberg | 1.9.1-amzn-0 | Apache Iceberg is an open table format for huge analytic datasets | | jupyterhub | 1.5.0 | Multi-user server for Jupyter notebooks | | livy-server | 0.8.0-incubating | REST interface for interacting with Apache Spark | | nginx | 1.12.1 | nginx [engine x] is an HTTP and reverse proxy server | | mariadb-server | 5.5.68\$1 | MariaDB database server. | | nvidia-cuda | 12.5.0 | Nvidia drivers and Cuda toolkit | | oozie-client | 5.2.1 | Oozie command-line client. | | oozie-server | 5.2.1 | Service for accepting Oozie workflow requests. | | opencv | 4.7.0 | Open Source Computer Vision Library. | | phoenix-library | 5.2.1 | The phoenix libraries for server and client | | phoenix-connectors | 6.0.0 | Apache Phoenix-Connectors for Spark-3 | | phoenix-query-server | 6.0.0 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.287-amzn-5 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.287-amzn-5 | Service for executing pieces of a query. | | presto-client | 0.287-amzn-5 | Presto command-line client which is installed on an HA cluster's stand-by masters where Presto server is not started. | | trino-coordinator | 475-amzn-0 | Service for accepting queries and managing query execution among trino-workers. | | trino-worker | 475-amzn-0 | Service for executing pieces of a query. | | trino-client | 475-amzn-0 | Trino command-line client which is installed on an HA cluster's stand-by masters where Trino server is not started. | | pig-client | 0.17.0 | Pig command-line client. | | r | 4.3.2 | The R Project for Statistical Computing | | ranger-kms-server | 2.0.0 | Apache Ranger Key Management System | | spark-client | 3.5.6-amzn-0 | Spark command-line clients. | | spark-history-server | 3.5.6-amzn-0 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 3.5.6-amzn-0 | In-memory execution engine for YARN. | | spark-yarn-slave | 3.5.6-amzn-0 | Apache Spark libraries needed by YARN slaves. | | spark-rapids | 25.06.0-amzn-0 | Nvidia Spark RAPIDS plugin that accelerates Apache Spark with GPUs. | | tensorflow | 2.19.0 | TensorFlow open source software library for high performance numerical computation. | | tez-on-yarn | 0.10.2-amzn-18 | The tez YARN application and libraries. | | tez-on-worker | 0.10.2-amzn-18 | The tez YARN application and libraries for worker nodes. | | zeppelin-server | 0.11.1 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.9.3-amzn-3 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.9.3-amzn-3 | ZooKeeper command line client. | ## 7.11.0 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). Reconfiguration actions occur when you specify a configuration for instance groups in a running cluster. Amazon EMR only initiates reconfiguration actions for the classifications that you modify. For more information, see [Reconfigure an instance group in a running cluster](emr-configure-apps-running-cluster.md). **emr-7.11.0 classifications** | Classifications | Description | Reconfiguration Actions | | --- | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | Restarts the ResourceManager service. | | container-executor | Change values in Hadoop YARN's container-executor.cfg file. | Not available. | | container-log4j | Change values in Hadoop YARN's container-log4j.properties file. | Not available. | | core-site | Change values in Hadoop's core-site.xml file. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts Hadoop KMS, Ranger KMS, HiveServer2, Hive MetaStore, Hadoop Httpfs, and MapReduce-HistoryServer. | | docker-conf | Change docker related settings. | Not available. | | emrfs-site | Change EMRFS settings. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts HBaseRegionserver, HBaseMaster, HBaseThrift, HBaseRest, HiveServer2, Hive MetaStore, Hadoop Httpfs, and MapReduce-HistoryServer. | | flink-conf | Change flink-conf.yaml settings. | Restarts Flink history server. | | flink-log4j | Change Flink log4j.properties settings. | Restarts Flink history server. | | flink-log4j-session | Change Flink log4j-session.properties settings for Kubernetes/Yarn session. | Restarts Flink history server. | | flink-log4j-cli | Change Flink log4j-cli.properties settings. | Restarts Flink history server. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts PhoenixQueryserver, HiveServer2, Hive MetaStore, and MapReduce-HistoryServer. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | Restarts the Hadoop HDFS services SecondaryNamenode, Datanode, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts Hadoop KMS, Hadoop Httpfs, and MapReduce-HistoryServer. | | hadoop-ssl-server | Change hadoop ssl server configuration | Not available. | | hadoop-ssl-client | Change hadoop ssl client configuration | Not available. | | hbase | Amazon EMR-curated settings for Apache HBase. | Custom EMR specific property. Sets emrfs-site and hbase-site configs. See those for their associated restarts. | | hbase-env | Change values in HBase's environment. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. | | hbase-log4j2 | Change values in HBase's hbase-log4j2.properties file. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbase.properties file. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | Not available. | | hbase-site | Change values in HBase's hbase-site.xml file. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. Additionally restarts Phoenix QueryServer. | | hdfs-encryption-zones | Configure HDFS encryption zones. | This classification should not be reconfigured. | | hdfs-env | Change values in the HDFS environment. | Restarts Hadoop HDFS services Namenode, Datanode, and ZKFC. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Additionally restarts Hadoop Httpfs. | | hcatalog-env | Change values in HCatalog's environment. | Restarts Hive HCatalog Server. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | Restarts Hive HCatalog Server. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | Restarts Hive HCatalog Server. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | Restarts Hive WebHCat server. | | hcatalog-webhcat-log4j2 | Change values in HCatalog WebHCat's log4j2.properties. | Restarts Hive WebHCat server. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | Restarts Hive WebHCat server. | | hive | Amazon EMR-curated settings for Apache Hive. | Sets configurations to launch Hive LLAP service. | | hive-beeline-log4j2 | Change values in Hive's beeline-log4j2.properties file. | Not available. | | hive-parquet-logging | Change values in Hive's parquet-logging.properties file. | Not available. | | hive-env | Change values in the Hive environment. | Restarts HiveServer2, HiveMetastore, and Hive HCatalog-Server. Runs Hive schemaTool CLI commands to verify hive-metastore. | | hive-exec-log4j2 | Change values in Hive's hive-exec-log4j2.properties file. | Not available. | | hive-llap-daemon-log4j2 | Change values in Hive's llap-daemon-log4j2.properties file. | Not available. | | hive-log4j2 | Change values in Hive's hive-log4j2.properties file. | Not available. | | hive-site | Change values in Hive's hive-site.xml file | Restarts HiveServer2, HiveMetastore, and Hive HCatalog-Server. Runs Hive schemaTool CLI commands to verify hive-metastore. Also restarts Oozie and Zeppelin. | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | Not available. | | hue-ini | Change values in Hue's ini file | Restarts Hue. Also activates Hue config override CLI commands to pick up new configurations. | | httpfs-env | Change values in the HTTPFS environment. | Restarts Hadoop Httpfs service. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | Restarts Hadoop Httpfs service. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | Not available. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | Restarts Hadoop-KMS service. | | hadoop-kms-java-home | Change Hadoop's KMS java home | Not available. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | Not available. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | Restarts Hadoop-KMS and Ranger-KMS service. | | hudi-env | Change values in the Hudi environment. | Not available. | | hudi-defaults | Change values in Hudi's hudi-defaults.conf file. | Not available. | | iceberg-defaults | Change values in Iceberg's iceberg-defaults.conf file. | Not available. | | delta-defaults | Change values in Delta's delta-defaults.conf file. | Not available. | | jupyter-notebook-conf | Change values in Jupyter Notebook's jupyter\$1notebook\$1config.py file. | Not available. | | jupyter-hub-conf | Change values in JupyterHubs's jupyterhub\$1config.py file. | Not available. | | jupyter-s3-conf | Configure Jupyter Notebook S3 persistence. | Not available. | | jupyter-sparkmagic-conf | Change values in Sparkmagic's config.json file. | Not available. | | livy-conf | Change values in Livy's livy.conf file. | Restarts Livy Server. | | livy-env | Change values in the Livy environment. | Restarts Livy Server. | | livy-log4j2 | Change Livy log4j2.properties settings. | Restarts Livy Server. | | mapred-env | Change values in the MapReduce application's environment. | Restarts Hadoop MapReduce-HistoryServer. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | Restarts Hadoop MapReduce-HistoryServer. | | oozie-env | Change values in Oozie's environment. | Restarts Oozie. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | Restarts Oozie. | | oozie-site | Change values in Oozie's oozie-site.xml file. | Restarts Oozie. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | Not available. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | Not available. | | phoenix-log4j2 | Change values in Phoenix's log4j2.properties file. | Restarts Phoenix-QueryServer. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | Not available. | | pig-env | Change values in the Pig environment. | Not available. | | pig-properties | Change values in Pig's pig.properties file. | Restarts Oozie. | | pig-log4j | Change values in Pig's log4j.properties file. | Not available. | | presto-log | Change values in Presto's log.properties file. | Restarts Presto-Server (for PrestoDB) | | presto-config | Change values in Presto's config.properties file. | Restarts Presto-Server (for PrestoDB) | | presto-password-authenticator | Change values in Presto's password-authenticator.properties file. | Not available. | | presto-env | Change values in Presto's presto-env.sh file. | Restarts Presto-Server (for PrestoDB) | | presto-node | Change values in Presto's node.properties file. | Not available. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | Not available. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | Not available. | | presto-connector-hive | Change values in Presto's hive.properties file. | Restarts Presto-Server (for PrestoDB) | | presto-connector-jmx | Change values in Presto's jmx.properties file. | Not available. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | Not available. | | presto-connector-lakeformation | Change values in Presto's lakeformation.properties file. | Restarts Presto-Server (for PrestoDB) | | presto-connector-localfile | Change values in Presto's localfile.properties file. | Not available. | | presto-connector-memory | Change values in Presto's memory.properties file. | Not available. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | Not available. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | Not available. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | Not available. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | Not available. | | presto-connector-redis | Change values in Presto's redis.properties file. | Not available. | | presto-connector-redshift | Change values in Presto's redshift.properties file. | Not available. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | Not available. | | presto-connector-tpcds | Change values in Presto's tpcds.properties file. | Not available. | | trino-log | Change values in Trino's log.properties file. | Restarts Trino-Server (for Trino) | | trino-config | Change values in Trino's config.properties file. | Restarts Trino-Server (for Trino) | | trino-password-authenticator | Change values in Trino's password-authenticator.properties file. | Restarts Trino-Server (for Trino) | | trino-env | Change values in Trino's trino-env.sh file. | Restarts Trino-Server (for Trino) | | trino-node | Change values in Trino's node.properties file. | Not available. | | trino-connector-blackhole | Change values in Trino's blackhole.properties file. | Not available. | | trino-connector-cassandra | Change values in Trino's cassandra.properties file. | Not available. | | trino-connector-delta | Change values in Trino's delta.properties file. | Restarts Trino-Server (for Trino) | | trino-connector-hive | Change values in Trino's hive.properties file. | Restarts Trino-Server (for Trino) | | trino-exchange-manager | Change values in Trino's exchange-manager.properties file. | Restarts Trino-Server (for Trino) | | trino-connector-iceberg | Change values in Trino's iceberg.properties file. | Restarts Trino-Server (for Trino) | | trino-connector-hudi | Change values in Trino's hudi.properties file. | Restarts Trino-Server (for Trino) | | trino-connector-jmx | Change values in Trino's jmx.properties file. | Not available. | | trino-connector-kafka | Change values in Trino's kafka.properties file. | Not available. | | trino-connector-localfile | Change values in Trino's localfile.properties file. | Not available. | | trino-connector-memory | Change values in Trino's memory.properties file. | Not available. | | trino-connector-mongodb | Change values in Trino's mongodb.properties file. | Not available. | | trino-connector-mysql | Change values in Trino's mysql.properties file. | Not available. | | trino-connector-postgresql | Change values in Trino's postgresql.properties file. | Not available. | | trino-connector-raptor | Change values in Trino's raptor.properties file. | Not available. | | trino-connector-redis | Change values in Trino's redis.properties file. | Not available. | | trino-connector-redshift | Change values in Trino's redshift.properties file. | Not available. | | trino-connector-tpch | Change values in Trino's tpch.properties file. | Not available. | | trino-connector-tpcds | Change values in Trino's tpcds.properties file. | Not available. | | ranger-kms-dbks-site | Change values in dbks-site.xml file of Ranger KMS. | Restarts Ranger KMS Server. | | ranger-kms-site | Change values in ranger-kms-site.xml file of Ranger KMS. | Restarts Ranger KMS Server. | | ranger-kms-env | Change values in the Ranger KMS environment. | Restarts Ranger KMS Server. | | ranger-kms-logback | Change values in kms-logback.xml file of Ranger KMS. | Not available. | | ranger-kms-db-ca | Change values for CA file on S3 for MySQL SSL connection with Ranger KMS. | Not available. | | spark | Amazon EMR-curated settings for Apache Spark. | This property modifies spark-defaults. See actions there. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | Restarts Spark history server and Spark thrift server. | | spark-env | Change values in the Spark environment. | Restarts Spark history server and Spark thrift server. | | spark-hive-site | Change values in Spark's hive-site.xml file | Not available. | | spark-log4j2 | Change values in Spark's log4j2.properties file. | Restarts Spark history server and Spark thrift server. | | spark-metrics | Change values in Spark's metrics.properties file. | Restarts Spark history server and Spark thrift server. | | tez-site | Change values in Tez's tez-site.xml file. | Restart Oozie and HiveServer2. | | yarn-env | Change values in the YARN environment. | Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts MapReduce-HistoryServer. | | yarn-site | Change values in YARN's yarn-site.xml file. | Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts Livy Server and MapReduce-HistoryServer. | | zeppelin-env | Change values in the Zeppelin environment. | Restarts Zeppelin. | | zeppelin-site | Change configuration settings in zeppelin-site.xml. | Restarts Zeppelin. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | Restarts Zookeeper server. | | zookeeper-logback | Change values in ZooKeeper's logback.xml file. | Restarts Zookeeper server. | | cloudwatch-logs | Configure CloudWatch Logs integration for EMR cluster nodes. | Not available. | | emr-metrics | Change emr metric settings for this node. | Restarts the CloudWatchAgent service. | ## 7.11.0 change log **Change log for 7.11.0 release and release notes** | Date | Event | Description | | --- | --- | --- | | 2025-11-12 | Docs publication | Amazon EMR 7.11.0 release notes first published | # Amazon EMR release 7.10.0 ## 7.10.0 supported lifecycle The following table describes the supported lifecycle dates for Amazon EMR release 7.10.0. | Support phase | Date | | --- | --- | | Initial release date | August 15, 2025 | | Standard support until | August 15, 2027 | | End of support | August 16, 2027 | | End of life | August 15, 2028 | ## 7.10.0 application versions This release includes the following applications: [https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/Install-CloudWatch-Agent.html](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/Install-CloudWatch-Agent.html), [https://delta.io/](https://delta.io/), [https://flink.apache.org/](https://flink.apache.org/), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [https://hudi.apache.org](https://hudi.apache.org), [http://gethue.com/](http://gethue.com/), [https://iceberg.apache.org/](https://iceberg.apache.org/), [https://jupyter-enterprise-gateway.readthedocs.io/en/latest/](https://jupyter-enterprise-gateway.readthedocs.io/en/latest/), [https://jupyterhub.readthedocs.io/en/latest/#](https://jupyterhub.readthedocs.io/en/latest/#), [https://livy.incubator.apache.org/](https://livy.incubator.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [https://www.tensorflow.org/](https://www.tensorflow.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://trino.io/](https://trino.io/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-7.10.0 | emr-7.9.0 | emr-7.8.0 | emr-7.7.0 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 2.31.48, 1.12.782 | 2.31.16, 1.12.782 | 2.29.52, 1.12.780 | 2.29.44, 1.12.780 | | Python | 3.9, 3.11 | 3.9, 3.11 | 3.9, 3.11 | 3.9, 3.11 | | Scala | 2.12.18 | 2.12.18 | 2.12.18 | 2.12.18 | | AmazonCloudWatchAgent | 1.300032.2-amzn-0 | 1.300032.2-amzn-0 | 1.300032.2-amzn-0 | 1.300032.2-amzn-0 | | Delta | 3.3.0-amzn-2 | 3.3.0-amzn-1 | 3.3.0-amzn-0 | 3.2.1-amzn-2 | | Flink | 1.20.0-amzn-4 | 1.20.0-amzn-3 | 1.20.0-amzn-2 | 1.20.0-amzn-1 | | Ganglia | - | - | - | - | | HBase | 2.6.2-amzn-1 | 2.6.2-amzn-0 | 2.6.1-amzn-2 | 2.6.1-amzn-1 | | HCatalog | 3.1.3-amzn-19 | 3.1.3-amzn-18 | 3.1.3-amzn-17 | 3.1.3-amzn-16 | | Hadoop | 3.4.1-amzn-2 | 3.4.1-amzn-1 | 3.4.1-amzn-0 | 3.4.0-amzn-3 | | Hive | 3.1.3-amzn-19 | 3.1.3-amzn-18 | 3.1.3-amzn-17 | 3.1.3-amzn-16 | | Hudi | 0.15.0-amzn-7 | 0.15.0-amzn-6 | 0.15.0-amzn-5 | 0.15.0-amzn-4 | | Hue | 4.11.0 | 4.11.0 | 4.11.0 | 4.11.0 | | Iceberg | 1.8.1-amzn-0 | 1.7.1-amzn-2 | 1.7.1-amzn-1 | 1.7.1-amzn-0 | | JupyterEnterpriseGateway | 2.6.0 | 2.6.0 | 2.6.0 | 2.6.0 | | JupyterHub | 1.5.0 | 1.5.0 | 1.5.0 | 1.5.0 | | Livy | 0.8.0-incubating | 0.8.0-incubating | 0.8.0-incubating | 0.8.0-incubating | | MXNet | - | - | - | - | | Mahout | - | - | - | - | | Oozie | 5.2.1 | 5.2.1 | 5.2.1 | 5.2.1 | | Phoenix | 5.2.1 | 5.2.1 | 5.2.1 | 5.2.1 | | Pig | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | | Presto | 0.287-amzn-4 | 0.287-amzn-3 | 0.287-amzn-2 | 0.287-amzn-1 | | Spark | 3.5.5-amzn-1 | 3.5.5-amzn-0 | 3.5.4-amzn-0 | 3.5.3-amzn-1 | | Sqoop | - | - | - | - | | TensorFlow | 2.18.0 | 2.16.1 | 2.16.1 | 2.16.1 | | Tez | 0.10.2-amzn-17 | 0.10.2-amzn-16 | 0.10.2-amzn-15 | 0.10.2-amzn-14 | | Trino (PrestoSQL) | 470-amzn-0 | 467-amzn-2 | 467-amzn-1 | 467-amzn-0 | | Zeppelin | 0.11.1 | 0.11.1 | 0.11.1 | 0.11.1 | | ZooKeeper | 3.9.3-amzn-2 | 3.9.3-amzn-1 | 3.9.3-amzn-0 | 3.9.2-amzn-1 | # 7.10.0 common vulnerabilities and exposures The following table lists all CVEs that don't impact EMR clusters that run on recommended configurations of Amazon EMR 7.10.0. Amazon EMR is dependent on upstream open-source for availability of fixes and will provide the latest stable version as part of the Core Engine components within 90 days of Amazon EMR verifying the fixes. This table was updated on September 25, 2025. | CVE ID | Severity | CVE details URL | | --- | --- | --- | | CVE-2015-1832 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2015-1832](https://nvd.nist.gov/vuln/detail/CVE-2015-1832) | | CVE-2016-5018 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2016-5018](https://nvd.nist.gov/vuln/detail/CVE-2016-5018) | | CVE-2017-15095 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2017-15095](https://nvd.nist.gov/vuln/detail/CVE-2017-15095) | | CVE-2017-17485 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2017-17485](https://nvd.nist.gov/vuln/detail/CVE-2017-17485) | | CVE-2017-7525 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2017-7525](https://nvd.nist.gov/vuln/detail/CVE-2017-7525) | | CVE-2018-11307 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-11307](https://nvd.nist.gov/vuln/detail/CVE-2018-11307) | | CVE-2018-11307 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-11307](https://nvd.nist.gov/vuln/detail/CVE-2018-11307) | | CVE-2018-14718 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-14718](https://nvd.nist.gov/vuln/detail/CVE-2018-14718) | | CVE-2018-14718 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-14718](https://nvd.nist.gov/vuln/detail/CVE-2018-14718) | | CVE-2018-14719 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-14719](https://nvd.nist.gov/vuln/detail/CVE-2018-14719) | | CVE-2018-14719 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-14719](https://nvd.nist.gov/vuln/detail/CVE-2018-14719) | | CVE-2018-14720 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-14720](https://nvd.nist.gov/vuln/detail/CVE-2018-14720) | | CVE-2018-14720 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-14720](https://nvd.nist.gov/vuln/detail/CVE-2018-14720) | | CVE-2018-14721 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-14721](https://nvd.nist.gov/vuln/detail/CVE-2018-14721) | | CVE-2018-14721 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-14721](https://nvd.nist.gov/vuln/detail/CVE-2018-14721) | | CVE-2018-19360 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-19360](https://nvd.nist.gov/vuln/detail/CVE-2018-19360) | | CVE-2018-19360 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-19360](https://nvd.nist.gov/vuln/detail/CVE-2018-19360) | | CVE-2018-19361 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-19361](https://nvd.nist.gov/vuln/detail/CVE-2018-19361) | | CVE-2018-19361 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-19361](https://nvd.nist.gov/vuln/detail/CVE-2018-19361) | | CVE-2018-19362 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-19362](https://nvd.nist.gov/vuln/detail/CVE-2018-19362) | | CVE-2018-19362 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-19362](https://nvd.nist.gov/vuln/detail/CVE-2018-19362) | | CVE-2018-7489 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-7489](https://nvd.nist.gov/vuln/detail/CVE-2018-7489) | | CVE-2018-7489 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-7489](https://nvd.nist.gov/vuln/detail/CVE-2018-7489) | | CVE-2019-10202 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-10202](https://nvd.nist.gov/vuln/detail/CVE-2019-10202) | | CVE-2019-14379 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-14379](https://nvd.nist.gov/vuln/detail/CVE-2019-14379) | | CVE-2019-14379 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-14379](https://nvd.nist.gov/vuln/detail/CVE-2019-14379) | | CVE-2019-14540 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-14540](https://nvd.nist.gov/vuln/detail/CVE-2019-14540) | | CVE-2019-14540 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-14540](https://nvd.nist.gov/vuln/detail/CVE-2019-14540) | | CVE-2019-14892 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-14892](https://nvd.nist.gov/vuln/detail/CVE-2019-14892) | | CVE-2019-14892 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-14892](https://nvd.nist.gov/vuln/detail/CVE-2019-14892) | | CVE-2019-14893 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-14893](https://nvd.nist.gov/vuln/detail/CVE-2019-14893) | | CVE-2019-14893 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-14893](https://nvd.nist.gov/vuln/detail/CVE-2019-14893) | | CVE-2019-16335 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-16335](https://nvd.nist.gov/vuln/detail/CVE-2019-16335) | | CVE-2019-16335 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-16335](https://nvd.nist.gov/vuln/detail/CVE-2019-16335) | | CVE-2019-16942 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-16942](https://nvd.nist.gov/vuln/detail/CVE-2019-16942) | | CVE-2019-16942 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-16942](https://nvd.nist.gov/vuln/detail/CVE-2019-16942) | | CVE-2019-16943 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-16943](https://nvd.nist.gov/vuln/detail/CVE-2019-16943) | | CVE-2019-16943 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-16943](https://nvd.nist.gov/vuln/detail/CVE-2019-16943) | | CVE-2019-17267 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-17267](https://nvd.nist.gov/vuln/detail/CVE-2019-17267) | | CVE-2019-17267 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-17267](https://nvd.nist.gov/vuln/detail/CVE-2019-17267) | | CVE-2019-17531 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-17531](https://nvd.nist.gov/vuln/detail/CVE-2019-17531) | | CVE-2019-17531 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-17531](https://nvd.nist.gov/vuln/detail/CVE-2019-17531) | | CVE-2019-20330 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-20330](https://nvd.nist.gov/vuln/detail/CVE-2019-20330) | | CVE-2019-20330 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-20330](https://nvd.nist.gov/vuln/detail/CVE-2019-20330) | | CVE-2020-8840 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2020-8840](https://nvd.nist.gov/vuln/detail/CVE-2020-8840) | | CVE-2020-8840 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2020-8840](https://nvd.nist.gov/vuln/detail/CVE-2020-8840) | | CVE-2020-9546 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2020-9546](https://nvd.nist.gov/vuln/detail/CVE-2020-9546) | | CVE-2020-9546 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2020-9546](https://nvd.nist.gov/vuln/detail/CVE-2020-9546) | | CVE-2020-9547 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2020-9547](https://nvd.nist.gov/vuln/detail/CVE-2020-9547) | | CVE-2020-9547 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2020-9547](https://nvd.nist.gov/vuln/detail/CVE-2020-9547) | | CVE-2020-9548 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2020-9548](https://nvd.nist.gov/vuln/detail/CVE-2020-9548) | | CVE-2020-9548 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2020-9548](https://nvd.nist.gov/vuln/detail/CVE-2020-9548) | | CVE-2022-1471 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2022-1471](https://nvd.nist.gov/vuln/detail/CVE-2022-1471) | | CVE-2022-1471 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2022-1471](https://nvd.nist.gov/vuln/detail/CVE-2022-1471) | | CVE-2022-37865 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2022-37865](https://nvd.nist.gov/vuln/detail/CVE-2022-37865) | | CVE-2022-39135 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2022-39135](https://nvd.nist.gov/vuln/detail/CVE-2022-39135) | | CVE-2022-42889 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2022-42889](https://nvd.nist.gov/vuln/detail/CVE-2022-42889) | | CVE-2022-46337 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2022-46337](https://nvd.nist.gov/vuln/detail/CVE-2022-46337) | | CVE-2022-46337 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2022-46337](https://nvd.nist.gov/vuln/detail/CVE-2022-46337) | | CVE-2024-51504 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2024-51504](https://nvd.nist.gov/vuln/detail/CVE-2024-51504) | | CVE-2024-51504 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2024-51504](https://nvd.nist.gov/vuln/detail/CVE-2024-51504) | | CVE-2024-52046 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2024-52046](https://nvd.nist.gov/vuln/detail/CVE-2024-52046) | | CVE-2024-52046 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2024-52046](https://nvd.nist.gov/vuln/detail/CVE-2024-52046) | | CVE-2024-52577 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2024-52577](https://nvd.nist.gov/vuln/detail/CVE-2024-52577) | | CVE-2024-53990 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2024-53990](https://nvd.nist.gov/vuln/detail/CVE-2024-53990) | | CVE-2024-56325 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2024-56325](https://nvd.nist.gov/vuln/detail/CVE-2024-56325) | | CVE-2025-30065 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2025-30065](https://nvd.nist.gov/vuln/detail/CVE-2025-30065) | | CVE-2014-3643 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2014-3643](https://nvd.nist.gov/vuln/detail/CVE-2014-3643) | | CVE-2015-5237 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2015-5237](https://nvd.nist.gov/vuln/detail/CVE-2015-5237) | | CVE-2015-5237 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2015-5237](https://nvd.nist.gov/vuln/detail/CVE-2015-5237) | | CVE-2016-6796 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2016-6796](https://nvd.nist.gov/vuln/detail/CVE-2016-6796) | | CVE-2018-1000180 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2018-1000180](https://nvd.nist.gov/vuln/detail/CVE-2018-1000180) | | CVE-2018-1000180 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2018-1000180](https://nvd.nist.gov/vuln/detail/CVE-2018-1000180) | | CVE-2018-12022 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2018-12022](https://nvd.nist.gov/vuln/detail/CVE-2018-12022) | | CVE-2018-12022 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2018-12022](https://nvd.nist.gov/vuln/detail/CVE-2018-12022) | | CVE-2018-12023 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2018-12023](https://nvd.nist.gov/vuln/detail/CVE-2018-12023) | | CVE-2018-12023 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2018-12023](https://nvd.nist.gov/vuln/detail/CVE-2018-12023) | | CVE-2018-5968 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2018-5968](https://nvd.nist.gov/vuln/detail/CVE-2018-5968) | | CVE-2019-0205 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2019-0205](https://nvd.nist.gov/vuln/detail/CVE-2019-0205) | | CVE-2019-0205 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2019-0205](https://nvd.nist.gov/vuln/detail/CVE-2019-0205) | | CVE-2019-10172 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2019-10172](https://nvd.nist.gov/vuln/detail/CVE-2019-10172) | | CVE-2019-12086 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2019-12086](https://nvd.nist.gov/vuln/detail/CVE-2019-12086) | | CVE-2019-12086 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2019-12086](https://nvd.nist.gov/vuln/detail/CVE-2019-12086) | | CVE-2019-14439 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2019-14439](https://nvd.nist.gov/vuln/detail/CVE-2019-14439) | | CVE-2019-14439 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2019-14439](https://nvd.nist.gov/vuln/detail/CVE-2019-14439) | | CVE-2020-10650 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-10650](https://nvd.nist.gov/vuln/detail/CVE-2020-10650) | | CVE-2020-10650 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-10650](https://nvd.nist.gov/vuln/detail/CVE-2020-10650) | | CVE-2020-10672 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-10672](https://nvd.nist.gov/vuln/detail/CVE-2020-10672) | | CVE-2020-10672 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-10672](https://nvd.nist.gov/vuln/detail/CVE-2020-10672) | | CVE-2020-10673 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-10673](https://nvd.nist.gov/vuln/detail/CVE-2020-10673) | | CVE-2020-10673 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-10673](https://nvd.nist.gov/vuln/detail/CVE-2020-10673) | | CVE-2020-10968 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-10968](https://nvd.nist.gov/vuln/detail/CVE-2020-10968) | | CVE-2020-10968 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-10968](https://nvd.nist.gov/vuln/detail/CVE-2020-10968) | | CVE-2020-10969 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-10969](https://nvd.nist.gov/vuln/detail/CVE-2020-10969) | | CVE-2020-10969 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-10969](https://nvd.nist.gov/vuln/detail/CVE-2020-10969) | | CVE-2020-11111 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-11111](https://nvd.nist.gov/vuln/detail/CVE-2020-11111) | | CVE-2020-11111 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-11111](https://nvd.nist.gov/vuln/detail/CVE-2020-11111) | | CVE-2020-11112 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-11112](https://nvd.nist.gov/vuln/detail/CVE-2020-11112) | | CVE-2020-11112 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-11112](https://nvd.nist.gov/vuln/detail/CVE-2020-11112) | | CVE-2020-11113 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-11113](https://nvd.nist.gov/vuln/detail/CVE-2020-11113) | | CVE-2020-11113 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-11113](https://nvd.nist.gov/vuln/detail/CVE-2020-11113) | | CVE-2020-11619 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-11619](https://nvd.nist.gov/vuln/detail/CVE-2020-11619) | | CVE-2020-11619 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-11619](https://nvd.nist.gov/vuln/detail/CVE-2020-11619) | | CVE-2020-11620 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-11620](https://nvd.nist.gov/vuln/detail/CVE-2020-11620) | | CVE-2020-11620 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-11620](https://nvd.nist.gov/vuln/detail/CVE-2020-11620) | | CVE-2020-13949 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-13949](https://nvd.nist.gov/vuln/detail/CVE-2020-13949) | | CVE-2020-13949 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-13949](https://nvd.nist.gov/vuln/detail/CVE-2020-13949) | | CVE-2020-13949 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-13949](https://nvd.nist.gov/vuln/detail/CVE-2020-13949) | | CVE-2020-14060 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-14060](https://nvd.nist.gov/vuln/detail/CVE-2020-14060) | | CVE-2020-14060 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-14060](https://nvd.nist.gov/vuln/detail/CVE-2020-14060) | | CVE-2020-14061 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-14061](https://nvd.nist.gov/vuln/detail/CVE-2020-14061) | | CVE-2020-14061 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-14061](https://nvd.nist.gov/vuln/detail/CVE-2020-14061) | | CVE-2020-14062 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-14062](https://nvd.nist.gov/vuln/detail/CVE-2020-14062) | | CVE-2020-14062 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-14062](https://nvd.nist.gov/vuln/detail/CVE-2020-14062) | | CVE-2020-14195 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-14195](https://nvd.nist.gov/vuln/detail/CVE-2020-14195) | | CVE-2020-14195 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-14195](https://nvd.nist.gov/vuln/detail/CVE-2020-14195) | | CVE-2020-24616 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-24616](https://nvd.nist.gov/vuln/detail/CVE-2020-24616) | | CVE-2020-24616 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-24616](https://nvd.nist.gov/vuln/detail/CVE-2020-24616) | | CVE-2020-24750 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-24750](https://nvd.nist.gov/vuln/detail/CVE-2020-24750) | | CVE-2020-24750 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-24750](https://nvd.nist.gov/vuln/detail/CVE-2020-24750) | | CVE-2020-25649 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-25649](https://nvd.nist.gov/vuln/detail/CVE-2020-25649) | | CVE-2020-28052 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-28052](https://nvd.nist.gov/vuln/detail/CVE-2020-28052) | | CVE-2020-35490 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-35490](https://nvd.nist.gov/vuln/detail/CVE-2020-35490) | | CVE-2020-35490 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-35490](https://nvd.nist.gov/vuln/detail/CVE-2020-35490) | | CVE-2020-35491 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-35491](https://nvd.nist.gov/vuln/detail/CVE-2020-35491) | | CVE-2020-35491 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-35491](https://nvd.nist.gov/vuln/detail/CVE-2020-35491) | | CVE-2020-35728 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-35728](https://nvd.nist.gov/vuln/detail/CVE-2020-35728) | | CVE-2020-35728 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-35728](https://nvd.nist.gov/vuln/detail/CVE-2020-35728) | | CVE-2020-36179 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36179](https://nvd.nist.gov/vuln/detail/CVE-2020-36179) | | CVE-2020-36179 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36179](https://nvd.nist.gov/vuln/detail/CVE-2020-36179) | | CVE-2020-36180 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36180](https://nvd.nist.gov/vuln/detail/CVE-2020-36180) | | CVE-2020-36180 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36180](https://nvd.nist.gov/vuln/detail/CVE-2020-36180) | | CVE-2020-36181 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36181](https://nvd.nist.gov/vuln/detail/CVE-2020-36181) | | CVE-2020-36181 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36181](https://nvd.nist.gov/vuln/detail/CVE-2020-36181) | | CVE-2020-36182 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36182](https://nvd.nist.gov/vuln/detail/CVE-2020-36182) | | CVE-2020-36182 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36182](https://nvd.nist.gov/vuln/detail/CVE-2020-36182) | | CVE-2020-36183 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36183](https://nvd.nist.gov/vuln/detail/CVE-2020-36183) | | CVE-2020-36183 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36183](https://nvd.nist.gov/vuln/detail/CVE-2020-36183) | | CVE-2020-36184 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36184](https://nvd.nist.gov/vuln/detail/CVE-2020-36184) | | CVE-2020-36184 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36184](https://nvd.nist.gov/vuln/detail/CVE-2020-36184) | | CVE-2020-36185 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36185](https://nvd.nist.gov/vuln/detail/CVE-2020-36185) | | CVE-2020-36185 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36185](https://nvd.nist.gov/vuln/detail/CVE-2020-36185) | | CVE-2020-36186 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36186](https://nvd.nist.gov/vuln/detail/CVE-2020-36186) | | CVE-2020-36186 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36186](https://nvd.nist.gov/vuln/detail/CVE-2020-36186) | | CVE-2020-36187 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36187](https://nvd.nist.gov/vuln/detail/CVE-2020-36187) | | CVE-2020-36187 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36187](https://nvd.nist.gov/vuln/detail/CVE-2020-36187) | | CVE-2020-36188 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36188](https://nvd.nist.gov/vuln/detail/CVE-2020-36188) | | CVE-2020-36188 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36188](https://nvd.nist.gov/vuln/detail/CVE-2020-36188) | | CVE-2020-36189 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36189](https://nvd.nist.gov/vuln/detail/CVE-2020-36189) | | CVE-2020-36189 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36189](https://nvd.nist.gov/vuln/detail/CVE-2020-36189) | | CVE-2020-36518 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36518](https://nvd.nist.gov/vuln/detail/CVE-2020-36518) | | CVE-2020-36518 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36518](https://nvd.nist.gov/vuln/detail/CVE-2020-36518) | | CVE-2021-20190 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-20190](https://nvd.nist.gov/vuln/detail/CVE-2021-20190) | | CVE-2021-20190 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-20190](https://nvd.nist.gov/vuln/detail/CVE-2021-20190) | | CVE-2021-31684 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-31684](https://nvd.nist.gov/vuln/detail/CVE-2021-31684) | | CVE-2021-34538 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-34538](https://nvd.nist.gov/vuln/detail/CVE-2021-34538) | | CVE-2021-34538 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-34538](https://nvd.nist.gov/vuln/detail/CVE-2021-34538) | | CVE-2021-34538 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-34538](https://nvd.nist.gov/vuln/detail/CVE-2021-34538) | | CVE-2022-25647 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-25647](https://nvd.nist.gov/vuln/detail/CVE-2022-25647) | | CVE-2022-25647 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-25647](https://nvd.nist.gov/vuln/detail/CVE-2022-25647) | | CVE-2022-25647 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-25647](https://nvd.nist.gov/vuln/detail/CVE-2022-25647) | | CVE-2022-25857 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-25857](https://nvd.nist.gov/vuln/detail/CVE-2022-25857) | | CVE-2022-3171 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-3171](https://nvd.nist.gov/vuln/detail/CVE-2022-3171) | | CVE-2022-3171 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-3171](https://nvd.nist.gov/vuln/detail/CVE-2022-3171) | | CVE-2022-3509 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-3509](https://nvd.nist.gov/vuln/detail/CVE-2022-3509) | | CVE-2022-3509 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-3509](https://nvd.nist.gov/vuln/detail/CVE-2022-3509) | | CVE-2022-3510 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-3510](https://nvd.nist.gov/vuln/detail/CVE-2022-3510) | | CVE-2022-36364 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-36364](https://nvd.nist.gov/vuln/detail/CVE-2022-36364) | | CVE-2022-37866 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-37866](https://nvd.nist.gov/vuln/detail/CVE-2022-37866) | | CVE-2022-40149 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-40149](https://nvd.nist.gov/vuln/detail/CVE-2022-40149) | | CVE-2022-40149 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-40149](https://nvd.nist.gov/vuln/detail/CVE-2022-40149) | | CVE-2022-40150 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-40150](https://nvd.nist.gov/vuln/detail/CVE-2022-40150) | | CVE-2022-40150 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-40150](https://nvd.nist.gov/vuln/detail/CVE-2022-40150) | | CVE-2022-41404 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-41404](https://nvd.nist.gov/vuln/detail/CVE-2022-41404) | | CVE-2022-42003 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-42003](https://nvd.nist.gov/vuln/detail/CVE-2022-42003) | | CVE-2022-42003 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-42003](https://nvd.nist.gov/vuln/detail/CVE-2022-42003) | | CVE-2022-42003 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-42003](https://nvd.nist.gov/vuln/detail/CVE-2022-42003) | | CVE-2022-42003 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-42003](https://nvd.nist.gov/vuln/detail/CVE-2022-42003) | | CVE-2022-42003 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-42003](https://nvd.nist.gov/vuln/detail/CVE-2022-42003) | | CVE-2022-42004 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-42004](https://nvd.nist.gov/vuln/detail/CVE-2022-42004) | | CVE-2022-42004 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-42004](https://nvd.nist.gov/vuln/detail/CVE-2022-42004) | | CVE-2022-42004 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-42004](https://nvd.nist.gov/vuln/detail/CVE-2022-42004) | | CVE-2022-42004 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-42004](https://nvd.nist.gov/vuln/detail/CVE-2022-42004) | | CVE-2022-45685 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-45685](https://nvd.nist.gov/vuln/detail/CVE-2022-45685) | | CVE-2022-45685 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-45685](https://nvd.nist.gov/vuln/detail/CVE-2022-45685) | | CVE-2022-45693 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-45693](https://nvd.nist.gov/vuln/detail/CVE-2022-45693) | | CVE-2022-45693 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-45693](https://nvd.nist.gov/vuln/detail/CVE-2022-45693) | | CVE-2022-46751 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-46751](https://nvd.nist.gov/vuln/detail/CVE-2022-46751) | | CVE-2022-46751 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-46751](https://nvd.nist.gov/vuln/detail/CVE-2022-46751) | | CVE-2023-1370 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-1370](https://nvd.nist.gov/vuln/detail/CVE-2023-1370) | | CVE-2023-1436 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-1436](https://nvd.nist.gov/vuln/detail/CVE-2023-1436) | | CVE-2023-1436 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-1436](https://nvd.nist.gov/vuln/detail/CVE-2023-1436) | | CVE-2023-2976 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-2976](https://nvd.nist.gov/vuln/detail/CVE-2023-2976) | | CVE-2023-2976 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-2976](https://nvd.nist.gov/vuln/detail/CVE-2023-2976) | | CVE-2023-2976 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-2976](https://nvd.nist.gov/vuln/detail/CVE-2023-2976) | | CVE-2023-2976 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-2976](https://nvd.nist.gov/vuln/detail/CVE-2023-2976) | | CVE-2023-2976 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-2976](https://nvd.nist.gov/vuln/detail/CVE-2023-2976) | | CVE-2023-2976 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-2976](https://nvd.nist.gov/vuln/detail/CVE-2023-2976) | | CVE-2023-2976 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-2976](https://nvd.nist.gov/vuln/detail/CVE-2023-2976) | | CVE-2023-2976 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-2976](https://nvd.nist.gov/vuln/detail/CVE-2023-2976) | | CVE-2023-2976 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-2976](https://nvd.nist.gov/vuln/detail/CVE-2023-2976) | | CVE-2023-2976 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-2976](https://nvd.nist.gov/vuln/detail/CVE-2023-2976) | | CVE-2023-39410 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-39410](https://nvd.nist.gov/vuln/detail/CVE-2023-39410) | | CVE-2023-6378 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-6378](https://nvd.nist.gov/vuln/detail/CVE-2023-6378) | | CVE-2023-6378 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-6378](https://nvd.nist.gov/vuln/detail/CVE-2023-6378) | | CVE-2023-6481 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-6481](https://nvd.nist.gov/vuln/detail/CVE-2023-6481) | | CVE-2023-6481 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-6481](https://nvd.nist.gov/vuln/detail/CVE-2023-6481) | | CVE-2024-13009 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-13009](https://nvd.nist.gov/vuln/detail/CVE-2024-13009) | | CVE-2024-13009 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-13009](https://nvd.nist.gov/vuln/detail/CVE-2024-13009) | | CVE-2024-29131 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-29131](https://nvd.nist.gov/vuln/detail/CVE-2024-29131) | | CVE-2024-29857 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-29857](https://nvd.nist.gov/vuln/detail/CVE-2024-29857) | | CVE-2024-29857 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-29857](https://nvd.nist.gov/vuln/detail/CVE-2024-29857) | | CVE-2024-29857 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-29857](https://nvd.nist.gov/vuln/detail/CVE-2024-29857) | | CVE-2024-29857 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-29857](https://nvd.nist.gov/vuln/detail/CVE-2024-29857) | | CVE-2024-30172 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-30172](https://nvd.nist.gov/vuln/detail/CVE-2024-30172) | | CVE-2024-30172 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-30172](https://nvd.nist.gov/vuln/detail/CVE-2024-30172) | | CVE-2024-34447 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-34447](https://nvd.nist.gov/vuln/detail/CVE-2024-34447) | | CVE-2024-34447 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-34447](https://nvd.nist.gov/vuln/detail/CVE-2024-34447) | | CVE-2024-36114 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-36114](https://nvd.nist.gov/vuln/detail/CVE-2024-36114) | | CVE-2024-36114 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-36114](https://nvd.nist.gov/vuln/detail/CVE-2024-36114) | | CVE-2024-47561 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-47561](https://nvd.nist.gov/vuln/detail/CVE-2024-47561) | | CVE-2024-57699 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-57699](https://nvd.nist.gov/vuln/detail/CVE-2024-57699) | | CVE-2024-7254 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-7254](https://nvd.nist.gov/vuln/detail/CVE-2024-7254) | | CVE-2024-7254 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-7254](https://nvd.nist.gov/vuln/detail/CVE-2024-7254) | | CVE-2024-7254 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-7254](https://nvd.nist.gov/vuln/detail/CVE-2024-7254) | | CVE-2024-7254 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-7254](https://nvd.nist.gov/vuln/detail/CVE-2024-7254) | | CVE-2024-7254 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-7254](https://nvd.nist.gov/vuln/detail/CVE-2024-7254) | | CVE-2024-7254 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-7254](https://nvd.nist.gov/vuln/detail/CVE-2024-7254) | | CVE-2024-7254 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-7254](https://nvd.nist.gov/vuln/detail/CVE-2024-7254) | | CVE-2024-9823 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-9823](https://nvd.nist.gov/vuln/detail/CVE-2024-9823) | | CVE-2025-1948 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2025-1948](https://nvd.nist.gov/vuln/detail/CVE-2025-1948) | | CVE-2025-24970 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2025-24970](https://nvd.nist.gov/vuln/detail/CVE-2025-24970) | | CVE-2025-24970 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2025-24970](https://nvd.nist.gov/vuln/detail/CVE-2025-24970) | | CVE-2025-27553 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2025-27553](https://nvd.nist.gov/vuln/detail/CVE-2025-27553) | | CVE-2025-27817 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2025-27817](https://nvd.nist.gov/vuln/detail/CVE-2025-27817) | | CVE-2025-27817 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2025-27817](https://nvd.nist.gov/vuln/detail/CVE-2025-27817) | | CVE-2025-27817 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2025-27817](https://nvd.nist.gov/vuln/detail/CVE-2025-27817) | | CVE-2025-27820 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2025-27820](https://nvd.nist.gov/vuln/detail/CVE-2025-27820) | | CVE-2025-27820 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2025-27820](https://nvd.nist.gov/vuln/detail/CVE-2025-27820) | | CVE-2025-30706 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2025-30706](https://nvd.nist.gov/vuln/detail/CVE-2025-30706) | | CVE-2025-30706 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2025-30706](https://nvd.nist.gov/vuln/detail/CVE-2025-30706) | | CVE-2025-46762 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2025-46762](https://nvd.nist.gov/vuln/detail/CVE-2025-46762) | | CVE-2025-48734 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2025-48734](https://nvd.nist.gov/vuln/detail/CVE-2025-48734) | | CVE-2025-48734 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2025-48734](https://nvd.nist.gov/vuln/detail/CVE-2025-48734) | | CVE-2025-49146 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2025-49146](https://nvd.nist.gov/vuln/detail/CVE-2025-49146) | | CVE-2025-5115 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2025-5115](https://nvd.nist.gov/vuln/detail/CVE-2025-5115) | | CVE-2025-52434 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2025-52434](https://nvd.nist.gov/vuln/detail/CVE-2025-52434) | | CVE-2025-52999 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2025-52999](https://nvd.nist.gov/vuln/detail/CVE-2025-52999) | | CVE-2025-52999 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2025-52999](https://nvd.nist.gov/vuln/detail/CVE-2025-52999) | | CVE-2025-52999 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2025-52999](https://nvd.nist.gov/vuln/detail/CVE-2025-52999) | | CVE-2025-52999 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2025-52999](https://nvd.nist.gov/vuln/detail/CVE-2025-52999) | | CVE-2025-52999 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2025-52999](https://nvd.nist.gov/vuln/detail/CVE-2025-52999) | | CVE-2025-52999 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2025-52999](https://nvd.nist.gov/vuln/detail/CVE-2025-52999) | | CVE-2025-55163 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2025-55163](https://nvd.nist.gov/vuln/detail/CVE-2025-55163) | | CVE-2025-55163 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2025-55163](https://nvd.nist.gov/vuln/detail/CVE-2025-55163) | | CVE-2025-58056 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2025-58056](https://nvd.nist.gov/vuln/detail/CVE-2025-58056) | | CVE-2025-58056 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2025-58056](https://nvd.nist.gov/vuln/detail/CVE-2025-58056) | | CVE-2025-58057 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2025-58057](https://nvd.nist.gov/vuln/detail/CVE-2025-58057) | | CVE-2025-58057 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2025-58057](https://nvd.nist.gov/vuln/detail/CVE-2025-58057) | | CVE-2025-58057 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2025-58057](https://nvd.nist.gov/vuln/detail/CVE-2025-58057) | | CVE-2025-58057 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2025-58057](https://nvd.nist.gov/vuln/detail/CVE-2025-58057) | | CVE-2012-5783 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2012-5783](https://nvd.nist.gov/vuln/detail/CVE-2012-5783) | | CVE-2012-5783 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2012-5783](https://nvd.nist.gov/vuln/detail/CVE-2012-5783) | | CVE-2012-6153 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2012-6153](https://nvd.nist.gov/vuln/detail/CVE-2012-6153) | | CVE-2012-6153 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2012-6153](https://nvd.nist.gov/vuln/detail/CVE-2012-6153) | | CVE-2018-10237 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2018-10237](https://nvd.nist.gov/vuln/detail/CVE-2018-10237) | | CVE-2018-10237 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2018-10237](https://nvd.nist.gov/vuln/detail/CVE-2018-10237) | | CVE-2018-10237 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2018-10237](https://nvd.nist.gov/vuln/detail/CVE-2018-10237) | | CVE-2018-10237 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2018-10237](https://nvd.nist.gov/vuln/detail/CVE-2018-10237) | | CVE-2018-10237 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2018-10237](https://nvd.nist.gov/vuln/detail/CVE-2018-10237) | | CVE-2018-11798 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2018-11798](https://nvd.nist.gov/vuln/detail/CVE-2018-11798) | | CVE-2018-1313 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2018-1313](https://nvd.nist.gov/vuln/detail/CVE-2018-1313) | | CVE-2019-12384 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2019-12384](https://nvd.nist.gov/vuln/detail/CVE-2019-12384) | | CVE-2019-12384 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2019-12384](https://nvd.nist.gov/vuln/detail/CVE-2019-12384) | | CVE-2019-12814 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2019-12814](https://nvd.nist.gov/vuln/detail/CVE-2019-12814) | | CVE-2019-12814 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2019-12814](https://nvd.nist.gov/vuln/detail/CVE-2019-12814) | | CVE-2020-13955 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2020-13955](https://nvd.nist.gov/vuln/detail/CVE-2020-13955) | | CVE-2020-13956 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2020-13956](https://nvd.nist.gov/vuln/detail/CVE-2020-13956) | | CVE-2020-15250 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2020-15250](https://nvd.nist.gov/vuln/detail/CVE-2020-15250) | | CVE-2020-15250 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2020-15250](https://nvd.nist.gov/vuln/detail/CVE-2020-15250) | | CVE-2020-15522 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2020-15522](https://nvd.nist.gov/vuln/detail/CVE-2020-15522) | | CVE-2020-15522 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2020-15522](https://nvd.nist.gov/vuln/detail/CVE-2020-15522) | | CVE-2020-15522 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2020-15522](https://nvd.nist.gov/vuln/detail/CVE-2020-15522) | | CVE-2020-17521 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2020-17521](https://nvd.nist.gov/vuln/detail/CVE-2020-17521) | | CVE-2020-17521 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2020-17521](https://nvd.nist.gov/vuln/detail/CVE-2020-17521) | | CVE-2020-1945 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2020-1945](https://nvd.nist.gov/vuln/detail/CVE-2020-1945) | | CVE-2020-26939 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2020-26939](https://nvd.nist.gov/vuln/detail/CVE-2020-26939) | | CVE-2020-26939 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2020-26939](https://nvd.nist.gov/vuln/detail/CVE-2020-26939) | | CVE-2020-26939 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2020-26939](https://nvd.nist.gov/vuln/detail/CVE-2020-26939) | | CVE-2021-22569 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2021-22569](https://nvd.nist.gov/vuln/detail/CVE-2021-22569) | | CVE-2021-22569 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2021-22569](https://nvd.nist.gov/vuln/detail/CVE-2021-22569) | | CVE-2021-27568 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2021-27568](https://nvd.nist.gov/vuln/detail/CVE-2021-27568) | | CVE-2021-28170 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2021-28170](https://nvd.nist.gov/vuln/detail/CVE-2021-28170) | | CVE-2021-29425 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2021-29425](https://nvd.nist.gov/vuln/detail/CVE-2021-29425) | | CVE-2021-36373 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2021-36373](https://nvd.nist.gov/vuln/detail/CVE-2021-36373) | | CVE-2021-36374 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2021-36374](https://nvd.nist.gov/vuln/detail/CVE-2021-36374) | | CVE-2022-38749 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2022-38749](https://nvd.nist.gov/vuln/detail/CVE-2022-38749) | | CVE-2022-38750 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2022-38750](https://nvd.nist.gov/vuln/detail/CVE-2022-38750) | | CVE-2022-38751 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2022-38751](https://nvd.nist.gov/vuln/detail/CVE-2022-38751) | | CVE-2022-38752 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2022-38752](https://nvd.nist.gov/vuln/detail/CVE-2022-38752) | | CVE-2022-41854 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2022-41854](https://nvd.nist.gov/vuln/detail/CVE-2022-41854) | | CVE-2023-0833 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2023-0833](https://nvd.nist.gov/vuln/detail/CVE-2023-0833) | | CVE-2023-33201 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2023-33201](https://nvd.nist.gov/vuln/detail/CVE-2023-33201) | | CVE-2023-33201 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2023-33201](https://nvd.nist.gov/vuln/detail/CVE-2023-33201) | | CVE-2023-33201 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2023-33201](https://nvd.nist.gov/vuln/detail/CVE-2023-33201) | | CVE-2023-33201 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2023-33201](https://nvd.nist.gov/vuln/detail/CVE-2023-33201) | | CVE-2023-33202 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2023-33202](https://nvd.nist.gov/vuln/detail/CVE-2023-33202) | | CVE-2023-33202 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2023-33202](https://nvd.nist.gov/vuln/detail/CVE-2023-33202) | | CVE-2023-33202 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2023-33202](https://nvd.nist.gov/vuln/detail/CVE-2023-33202) | | CVE-2023-33202 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2023-33202](https://nvd.nist.gov/vuln/detail/CVE-2023-33202) | | CVE-2023-42503 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2023-42503](https://nvd.nist.gov/vuln/detail/CVE-2023-42503) | | CVE-2023-51074 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2023-51074](https://nvd.nist.gov/vuln/detail/CVE-2023-51074) | | CVE-2024-12798 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-12798](https://nvd.nist.gov/vuln/detail/CVE-2024-12798) | | CVE-2024-12798 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-12798](https://nvd.nist.gov/vuln/detail/CVE-2024-12798) | | CVE-2024-12798 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-12798](https://nvd.nist.gov/vuln/detail/CVE-2024-12798) | | CVE-2024-12798 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-12798](https://nvd.nist.gov/vuln/detail/CVE-2024-12798) | | CVE-2024-12798 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-12798](https://nvd.nist.gov/vuln/detail/CVE-2024-12798) | | CVE-2024-12798 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-12798](https://nvd.nist.gov/vuln/detail/CVE-2024-12798) | | CVE-2024-23944 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-23944](https://nvd.nist.gov/vuln/detail/CVE-2024-23944) | | CVE-2024-23945 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-23945](https://nvd.nist.gov/vuln/detail/CVE-2024-23945) | | CVE-2024-23945 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-23945](https://nvd.nist.gov/vuln/detail/CVE-2024-23945) | | CVE-2024-23953 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-23953](https://nvd.nist.gov/vuln/detail/CVE-2024-23953) | | CVE-2024-23953 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-23953](https://nvd.nist.gov/vuln/detail/CVE-2024-23953) | | CVE-2024-25710 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-25710](https://nvd.nist.gov/vuln/detail/CVE-2024-25710) | | CVE-2024-25710 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-25710](https://nvd.nist.gov/vuln/detail/CVE-2024-25710) | | CVE-2024-26308 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-26308](https://nvd.nist.gov/vuln/detail/CVE-2024-26308) | | CVE-2024-26308 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-26308](https://nvd.nist.gov/vuln/detail/CVE-2024-26308) | | CVE-2024-29025 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-29025](https://nvd.nist.gov/vuln/detail/CVE-2024-29025) | | CVE-2024-29133 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-29133](https://nvd.nist.gov/vuln/detail/CVE-2024-29133) | | CVE-2024-29869 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-29869](https://nvd.nist.gov/vuln/detail/CVE-2024-29869) | | CVE-2024-29869 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-29869](https://nvd.nist.gov/vuln/detail/CVE-2024-29869) | | CVE-2024-30171 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-30171](https://nvd.nist.gov/vuln/detail/CVE-2024-30171) | | CVE-2024-30171 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-30171](https://nvd.nist.gov/vuln/detail/CVE-2024-30171) | | CVE-2024-30171 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-30171](https://nvd.nist.gov/vuln/detail/CVE-2024-30171) | | CVE-2024-30171 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-30171](https://nvd.nist.gov/vuln/detail/CVE-2024-30171) | | CVE-2024-31141 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-31141](https://nvd.nist.gov/vuln/detail/CVE-2024-31141) | | CVE-2024-47535 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-47535](https://nvd.nist.gov/vuln/detail/CVE-2024-47535) | | CVE-2024-47535 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-47535](https://nvd.nist.gov/vuln/detail/CVE-2024-47535) | | CVE-2024-47554 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-47554](https://nvd.nist.gov/vuln/detail/CVE-2024-47554) | | CVE-2024-47554 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-47554](https://nvd.nist.gov/vuln/detail/CVE-2024-47554) | | CVE-2024-47554 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-47554](https://nvd.nist.gov/vuln/detail/CVE-2024-47554) | | CVE-2024-47554 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-47554](https://nvd.nist.gov/vuln/detail/CVE-2024-47554) | | CVE-2024-52980 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-52980](https://nvd.nist.gov/vuln/detail/CVE-2024-52980) | | CVE-2024-56128 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-56128](https://nvd.nist.gov/vuln/detail/CVE-2024-56128) | | CVE-2024-6763 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-6763](https://nvd.nist.gov/vuln/detail/CVE-2024-6763) | | CVE-2024-6763 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-6763](https://nvd.nist.gov/vuln/detail/CVE-2024-6763) | | CVE-2024-6763 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-6763](https://nvd.nist.gov/vuln/detail/CVE-2024-6763) | | CVE-2024-6763 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-6763](https://nvd.nist.gov/vuln/detail/CVE-2024-6763) | | CVE-2024-6763 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-6763](https://nvd.nist.gov/vuln/detail/CVE-2024-6763) | | CVE-2024-8184 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-8184](https://nvd.nist.gov/vuln/detail/CVE-2024-8184) | | CVE-2025-22227 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2025-22227](https://nvd.nist.gov/vuln/detail/CVE-2025-22227) | | CVE-2025-25193 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2025-25193](https://nvd.nist.gov/vuln/detail/CVE-2025-25193) | | CVE-2025-25193 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2025-25193](https://nvd.nist.gov/vuln/detail/CVE-2025-25193) | | CVE-2025-25193 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2025-25193](https://nvd.nist.gov/vuln/detail/CVE-2025-25193) | | CVE-2025-30474 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2025-30474](https://nvd.nist.gov/vuln/detail/CVE-2025-30474) | | CVE-2025-3588 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2025-3588](https://nvd.nist.gov/vuln/detail/CVE-2025-3588) | | CVE-2025-46392 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2025-46392](https://nvd.nist.gov/vuln/detail/CVE-2025-46392) | | CVE-2025-46551 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2025-46551](https://nvd.nist.gov/vuln/detail/CVE-2025-46551) | | CVE-2025-48924 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2025-48924](https://nvd.nist.gov/vuln/detail/CVE-2025-48924) | | CVE-2025-48924 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2025-48924](https://nvd.nist.gov/vuln/detail/CVE-2025-48924) | | CVE-2025-48924 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2025-48924](https://nvd.nist.gov/vuln/detail/CVE-2025-48924) | | CVE-2025-48924 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2025-48924](https://nvd.nist.gov/vuln/detail/CVE-2025-48924) | | CVE-2025-48924 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2025-48924](https://nvd.nist.gov/vuln/detail/CVE-2025-48924) | | CVE-2025-48924 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2025-48924](https://nvd.nist.gov/vuln/detail/CVE-2025-48924) | | CVE-2025-48924 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2025-48924](https://nvd.nist.gov/vuln/detail/CVE-2025-48924) | | CVE-2025-48924 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2025-48924](https://nvd.nist.gov/vuln/detail/CVE-2025-48924) | | CVE-2025-48924 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2025-48924](https://nvd.nist.gov/vuln/detail/CVE-2025-48924) | | CVE-2025-48924 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2025-48924](https://nvd.nist.gov/vuln/detail/CVE-2025-48924) | | CVE-2025-49128 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2025-49128](https://nvd.nist.gov/vuln/detail/CVE-2025-49128) | | CVE-2025-49128 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2025-49128](https://nvd.nist.gov/vuln/detail/CVE-2025-49128) | | CVE-2025-49128 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2025-49128](https://nvd.nist.gov/vuln/detail/CVE-2025-49128) | | CVE-2025-49128 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2025-49128](https://nvd.nist.gov/vuln/detail/CVE-2025-49128) | | CVE-2025-53864 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2025-53864](https://nvd.nist.gov/vuln/detail/CVE-2025-53864) | | CVE-2025-53864 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2025-53864](https://nvd.nist.gov/vuln/detail/CVE-2025-53864) | | CVE-2025-58457 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2025-58457](https://nvd.nist.gov/vuln/detail/CVE-2025-58457) | | CVE-2025-58457 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2025-58457](https://nvd.nist.gov/vuln/detail/CVE-2025-58457) | | CVE-2025-58457 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2025-58457](https://nvd.nist.gov/vuln/detail/CVE-2025-58457) | | CVE-2025-8885 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2025-8885](https://nvd.nist.gov/vuln/detail/CVE-2025-8885) | | CVE-2025-8885 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2025-8885](https://nvd.nist.gov/vuln/detail/CVE-2025-8885) | | CVE-2025-8916 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2025-8916](https://nvd.nist.gov/vuln/detail/CVE-2025-8916) | | CVE-2025-8916 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2025-8916](https://nvd.nist.gov/vuln/detail/CVE-2025-8916) | | CVE-2025-8916 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2025-8916](https://nvd.nist.gov/vuln/detail/CVE-2025-8916) | | CVE-2025-8916 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2025-8916](https://nvd.nist.gov/vuln/detail/CVE-2025-8916) | | CVE-2020-8908 | LOW | [https://nvd.nist.gov/vuln/detail/CVE-2020-8908](https://nvd.nist.gov/vuln/detail/CVE-2020-8908) | | CVE-2020-8908 | LOW | [https://nvd.nist.gov/vuln/detail/CVE-2020-8908](https://nvd.nist.gov/vuln/detail/CVE-2020-8908) | | CVE-2020-8908 | LOW | [https://nvd.nist.gov/vuln/detail/CVE-2020-8908](https://nvd.nist.gov/vuln/detail/CVE-2020-8908) | | CVE-2020-8908 | LOW | [https://nvd.nist.gov/vuln/detail/CVE-2020-8908](https://nvd.nist.gov/vuln/detail/CVE-2020-8908) | | CVE-2020-8908 | LOW | [https://nvd.nist.gov/vuln/detail/CVE-2020-8908](https://nvd.nist.gov/vuln/detail/CVE-2020-8908) | | CVE-2020-8908 | LOW | [https://nvd.nist.gov/vuln/detail/CVE-2020-8908](https://nvd.nist.gov/vuln/detail/CVE-2020-8908) | | CVE-2020-8908 | LOW | [https://nvd.nist.gov/vuln/detail/CVE-2020-8908](https://nvd.nist.gov/vuln/detail/CVE-2020-8908) | | CVE-2020-8908 | LOW | [https://nvd.nist.gov/vuln/detail/CVE-2020-8908](https://nvd.nist.gov/vuln/detail/CVE-2020-8908) | | CVE-2020-8908 | LOW | [https://nvd.nist.gov/vuln/detail/CVE-2020-8908](https://nvd.nist.gov/vuln/detail/CVE-2020-8908) | | CVE-2020-8908 | LOW | [https://nvd.nist.gov/vuln/detail/CVE-2020-8908](https://nvd.nist.gov/vuln/detail/CVE-2020-8908) | | CVE-2024-12801 | LOW | [https://nvd.nist.gov/vuln/detail/CVE-2024-12801](https://nvd.nist.gov/vuln/detail/CVE-2024-12801) | | CVE-2024-12801 | LOW | [https://nvd.nist.gov/vuln/detail/CVE-2024-12801](https://nvd.nist.gov/vuln/detail/CVE-2024-12801) | | CVE-2024-12801 | LOW | [https://nvd.nist.gov/vuln/detail/CVE-2024-12801](https://nvd.nist.gov/vuln/detail/CVE-2024-12801) | | CVE-2025-27496 | LOW | [https://nvd.nist.gov/vuln/detail/CVE-2025-27496](https://nvd.nist.gov/vuln/detail/CVE-2025-27496) | ## 7.10.0 release notes The following release notes include information for Amazon EMR release 7.10.0. **New features** + **Application upgrades** – Amazon EMR 7.10.0 application upgrades include AWS SDK for Java 2.31.48/1.12.782, Delta 3.3.0-amzn-2, Hadoop 3.4.1-amzn-2, Hudi 0.15.0-amzn-7, Iceberg 1.8.1-amzn-0, Spark 3.5.5-amzn-1, and Zookeeper 3.9.3-amzn-2. + **S3A filesystem** – Starting from the EMR 7.10.0 release, the S3A filesystem has replaced EMRFS as the default EMR S3 connector. For more information, see [Migration Guide: EMRFS to S3A Filesystem](https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-s3a-migrate.html). + **Spark Kafka streaming support** – Starting from the EMR 7.10.0 release, aws-msk-iam-auth 2.3.2 and commons-pool2 2.11.1 dependencies are included for Spark streaming workloads with Kafka data sources. These JARs are available when Spark application is selected at cluster launch. + **Flink streaming connectors** – Starting from the EMR 7.10.0 release, flink-sql-connector-kafka 3.4.0, flink-sql-connector-kinesis 5.0.0, and aws-msk-iam-auth 2.3.2 are included for Flink streaming workloads with Kafka and Kinesis data sources. These connector JARs are available when Flink application is selected at cluster launch. **Known issues** + **ACCEPTED State** – YARN Application stuck In ACCEPTED State due to open-source bug [YARN-11834](https://issues.apache.org/jira/browse/YARN-11834). Reach out to the AWS support team if you encounter this issue. + **ConcurrentModificationException exception** – YARN restarting frequently with ConcurrentModificationException exception due to open-source bug [YARN-11838](https://issues.apache.org/jira/browse/YARN-11838). Reach out to the AWS support team if you encounter this issue. + **YARN application logs** – YARN application logs not visible through Resource Manager UI. In order to fix the issue, follow these steps: 1. Update the `yarn.log.server.url` configuration in yarn-site.xml to the following value: ``` yarn.log.server.url https://:19890/jobhistory/logs ``` 1. Restart the hadoop resource manager service: `systemctl restart hadoop-resourcemanager.service` + **Uploading step logs** – Some clusters may not be able to upload step logs to your S3 bucket due to a race condition. Reach out to the AWS support team if you encounter this issue. + **HBase table creation in Hive with Glue catalog** – From EMR-7.3.0 to EMR-7.10.0, there is a bug due to Hive Iceberg integration, which causes HBase table creation in Hive to fail when Glue Data Catalog is used as the metastore. Please reach out to the AWS support team if you encounter this issue. + **Tez UI** – From EMR-7.3 to EMR-7.10, there is a bug where accessing Tez UI through YARN Resource Manager Web UI fails when in-transit encryption is enabled. This occurs because YARN proxy attempts to use HTTP protocol while Tez UI requires HTTPS. Please modify the `tez.tez-ui.history-url.base` configuration in `tez-site` from `http` to `https` to resolve this issue. + **Missing step and container logs** - From Amazon EMR 7.9 to Amazon EMR 7.12, there is a bug where step logs and/or container logs may not get uploaded to S3 if the step or application is running for longer than 3 hours. Please upgrade to Amazon EMR 7.13 to resolve this issue. + Release labels and supported regions. [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-7100-release.html) ## 7.10.0 default Java versions Amazon EMR release 7.10 and higher ships with Amazon Corretto 17 (built on OpenJDK) by default for applications that support Corretto 17 (JDK 17), with the exception of Apache Livy. The following table shows the default Java versions for applications in Amazon EMR 7.10.0. If you want to change the default JVM on your cluster, follow the instructions in [Configure applications to use a specific Java Virtual Machine](configuring-java8.md) for each application that runs on the cluster. You can only use one Java runtime version for a cluster. Amazon EMR doesn't support running different nodes or applications on different runtime versions on the same cluster. | Application | Java / Amazon Corretto version (default is bold) | | --- | --- | | Delta | 17, 11, 8 | | Flink | 17, 11, 8 | | HBase | 17, 11, 8 | | HCatalog | 17, 11, 8 | | Hadoop | 17, 11, 8 | | Hive | 17, 11, 8 | | Hudi | 17, 11, 8 | | Iceberg | 17, 11 | | Livy | 17, 11, 8 | | Oozie | 17, 11, 8 | | Phoenix | 17, 11, 8 | | PrestoDB | 8 | | Spark | 17, 11, 8 | | Spark RAPIDS | 17, 11, 8 | | Tez | 17, 11, 8 | | Trino | 21, 17 | | Zeppelin | 17, 11, 8 | | Pig | 17, 11, 8 | | Zookeeper | 17, 11, 8 | ## 7.10.0 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | adot-java-agent | 1.31.0 | A Java Agent that collects metrics from application daemons. | | delta | 3.3.0-amzn-2 | Delta lake is an open table format for huge analytic datasets | | emr-amazon-cloudwatch-agent | 1.300032.2-amzn-0 | An application that collects internal system-level metrics and custom application metrics from Amazon EC2 instances. | | emr-ddb | 5.7.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 3.19.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.23.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-notebook-env | 1.18.0 | Conda env for emr notebook which includes jupyter enterprise gateway | | emr-s3-dist-cp | 2.40.0 | Distributed copy application optimized for Amazon S3. | | emr-s3-select | 2.19.0 | EMR S3Select Connector | | emr-wal-cli | 1.3.4 | Cli used for emrwal list/deletion. | | emrfs | 2.71.0 | Amazon S3 connector for Hadoop ecosystem applications. | | flink-client | 1.20.0-amzn-4 | Apache Flink command line client scripts and applications. | | flink-jobmanager-config | 1.20.0-amzn-4 | Managing resources on EMR nodes for Apache Flink JobManager. | | hadoop-client | 3.4.1-amzn-2 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 3.4.1-amzn-2 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 3.4.1-amzn-2 | HDFS command-line client and library | | hadoop-hdfs-namenode | 3.4.1-amzn-2 | HDFS service for tracking file names and block locations. | | hadoop-hdfs-journalnode | 3.4.1-amzn-2 | HDFS service for managing the Hadoop filesystem journal on HA clusters. | | hadoop-httpfs-server | 3.4.1-amzn-2 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 3.4.1-amzn-2 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 3.4.1-amzn-2 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 3.4.1-amzn-2 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 3.4.1-amzn-2 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 3.4.1-amzn-2 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 2.6.2-amzn-1 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 2.6.2-amzn-1 | Service for serving one or more HBase regions. | | hbase-client | 2.6.2-amzn-1 | HBase command-line client. | | hbase-rest-server | 2.6.2-amzn-1 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 2.6.2-amzn-1 | Service providing a Thrift endpoint to HBase. | | hbase-operator-tools | 2.6.2-amzn-1 | Repair tool for Apache HBase clusters. | | hcatalog-client | 3.1.3-amzn-19 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 3.1.3-amzn-19 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 3.1.3-amzn-19 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 3.1.3-amzn-19 | Hive command line client. | | hive-hbase | 3.1.3-amzn-19 | Hive-hbase client. | | hive-metastore-server | 3.1.3-amzn-19 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server2 | 3.1.3-amzn-19 | Service for accepting Hive queries as web requests. | | hudi | 0.15.0-amzn-7 | Incremental processing framework to power data pipeline at low latency and high efficiency. | | hudi-presto | 0.15.0-amzn-7 | Bundle library for running Presto with Hudi. | | hudi-trino | 0.15.0-amzn-7 | Bundle library for running Trino with Hudi. | | hudi-spark | 0.15.0-amzn-7 | Bundle library for running Spark with Hudi. | | hue-server | 4.11.0 | Web application for analyzing data using Hadoop ecosystem applications | | iceberg | 1.8.1-amzn-0 | Apache Iceberg is an open table format for huge analytic datasets | | jupyterhub | 1.5.0 | Multi-user server for Jupyter notebooks | | livy-server | 0.8.0-incubating | REST interface for interacting with Apache Spark | | nginx | 1.12.1 | nginx [engine x] is an HTTP and reverse proxy server | | mariadb-server | 5.5.68\$1 | MariaDB database server. | | nvidia-cuda | 12.5.0 | Nvidia drivers and Cuda toolkit | | oozie-client | 5.2.1 | Oozie command-line client. | | oozie-server | 5.2.1 | Service for accepting Oozie workflow requests. | | opencv | 4.7.0 | Open Source Computer Vision Library. | | phoenix-library | 5.2.1 | The phoenix libraries for server and client | | phoenix-connectors | 6.0.0 | Apache Phoenix-Connectors for Spark-3 | | phoenix-query-server | 6.0.0 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.287-amzn-4 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.287-amzn-4 | Service for executing pieces of a query. | | presto-client | 0.287-amzn-4 | Presto command-line client which is installed on an HA cluster's stand-by masters where Presto server is not started. | | trino-coordinator | 470-amzn-0 | Service for accepting queries and managing query execution among trino-workers. | | trino-worker | 470-amzn-0 | Service for executing pieces of a query. | | trino-client | 470-amzn-0 | Trino command-line client which is installed on an HA cluster's stand-by masters where Trino server is not started. | | pig-client | 0.17.0 | Pig command-line client. | | r | 4.3.2 | The R Project for Statistical Computing | | ranger-kms-server | 2.0.0 | Apache Ranger Key Management System | | spark-client | 3.5.5-amzn-1 | Spark command-line clients. | | spark-history-server | 3.5.5-amzn-1 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 3.5.5-amzn-1 | In-memory execution engine for YARN. | | spark-yarn-slave | 3.5.5-amzn-1 | Apache Spark libraries needed by YARN slaves. | | spark-rapids | 3.5.5-amzn-1 | Nvidia Spark RAPIDS plugin that accelerates Apache Spark with GPUs. | | tensorflow | 2.18.0 | TensorFlow open source software library for high performance numerical computation. | | tez-on-yarn | 0.10.2-amzn-17 | The tez YARN application and libraries. | | tez-on-worker | 0.10.2-amzn-17 | The tez YARN application and libraries for worker nodes. | | zeppelin-server | 0.11.1 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.9.3-amzn-2 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.9.3-amzn-2 | ZooKeeper command line client. | ## 7.10.0 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). Reconfiguration actions occur when you specify a configuration for instance groups in a running cluster. Amazon EMR only initiates reconfiguration actions for the classifications that you modify. For more information, see [Reconfigure an instance group in a running cluster](emr-configure-apps-running-cluster.md). **emr-7.10.0 classifications** | Classifications | Description | Reconfiguration Actions | | --- | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | Restarts the ResourceManager service. | | container-executor | Change values in Hadoop YARN's container-executor.cfg file. | Not available. | | container-log4j | Change values in Hadoop YARN's container-log4j.properties file. | Not available. | | core-site | Change values in Hadoop's core-site.xml file. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts Hadoop KMS, Ranger KMS, HiveServer2, Hive MetaStore, Hadoop Httpfs, and MapReduce-HistoryServer. | | docker-conf | Change docker related settings. | Not available. | | emrfs-site | Change EMRFS settings. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts HBaseRegionserver, HBaseMaster, HBaseThrift, HBaseRest, HiveServer2, Hive MetaStore, Hadoop Httpfs, and MapReduce-HistoryServer. | | flink-conf | Change flink-conf.yaml settings. | Restarts Flink history server. | | flink-log4j | Change Flink log4j.properties settings. | Restarts Flink history server. | | flink-log4j-session | Change Flink log4j-session.properties settings for Kubernetes/Yarn session. | Restarts Flink history server. | | flink-log4j-cli | Change Flink log4j-cli.properties settings. | Restarts Flink history server. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts PhoenixQueryserver, HiveServer2, Hive MetaStore, and MapReduce-HistoryServer. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | Restarts the Hadoop HDFS services SecondaryNamenode, Datanode, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts Hadoop KMS, Hadoop Httpfs, and MapReduce-HistoryServer. | | hadoop-ssl-server | Change hadoop ssl server configuration | Not available. | | hadoop-ssl-client | Change hadoop ssl client configuration | Not available. | | hbase | Amazon EMR-curated settings for Apache HBase. | Custom EMR specific property. Sets emrfs-site and hbase-site configs. See those for their associated restarts. | | hbase-env | Change values in HBase's environment. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. | | hbase-log4j2 | Change values in HBase's hbase-log4j2.properties file. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbase.properties file. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | Not available. | | hbase-site | Change values in HBase's hbase-site.xml file. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. Additionally restarts Phoenix QueryServer. | | hdfs-encryption-zones | Configure HDFS encryption zones. | This classification should not be reconfigured. | | hdfs-env | Change values in the HDFS environment. | Restarts Hadoop HDFS services Namenode, Datanode, and ZKFC. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Additionally restarts Hadoop Httpfs. | | hcatalog-env | Change values in HCatalog's environment. | Restarts Hive HCatalog Server. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | Restarts Hive HCatalog Server. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | Restarts Hive HCatalog Server. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | Restarts Hive WebHCat server. | | hcatalog-webhcat-log4j2 | Change values in HCatalog WebHCat's log4j2.properties. | Restarts Hive WebHCat server. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | Restarts Hive WebHCat server. | | hive | Amazon EMR-curated settings for Apache Hive. | Sets configurations to launch Hive LLAP service. | | hive-beeline-log4j2 | Change values in Hive's beeline-log4j2.properties file. | Not available. | | hive-parquet-logging | Change values in Hive's parquet-logging.properties file. | Not available. | | hive-env | Change values in the Hive environment. | Restarts HiveServer2, HiveMetastore, and Hive HCatalog-Server. Runs Hive schemaTool CLI commands to verify hive-metastore. | | hive-exec-log4j2 | Change values in Hive's hive-exec-log4j2.properties file. | Not available. | | hive-llap-daemon-log4j2 | Change values in Hive's llap-daemon-log4j2.properties file. | Not available. | | hive-log4j2 | Change values in Hive's hive-log4j2.properties file. | Not available. | | hive-site | Change values in Hive's hive-site.xml file | Restarts HiveServer2, HiveMetastore, and Hive HCatalog-Server. Runs Hive schemaTool CLI commands to verify hive-metastore. Also restarts Oozie and Zeppelin. | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | Not available. | | hue-ini | Change values in Hue's ini file | Restarts Hue. Also activates Hue config override CLI commands to pick up new configurations. | | httpfs-env | Change values in the HTTPFS environment. | Restarts Hadoop Httpfs service. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | Restarts Hadoop Httpfs service. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | Not available. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | Restarts Hadoop-KMS service. | | hadoop-kms-java-home | Change Hadoop's KMS java home | Not available. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | Not available. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | Restarts Hadoop-KMS and Ranger-KMS service. | | hudi-env | Change values in the Hudi environment. | Not available. | | hudi-defaults | Change values in Hudi's hudi-defaults.conf file. | Not available. | | iceberg-defaults | Change values in Iceberg's iceberg-defaults.conf file. | Not available. | | delta-defaults | Change values in Delta's delta-defaults.conf file. | Not available. | | jupyter-notebook-conf | Change values in Jupyter Notebook's jupyter\$1notebook\$1config.py file. | Not available. | | jupyter-hub-conf | Change values in JupyterHubs's jupyterhub\$1config.py file. | Not available. | | jupyter-s3-conf | Configure Jupyter Notebook S3 persistence. | Not available. | | jupyter-sparkmagic-conf | Change values in Sparkmagic's config.json file. | Not available. | | livy-conf | Change values in Livy's livy.conf file. | Restarts Livy Server. | | livy-env | Change values in the Livy environment. | Restarts Livy Server. | | livy-log4j2 | Change Livy log4j2.properties settings. | Restarts Livy Server. | | mapred-env | Change values in the MapReduce application's environment. | Restarts Hadoop MapReduce-HistoryServer. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | Restarts Hadoop MapReduce-HistoryServer. | | oozie-env | Change values in Oozie's environment. | Restarts Oozie. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | Restarts Oozie. | | oozie-site | Change values in Oozie's oozie-site.xml file. | Restarts Oozie. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | Not available. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | Not available. | | phoenix-log4j2 | Change values in Phoenix's log4j2.properties file. | Restarts Phoenix-QueryServer. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | Not available. | | pig-env | Change values in the Pig environment. | Not available. | | pig-properties | Change values in Pig's pig.properties file. | Restarts Oozie. | | pig-log4j | Change values in Pig's log4j.properties file. | Not available. | | presto-log | Change values in Presto's log.properties file. | Restarts Presto-Server (for PrestoDB) | | presto-config | Change values in Presto's config.properties file. | Restarts Presto-Server (for PrestoDB) | | presto-password-authenticator | Change values in Presto's password-authenticator.properties file. | Not available. | | presto-env | Change values in Presto's presto-env.sh file. | Restarts Presto-Server (for PrestoDB) | | presto-node | Change values in Presto's node.properties file. | Not available. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | Not available. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | Not available. | | presto-connector-hive | Change values in Presto's hive.properties file. | Restarts Presto-Server (for PrestoDB) | | presto-connector-jmx | Change values in Presto's jmx.properties file. | Not available. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | Not available. | | presto-connector-lakeformation | Change values in Presto's lakeformation.properties file. | Restarts Presto-Server (for PrestoDB) | | presto-connector-localfile | Change values in Presto's localfile.properties file. | Not available. | | presto-connector-memory | Change values in Presto's memory.properties file. | Not available. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | Not available. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | Not available. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | Not available. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | Not available. | | presto-connector-redis | Change values in Presto's redis.properties file. | Not available. | | presto-connector-redshift | Change values in Presto's redshift.properties file. | Not available. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | Not available. | | presto-connector-tpcds | Change values in Presto's tpcds.properties file. | Not available. | | trino-log | Change values in Trino's log.properties file. | Restarts Trino-Server (for Trino) | | trino-config | Change values in Trino's config.properties file. | Restarts Trino-Server (for Trino) | | trino-password-authenticator | Change values in Trino's password-authenticator.properties file. | Restarts Trino-Server (for Trino) | | trino-env | Change values in Trino's trino-env.sh file. | Restarts Trino-Server (for Trino) | | trino-node | Change values in Trino's node.properties file. | Not available. | | trino-connector-blackhole | Change values in Trino's blackhole.properties file. | Not available. | | trino-connector-cassandra | Change values in Trino's cassandra.properties file. | Not available. | | trino-connector-delta | Change values in Trino's delta.properties file. | Restarts Trino-Server (for Trino) | | trino-connector-hive | Change values in Trino's hive.properties file. | Restarts Trino-Server (for Trino) | | trino-exchange-manager | Change values in Trino's exchange-manager.properties file. | Restarts Trino-Server (for Trino) | | trino-connector-iceberg | Change values in Trino's iceberg.properties file. | Restarts Trino-Server (for Trino) | | trino-connector-hudi | Change values in Trino's hudi.properties file. | Restarts Trino-Server (for Trino) | | trino-connector-jmx | Change values in Trino's jmx.properties file. | Not available. | | trino-connector-kafka | Change values in Trino's kafka.properties file. | Not available. | | trino-connector-localfile | Change values in Trino's localfile.properties file. | Not available. | | trino-connector-memory | Change values in Trino's memory.properties file. | Not available. | | trino-connector-mongodb | Change values in Trino's mongodb.properties file. | Not available. | | trino-connector-mysql | Change values in Trino's mysql.properties file. | Not available. | | trino-connector-postgresql | Change values in Trino's postgresql.properties file. | Not available. | | trino-connector-raptor | Change values in Trino's raptor.properties file. | Not available. | | trino-connector-redis | Change values in Trino's redis.properties file. | Not available. | | trino-connector-redshift | Change values in Trino's redshift.properties file. | Not available. | | trino-connector-tpch | Change values in Trino's tpch.properties file. | Not available. | | trino-connector-tpcds | Change values in Trino's tpcds.properties file. | Not available. | | ranger-kms-dbks-site | Change values in dbks-site.xml file of Ranger KMS. | Restarts Ranger KMS Server. | | ranger-kms-site | Change values in ranger-kms-site.xml file of Ranger KMS. | Restarts Ranger KMS Server. | | ranger-kms-env | Change values in the Ranger KMS environment. | Restarts Ranger KMS Server. | | ranger-kms-logback | Change values in kms-logback.xml file of Ranger KMS. | Not available. | | ranger-kms-db-ca | Change values for CA file on S3 for MySQL SSL connection with Ranger KMS. | Not available. | | spark | Amazon EMR-curated settings for Apache Spark. | This property modifies spark-defaults. See actions there. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | Restarts Spark history server and Spark thrift server. | | spark-env | Change values in the Spark environment. | Restarts Spark history server and Spark thrift server. | | spark-hive-site | Change values in Spark's hive-site.xml file | Not available. | | spark-log4j2 | Change values in Spark's log4j2.properties file. | Restarts Spark history server and Spark thrift server. | | spark-metrics | Change values in Spark's metrics.properties file. | Restarts Spark history server and Spark thrift server. | | tez-site | Change values in Tez's tez-site.xml file. | Restart Oozie and HiveServer2. | | yarn-env | Change values in the YARN environment. | Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts MapReduce-HistoryServer. | | yarn-site | Change values in YARN's yarn-site.xml file. | Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts Livy Server and MapReduce-HistoryServer. | | zeppelin-env | Change values in the Zeppelin environment. | Restarts Zeppelin. | | zeppelin-site | Change configuration settings in zeppelin-site.xml. | Restarts Zeppelin. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | Restarts Zookeeper server. | | zookeeper-logback | Change values in ZooKeeper's logback.xml file. | Restarts Zookeeper server. | | emr-metrics | Change emr metric settings for this node. | Restarts the CloudWatchAgent service. | ## 7.10.0 change log **Change log for 7.10.0 release and release notes** | Date | Event | Description | | --- | --- | --- | | 2025-08-15 | Docs publication | Amazon EMR 7.10.0 release notes first published | | 2025-08-15 | Deployment complete | Amazon EMR 7.10.0 fully deployed to all [supported Regions](https://aws.amazon.com/about-aws/global-infrastructure/regional-product-services/) | | 2025-08-15 | Initial limited release | Amazon EMR 7.10.0 first deployed to initial commercial Regions | # Amazon EMR release 7.9.0 ## 7.9.0 supported lifecycle The following table describes the supported lifecycle dates for Amazon EMR release 7.9.0. | Support phase | Date | | --- | --- | | Initial release date | May 19, 2025 | | Standard support until | May 19, 2027 | | End of support | May 20, 2027 | | End of life | May 19, 2028 | ## 7.9.0 application versions This release includes the following applications: [https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/Install-CloudWatch-Agent.html](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/Install-CloudWatch-Agent.html), [https://delta.io/](https://delta.io/), [https://flink.apache.org/](https://flink.apache.org/), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [https://hudi.apache.org](https://hudi.apache.org), [http://gethue.com/](http://gethue.com/), [https://iceberg.apache.org/](https://iceberg.apache.org/), [https://jupyter-enterprise-gateway.readthedocs.io/en/latest/](https://jupyter-enterprise-gateway.readthedocs.io/en/latest/), [https://jupyterhub.readthedocs.io/en/latest/#](https://jupyterhub.readthedocs.io/en/latest/#), [https://livy.incubator.apache.org/](https://livy.incubator.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [https://www.tensorflow.org/](https://www.tensorflow.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://trino.io/](https://trino.io/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-7.9.0 | emr-7.8.0 | emr-7.7.0 | emr-7.6.0 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 2.31.16, 1.12.782 | 2.29.52, 1.12.780 | 2.29.44, 1.12.780 | 2.29.25, 1.12.779 | | Python | 3.9, 3.11 | 3.9, 3.11 | 3.9, 3.11 | 3.9, 3.11 | | Scala | 2.12.18 | 2.12.18 | 2.12.18 | 2.12.18 | | AmazonCloudWatchAgent | 1.300032.2-amzn-0 | 1.300032.2-amzn-0 | 1.300032.2-amzn-0 | 1.300032.2-amzn-0 | | Delta | 3.3.0-amzn-1 | 3.3.0-amzn-0 | 3.2.1-amzn-2 | 3.2.1-amzn-1 | | Flink | 1.20.0-amzn-3 | 1.20.0-amzn-2 | 1.20.0-amzn-1 | 1.20.0-amzn-0 | | Ganglia | - | - | - | - | | HBase | 2.6.2-amzn-0 | 2.6.1-amzn-2 | 2.6.1-amzn-1 | 2.6.1-amzn-0 | | HCatalog | 3.1.3-amzn-18 | 3.1.3-amzn-17 | 3.1.3-amzn-16 | 3.1.3-amzn-15 | | Hadoop | 3.4.1-amzn-1 | 3.4.1-amzn-0 | 3.4.0-amzn-3 | 3.4.0-amzn-2 | | Hive | 3.1.3-amzn-18 | 3.1.3-amzn-17 | 3.1.3-amzn-16 | 3.1.3-amzn-15 | | Hudi | 0.15.0-amzn-6 | 0.15.0-amzn-5 | 0.15.0-amzn-4 | 0.15.0-amzn-3 | | Hue | 4.11.0 | 4.11.0 | 4.11.0 | 4.11.0 | | Iceberg | 1.7.1-amzn-2 | 1.7.1-amzn-1 | 1.7.1-amzn-0 | 1.6.1-amzn-2 | | JupyterEnterpriseGateway | 2.6.0 | 2.6.0 | 2.6.0 | 2.6.0 | | JupyterHub | 1.5.0 | 1.5.0 | 1.5.0 | 1.5.0 | | Livy | 0.8.0-incubating | 0.8.0-incubating | 0.8.0-incubating | 0.8.0-incubating | | MXNet | - | - | - | - | | Mahout | - | - | - | - | | Oozie | 5.2.1 | 5.2.1 | 5.2.1 | 5.2.1 | | Phoenix | 5.2.1 | 5.2.1 | 5.2.1 | 5.2.0 | | Pig | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | | Presto | 0.287-amzn-3 | 0.287-amzn-2 | 0.287-amzn-1 | 0.287-amzn-1 | | Spark | 3.5.5-amzn-0 | 3.5.4-amzn-0 | 3.5.3-amzn-1 | 3.5.3-amzn-0 | | Sqoop | - | - | - | - | | TensorFlow | 2.16.1 | 2.16.1 | 2.16.1 | 2.16.1 | | Tez | 0.10.2-amzn-16 | 0.10.2-amzn-15 | 0.10.2-amzn-14 | 0.10.2-amzn-13 | | Trino (PrestoSQL) | 467-amzn-2 | 467-amzn-1 | 467-amzn-0 | 457-amzn-0 | | Zeppelin | 0.11.1 | 0.11.1 | 0.11.1 | 0.11.1 | | ZooKeeper | 3.9.3-amzn-1 | 3.9.3-amzn-0 | 3.9.2-amzn-1 | 3.9.2-amzn-0 | # 7.9.0 common vulnerabilities and exposures The following table lists all CVEs that don't impact EMR clusters that run on recommended configurations of Amazon EMR 7.9.0. Amazon EMR is dependent on upstream open-source for availability of fixes and will provide the latest stable version as part of the Core Engine components within 90 days of Amazon EMR verifying the fixes. This table was updated on May 12, 2025. | CVE ID | Severity | CVE details URL | | --- | --- | --- | | CVE-2015-1832 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2015-1832](https://nvd.nist.gov/vuln/detail/CVE-2015-1832) | | CVE-2016-5018 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2016-5018](https://nvd.nist.gov/vuln/detail/CVE-2016-5018) | | CVE-2017-15095 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2017-15095](https://nvd.nist.gov/vuln/detail/CVE-2017-15095) | | CVE-2017-17485 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2017-17485](https://nvd.nist.gov/vuln/detail/CVE-2017-17485) | | CVE-2017-7525 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2017-7525](https://nvd.nist.gov/vuln/detail/CVE-2017-7525) | | CVE-2018-11307 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-11307](https://nvd.nist.gov/vuln/detail/CVE-2018-11307) | | CVE-2018-11307 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-11307](https://nvd.nist.gov/vuln/detail/CVE-2018-11307) | | CVE-2018-14718 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-14718](https://nvd.nist.gov/vuln/detail/CVE-2018-14718) | | CVE-2018-14718 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-14718](https://nvd.nist.gov/vuln/detail/CVE-2018-14718) | | CVE-2018-14719 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-14719](https://nvd.nist.gov/vuln/detail/CVE-2018-14719) | | CVE-2018-14719 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-14719](https://nvd.nist.gov/vuln/detail/CVE-2018-14719) | | CVE-2018-14720 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-14720](https://nvd.nist.gov/vuln/detail/CVE-2018-14720) | | CVE-2018-14720 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-14720](https://nvd.nist.gov/vuln/detail/CVE-2018-14720) | | CVE-2018-14721 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-14721](https://nvd.nist.gov/vuln/detail/CVE-2018-14721) | | CVE-2018-14721 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-14721](https://nvd.nist.gov/vuln/detail/CVE-2018-14721) | | CVE-2018-19360 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-19360](https://nvd.nist.gov/vuln/detail/CVE-2018-19360) | | CVE-2018-19360 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-19360](https://nvd.nist.gov/vuln/detail/CVE-2018-19360) | | CVE-2018-19361 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-19361](https://nvd.nist.gov/vuln/detail/CVE-2018-19361) | | CVE-2018-19361 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-19361](https://nvd.nist.gov/vuln/detail/CVE-2018-19361) | | CVE-2018-19362 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-19362](https://nvd.nist.gov/vuln/detail/CVE-2018-19362) | | CVE-2018-19362 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-19362](https://nvd.nist.gov/vuln/detail/CVE-2018-19362) | | CVE-2018-7489 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-7489](https://nvd.nist.gov/vuln/detail/CVE-2018-7489) | | CVE-2018-7489 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-7489](https://nvd.nist.gov/vuln/detail/CVE-2018-7489) | | CVE-2019-10202 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-10202](https://nvd.nist.gov/vuln/detail/CVE-2019-10202) | | CVE-2019-14379 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-14379](https://nvd.nist.gov/vuln/detail/CVE-2019-14379) | | CVE-2019-14379 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-14379](https://nvd.nist.gov/vuln/detail/CVE-2019-14379) | | CVE-2019-14540 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-14540](https://nvd.nist.gov/vuln/detail/CVE-2019-14540) | | CVE-2019-14540 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-14540](https://nvd.nist.gov/vuln/detail/CVE-2019-14540) | | CVE-2019-14892 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-14892](https://nvd.nist.gov/vuln/detail/CVE-2019-14892) | | CVE-2019-14892 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-14892](https://nvd.nist.gov/vuln/detail/CVE-2019-14892) | | CVE-2019-14893 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-14893](https://nvd.nist.gov/vuln/detail/CVE-2019-14893) | | CVE-2019-14893 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-14893](https://nvd.nist.gov/vuln/detail/CVE-2019-14893) | | CVE-2019-16335 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-16335](https://nvd.nist.gov/vuln/detail/CVE-2019-16335) | | CVE-2019-16335 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-16335](https://nvd.nist.gov/vuln/detail/CVE-2019-16335) | | CVE-2019-16942 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-16942](https://nvd.nist.gov/vuln/detail/CVE-2019-16942) | | CVE-2019-16942 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-16942](https://nvd.nist.gov/vuln/detail/CVE-2019-16942) | | CVE-2019-16943 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-16943](https://nvd.nist.gov/vuln/detail/CVE-2019-16943) | | CVE-2019-16943 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-16943](https://nvd.nist.gov/vuln/detail/CVE-2019-16943) | | CVE-2019-17267 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-17267](https://nvd.nist.gov/vuln/detail/CVE-2019-17267) | | CVE-2019-17267 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-17267](https://nvd.nist.gov/vuln/detail/CVE-2019-17267) | | CVE-2019-17531 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-17531](https://nvd.nist.gov/vuln/detail/CVE-2019-17531) | | CVE-2019-17531 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-17531](https://nvd.nist.gov/vuln/detail/CVE-2019-17531) | | CVE-2019-20330 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-20330](https://nvd.nist.gov/vuln/detail/CVE-2019-20330) | | CVE-2019-20330 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-20330](https://nvd.nist.gov/vuln/detail/CVE-2019-20330) | | CVE-2020-8840 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2020-8840](https://nvd.nist.gov/vuln/detail/CVE-2020-8840) | | CVE-2020-8840 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2020-8840](https://nvd.nist.gov/vuln/detail/CVE-2020-8840) | | CVE-2020-9546 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2020-9546](https://nvd.nist.gov/vuln/detail/CVE-2020-9546) | | CVE-2020-9546 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2020-9546](https://nvd.nist.gov/vuln/detail/CVE-2020-9546) | | CVE-2020-9547 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2020-9547](https://nvd.nist.gov/vuln/detail/CVE-2020-9547) | | CVE-2020-9547 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2020-9547](https://nvd.nist.gov/vuln/detail/CVE-2020-9547) | | CVE-2020-9548 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2020-9548](https://nvd.nist.gov/vuln/detail/CVE-2020-9548) | | CVE-2020-9548 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2020-9548](https://nvd.nist.gov/vuln/detail/CVE-2020-9548) | | CVE-2022-1471 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2022-1471](https://nvd.nist.gov/vuln/detail/CVE-2022-1471) | | CVE-2022-1471 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2022-1471](https://nvd.nist.gov/vuln/detail/CVE-2022-1471) | | CVE-2022-37865 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2022-37865](https://nvd.nist.gov/vuln/detail/CVE-2022-37865) | | CVE-2022-39135 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2022-39135](https://nvd.nist.gov/vuln/detail/CVE-2022-39135) | | CVE-2022-42889 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2022-42889](https://nvd.nist.gov/vuln/detail/CVE-2022-42889) | | CVE-2022-46337 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2022-46337](https://nvd.nist.gov/vuln/detail/CVE-2022-46337) | | CVE-2022-46337 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2022-46337](https://nvd.nist.gov/vuln/detail/CVE-2022-46337) | | CVE-2024-51504 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2024-51504](https://nvd.nist.gov/vuln/detail/CVE-2024-51504) | | CVE-2024-51504 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2024-51504](https://nvd.nist.gov/vuln/detail/CVE-2024-51504) | | CVE-2024-52046 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2024-52046](https://nvd.nist.gov/vuln/detail/CVE-2024-52046) | | CVE-2024-52046 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2024-52046](https://nvd.nist.gov/vuln/detail/CVE-2024-52046) | | CVE-2024-52577 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2024-52577](https://nvd.nist.gov/vuln/detail/CVE-2024-52577) | | CVE-2024-53990 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2024-53990](https://nvd.nist.gov/vuln/detail/CVE-2024-53990) | | CVE-2024-56325 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2024-56325](https://nvd.nist.gov/vuln/detail/CVE-2024-56325) | | CVE-2025-30065 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2025-30065](https://nvd.nist.gov/vuln/detail/CVE-2025-30065) | | CVE-2015-5237 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2015-5237](https://nvd.nist.gov/vuln/detail/CVE-2015-5237) | | CVE-2015-5237 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2015-5237](https://nvd.nist.gov/vuln/detail/CVE-2015-5237) | | CVE-2016-6796 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2016-6796](https://nvd.nist.gov/vuln/detail/CVE-2016-6796) | | CVE-2018-1000180 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2018-1000180](https://nvd.nist.gov/vuln/detail/CVE-2018-1000180) | | CVE-2018-12022 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2018-12022](https://nvd.nist.gov/vuln/detail/CVE-2018-12022) | | CVE-2018-12022 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2018-12022](https://nvd.nist.gov/vuln/detail/CVE-2018-12022) | | CVE-2018-12023 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2018-12023](https://nvd.nist.gov/vuln/detail/CVE-2018-12023) | | CVE-2018-12023 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2018-12023](https://nvd.nist.gov/vuln/detail/CVE-2018-12023) | | CVE-2018-5968 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2018-5968](https://nvd.nist.gov/vuln/detail/CVE-2018-5968) | | CVE-2019-0205 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2019-0205](https://nvd.nist.gov/vuln/detail/CVE-2019-0205) | | CVE-2019-0205 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2019-0205](https://nvd.nist.gov/vuln/detail/CVE-2019-0205) | | CVE-2019-10172 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2019-10172](https://nvd.nist.gov/vuln/detail/CVE-2019-10172) | | CVE-2019-12086 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2019-12086](https://nvd.nist.gov/vuln/detail/CVE-2019-12086) | | CVE-2019-12086 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2019-12086](https://nvd.nist.gov/vuln/detail/CVE-2019-12086) | | CVE-2019-14439 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2019-14439](https://nvd.nist.gov/vuln/detail/CVE-2019-14439) | | CVE-2019-14439 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2019-14439](https://nvd.nist.gov/vuln/detail/CVE-2019-14439) | | CVE-2020-10650 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-10650](https://nvd.nist.gov/vuln/detail/CVE-2020-10650) | | CVE-2020-10650 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-10650](https://nvd.nist.gov/vuln/detail/CVE-2020-10650) | | CVE-2020-10672 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-10672](https://nvd.nist.gov/vuln/detail/CVE-2020-10672) | | CVE-2020-10672 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-10672](https://nvd.nist.gov/vuln/detail/CVE-2020-10672) | | CVE-2020-10673 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-10673](https://nvd.nist.gov/vuln/detail/CVE-2020-10673) | | CVE-2020-10673 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-10673](https://nvd.nist.gov/vuln/detail/CVE-2020-10673) | | CVE-2020-10968 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-10968](https://nvd.nist.gov/vuln/detail/CVE-2020-10968) | | CVE-2020-10968 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-10968](https://nvd.nist.gov/vuln/detail/CVE-2020-10968) | | CVE-2020-10969 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-10969](https://nvd.nist.gov/vuln/detail/CVE-2020-10969) | | CVE-2020-10969 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-10969](https://nvd.nist.gov/vuln/detail/CVE-2020-10969) | | CVE-2020-11111 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-11111](https://nvd.nist.gov/vuln/detail/CVE-2020-11111) | | CVE-2020-11111 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-11111](https://nvd.nist.gov/vuln/detail/CVE-2020-11111) | | CVE-2020-11112 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-11112](https://nvd.nist.gov/vuln/detail/CVE-2020-11112) | | CVE-2020-11112 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-11112](https://nvd.nist.gov/vuln/detail/CVE-2020-11112) | | CVE-2020-11113 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-11113](https://nvd.nist.gov/vuln/detail/CVE-2020-11113) | | CVE-2020-11113 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-11113](https://nvd.nist.gov/vuln/detail/CVE-2020-11113) | | CVE-2020-11619 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-11619](https://nvd.nist.gov/vuln/detail/CVE-2020-11619) | | CVE-2020-11619 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-11619](https://nvd.nist.gov/vuln/detail/CVE-2020-11619) | | CVE-2020-11620 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-11620](https://nvd.nist.gov/vuln/detail/CVE-2020-11620) | | CVE-2020-11620 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-11620](https://nvd.nist.gov/vuln/detail/CVE-2020-11620) | | CVE-2020-13949 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-13949](https://nvd.nist.gov/vuln/detail/CVE-2020-13949) | | CVE-2020-13949 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-13949](https://nvd.nist.gov/vuln/detail/CVE-2020-13949) | | CVE-2020-13949 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-13949](https://nvd.nist.gov/vuln/detail/CVE-2020-13949) | | CVE-2020-14060 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-14060](https://nvd.nist.gov/vuln/detail/CVE-2020-14060) | | CVE-2020-14060 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-14060](https://nvd.nist.gov/vuln/detail/CVE-2020-14060) | | CVE-2020-14061 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-14061](https://nvd.nist.gov/vuln/detail/CVE-2020-14061) | | CVE-2020-14061 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-14061](https://nvd.nist.gov/vuln/detail/CVE-2020-14061) | | CVE-2020-14062 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-14062](https://nvd.nist.gov/vuln/detail/CVE-2020-14062) | | CVE-2020-14062 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-14062](https://nvd.nist.gov/vuln/detail/CVE-2020-14062) | | CVE-2020-14195 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-14195](https://nvd.nist.gov/vuln/detail/CVE-2020-14195) | | CVE-2020-14195 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-14195](https://nvd.nist.gov/vuln/detail/CVE-2020-14195) | | CVE-2020-24616 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-24616](https://nvd.nist.gov/vuln/detail/CVE-2020-24616) | | CVE-2020-24616 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-24616](https://nvd.nist.gov/vuln/detail/CVE-2020-24616) | | CVE-2020-24750 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-24750](https://nvd.nist.gov/vuln/detail/CVE-2020-24750) | | CVE-2020-24750 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-24750](https://nvd.nist.gov/vuln/detail/CVE-2020-24750) | | CVE-2020-25649 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-25649](https://nvd.nist.gov/vuln/detail/CVE-2020-25649) | | CVE-2020-28052 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-28052](https://nvd.nist.gov/vuln/detail/CVE-2020-28052) | | CVE-2020-35490 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-35490](https://nvd.nist.gov/vuln/detail/CVE-2020-35490) | | CVE-2020-35490 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-35490](https://nvd.nist.gov/vuln/detail/CVE-2020-35490) | | CVE-2020-35491 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-35491](https://nvd.nist.gov/vuln/detail/CVE-2020-35491) | | CVE-2020-35491 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-35491](https://nvd.nist.gov/vuln/detail/CVE-2020-35491) | | CVE-2020-35728 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-35728](https://nvd.nist.gov/vuln/detail/CVE-2020-35728) | | CVE-2020-35728 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-35728](https://nvd.nist.gov/vuln/detail/CVE-2020-35728) | | CVE-2020-36179 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36179](https://nvd.nist.gov/vuln/detail/CVE-2020-36179) | | CVE-2020-36179 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36179](https://nvd.nist.gov/vuln/detail/CVE-2020-36179) | | CVE-2020-36180 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36180](https://nvd.nist.gov/vuln/detail/CVE-2020-36180) | | CVE-2020-36180 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36180](https://nvd.nist.gov/vuln/detail/CVE-2020-36180) | | CVE-2020-36181 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36181](https://nvd.nist.gov/vuln/detail/CVE-2020-36181) | | CVE-2020-36181 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36181](https://nvd.nist.gov/vuln/detail/CVE-2020-36181) | | CVE-2020-36182 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36182](https://nvd.nist.gov/vuln/detail/CVE-2020-36182) | | CVE-2020-36182 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36182](https://nvd.nist.gov/vuln/detail/CVE-2020-36182) | | CVE-2020-36183 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36183](https://nvd.nist.gov/vuln/detail/CVE-2020-36183) | | CVE-2020-36183 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36183](https://nvd.nist.gov/vuln/detail/CVE-2020-36183) | | CVE-2020-36184 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36184](https://nvd.nist.gov/vuln/detail/CVE-2020-36184) | | CVE-2020-36184 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36184](https://nvd.nist.gov/vuln/detail/CVE-2020-36184) | | CVE-2020-36185 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36185](https://nvd.nist.gov/vuln/detail/CVE-2020-36185) | | CVE-2020-36185 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36185](https://nvd.nist.gov/vuln/detail/CVE-2020-36185) | | CVE-2020-36186 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36186](https://nvd.nist.gov/vuln/detail/CVE-2020-36186) | | CVE-2020-36186 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36186](https://nvd.nist.gov/vuln/detail/CVE-2020-36186) | | CVE-2020-36187 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36187](https://nvd.nist.gov/vuln/detail/CVE-2020-36187) | | CVE-2020-36187 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36187](https://nvd.nist.gov/vuln/detail/CVE-2020-36187) | | CVE-2020-36188 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36188](https://nvd.nist.gov/vuln/detail/CVE-2020-36188) | | CVE-2020-36188 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36188](https://nvd.nist.gov/vuln/detail/CVE-2020-36188) | | CVE-2020-36189 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36189](https://nvd.nist.gov/vuln/detail/CVE-2020-36189) | | CVE-2020-36189 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36189](https://nvd.nist.gov/vuln/detail/CVE-2020-36189) | | CVE-2020-36518 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36518](https://nvd.nist.gov/vuln/detail/CVE-2020-36518) | | CVE-2020-36518 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36518](https://nvd.nist.gov/vuln/detail/CVE-2020-36518) | | CVE-2021-20190 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-20190](https://nvd.nist.gov/vuln/detail/CVE-2021-20190) | | CVE-2021-20190 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-20190](https://nvd.nist.gov/vuln/detail/CVE-2021-20190) | | CVE-2021-31684 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-31684](https://nvd.nist.gov/vuln/detail/CVE-2021-31684) | | CVE-2021-34538 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-34538](https://nvd.nist.gov/vuln/detail/CVE-2021-34538) | | CVE-2021-34538 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-34538](https://nvd.nist.gov/vuln/detail/CVE-2021-34538) | | CVE-2021-34538 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-34538](https://nvd.nist.gov/vuln/detail/CVE-2021-34538) | | CVE-2022-25647 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-25647](https://nvd.nist.gov/vuln/detail/CVE-2022-25647) | | CVE-2022-25647 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-25647](https://nvd.nist.gov/vuln/detail/CVE-2022-25647) | | CVE-2022-25647 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-25647](https://nvd.nist.gov/vuln/detail/CVE-2022-25647) | | CVE-2022-25857 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-25857](https://nvd.nist.gov/vuln/detail/CVE-2022-25857) | | CVE-2022-3171 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-3171](https://nvd.nist.gov/vuln/detail/CVE-2022-3171) | | CVE-2022-3171 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-3171](https://nvd.nist.gov/vuln/detail/CVE-2022-3171) | | CVE-2022-3171 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-3171](https://nvd.nist.gov/vuln/detail/CVE-2022-3171) | | CVE-2022-3509 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-3509](https://nvd.nist.gov/vuln/detail/CVE-2022-3509) | | CVE-2022-3509 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-3509](https://nvd.nist.gov/vuln/detail/CVE-2022-3509) | | CVE-2022-3509 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-3509](https://nvd.nist.gov/vuln/detail/CVE-2022-3509) | | CVE-2022-3510 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-3510](https://nvd.nist.gov/vuln/detail/CVE-2022-3510) | | CVE-2022-3510 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-3510](https://nvd.nist.gov/vuln/detail/CVE-2022-3510) | | CVE-2022-3510 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-3510](https://nvd.nist.gov/vuln/detail/CVE-2022-3510) | | CVE-2022-36364 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-36364](https://nvd.nist.gov/vuln/detail/CVE-2022-36364) | | CVE-2022-37866 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-37866](https://nvd.nist.gov/vuln/detail/CVE-2022-37866) | | CVE-2022-40149 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-40149](https://nvd.nist.gov/vuln/detail/CVE-2022-40149) | | CVE-2022-40149 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-40149](https://nvd.nist.gov/vuln/detail/CVE-2022-40149) | | CVE-2022-40150 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-40150](https://nvd.nist.gov/vuln/detail/CVE-2022-40150) | | CVE-2022-40150 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-40150](https://nvd.nist.gov/vuln/detail/CVE-2022-40150) | | CVE-2022-42003 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-42003](https://nvd.nist.gov/vuln/detail/CVE-2022-42003) | | CVE-2022-42003 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-42003](https://nvd.nist.gov/vuln/detail/CVE-2022-42003) | | CVE-2022-42003 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-42003](https://nvd.nist.gov/vuln/detail/CVE-2022-42003) | | CVE-2022-42003 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-42003](https://nvd.nist.gov/vuln/detail/CVE-2022-42003) | | CVE-2022-42003 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-42003](https://nvd.nist.gov/vuln/detail/CVE-2022-42003) | | CVE-2022-42004 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-42004](https://nvd.nist.gov/vuln/detail/CVE-2022-42004) | | CVE-2022-42004 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-42004](https://nvd.nist.gov/vuln/detail/CVE-2022-42004) | | CVE-2022-42004 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-42004](https://nvd.nist.gov/vuln/detail/CVE-2022-42004) | | CVE-2022-42004 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-42004](https://nvd.nist.gov/vuln/detail/CVE-2022-42004) | | CVE-2022-45685 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-45685](https://nvd.nist.gov/vuln/detail/CVE-2022-45685) | | CVE-2022-45685 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-45685](https://nvd.nist.gov/vuln/detail/CVE-2022-45685) | | CVE-2022-45693 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-45693](https://nvd.nist.gov/vuln/detail/CVE-2022-45693) | | CVE-2022-45693 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-45693](https://nvd.nist.gov/vuln/detail/CVE-2022-45693) | | CVE-2022-46751 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-46751](https://nvd.nist.gov/vuln/detail/CVE-2022-46751) | | CVE-2022-46751 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-46751](https://nvd.nist.gov/vuln/detail/CVE-2022-46751) | | CVE-2023-1370 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-1370](https://nvd.nist.gov/vuln/detail/CVE-2023-1370) | | CVE-2023-1436 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-1436](https://nvd.nist.gov/vuln/detail/CVE-2023-1436) | | CVE-2023-1436 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-1436](https://nvd.nist.gov/vuln/detail/CVE-2023-1436) | | CVE-2023-2976 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-2976](https://nvd.nist.gov/vuln/detail/CVE-2023-2976) | | CVE-2023-2976 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-2976](https://nvd.nist.gov/vuln/detail/CVE-2023-2976) | | CVE-2023-2976 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-2976](https://nvd.nist.gov/vuln/detail/CVE-2023-2976) | | CVE-2023-2976 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-2976](https://nvd.nist.gov/vuln/detail/CVE-2023-2976) | | CVE-2023-2976 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-2976](https://nvd.nist.gov/vuln/detail/CVE-2023-2976) | | CVE-2023-2976 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-2976](https://nvd.nist.gov/vuln/detail/CVE-2023-2976) | | CVE-2023-2976 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-2976](https://nvd.nist.gov/vuln/detail/CVE-2023-2976) | | CVE-2023-2976 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-2976](https://nvd.nist.gov/vuln/detail/CVE-2023-2976) | | CVE-2023-2976 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-2976](https://nvd.nist.gov/vuln/detail/CVE-2023-2976) | | CVE-2023-2976 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-2976](https://nvd.nist.gov/vuln/detail/CVE-2023-2976) | | CVE-2023-3635 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-3635](https://nvd.nist.gov/vuln/detail/CVE-2023-3635) | | CVE-2023-39410 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-39410](https://nvd.nist.gov/vuln/detail/CVE-2023-39410) | | CVE-2023-5072 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-5072](https://nvd.nist.gov/vuln/detail/CVE-2023-5072) | | CVE-2023-6378 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-6378](https://nvd.nist.gov/vuln/detail/CVE-2023-6378) | | CVE-2023-6378 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-6378](https://nvd.nist.gov/vuln/detail/CVE-2023-6378) | | CVE-2023-6481 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-6481](https://nvd.nist.gov/vuln/detail/CVE-2023-6481) | | CVE-2023-6481 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-6481](https://nvd.nist.gov/vuln/detail/CVE-2023-6481) | | CVE-2024-13009 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-13009](https://nvd.nist.gov/vuln/detail/CVE-2024-13009) | | CVE-2024-13009 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-13009](https://nvd.nist.gov/vuln/detail/CVE-2024-13009) | | CVE-2024-29131 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-29131](https://nvd.nist.gov/vuln/detail/CVE-2024-29131) | | CVE-2024-29857 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-29857](https://nvd.nist.gov/vuln/detail/CVE-2024-29857) | | CVE-2024-29857 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-29857](https://nvd.nist.gov/vuln/detail/CVE-2024-29857) | | CVE-2024-29857 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-29857](https://nvd.nist.gov/vuln/detail/CVE-2024-29857) | | CVE-2024-30172 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-30172](https://nvd.nist.gov/vuln/detail/CVE-2024-30172) | | CVE-2024-30172 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-30172](https://nvd.nist.gov/vuln/detail/CVE-2024-30172) | | CVE-2024-34447 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-34447](https://nvd.nist.gov/vuln/detail/CVE-2024-34447) | | CVE-2024-34447 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-34447](https://nvd.nist.gov/vuln/detail/CVE-2024-34447) | | CVE-2024-36114 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-36114](https://nvd.nist.gov/vuln/detail/CVE-2024-36114) | | CVE-2024-36114 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-36114](https://nvd.nist.gov/vuln/detail/CVE-2024-36114) | | CVE-2024-47561 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-47561](https://nvd.nist.gov/vuln/detail/CVE-2024-47561) | | CVE-2024-57699 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-57699](https://nvd.nist.gov/vuln/detail/CVE-2024-57699) | | CVE-2024-7254 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-7254](https://nvd.nist.gov/vuln/detail/CVE-2024-7254) | | CVE-2024-7254 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-7254](https://nvd.nist.gov/vuln/detail/CVE-2024-7254) | | CVE-2024-7254 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-7254](https://nvd.nist.gov/vuln/detail/CVE-2024-7254) | | CVE-2024-7254 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-7254](https://nvd.nist.gov/vuln/detail/CVE-2024-7254) | | CVE-2024-7254 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-7254](https://nvd.nist.gov/vuln/detail/CVE-2024-7254) | | CVE-2024-7254 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-7254](https://nvd.nist.gov/vuln/detail/CVE-2024-7254) | | CVE-2025-1948 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2025-1948](https://nvd.nist.gov/vuln/detail/CVE-2025-1948) | | CVE-2025-1948 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2025-1948](https://nvd.nist.gov/vuln/detail/CVE-2025-1948) | | CVE-2025-24789 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2025-24789](https://nvd.nist.gov/vuln/detail/CVE-2025-24789) | | CVE-2025-24970 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2025-24970](https://nvd.nist.gov/vuln/detail/CVE-2025-24970) | | CVE-2025-24970 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2025-24970](https://nvd.nist.gov/vuln/detail/CVE-2025-24970) | | CVE-2025-24970 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2025-24970](https://nvd.nist.gov/vuln/detail/CVE-2025-24970) | | CVE-2025-27553 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2025-27553](https://nvd.nist.gov/vuln/detail/CVE-2025-27553) | | CVE-2025-27820 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2025-27820](https://nvd.nist.gov/vuln/detail/CVE-2025-27820) | | CVE-2025-27820 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2025-27820](https://nvd.nist.gov/vuln/detail/CVE-2025-27820) | | CVE-2025-46762 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2025-46762](https://nvd.nist.gov/vuln/detail/CVE-2025-46762) | | CVE-2012-5783 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2012-5783](https://nvd.nist.gov/vuln/detail/CVE-2012-5783) | | CVE-2012-6153 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2012-6153](https://nvd.nist.gov/vuln/detail/CVE-2012-6153) | | CVE-2018-10237 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2018-10237](https://nvd.nist.gov/vuln/detail/CVE-2018-10237) | | CVE-2018-10237 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2018-10237](https://nvd.nist.gov/vuln/detail/CVE-2018-10237) | | CVE-2018-10237 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2018-10237](https://nvd.nist.gov/vuln/detail/CVE-2018-10237) | | CVE-2018-10237 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2018-10237](https://nvd.nist.gov/vuln/detail/CVE-2018-10237) | | CVE-2018-10237 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2018-10237](https://nvd.nist.gov/vuln/detail/CVE-2018-10237) | | CVE-2018-11798 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2018-11798](https://nvd.nist.gov/vuln/detail/CVE-2018-11798) | | CVE-2018-1313 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2018-1313](https://nvd.nist.gov/vuln/detail/CVE-2018-1313) | | CVE-2019-12384 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2019-12384](https://nvd.nist.gov/vuln/detail/CVE-2019-12384) | | CVE-2019-12384 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2019-12384](https://nvd.nist.gov/vuln/detail/CVE-2019-12384) | | CVE-2019-12814 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2019-12814](https://nvd.nist.gov/vuln/detail/CVE-2019-12814) | | CVE-2019-12814 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2019-12814](https://nvd.nist.gov/vuln/detail/CVE-2019-12814) | | CVE-2020-13955 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2020-13955](https://nvd.nist.gov/vuln/detail/CVE-2020-13955) | | CVE-2020-13956 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2020-13956](https://nvd.nist.gov/vuln/detail/CVE-2020-13956) | | CVE-2020-15250 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2020-15250](https://nvd.nist.gov/vuln/detail/CVE-2020-15250) | | CVE-2020-15250 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2020-15250](https://nvd.nist.gov/vuln/detail/CVE-2020-15250) | | CVE-2020-15522 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2020-15522](https://nvd.nist.gov/vuln/detail/CVE-2020-15522) | | CVE-2020-15522 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2020-15522](https://nvd.nist.gov/vuln/detail/CVE-2020-15522) | | CVE-2020-17521 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2020-17521](https://nvd.nist.gov/vuln/detail/CVE-2020-17521) | | CVE-2020-17521 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2020-17521](https://nvd.nist.gov/vuln/detail/CVE-2020-17521) | | CVE-2020-1945 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2020-1945](https://nvd.nist.gov/vuln/detail/CVE-2020-1945) | | CVE-2020-26939 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2020-26939](https://nvd.nist.gov/vuln/detail/CVE-2020-26939) | | CVE-2020-26939 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2020-26939](https://nvd.nist.gov/vuln/detail/CVE-2020-26939) | | CVE-2021-22569 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2021-22569](https://nvd.nist.gov/vuln/detail/CVE-2021-22569) | | CVE-2021-22569 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2021-22569](https://nvd.nist.gov/vuln/detail/CVE-2021-22569) | | CVE-2021-22569 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2021-22569](https://nvd.nist.gov/vuln/detail/CVE-2021-22569) | | CVE-2021-22570 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2021-22570](https://nvd.nist.gov/vuln/detail/CVE-2021-22570) | | CVE-2021-22570 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2021-22570](https://nvd.nist.gov/vuln/detail/CVE-2021-22570) | | CVE-2021-27568 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2021-27568](https://nvd.nist.gov/vuln/detail/CVE-2021-27568) | | CVE-2021-28170 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2021-28170](https://nvd.nist.gov/vuln/detail/CVE-2021-28170) | | CVE-2021-29425 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2021-29425](https://nvd.nist.gov/vuln/detail/CVE-2021-29425) | | CVE-2021-36373 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2021-36373](https://nvd.nist.gov/vuln/detail/CVE-2021-36373) | | CVE-2021-36374 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2021-36374](https://nvd.nist.gov/vuln/detail/CVE-2021-36374) | | CVE-2022-38749 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2022-38749](https://nvd.nist.gov/vuln/detail/CVE-2022-38749) | | CVE-2022-38750 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2022-38750](https://nvd.nist.gov/vuln/detail/CVE-2022-38750) | | CVE-2022-38751 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2022-38751](https://nvd.nist.gov/vuln/detail/CVE-2022-38751) | | CVE-2022-38752 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2022-38752](https://nvd.nist.gov/vuln/detail/CVE-2022-38752) | | CVE-2022-41854 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2022-41854](https://nvd.nist.gov/vuln/detail/CVE-2022-41854) | | CVE-2023-0833 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2023-0833](https://nvd.nist.gov/vuln/detail/CVE-2023-0833) | | CVE-2023-33201 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2023-33201](https://nvd.nist.gov/vuln/detail/CVE-2023-33201) | | CVE-2023-33201 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2023-33201](https://nvd.nist.gov/vuln/detail/CVE-2023-33201) | | CVE-2023-33201 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2023-33201](https://nvd.nist.gov/vuln/detail/CVE-2023-33201) | | CVE-2023-33202 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2023-33202](https://nvd.nist.gov/vuln/detail/CVE-2023-33202) | | CVE-2023-33202 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2023-33202](https://nvd.nist.gov/vuln/detail/CVE-2023-33202) | | CVE-2023-33202 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2023-33202](https://nvd.nist.gov/vuln/detail/CVE-2023-33202) | | CVE-2023-42503 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2023-42503](https://nvd.nist.gov/vuln/detail/CVE-2023-42503) | | CVE-2023-51074 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2023-51074](https://nvd.nist.gov/vuln/detail/CVE-2023-51074) | | CVE-2024-12798 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-12798](https://nvd.nist.gov/vuln/detail/CVE-2024-12798) | | CVE-2024-12798 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-12798](https://nvd.nist.gov/vuln/detail/CVE-2024-12798) | | CVE-2024-12798 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-12798](https://nvd.nist.gov/vuln/detail/CVE-2024-12798) | | CVE-2024-12798 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-12798](https://nvd.nist.gov/vuln/detail/CVE-2024-12798) | | CVE-2024-12798 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-12798](https://nvd.nist.gov/vuln/detail/CVE-2024-12798) | | CVE-2024-12798 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-12798](https://nvd.nist.gov/vuln/detail/CVE-2024-12798) | | CVE-2024-12798 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-12798](https://nvd.nist.gov/vuln/detail/CVE-2024-12798) | | CVE-2024-23944 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-23944](https://nvd.nist.gov/vuln/detail/CVE-2024-23944) | | CVE-2024-23945 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-23945](https://nvd.nist.gov/vuln/detail/CVE-2024-23945) | | CVE-2024-23945 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-23945](https://nvd.nist.gov/vuln/detail/CVE-2024-23945) | | CVE-2024-23953 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-23953](https://nvd.nist.gov/vuln/detail/CVE-2024-23953) | | CVE-2024-23953 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-23953](https://nvd.nist.gov/vuln/detail/CVE-2024-23953) | | CVE-2024-25710 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-25710](https://nvd.nist.gov/vuln/detail/CVE-2024-25710) | | CVE-2024-25710 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-25710](https://nvd.nist.gov/vuln/detail/CVE-2024-25710) | | CVE-2024-26308 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-26308](https://nvd.nist.gov/vuln/detail/CVE-2024-26308) | | CVE-2024-26308 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-26308](https://nvd.nist.gov/vuln/detail/CVE-2024-26308) | | CVE-2024-29025 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-29025](https://nvd.nist.gov/vuln/detail/CVE-2024-29025) | | CVE-2024-29133 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-29133](https://nvd.nist.gov/vuln/detail/CVE-2024-29133) | | CVE-2024-29869 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-29869](https://nvd.nist.gov/vuln/detail/CVE-2024-29869) | | CVE-2024-29869 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-29869](https://nvd.nist.gov/vuln/detail/CVE-2024-29869) | | CVE-2024-30171 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-30171](https://nvd.nist.gov/vuln/detail/CVE-2024-30171) | | CVE-2024-30171 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-30171](https://nvd.nist.gov/vuln/detail/CVE-2024-30171) | | CVE-2024-30171 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-30171](https://nvd.nist.gov/vuln/detail/CVE-2024-30171) | | CVE-2024-31141 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-31141](https://nvd.nist.gov/vuln/detail/CVE-2024-31141) | | CVE-2024-43382 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-43382](https://nvd.nist.gov/vuln/detail/CVE-2024-43382) | | CVE-2024-47535 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-47535](https://nvd.nist.gov/vuln/detail/CVE-2024-47535) | | CVE-2024-47535 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-47535](https://nvd.nist.gov/vuln/detail/CVE-2024-47535) | | CVE-2024-47535 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-47535](https://nvd.nist.gov/vuln/detail/CVE-2024-47535) | | CVE-2024-47554 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-47554](https://nvd.nist.gov/vuln/detail/CVE-2024-47554) | | CVE-2024-47554 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-47554](https://nvd.nist.gov/vuln/detail/CVE-2024-47554) | | CVE-2024-47554 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-47554](https://nvd.nist.gov/vuln/detail/CVE-2024-47554) | | CVE-2024-47554 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-47554](https://nvd.nist.gov/vuln/detail/CVE-2024-47554) | | CVE-2024-52980 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-52980](https://nvd.nist.gov/vuln/detail/CVE-2024-52980) | | CVE-2024-56128 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-56128](https://nvd.nist.gov/vuln/detail/CVE-2024-56128) | | CVE-2024-6763 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-6763](https://nvd.nist.gov/vuln/detail/CVE-2024-6763) | | CVE-2024-6763 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-6763](https://nvd.nist.gov/vuln/detail/CVE-2024-6763) | | CVE-2024-6763 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-6763](https://nvd.nist.gov/vuln/detail/CVE-2024-6763) | | CVE-2024-6763 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-6763](https://nvd.nist.gov/vuln/detail/CVE-2024-6763) | | CVE-2024-6763 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-6763](https://nvd.nist.gov/vuln/detail/CVE-2024-6763) | | CVE-2024-8184 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-8184](https://nvd.nist.gov/vuln/detail/CVE-2024-8184) | | CVE-2024-9823 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-9823](https://nvd.nist.gov/vuln/detail/CVE-2024-9823) | | CVE-2025-24790 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2025-24790](https://nvd.nist.gov/vuln/detail/CVE-2025-24790) | | CVE-2025-25193 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2025-25193](https://nvd.nist.gov/vuln/detail/CVE-2025-25193) | | CVE-2025-25193 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2025-25193](https://nvd.nist.gov/vuln/detail/CVE-2025-25193) | | CVE-2025-25193 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2025-25193](https://nvd.nist.gov/vuln/detail/CVE-2025-25193) | | CVE-2025-25193 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2025-25193](https://nvd.nist.gov/vuln/detail/CVE-2025-25193) | | CVE-2025-30474 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2025-30474](https://nvd.nist.gov/vuln/detail/CVE-2025-30474) | | CVE-2025-3588 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2025-3588](https://nvd.nist.gov/vuln/detail/CVE-2025-3588) | | CVE-2025-46551 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2025-46551](https://nvd.nist.gov/vuln/detail/CVE-2025-46551) | | CVE-2020-8908 | LOW | [https://nvd.nist.gov/vuln/detail/CVE-2020-8908](https://nvd.nist.gov/vuln/detail/CVE-2020-8908) | | CVE-2020-8908 | LOW | [https://nvd.nist.gov/vuln/detail/CVE-2020-8908](https://nvd.nist.gov/vuln/detail/CVE-2020-8908) | | CVE-2020-8908 | LOW | [https://nvd.nist.gov/vuln/detail/CVE-2020-8908](https://nvd.nist.gov/vuln/detail/CVE-2020-8908) | | CVE-2020-8908 | LOW | [https://nvd.nist.gov/vuln/detail/CVE-2020-8908](https://nvd.nist.gov/vuln/detail/CVE-2020-8908) | | CVE-2020-8908 | LOW | [https://nvd.nist.gov/vuln/detail/CVE-2020-8908](https://nvd.nist.gov/vuln/detail/CVE-2020-8908) | | CVE-2020-8908 | LOW | [https://nvd.nist.gov/vuln/detail/CVE-2020-8908](https://nvd.nist.gov/vuln/detail/CVE-2020-8908) | | CVE-2020-8908 | LOW | [https://nvd.nist.gov/vuln/detail/CVE-2020-8908](https://nvd.nist.gov/vuln/detail/CVE-2020-8908) | | CVE-2020-8908 | LOW | [https://nvd.nist.gov/vuln/detail/CVE-2020-8908](https://nvd.nist.gov/vuln/detail/CVE-2020-8908) | | CVE-2020-8908 | LOW | [https://nvd.nist.gov/vuln/detail/CVE-2020-8908](https://nvd.nist.gov/vuln/detail/CVE-2020-8908) | | CVE-2020-8908 | LOW | [https://nvd.nist.gov/vuln/detail/CVE-2020-8908](https://nvd.nist.gov/vuln/detail/CVE-2020-8908) | | CVE-2024-12801 | LOW | [https://nvd.nist.gov/vuln/detail/CVE-2024-12801](https://nvd.nist.gov/vuln/detail/CVE-2024-12801) | | CVE-2024-12801 | LOW | [https://nvd.nist.gov/vuln/detail/CVE-2024-12801](https://nvd.nist.gov/vuln/detail/CVE-2024-12801) | | CVE-2024-12801 | LOW | [https://nvd.nist.gov/vuln/detail/CVE-2024-12801](https://nvd.nist.gov/vuln/detail/CVE-2024-12801) | | CVE-2024-12801 | LOW | [https://nvd.nist.gov/vuln/detail/CVE-2024-12801](https://nvd.nist.gov/vuln/detail/CVE-2024-12801) | | CVE-2025-27496 | LOW | [https://nvd.nist.gov/vuln/detail/CVE-2025-27496](https://nvd.nist.gov/vuln/detail/CVE-2025-27496) | | CVE-2025-46392 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2025-46392](https://nvd.nist.gov/vuln/detail/CVE-2025-46392) | ## 7.9.0 release notes The following release notes include information for Amazon EMR release 7.9.0. **New features** + **Application upgrades** – Amazon EMR 7.9.0 application upgrades include AWS SDK for Java 2.31.16/1.12.782, Delta 3.3.0-amzn-1, Hadoop 3.4.1, Hudi 0.15.0-amzn-6, Iceberg 1.7.1-amzn-2, Spark 3.5.5, and Zookeeper 3.9.3. + **CM-CMK support for Persistent App UI** – This feature extends SSE encryption applied to logs managed in the EMR service bucket to honor Customer Managed Custom Master Key (CM-CMK). This feature enables customers to manage accessibility of logs after their clusters terminate. **Known issues** + When the logging option for EMR on EC2 is disabled, logs are not available after the cluster terminates and persistent application UIs will not contain any logs. + **KMS encryption using the EMR-EC2 SecretAgent component** – There is an error that occurs when KMS encryption at rest is enabled for Lake Formation-enabled clusters in EMR versions 7.8 and 7.9. Lake Formation-enabled jobs will fail with a *NoSuchMethodError*. There are two workarounds available: + Temporarily disable KMS encryption at rest for affected clusters. + Contact AWS Support to request a Bootstrap Action artifact if KMS encryption must be maintained. **Note** **Permanent fix** – In August 2025 an update to address the above issue was deployed to all regions. There is no need to disable KMS encryption at rest. + **Uploading step logs** – Some clusters may not be able to upload step logs to your S3 bucket due to a race condition. Reach out to AWS support team if you encounter this issue. + **HBase table creation in Hive with Glue catalog** – From EMR-7.3.0 to EMR-7.10.0, there is a bug due to Hive Iceberg integration, which causes HBase table creation in Hive to fail when Glue Data Catalog is used as the metastore. Please reach out to the AWS support team if you encounter this issue. + **Tez UI** – From EMR-7.3 to EMR-7.10, there is a bug where accessing Tez UI through YARN Resource Manager Web UI fails when in-transit encryption is enabled. This occurs because YARN proxy attempts to use HTTP protocol while Tez UI requires HTTPS. Please modify the `tez.tez-ui.history-url.base` configuration in `tez-site` from `http` to `https` to resolve this issue. + **Missing step and container logs** - From Amazon EMR 7.9 to Amazon EMR 7.12, there is a bug where step logs and/or container logs may not get uploaded to S3 if the step or application is running for longer than 3 hours. Please upgrade to Amazon EMR 7.13 to resolve this issue. **Changes, enhancements, and resolved issues** + Starting with Spark 3.3.1 (supported in EMR versions 6.10 and above), all executors in a decommissioning host are set to a new `ExecutorState`, called *DECOMMISSIONING* state. The executors being decommissioned cannot be used by Yarn to allocate tasks and thus it will request for new executors, if needed, for the tasks being executed. Thus, if you disable Spark DRA while using EMR Managed Scaling, EMR Auto Scaling, or any custom scaling mechanism on EMR-EC2 clusters, then Yarn may request maximum permissible executors for each job. In order to avoid this issue, leave the `spark.dynamicAllocation.enabled` property set to `TRUE` (which is the default) when you are using the above combination of features. In addition, you can also set minimum and maximum executor constraints by setting values for `spark.dynamicAllocation.maxExecutors` and `spark.dynamicAllocation.minExecutors` properties for your Spark jobs, to restrict the number of executors allocated during the job’s execution. + *Oozie deprecation* – Because OSS Oozie has gone into attic mode, Apache Oozie will be removed from Amazon EMR release 8.x series. Oozie will still be included as part of 7.x series. We recommend that you use MWAA as an alternative to Oozie. Customers may refer to the utility [oozie-to-airflow-emr](https://github.com/dgghosalaws/oozie-to-airflow-emr) on GitHub to migrate existing Oozie workflows to MWAA. + *YARN container bin packing* – Starting with Amazon EMR version 7.9.0, container bin-packing policy is now available for the YARN capacity scheduler, which is built on top of YARN's multi-node placement policy. + The *hudi-cli.sh* – The **hudi-cli.sh** script has been deprecated in EMR release 7.9.0 and higher. Amazon EMR release 7.9.0 and higher uses **hudi-cli-bundle.jar**. + *EMR clusters on FIPS-enabled AL2023 AMI* – Starting from EMR 7.9, customers can launch EMR clusters on [FIPS-enabled AL2023 AMI](https://docs.aws.amazon.com/linux/al2023/ug/fips-mode.html). This change updates EMR’s digital signatures and hash algorithms to ensure all EMR packages adhere to recommended cryptographic policies for FIPS 140-3. Note: This does not cover the FIPS certification status of EMR or AL2023 - refer to [AL2023 FIPS FAQ](https://aws.amazon.com/linux/amazon-linux-2023/faqs/) for latest information on FIPS 140-3 validation status. + *Step latency* – This release resolves an issue where step latency increased when using in-transit encryption without explicitly selecting the Hadoop application for installation. + *Cluster launch with EBS-encrypted volumes* – This release fixes an issue where a KMS call on EBS-encrypted volumes could cause an EMR cluster launch to fail. + [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-790-release.html) ## 7.9.0 default Java versions Amazon EMR release 7.9 and higher ships with Amazon Corretto 17 (built on OpenJDK) by default for applications that support Corretto 17 (JDK 17), with the exception of Apache Livy. The following table shows the default Java versions for applications in Amazon EMR 7.9.0. If you want to change the default JVM on your cluster, follow the instructions in [Configure applications to use a specific Java Virtual Machine](configuring-java8.md) for each application that runs on the cluster. You can only use one Java runtime version for a cluster. Amazon EMR doesn't support running different nodes or applications on different runtime versions on the same cluster. | Application | Java / Amazon Corretto version (default is bold) | | --- | --- | | Delta | 17, 11, 8 | | Flink | 17, 11, 8 | | HBase | 17, 11, 8 | | HCatalog | 17, 11, 8 | | Hadoop | 17, 11, 8 | | Hive | 17, 11, 8 | | Hudi | 17, 11, 8 | | Iceberg | 17, 11 | | Livy | 17, 11, 8 | | Oozie | 17, 11, 8 | | Phoenix | 17, 11, 8 | | PrestoDB | 8 | | Spark | 17, 11, 8 | | Spark RAPIDS | 17, 11, 8 | | Tez | 17, 11, 8 | | Trino | 21, 17 | | Zeppelin | 17, 11, 8 | | Pig | 17, 11, 8 | | Zookeeper | 17, 11, 8 | ## 7.9.0 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | adot-java-agent | 1.31.0 | A Java Agent that collects metrics from application daemons. | | delta | 3.3.0-amzn-1 | Delta lake is an open table format for huge analytic datasets | | emr-amazon-cloudwatch-agent | 1.300032.2-amzn-0 | An application that collects internal system-level metrics and custom application metrics from Amazon EC2 instances. | | emr-ddb | 5.7.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 3.18.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.22.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-notebook-env | 1.18.0 | Conda env for emr notebook which includes jupyter enterprise gateway | | emr-s3-dist-cp | 2.39.0 | Distributed copy application optimized for Amazon S3. | | emr-s3-select | 2.18.0 | EMR S3Select Connector | | emr-wal-cli | 1.3.4 | Cli used for emrwal list/deletion. | | emrfs | 2.70.0 | Amazon S3 connector for Hadoop ecosystem applications. | | flink-client | 1.20.0-amzn-3 | Apache Flink command line client scripts and applications. | | flink-jobmanager-config | 1.20.0-amzn-3 | Managing resources on EMR nodes for Apache Flink JobManager. | | hadoop-client | 3.4.1-amzn-1 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 3.4.1-amzn-1 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 3.4.1-amzn-1 | HDFS command-line client and library | | hadoop-hdfs-namenode | 3.4.1-amzn-1 | HDFS service for tracking file names and block locations. | | hadoop-hdfs-journalnode | 3.4.1-amzn-1 | HDFS service for managing the Hadoop filesystem journal on HA clusters. | | hadoop-httpfs-server | 3.4.1-amzn-1 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 3.4.1-amzn-1 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 3.4.1-amzn-1 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 3.4.1-amzn-1 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 3.4.1-amzn-1 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 3.4.1-amzn-1 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 2.6.2-amzn-0 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 2.6.2-amzn-0 | Service for serving one or more HBase regions. | | hbase-client | 2.6.2-amzn-0 | HBase command-line client. | | hbase-rest-server | 2.6.2-amzn-0 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 2.6.2-amzn-0 | Service providing a Thrift endpoint to HBase. | | hbase-operator-tools | 2.6.2-amzn-0 | Repair tool for Apache HBase clusters. | | hcatalog-client | 3.1.3-amzn-18 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 3.1.3-amzn-18 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 3.1.3-amzn-18 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 3.1.3-amzn-18 | Hive command line client. | | hive-hbase | 3.1.3-amzn-18 | Hive-hbase client. | | hive-metastore-server | 3.1.3-amzn-18 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server2 | 3.1.3-amzn-18 | Service for accepting Hive queries as web requests. | | hudi | 0.15.0-amzn-6 | Incremental processing framework to power data pipeline at low latency and high efficiency. | | hudi-presto | 0.15.0-amzn-6 | Bundle library for running Presto with Hudi. | | hudi-trino | 0.15.0-amzn-6 | Bundle library for running Trino with Hudi. | | hudi-spark | 0.15.0-amzn-6 | Bundle library for running Spark with Hudi. | | hue-server | 4.11.0 | Web application for analyzing data using Hadoop ecosystem applications | | iceberg | 1.7.1-amzn-2 | Apache Iceberg is an open table format for huge analytic datasets | | jupyterhub | 1.5.0 | Multi-user server for Jupyter notebooks | | livy-server | 0.8.0-incubating | REST interface for interacting with Apache Spark | | nginx | 1.12.1 | nginx [engine x] is an HTTP and reverse proxy server | | mariadb-server | 5.5.68\$1 | MariaDB database server. | | nvidia-cuda | 12.3.0 | Nvidia drivers and Cuda toolkit | | oozie-client | 5.2.1 | Oozie command-line client. | | oozie-server | 5.2.1 | Service for accepting Oozie workflow requests. | | opencv | 4.7.0 | Open Source Computer Vision Library. | | phoenix-library | 5.2.1 | The phoenix libraries for server and client | | phoenix-connectors | 6.0.0 | Apache Phoenix-Connectors for Spark-3 | | phoenix-query-server | 6.0.0 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.287-amzn-3 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.287-amzn-3 | Service for executing pieces of a query. | | presto-client | 0.287-amzn-3 | Presto command-line client which is installed on an HA cluster's stand-by masters where Presto server is not started. | | trino-coordinator | 467-amzn-2 | Service for accepting queries and managing query execution among trino-workers. | | trino-worker | 467-amzn-2 | Service for executing pieces of a query. | | trino-client | 467-amzn-2 | Trino command-line client which is installed on an HA cluster's stand-by masters where Trino server is not started. | | pig-client | 0.17.0 | Pig command-line client. | | r | 4.3.2 | The R Project for Statistical Computing | | ranger-kms-server | 2.0.0 | Apache Ranger Key Management System | | spark-client | 3.5.5-amzn-0 | Spark command-line clients. | | spark-history-server | 3.5.5-amzn-0 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 3.5.5-amzn-0 | In-memory execution engine for YARN. | | spark-yarn-slave | 3.5.5-amzn-0 | Apache Spark libraries needed by YARN slaves. | | spark-rapids | 25.02.1-amzn-0 | Nvidia Spark RAPIDS plugin that accelerates Apache Spark with GPUs. | | tensorflow | 2.16.1 | TensorFlow open source software library for high performance numerical computation. | | tez-on-yarn | 0.10.2-amzn-16 | The tez YARN application and libraries. | | tez-on-worker | 0.10.2-amzn-16 | The tez YARN application and libraries for worker nodes. | | zeppelin-server | 0.11.1 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.9.3-amzn-1 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.9.3-amzn-1 | ZooKeeper command line client. | ## 7.9.0 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). Reconfiguration actions occur when you specify a configuration for instance groups in a running cluster. Amazon EMR only initiates reconfiguration actions for the classifications that you modify. For more information, see [Reconfigure an instance group in a running cluster](emr-configure-apps-running-cluster.md). **emr-7.9.0 classifications** | Classifications | Description | Reconfiguration Actions | | --- | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | Restarts the ResourceManager service. | | container-executor | Change values in Hadoop YARN's container-executor.cfg file. | Not available. | | container-log4j | Change values in Hadoop YARN's container-log4j.properties file. | Not available. | | core-site | Change values in Hadoop's core-site.xml file. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts Hadoop KMS, Ranger KMS, HiveServer2, Hive MetaStore, Hadoop Httpfs, and MapReduce-HistoryServer. | | docker-conf | Change docker related settings. | Not available. | | emrfs-site | Change EMRFS settings. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts HBaseRegionserver, HBaseMaster, HBaseThrift, HBaseRest, HiveServer2, Hive MetaStore, Hadoop Httpfs, and MapReduce-HistoryServer. | | flink-conf | Change flink-conf.yaml settings. | Restarts Flink history server. | | flink-log4j | Change Flink log4j.properties settings. | Restarts Flink history server. | | flink-log4j-session | Change Flink log4j-session.properties settings for Kubernetes/Yarn session. | Restarts Flink history server. | | flink-log4j-cli | Change Flink log4j-cli.properties settings. | Restarts Flink history server. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts PhoenixQueryserver, HiveServer2, Hive MetaStore, and MapReduce-HistoryServer. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | Restarts the Hadoop HDFS services SecondaryNamenode, Datanode, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts Hadoop KMS, Hadoop Httpfs, and MapReduce-HistoryServer. | | hadoop-ssl-server | Change hadoop ssl server configuration | Not available. | | hadoop-ssl-client | Change hadoop ssl client configuration | Not available. | | hbase | Amazon EMR-curated settings for Apache HBase. | Custom EMR specific property. Sets emrfs-site and hbase-site configs. See those for their associated restarts. | | hbase-env | Change values in HBase's environment. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. | | hbase-log4j2 | Change values in HBase's hbase-log4j2.properties file. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbase.properties file. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | Not available. | | hbase-site | Change values in HBase's hbase-site.xml file. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. Additionally restarts Phoenix QueryServer. | | hdfs-encryption-zones | Configure HDFS encryption zones. | This classification should not be reconfigured. | | hdfs-env | Change values in the HDFS environment. | Restarts Hadoop HDFS services Namenode, Datanode, and ZKFC. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Additionally restarts Hadoop Httpfs. | | hcatalog-env | Change values in HCatalog's environment. | Restarts Hive HCatalog Server. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | Restarts Hive HCatalog Server. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | Restarts Hive HCatalog Server. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | Restarts Hive WebHCat server. | | hcatalog-webhcat-log4j2 | Change values in HCatalog WebHCat's log4j2.properties. | Restarts Hive WebHCat server. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | Restarts Hive WebHCat server. | | hive | Amazon EMR-curated settings for Apache Hive. | Sets configurations to launch Hive LLAP service. | | hive-beeline-log4j2 | Change values in Hive's beeline-log4j2.properties file. | Not available. | | hive-parquet-logging | Change values in Hive's parquet-logging.properties file. | Not available. | | hive-env | Change values in the Hive environment. | Restarts HiveServer2, HiveMetastore, and Hive HCatalog-Server. Runs Hive schemaTool CLI commands to verify hive-metastore. | | hive-exec-log4j2 | Change values in Hive's hive-exec-log4j2.properties file. | Not available. | | hive-llap-daemon-log4j2 | Change values in Hive's llap-daemon-log4j2.properties file. | Not available. | | hive-log4j2 | Change values in Hive's hive-log4j2.properties file. | Not available. | | hive-site | Change values in Hive's hive-site.xml file | Restarts HiveServer2, HiveMetastore, and Hive HCatalog-Server. Runs Hive schemaTool CLI commands to verify hive-metastore. Also restarts Oozie and Zeppelin. | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | Not available. | | hue-ini | Change values in Hue's ini file | Restarts Hue. Also activates Hue config override CLI commands to pick up new configurations. | | httpfs-env | Change values in the HTTPFS environment. | Restarts Hadoop Httpfs service. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | Restarts Hadoop Httpfs service. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | Not available. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | Restarts Hadoop-KMS service. | | hadoop-kms-java-home | Change Hadoop's KMS java home | Not available. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | Not available. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | Restarts Hadoop-KMS and Ranger-KMS service. | | hudi-env | Change values in the Hudi environment. | Not available. | | hudi-defaults | Change values in Hudi's hudi-defaults.conf file. | Not available. | | iceberg-defaults | Change values in Iceberg's iceberg-defaults.conf file. | Not available. | | delta-defaults | Change values in Delta's delta-defaults.conf file. | Not available. | | jupyter-notebook-conf | Change values in Jupyter Notebook's jupyter\$1notebook\$1config.py file. | Not available. | | jupyter-hub-conf | Change values in JupyterHubs's jupyterhub\$1config.py file. | Not available. | | jupyter-s3-conf | Configure Jupyter Notebook S3 persistence. | Not available. | | jupyter-sparkmagic-conf | Change values in Sparkmagic's config.json file. | Not available. | | livy-conf | Change values in Livy's livy.conf file. | Restarts Livy Server. | | livy-env | Change values in the Livy environment. | Restarts Livy Server. | | livy-log4j2 | Change Livy log4j2.properties settings. | Restarts Livy Server. | | mapred-env | Change values in the MapReduce application's environment. | Restarts Hadoop MapReduce-HistoryServer. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | Restarts Hadoop MapReduce-HistoryServer. | | oozie-env | Change values in Oozie's environment. | Restarts Oozie. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | Restarts Oozie. | | oozie-site | Change values in Oozie's oozie-site.xml file. | Restarts Oozie. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | Not available. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | Not available. | | phoenix-log4j2 | Change values in Phoenix's log4j2.properties file. | Restarts Phoenix-QueryServer. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | Not available. | | pig-env | Change values in the Pig environment. | Not available. | | pig-properties | Change values in Pig's pig.properties file. | Restarts Oozie. | | pig-log4j | Change values in Pig's log4j.properties file. | Not available. | | presto-log | Change values in Presto's log.properties file. | Restarts Presto-Server (for PrestoDB) | | presto-config | Change values in Presto's config.properties file. | Restarts Presto-Server (for PrestoDB) | | presto-password-authenticator | Change values in Presto's password-authenticator.properties file. | Not available. | | presto-env | Change values in Presto's presto-env.sh file. | Restarts Presto-Server (for PrestoDB) | | presto-node | Change values in Presto's node.properties file. | Not available. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | Not available. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | Not available. | | presto-connector-hive | Change values in Presto's hive.properties file. | Restarts Presto-Server (for PrestoDB) | | presto-connector-jmx | Change values in Presto's jmx.properties file. | Not available. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | Not available. | | presto-connector-lakeformation | Change values in Presto's lakeformation.properties file. | Restarts Presto-Server (for PrestoDB) | | presto-connector-localfile | Change values in Presto's localfile.properties file. | Not available. | | presto-connector-memory | Change values in Presto's memory.properties file. | Not available. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | Not available. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | Not available. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | Not available. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | Not available. | | presto-connector-redis | Change values in Presto's redis.properties file. | Not available. | | presto-connector-redshift | Change values in Presto's redshift.properties file. | Not available. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | Not available. | | presto-connector-tpcds | Change values in Presto's tpcds.properties file. | Not available. | | trino-log | Change values in Trino's log.properties file. | Restarts Trino-Server (for Trino) | | trino-config | Change values in Trino's config.properties file. | Restarts Trino-Server (for Trino) | | trino-password-authenticator | Change values in Trino's password-authenticator.properties file. | Restarts Trino-Server (for Trino) | | trino-env | Change values in Trino's trino-env.sh file. | Restarts Trino-Server (for Trino) | | trino-node | Change values in Trino's node.properties file. | Not available. | | trino-connector-blackhole | Change values in Trino's blackhole.properties file. | Not available. | | trino-connector-cassandra | Change values in Trino's cassandra.properties file. | Not available. | | trino-connector-delta | Change values in Trino's delta.properties file. | Restarts Trino-Server (for Trino) | | trino-connector-hive | Change values in Trino's hive.properties file. | Restarts Trino-Server (for Trino) | | trino-exchange-manager | Change values in Trino's exchange-manager.properties file. | Restarts Trino-Server (for Trino) | | trino-connector-iceberg | Change values in Trino's iceberg.properties file. | Restarts Trino-Server (for Trino) | | trino-connector-hudi | Change values in Trino's hudi.properties file. | Restarts Trino-Server (for Trino) | | trino-connector-jmx | Change values in Trino's jmx.properties file. | Not available. | | trino-connector-kafka | Change values in Trino's kafka.properties file. | Not available. | | trino-connector-localfile | Change values in Trino's localfile.properties file. | Not available. | | trino-connector-memory | Change values in Trino's memory.properties file. | Not available. | | trino-connector-mongodb | Change values in Trino's mongodb.properties file. | Not available. | | trino-connector-mysql | Change values in Trino's mysql.properties file. | Not available. | | trino-connector-postgresql | Change values in Trino's postgresql.properties file. | Not available. | | trino-connector-raptor | Change values in Trino's raptor.properties file. | Not available. | | trino-connector-redis | Change values in Trino's redis.properties file. | Not available. | | trino-connector-redshift | Change values in Trino's redshift.properties file. | Not available. | | trino-connector-tpch | Change values in Trino's tpch.properties file. | Not available. | | trino-connector-tpcds | Change values in Trino's tpcds.properties file. | Not available. | | ranger-kms-dbks-site | Change values in dbks-site.xml file of Ranger KMS. | Restarts Ranger KMS Server. | | ranger-kms-site | Change values in ranger-kms-site.xml file of Ranger KMS. | Restarts Ranger KMS Server. | | ranger-kms-env | Change values in the Ranger KMS environment. | Restarts Ranger KMS Server. | | ranger-kms-logback | Change values in kms-logback.xml file of Ranger KMS. | Not available. | | ranger-kms-db-ca | Change values for CA file on S3 for MySQL SSL connection with Ranger KMS. | Not available. | | spark | Amazon EMR-curated settings for Apache Spark. | This property modifies spark-defaults. See actions there. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | Restarts Spark history server and Spark thrift server. | | spark-env | Change values in the Spark environment. | Restarts Spark history server and Spark thrift server. | | spark-hive-site | Change values in Spark's hive-site.xml file | Not available. | | spark-log4j2 | Change values in Spark's log4j2.properties file. | Restarts Spark history server and Spark thrift server. | | spark-metrics | Change values in Spark's metrics.properties file. | Restarts Spark history server and Spark thrift server. | | tez-site | Change values in Tez's tez-site.xml file. | Restart Oozie and HiveServer2. | | yarn-env | Change values in the YARN environment. | Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts MapReduce-HistoryServer. | | yarn-site | Change values in YARN's yarn-site.xml file. | Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts Livy Server and MapReduce-HistoryServer. | | zeppelin-env | Change values in the Zeppelin environment. | Restarts Zeppelin. | | zeppelin-site | Change configuration settings in zeppelin-site.xml. | Restarts Zeppelin. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | Restarts Zookeeper server. | | zookeeper-logback | Change values in ZooKeeper's logback.xml file. | Restarts Zookeeper server. | | emr-metrics | Change emr metric settings for this node. | Restarts the CloudWatchAgent service. | ## 7.9.0 change log **Change log for 7.9.0 release and release notes** | Date | Event | Description | | --- | --- | --- | | 2025-05-19 | Docs publication | Amazon EMR 7.9.0 release notes first published | | 2025-05-19 | Deployment complete | Amazon EMR 7.9.0 fully deployed to all [supported Regions](https://aws.amazon.com/about-aws/global-infrastructure/regional-product-services/) | | 2025-05-19 | Initial limited release | Amazon EMR 7.9.0 first deployed to initial commercial Regions | # Amazon EMR release 7.8.0 ## 7.8.0 supported lifecycle The following table describes the supported lifecycle dates for Amazon EMR release 7.8.0. | Support phase | Date | | --- | --- | | Initial release date | March 7, 2025 | | Standard support until | March 6, 2027 | | End of support | March 7, 2027 | | End of life | March 7, 2028 | ## 7.8.0 application versions This release includes the following applications: [https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/Install-CloudWatch-Agent.html](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/Install-CloudWatch-Agent.html), [https://delta.io/](https://delta.io/), [https://flink.apache.org/](https://flink.apache.org/), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [https://hudi.apache.org](https://hudi.apache.org), [http://gethue.com/](http://gethue.com/), [https://iceberg.apache.org/](https://iceberg.apache.org/), [https://jupyter-enterprise-gateway.readthedocs.io/en/latest/](https://jupyter-enterprise-gateway.readthedocs.io/en/latest/), [https://jupyterhub.readthedocs.io/en/latest/#](https://jupyterhub.readthedocs.io/en/latest/#), [https://livy.incubator.apache.org/](https://livy.incubator.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [https://www.tensorflow.org/](https://www.tensorflow.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://trino.io/](https://trino.io/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-7.8.0 | emr-7.7.0 | emr-7.6.0 | emr-7.5.0 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 2.29.52, 1.12.780 | 2.29.44, 1.12.780 | 2.29.25, 1.12.779 | 2.28.8, 1.12.772 | | Python | 3.9, 3.11 | 3.9, 3.11 | 3.9, 3.11 | 3.9, 3.11 | | Scala | 2.12.18 | 2.12.18 | 2.12.18 | 2.12.18 | | AmazonCloudWatchAgent | 1.300032.2-amzn-0 | 1.300032.2-amzn-0 | 1.300032.2-amzn-0 | 1.300032.2-amzn-0 | | Delta | 3.3.0-amzn-0 | 3.2.1-amzn-2 | 3.2.1-amzn-1 | 3.2.1-amzn-0 | | Flink | 1.20.0-amzn-2 | 1.20.0-amzn-1 | 1.20.0-amzn-0 | 1.19.1-amzn-1 | | Ganglia | - | - | - | - | | HBase | 2.6.1-amzn-2 | 2.6.1-amzn-1 | 2.6.1-amzn-0 | 2.5.10-amzn-0 | | HCatalog | 3.1.3-amzn-17 | 3.1.3-amzn-16 | 3.1.3-amzn-15 | 3.1.3-amzn-14 | | Hadoop | 3.4.1-amzn-0 | 3.4.0-amzn-3 | 3.4.0-amzn-2 | 3.4.0-amzn-1 | | Hive | 3.1.3-amzn-17 | 3.1.3-amzn-16 | 3.1.3-amzn-15 | 3.1.3-amzn-14 | | Hudi | 0.15.0-amzn-5 | 0.15.0-amzn-4 | 0.15.0-amzn-3 | 0.15.0-amzn-2 | | Hue | 4.11.0 | 4.11.0 | 4.11.0 | 4.11.0 | | Iceberg | 1.7.1-amzn-1 | 1.7.1-amzn-0 | 1.6.1-amzn-2 | 1.6.1-amzn-1 | | JupyterEnterpriseGateway | 2.6.0 | 2.6.0 | 2.6.0 | 2.6.0 | | JupyterHub | 1.5.0 | 1.5.0 | 1.5.0 | 1.5.0 | | Livy | 0.8.0-incubating | 0.8.0-incubating | 0.8.0-incubating | 0.8.0-incubating | | MXNet | - | - | - | - | | Mahout | - | - | - | - | | Oozie | 5.2.1 | 5.2.1 | 5.2.1 | 5.2.1 | | Phoenix | 5.2.1 | 5.2.1 | 5.2.0 | 5.2.0 | | Pig | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | | Presto | 0.287-amzn-2 | 0.287-amzn-1 | 0.287-amzn-1 | 0.287-amzn-0 | | Spark | 3.5.4-amzn-0 | 3.5.3-amzn-1 | 3.5.3-amzn-0 | 3.5.2-amzn-1 | | Sqoop | - | - | - | - | | TensorFlow | 2.16.1 | 2.16.1 | 2.16.1 | 2.16.1 | | Tez | 0.10.2-amzn-15 | 0.10.2-amzn-14 | 0.10.2-amzn-13 | 0.10.2-amzn-12 | | Trino (PrestoSQL) | 467-amzn-1 | 467-amzn-0 | 457-amzn-0 | 446-amzn-0 | | Zeppelin | 0.11.1 | 0.11.1 | 0.11.1 | 0.11.1 | | ZooKeeper | 3.9.3-amzn-0 | 3.9.2-amzn-1 | 3.9.2-amzn-0 | 3.9.2-amzn-0 | # 7.8.0 release components details See the following table for more information about the *Extras* packages in Amazon EMR 7.7.0. | Application | Application Type | Package | Package Type | Package Version | | --- | --- | --- | --- | --- | | AmazonCloudWatchAgent | Core | hadoop-hdfs-journalnode | Extras | 3.4.1-amzn-0-1.amzn2023 | | AmazonCloudWatchAgent | Core | ranger-kms | Extras | 2.0.0-1.amzn2023 | | AmazonCloudWatchAgent | Core | s3-dist-cp | Extras | 2.38.0-1.amzn2023 | | AmazonCloudWatchAgent | Core | zookeeper | Extras | 3.9.3-amzn-0-1.amzn2023 | | AmazonCloudWatchAgent | Core | zookeeper-server | Extras | 3.9.3-amzn-0-1.amzn2023 | | Flink | Core | hadoop-client | Extras | 3.4.0.amzn.3-1.amzn2023 | | Flink | Core | hadoop-hdfs | Extras | 3.4.1-amzn-0-1.amzn2023 | | Flink | Core | hadoop-hdfs-datanode | Extras | 3.4.1-amzn-0-1.amzn2023 | | Flink | Core | hadoop-hdfs-journalnode | Extras | 3.4.1-amzn-0-1.amzn2023 | | Flink | Core | hadoop-hdfs-namenode | Extras | 3.4.1-amzn-0-1.amzn2023 | | Flink | Core | hadoop-httpfs | Extras | 3.4.1-amzn-0-1.amzn2023 | | Flink | Core | hadoop-kms | Extras | 3.4.1-amzn-0-1.amzn2023 | | Flink | Core | hadoop-mapreduce | Extras | 3.4.1-amzn-0-1.amzn2023 | | Flink | Core | hadoop-mapreduce-historyserver | Extras | 3.4.1-amzn-0-1.amzn2023 | | Flink | Core | hadoop-yarn | Extras | 3.4.1-amzn-0-1.amzn2023 | | Flink | Core | hadoop-yarn-nodemanager | Extras | 3.4.1-amzn-0-1.amzn2023 | | Flink | Core | hadoop-yarn-proxyserver | Extras | 3.4.1-amzn-0-1.amzn2023 | | Flink | Core | hadoop-yarn-resourcemanager | Extras | 3.4.1-amzn-0-1.amzn2023 | | Flink | Core | hadoop-yarn-timelineserver | Extras | 3.4.1-amzn-0-1.amzn2023 | | Flink | Core | hudi | Extras | 0.15.0-amzn-5-1.amzn2023 | | Flink | Core | ranger-kms | Extras | 2.0.0-1.amzn2023 | | Flink | Core | hadoop | Extras | 3.4.1-amzn-0-1.amzn2023 | | HBase | Core | s3-dist-cp | Extras | 2.38.0-1.amzn2023 | | HCatalog | Core | hadoop-yarn-proxyserver | Extras | 3.4.1-amzn-0-1.amzn2023 | | HCatalog | Core | mariadb-connector-java | Extras | | | Hive | Core | hadoop-httpfs | Extras | 3.4.1-amzn-0-1.amzn2023 | | Hive | Core | hadoop-yarn-proxyserver | Extras | 3.4.1-amzn-0-1.amzn2023 | | Hive | Core | hive-hbase | Extras | 3.1.3-amzn-17-1.amzn2023 | | Hive | Core | mariadb-connector-java | Extras | 3.1.3-amzn-17-1 | | Hive | Core | s3-dist-cp | Extras | 2.38.0-1.amzn2023 | | Phoenix | Core | s3-dist-cp | Extras | 2.38.0-1.amzn2023 | | Spark | Core | spark-datanucleus | Extras | 3.5.3.amzn.1-1.amzn2023 | | Tez | Core | hadoop-mapreduce | Extras | 3.4.1-amzn-0-1.amzn2023 | | Tez | Core | hadoop-mapreduce-historyserver | Extras | 3.4.1-amzn-0-1.amzn2023 | | Tez | Core | hadoop-yarn-proxyserver | Extras | 3.4.1-amzn-0-1.amzn2023 | | Trino | Core | bigtop-utils | Extras | 1.2.0-1.amzn2023 | | Trino | Core | hadoop-client | Extras | 3.4.1.amzn.0-1.amzn2023 | | Trino | Core | hadoop-hdfs | Extras | 3.4.1.amzn.0-1.amzn2023 | | Trino | Core | hadoop-hdfs-datanode | Extras | 3.4.1.amzn.0-1.amzn2023 | | Trino | Core | hadoop-hdfs-journalnode | Extras | 3.4.1.amzn.0-1.amzn2023 | | Trino | Core | hadoop-hdfs-namenode | Extras | 3.4.1.amzn.0-1.amzn2023 | | Trino | Core | hadoop-kms | Extras | 3.4.1.amzn.0-1.amzn2023 | | Trino | Core | hadoop-mapreduce | Extras | 3.4.1.amzn.0-1.amzn2023 | | Trino | Core | hadoop-mapreduce-historyserver | Extras | 3.4.1.amzn.0-1.amzn2023 | | Trino | Core | hadoop-yarn | Extras | 3.4.1.amzn.0-1.amzn2023 | | Trino | Core | hadoop-yarn-nodemanager | Extras | 3.4.1.amzn.0-1.amzn2023 | | Trino | Core | hadoop-yarn-proxyserver | Extras | 3.4.1.amzn.0-1.amzn2023 | | Trino | Core | hadoop-yarn-resourcemanager | Extras | 3.4.1.amzn.0-1.amzn2023 | | Trino | Core | hadoop-yarn-timelineserver | Extras | 3.4.1.amzn.0-1.amzn2023 | | Trino | Core | hive | Extras | 3.1.3.amzn.17-1.amzn2023 | | Trino | Core | hive-hcatalog-server | Extras | 3.1.3.amzn.17-1.amzn2023 | | Trino | Core | mariadb-connector-java | Extras | 2.7.2-1 | | Trino | Core | ranger-kms | Extras | 2.0.0-1.amzn2023 | | Trino | Core | zookeeper | Extras | 3.9.3.amzn.0-1.amzn2023 | | Trino | Core | zookeeper-server | Extras | 3.9.3.amzn.0-1.amzn2023 | | Trino | Core | bigtop-groovy | Extras | 2.5.4-1.amzn2023 | | Trino | Core | bigtop-jsvc | Extras | 1.2.4-1.amzn2023 | | Trino | Core | hadoop | Extras | 3.4.1.amzn.0-1.amzn2023 | | Trino | Core | hadoop-lzo | Extras | 0.4.19-1.amzn2023 | | Trino | Core | hive-hcatalog | Extras | 3.1.3.amzn.17-1.amzn2023 | | Trino | Core | hive-jdbc | Extras | 3.1.3.amzn.17-1.amzn2023 | | Zookeeper | Core | emrfs | Extras | 3.9.3.amzn.0-1.amzn2023 | | Zookeeper | Core | hadoop-client | Extras | 3.9.3.amzn.0-1.amzn2023 | | Zookeeper | Core | hadoop-hdfs | Extras | 3.9.3.amzn.0-1.amzn2023 | | Zookeeper | Core | hadoop-hdfs-datanode | Extras | 3.9.3.amzn.0-1.amzn2023 | | Zookeeper | Core | hadoop-hdfs-journalnode | Extras | 3.9.3.amzn.0-1.amzn2023 | | Zookeeper | Core | hadoop-hdfs-namenode | Extras | 3.9.3.amzn.0-1.amzn2023 | | Zookeeper | Core | hadoop-httpfs | Extras | 3.9.3.amzn.0-1.amzn2023 | | Zookeeper | Core | hadoop-kms | Extras | 3.9.3.amzn.0-1.amzn2023 | | Zookeeper | Core | hadoop-mapreduce | Extras | 3.4.1.amzn.0-1.amzn2023 | | Zookeeper | Core | hadoop-mapreduce-historyserver | Extras | 3.4.1.amzn.0-1.amzn2023 | | Zookeeper | Core | hadoop-yarn | Extras | 3.4.1.amzn.0-1.amzn2023 | | Zookeeper | Core | hadoop-yarn-nodemanager | Extras | 3.4.1.amzn.0-1.amzn2023 | | Zookeeper | Core | hadoop-yarn-proxyserver | Extras | 3.4.1.amzn.0-1.amzn2023 | | Zookeeper | Core | hadoop-yarn-resourcemanager | Extras | 3.4.1.amzn.0-1.amzn2023 | | Zookeeper | Core | hadoop-yarn-timelineserver | Extras | 3.4.1.amzn.0-1.amzn2023 | | Zookeeper | Core | ranger-kms | Extras | 2.0.0-1.amzn2023 | # 7.8.0 common vulnerabilities and exposures The following table lists all CVEs that don't impact EMR clusters that run on recommended configurations of Amazon EMR 7.8.0. Amazon EMR is dependent on upstream open-source for availability of fixes and will provide the latest stable version as part of the Core Engine components within 90 days of Amazon EMR verifying the fixes. This table was updated on March 6, 2025. | CVE ID | Severity | CVE details URL | | --- | --- | --- | | CVE-2015-1832 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2015-1832](https://nvd.nist.gov/vuln/detail/CVE-2015-1832) | | CVE-2016-5018 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2016-5018](https://nvd.nist.gov/vuln/detail/CVE-2016-5018) | | CVE-2017-15095 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2017-15095](https://nvd.nist.gov/vuln/detail/CVE-2017-15095) | | CVE-2017-17485 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2017-17485](https://nvd.nist.gov/vuln/detail/CVE-2017-17485) | | CVE-2017-7525 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2017-7525](https://nvd.nist.gov/vuln/detail/CVE-2017-7525) | | CVE-2018-11307 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-11307](https://nvd.nist.gov/vuln/detail/CVE-2018-11307) | | CVE-2018-11307 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-11307](https://nvd.nist.gov/vuln/detail/CVE-2018-11307) | | CVE-2018-14718 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-14718](https://nvd.nist.gov/vuln/detail/CVE-2018-14718) | | CVE-2018-14718 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-14718](https://nvd.nist.gov/vuln/detail/CVE-2018-14718) | | CVE-2018-14719 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-14719](https://nvd.nist.gov/vuln/detail/CVE-2018-14719) | | CVE-2018-14719 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-14719](https://nvd.nist.gov/vuln/detail/CVE-2018-14719) | | CVE-2018-14720 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-14720](https://nvd.nist.gov/vuln/detail/CVE-2018-14720) | | CVE-2018-14720 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-14720](https://nvd.nist.gov/vuln/detail/CVE-2018-14720) | | CVE-2018-14721 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-14721](https://nvd.nist.gov/vuln/detail/CVE-2018-14721) | | CVE-2018-14721 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-14721](https://nvd.nist.gov/vuln/detail/CVE-2018-14721) | | CVE-2018-19360 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-19360](https://nvd.nist.gov/vuln/detail/CVE-2018-19360) | | CVE-2018-19360 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-19360](https://nvd.nist.gov/vuln/detail/CVE-2018-19360) | | CVE-2018-19361 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-19361](https://nvd.nist.gov/vuln/detail/CVE-2018-19361) | | CVE-2018-19361 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-19361](https://nvd.nist.gov/vuln/detail/CVE-2018-19361) | | CVE-2018-19362 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-19362](https://nvd.nist.gov/vuln/detail/CVE-2018-19362) | | CVE-2018-19362 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-19362](https://nvd.nist.gov/vuln/detail/CVE-2018-19362) | | CVE-2018-7489 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-7489](https://nvd.nist.gov/vuln/detail/CVE-2018-7489) | | CVE-2018-7489 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-7489](https://nvd.nist.gov/vuln/detail/CVE-2018-7489) | | CVE-2019-10202 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-10202](https://nvd.nist.gov/vuln/detail/CVE-2019-10202) | | CVE-2019-14379 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-14379](https://nvd.nist.gov/vuln/detail/CVE-2019-14379) | | CVE-2019-14379 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-14379](https://nvd.nist.gov/vuln/detail/CVE-2019-14379) | | CVE-2019-14540 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-14540](https://nvd.nist.gov/vuln/detail/CVE-2019-14540) | | CVE-2019-14540 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-14540](https://nvd.nist.gov/vuln/detail/CVE-2019-14540) | | CVE-2019-14892 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-14892](https://nvd.nist.gov/vuln/detail/CVE-2019-14892) | | CVE-2019-14892 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-14892](https://nvd.nist.gov/vuln/detail/CVE-2019-14892) | | CVE-2019-14893 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-14893](https://nvd.nist.gov/vuln/detail/CVE-2019-14893) | | CVE-2019-14893 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-14893](https://nvd.nist.gov/vuln/detail/CVE-2019-14893) | | CVE-2019-16335 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-16335](https://nvd.nist.gov/vuln/detail/CVE-2019-16335) | | CVE-2019-16335 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-16335](https://nvd.nist.gov/vuln/detail/CVE-2019-16335) | | CVE-2019-16942 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-16942](https://nvd.nist.gov/vuln/detail/CVE-2019-16942) | | CVE-2019-16942 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-16942](https://nvd.nist.gov/vuln/detail/CVE-2019-16942) | | CVE-2019-16943 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-16943](https://nvd.nist.gov/vuln/detail/CVE-2019-16943) | | CVE-2019-16943 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-16943](https://nvd.nist.gov/vuln/detail/CVE-2019-16943) | | CVE-2019-17267 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-17267](https://nvd.nist.gov/vuln/detail/CVE-2019-17267) | | CVE-2019-17267 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-17267](https://nvd.nist.gov/vuln/detail/CVE-2019-17267) | | CVE-2019-17531 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-17531](https://nvd.nist.gov/vuln/detail/CVE-2019-17531) | | CVE-2019-17531 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-17531](https://nvd.nist.gov/vuln/detail/CVE-2019-17531) | | CVE-2019-20330 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-20330](https://nvd.nist.gov/vuln/detail/CVE-2019-20330) | | CVE-2019-20330 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-20330](https://nvd.nist.gov/vuln/detail/CVE-2019-20330) | | CVE-2020-8840 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2020-8840](https://nvd.nist.gov/vuln/detail/CVE-2020-8840) | | CVE-2020-8840 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2020-8840](https://nvd.nist.gov/vuln/detail/CVE-2020-8840) | | CVE-2020-9546 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2020-9546](https://nvd.nist.gov/vuln/detail/CVE-2020-9546) | | CVE-2020-9546 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2020-9546](https://nvd.nist.gov/vuln/detail/CVE-2020-9546) | | CVE-2020-9547 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2020-9547](https://nvd.nist.gov/vuln/detail/CVE-2020-9547) | | CVE-2020-9547 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2020-9547](https://nvd.nist.gov/vuln/detail/CVE-2020-9547) | | CVE-2020-9548 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2020-9548](https://nvd.nist.gov/vuln/detail/CVE-2020-9548) | | CVE-2020-9548 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2020-9548](https://nvd.nist.gov/vuln/detail/CVE-2020-9548) | | CVE-2022-1471 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2022-1471](https://nvd.nist.gov/vuln/detail/CVE-2022-1471) | | CVE-2022-1471 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2022-1471](https://nvd.nist.gov/vuln/detail/CVE-2022-1471) | | CVE-2022-37865 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2022-37865](https://nvd.nist.gov/vuln/detail/CVE-2022-37865) | | CVE-2022-39135 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2022-39135](https://nvd.nist.gov/vuln/detail/CVE-2022-39135) | | CVE-2022-42889 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2022-42889](https://nvd.nist.gov/vuln/detail/CVE-2022-42889) | | CVE-2022-46337 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2022-46337](https://nvd.nist.gov/vuln/detail/CVE-2022-46337) | | CVE-2022-46337 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2022-46337](https://nvd.nist.gov/vuln/detail/CVE-2022-46337) | | CVE-2024-47561 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2024-47561](https://nvd.nist.gov/vuln/detail/CVE-2024-47561) | | CVE-2024-47561 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2024-47561](https://nvd.nist.gov/vuln/detail/CVE-2024-47561) | | CVE-2024-47561 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2024-47561](https://nvd.nist.gov/vuln/detail/CVE-2024-47561) | | CVE-2024-47561 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2024-47561](https://nvd.nist.gov/vuln/detail/CVE-2024-47561) | | CVE-2024-47561 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2024-47561](https://nvd.nist.gov/vuln/detail/CVE-2024-47561) | | CVE-2024-52046 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2024-52046](https://nvd.nist.gov/vuln/detail/CVE-2024-52046) | | CVE-2024-52046 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2024-52046](https://nvd.nist.gov/vuln/detail/CVE-2024-52046) | | CVE-2024-53990 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2024-53990](https://nvd.nist.gov/vuln/detail/CVE-2024-53990) | | CVE-2024-53990 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2024-53990](https://nvd.nist.gov/vuln/detail/CVE-2024-53990) | | CVE-2015-5237 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2015-5237](https://nvd.nist.gov/vuln/detail/CVE-2015-5237) | | CVE-2015-5237 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2015-5237](https://nvd.nist.gov/vuln/detail/CVE-2015-5237) | | CVE-2016-6796 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2016-6796](https://nvd.nist.gov/vuln/detail/CVE-2016-6796) | | CVE-2018-1000180 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2018-1000180](https://nvd.nist.gov/vuln/detail/CVE-2018-1000180) | | CVE-2018-12022 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2018-12022](https://nvd.nist.gov/vuln/detail/CVE-2018-12022) | | CVE-2018-12022 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2018-12022](https://nvd.nist.gov/vuln/detail/CVE-2018-12022) | | CVE-2018-12023 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2018-12023](https://nvd.nist.gov/vuln/detail/CVE-2018-12023) | | CVE-2018-12023 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2018-12023](https://nvd.nist.gov/vuln/detail/CVE-2018-12023) | | CVE-2018-5968 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2018-5968](https://nvd.nist.gov/vuln/detail/CVE-2018-5968) | | CVE-2019-0205 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2019-0205](https://nvd.nist.gov/vuln/detail/CVE-2019-0205) | | CVE-2019-0205 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2019-0205](https://nvd.nist.gov/vuln/detail/CVE-2019-0205) | | CVE-2019-10172 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2019-10172](https://nvd.nist.gov/vuln/detail/CVE-2019-10172) | | CVE-2019-12086 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2019-12086](https://nvd.nist.gov/vuln/detail/CVE-2019-12086) | | CVE-2019-12086 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2019-12086](https://nvd.nist.gov/vuln/detail/CVE-2019-12086) | | CVE-2019-14439 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2019-14439](https://nvd.nist.gov/vuln/detail/CVE-2019-14439) | | CVE-2019-14439 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2019-14439](https://nvd.nist.gov/vuln/detail/CVE-2019-14439) | | CVE-2020-10650 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-10650](https://nvd.nist.gov/vuln/detail/CVE-2020-10650) | | CVE-2020-10650 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-10650](https://nvd.nist.gov/vuln/detail/CVE-2020-10650) | | CVE-2020-10672 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-10672](https://nvd.nist.gov/vuln/detail/CVE-2020-10672) | | CVE-2020-10672 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-10672](https://nvd.nist.gov/vuln/detail/CVE-2020-10672) | | CVE-2020-10673 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-10673](https://nvd.nist.gov/vuln/detail/CVE-2020-10673) | | CVE-2020-10673 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-10673](https://nvd.nist.gov/vuln/detail/CVE-2020-10673) | | CVE-2020-10968 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-10968](https://nvd.nist.gov/vuln/detail/CVE-2020-10968) | | CVE-2020-10968 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-10968](https://nvd.nist.gov/vuln/detail/CVE-2020-10968) | | CVE-2020-10969 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-10969](https://nvd.nist.gov/vuln/detail/CVE-2020-10969) | | CVE-2020-10969 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-10969](https://nvd.nist.gov/vuln/detail/CVE-2020-10969) | | CVE-2020-11111 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-11111](https://nvd.nist.gov/vuln/detail/CVE-2020-11111) | | CVE-2020-11111 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-11111](https://nvd.nist.gov/vuln/detail/CVE-2020-11111) | | CVE-2020-11112 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-11112](https://nvd.nist.gov/vuln/detail/CVE-2020-11112) | | CVE-2020-11112 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-11112](https://nvd.nist.gov/vuln/detail/CVE-2020-11112) | | CVE-2020-11113 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-11113](https://nvd.nist.gov/vuln/detail/CVE-2020-11113) | | CVE-2020-11113 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-11113](https://nvd.nist.gov/vuln/detail/CVE-2020-11113) | | CVE-2020-11619 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-11619](https://nvd.nist.gov/vuln/detail/CVE-2020-11619) | | CVE-2020-11619 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-11619](https://nvd.nist.gov/vuln/detail/CVE-2020-11619) | | CVE-2020-11620 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-11620](https://nvd.nist.gov/vuln/detail/CVE-2020-11620) | | CVE-2020-11620 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-11620](https://nvd.nist.gov/vuln/detail/CVE-2020-11620) | | CVE-2020-13936 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-13936](https://nvd.nist.gov/vuln/detail/CVE-2020-13936) | | CVE-2020-13949 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-13949](https://nvd.nist.gov/vuln/detail/CVE-2020-13949) | | CVE-2020-13949 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-13949](https://nvd.nist.gov/vuln/detail/CVE-2020-13949) | | CVE-2020-13949 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-13949](https://nvd.nist.gov/vuln/detail/CVE-2020-13949) | | CVE-2020-14060 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-14060](https://nvd.nist.gov/vuln/detail/CVE-2020-14060) | | CVE-2020-14060 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-14060](https://nvd.nist.gov/vuln/detail/CVE-2020-14060) | | CVE-2020-14061 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-14061](https://nvd.nist.gov/vuln/detail/CVE-2020-14061) | | CVE-2020-14061 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-14061](https://nvd.nist.gov/vuln/detail/CVE-2020-14061) | | CVE-2020-14062 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-14062](https://nvd.nist.gov/vuln/detail/CVE-2020-14062) | | CVE-2020-14062 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-14062](https://nvd.nist.gov/vuln/detail/CVE-2020-14062) | | CVE-2020-14195 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-14195](https://nvd.nist.gov/vuln/detail/CVE-2020-14195) | | CVE-2020-14195 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-14195](https://nvd.nist.gov/vuln/detail/CVE-2020-14195) | | CVE-2020-24616 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-24616](https://nvd.nist.gov/vuln/detail/CVE-2020-24616) | | CVE-2020-24616 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-24616](https://nvd.nist.gov/vuln/detail/CVE-2020-24616) | | CVE-2020-24750 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-24750](https://nvd.nist.gov/vuln/detail/CVE-2020-24750) | | CVE-2020-24750 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-24750](https://nvd.nist.gov/vuln/detail/CVE-2020-24750) | | CVE-2020-25649 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-25649](https://nvd.nist.gov/vuln/detail/CVE-2020-25649) | | CVE-2020-28052 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-28052](https://nvd.nist.gov/vuln/detail/CVE-2020-28052) | | CVE-2020-35490 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-35490](https://nvd.nist.gov/vuln/detail/CVE-2020-35490) | | CVE-2020-35490 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-35490](https://nvd.nist.gov/vuln/detail/CVE-2020-35490) | | CVE-2020-35491 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-35491](https://nvd.nist.gov/vuln/detail/CVE-2020-35491) | | CVE-2020-35491 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-35491](https://nvd.nist.gov/vuln/detail/CVE-2020-35491) | | CVE-2020-35728 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-35728](https://nvd.nist.gov/vuln/detail/CVE-2020-35728) | | CVE-2020-35728 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-35728](https://nvd.nist.gov/vuln/detail/CVE-2020-35728) | | CVE-2020-36179 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36179](https://nvd.nist.gov/vuln/detail/CVE-2020-36179) | | CVE-2020-36179 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36179](https://nvd.nist.gov/vuln/detail/CVE-2020-36179) | | CVE-2020-36180 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36180](https://nvd.nist.gov/vuln/detail/CVE-2020-36180) | | CVE-2020-36180 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36180](https://nvd.nist.gov/vuln/detail/CVE-2020-36180) | | CVE-2020-36181 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36181](https://nvd.nist.gov/vuln/detail/CVE-2020-36181) | | CVE-2020-36181 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36181](https://nvd.nist.gov/vuln/detail/CVE-2020-36181) | | CVE-2020-36182 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36182](https://nvd.nist.gov/vuln/detail/CVE-2020-36182) | | CVE-2020-36182 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36182](https://nvd.nist.gov/vuln/detail/CVE-2020-36182) | | CVE-2020-36183 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36183](https://nvd.nist.gov/vuln/detail/CVE-2020-36183) | | CVE-2020-36183 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36183](https://nvd.nist.gov/vuln/detail/CVE-2020-36183) | | CVE-2020-36184 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36184](https://nvd.nist.gov/vuln/detail/CVE-2020-36184) | | CVE-2020-36184 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36184](https://nvd.nist.gov/vuln/detail/CVE-2020-36184) | | CVE-2020-36185 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36185](https://nvd.nist.gov/vuln/detail/CVE-2020-36185) | | CVE-2020-36185 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36185](https://nvd.nist.gov/vuln/detail/CVE-2020-36185) | | CVE-2020-36186 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36186](https://nvd.nist.gov/vuln/detail/CVE-2020-36186) | | CVE-2020-36186 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36186](https://nvd.nist.gov/vuln/detail/CVE-2020-36186) | | CVE-2020-36187 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36187](https://nvd.nist.gov/vuln/detail/CVE-2020-36187) | | CVE-2020-36187 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36187](https://nvd.nist.gov/vuln/detail/CVE-2020-36187) | | CVE-2020-36188 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36188](https://nvd.nist.gov/vuln/detail/CVE-2020-36188) | | CVE-2020-36188 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36188](https://nvd.nist.gov/vuln/detail/CVE-2020-36188) | | CVE-2020-36189 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36189](https://nvd.nist.gov/vuln/detail/CVE-2020-36189) | | CVE-2020-36189 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36189](https://nvd.nist.gov/vuln/detail/CVE-2020-36189) | | CVE-2020-36518 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36518](https://nvd.nist.gov/vuln/detail/CVE-2020-36518) | | CVE-2020-36518 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36518](https://nvd.nist.gov/vuln/detail/CVE-2020-36518) | | CVE-2021-20190 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-20190](https://nvd.nist.gov/vuln/detail/CVE-2021-20190) | | CVE-2021-20190 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-20190](https://nvd.nist.gov/vuln/detail/CVE-2021-20190) | | CVE-2021-31684 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-31684](https://nvd.nist.gov/vuln/detail/CVE-2021-31684) | | CVE-2021-34538 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-34538](https://nvd.nist.gov/vuln/detail/CVE-2021-34538) | | CVE-2021-34538 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-34538](https://nvd.nist.gov/vuln/detail/CVE-2021-34538) | | CVE-2021-34538 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-34538](https://nvd.nist.gov/vuln/detail/CVE-2021-34538) | | CVE-2022-25647 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-25647](https://nvd.nist.gov/vuln/detail/CVE-2022-25647) | | CVE-2022-25647 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-25647](https://nvd.nist.gov/vuln/detail/CVE-2022-25647) | | CVE-2022-25647 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-25647](https://nvd.nist.gov/vuln/detail/CVE-2022-25647) | | CVE-2022-25857 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-25857](https://nvd.nist.gov/vuln/detail/CVE-2022-25857) | | CVE-2022-3171 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-3171](https://nvd.nist.gov/vuln/detail/CVE-2022-3171) | | CVE-2022-3171 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-3171](https://nvd.nist.gov/vuln/detail/CVE-2022-3171) | | CVE-2022-3509 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-3509](https://nvd.nist.gov/vuln/detail/CVE-2022-3509) | | CVE-2022-3509 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-3509](https://nvd.nist.gov/vuln/detail/CVE-2022-3509) | | CVE-2022-3510 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-3510](https://nvd.nist.gov/vuln/detail/CVE-2022-3510) | | CVE-2022-3510 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-3510](https://nvd.nist.gov/vuln/detail/CVE-2022-3510) | | CVE-2022-36364 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-36364](https://nvd.nist.gov/vuln/detail/CVE-2022-36364) | | CVE-2022-37866 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-37866](https://nvd.nist.gov/vuln/detail/CVE-2022-37866) | | CVE-2022-40149 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-40149](https://nvd.nist.gov/vuln/detail/CVE-2022-40149) | | CVE-2022-40149 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-40149](https://nvd.nist.gov/vuln/detail/CVE-2022-40149) | | CVE-2022-40150 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-40150](https://nvd.nist.gov/vuln/detail/CVE-2022-40150) | | CVE-2022-40150 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-40150](https://nvd.nist.gov/vuln/detail/CVE-2022-40150) | | CVE-2022-42003 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-42003](https://nvd.nist.gov/vuln/detail/CVE-2022-42003) | | CVE-2022-42003 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-42003](https://nvd.nist.gov/vuln/detail/CVE-2022-42003) | | CVE-2022-42003 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-42003](https://nvd.nist.gov/vuln/detail/CVE-2022-42003) | | CVE-2022-42003 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-42003](https://nvd.nist.gov/vuln/detail/CVE-2022-42003) | | CVE-2022-42003 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-42003](https://nvd.nist.gov/vuln/detail/CVE-2022-42003) | | CVE-2022-42004 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-42004](https://nvd.nist.gov/vuln/detail/CVE-2022-42004) | | CVE-2022-42004 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-42004](https://nvd.nist.gov/vuln/detail/CVE-2022-42004) | | CVE-2022-42004 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-42004](https://nvd.nist.gov/vuln/detail/CVE-2022-42004) | | CVE-2022-42004 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-42004](https://nvd.nist.gov/vuln/detail/CVE-2022-42004) | | CVE-2022-45685 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-45685](https://nvd.nist.gov/vuln/detail/CVE-2022-45685) | | CVE-2022-45685 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-45685](https://nvd.nist.gov/vuln/detail/CVE-2022-45685) | | CVE-2022-45693 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-45693](https://nvd.nist.gov/vuln/detail/CVE-2022-45693) | | CVE-2022-45693 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-45693](https://nvd.nist.gov/vuln/detail/CVE-2022-45693) | | CVE-2022-46751 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-46751](https://nvd.nist.gov/vuln/detail/CVE-2022-46751) | | CVE-2022-46751 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-46751](https://nvd.nist.gov/vuln/detail/CVE-2022-46751) | | CVE-2023-1370 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-1370](https://nvd.nist.gov/vuln/detail/CVE-2023-1370) | | CVE-2023-1370 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-1370](https://nvd.nist.gov/vuln/detail/CVE-2023-1370) | | CVE-2023-1436 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-1436](https://nvd.nist.gov/vuln/detail/CVE-2023-1436) | | CVE-2023-1436 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-1436](https://nvd.nist.gov/vuln/detail/CVE-2023-1436) | | CVE-2023-20883 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-20883](https://nvd.nist.gov/vuln/detail/CVE-2023-20883) | | CVE-2023-25194 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-25194](https://nvd.nist.gov/vuln/detail/CVE-2023-25194) | | CVE-2023-2976 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-2976](https://nvd.nist.gov/vuln/detail/CVE-2023-2976) | | CVE-2023-2976 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-2976](https://nvd.nist.gov/vuln/detail/CVE-2023-2976) | | CVE-2023-2976 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-2976](https://nvd.nist.gov/vuln/detail/CVE-2023-2976) | | CVE-2023-2976 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-2976](https://nvd.nist.gov/vuln/detail/CVE-2023-2976) | | CVE-2023-2976 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-2976](https://nvd.nist.gov/vuln/detail/CVE-2023-2976) | | CVE-2023-2976 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-2976](https://nvd.nist.gov/vuln/detail/CVE-2023-2976) | | CVE-2023-2976 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-2976](https://nvd.nist.gov/vuln/detail/CVE-2023-2976) | | CVE-2023-2976 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-2976](https://nvd.nist.gov/vuln/detail/CVE-2023-2976) | | CVE-2023-2976 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-2976](https://nvd.nist.gov/vuln/detail/CVE-2023-2976) | | CVE-2023-2976 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-2976](https://nvd.nist.gov/vuln/detail/CVE-2023-2976) | | CVE-2023-34453 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-34453](https://nvd.nist.gov/vuln/detail/CVE-2023-34453) | | CVE-2023-34454 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-34454](https://nvd.nist.gov/vuln/detail/CVE-2023-34454) | | CVE-2023-34455 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-34455](https://nvd.nist.gov/vuln/detail/CVE-2023-34455) | | CVE-2023-3635 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-3635](https://nvd.nist.gov/vuln/detail/CVE-2023-3635) | | CVE-2023-3635 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-3635](https://nvd.nist.gov/vuln/detail/CVE-2023-3635) | | CVE-2023-39410 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-39410](https://nvd.nist.gov/vuln/detail/CVE-2023-39410) | | CVE-2023-39410 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-39410](https://nvd.nist.gov/vuln/detail/CVE-2023-39410) | | CVE-2023-39410 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-39410](https://nvd.nist.gov/vuln/detail/CVE-2023-39410) | | CVE-2023-39410 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-39410](https://nvd.nist.gov/vuln/detail/CVE-2023-39410) | | CVE-2023-43642 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-43642](https://nvd.nist.gov/vuln/detail/CVE-2023-43642) | | CVE-2023-5072 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-5072](https://nvd.nist.gov/vuln/detail/CVE-2023-5072) | | CVE-2023-6378 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-6378](https://nvd.nist.gov/vuln/detail/CVE-2023-6378) | | CVE-2023-6378 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-6378](https://nvd.nist.gov/vuln/detail/CVE-2023-6378) | | CVE-2023-6481 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-6481](https://nvd.nist.gov/vuln/detail/CVE-2023-6481) | | CVE-2023-6481 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-6481](https://nvd.nist.gov/vuln/detail/CVE-2023-6481) | | CVE-2024-7254 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-7254](https://nvd.nist.gov/vuln/detail/CVE-2024-7254) | | CVE-2024-7254 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-7254](https://nvd.nist.gov/vuln/detail/CVE-2024-7254) | | CVE-2024-7254 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-7254](https://nvd.nist.gov/vuln/detail/CVE-2024-7254) | | CVE-2024-7254 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-7254](https://nvd.nist.gov/vuln/detail/CVE-2024-7254) | | CVE-2024-7254 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-7254](https://nvd.nist.gov/vuln/detail/CVE-2024-7254) | | CVE-2024-7254 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-7254](https://nvd.nist.gov/vuln/detail/CVE-2024-7254) | | CVE-2024-7254 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-7254](https://nvd.nist.gov/vuln/detail/CVE-2024-7254) | | CVE-2012-5783 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2012-5783](https://nvd.nist.gov/vuln/detail/CVE-2012-5783) | | CVE-2012-6153 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2012-6153](https://nvd.nist.gov/vuln/detail/CVE-2012-6153) | | CVE-2018-10237 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2018-10237](https://nvd.nist.gov/vuln/detail/CVE-2018-10237) | | CVE-2018-10237 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2018-10237](https://nvd.nist.gov/vuln/detail/CVE-2018-10237) | | CVE-2018-10237 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2018-10237](https://nvd.nist.gov/vuln/detail/CVE-2018-10237) | | CVE-2018-10237 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2018-10237](https://nvd.nist.gov/vuln/detail/CVE-2018-10237) | | CVE-2018-10237 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2018-10237](https://nvd.nist.gov/vuln/detail/CVE-2018-10237) | | CVE-2018-10237 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2018-10237](https://nvd.nist.gov/vuln/detail/CVE-2018-10237) | | CVE-2018-11798 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2018-11798](https://nvd.nist.gov/vuln/detail/CVE-2018-11798) | | CVE-2018-1313 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2018-1313](https://nvd.nist.gov/vuln/detail/CVE-2018-1313) | | CVE-2019-10219 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2019-10219](https://nvd.nist.gov/vuln/detail/CVE-2019-10219) | | CVE-2019-12384 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2019-12384](https://nvd.nist.gov/vuln/detail/CVE-2019-12384) | | CVE-2019-12384 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2019-12384](https://nvd.nist.gov/vuln/detail/CVE-2019-12384) | | CVE-2019-12814 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2019-12814](https://nvd.nist.gov/vuln/detail/CVE-2019-12814) | | CVE-2019-12814 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2019-12814](https://nvd.nist.gov/vuln/detail/CVE-2019-12814) | | CVE-2020-10693 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2020-10693](https://nvd.nist.gov/vuln/detail/CVE-2020-10693) | | CVE-2020-13955 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2020-13955](https://nvd.nist.gov/vuln/detail/CVE-2020-13955) | | CVE-2020-13956 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2020-13956](https://nvd.nist.gov/vuln/detail/CVE-2020-13956) | | CVE-2020-15250 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2020-15250](https://nvd.nist.gov/vuln/detail/CVE-2020-15250) | | CVE-2020-15250 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2020-15250](https://nvd.nist.gov/vuln/detail/CVE-2020-15250) | | CVE-2020-15522 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2020-15522](https://nvd.nist.gov/vuln/detail/CVE-2020-15522) | | CVE-2020-15522 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2020-15522](https://nvd.nist.gov/vuln/detail/CVE-2020-15522) | | CVE-2020-17521 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2020-17521](https://nvd.nist.gov/vuln/detail/CVE-2020-17521) | | CVE-2020-17521 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2020-17521](https://nvd.nist.gov/vuln/detail/CVE-2020-17521) | | CVE-2020-1945 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2020-1945](https://nvd.nist.gov/vuln/detail/CVE-2020-1945) | | CVE-2020-26939 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2020-26939](https://nvd.nist.gov/vuln/detail/CVE-2020-26939) | | CVE-2020-26939 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2020-26939](https://nvd.nist.gov/vuln/detail/CVE-2020-26939) | | CVE-2020-29582 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2020-29582](https://nvd.nist.gov/vuln/detail/CVE-2020-29582) | | CVE-2021-22569 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2021-22569](https://nvd.nist.gov/vuln/detail/CVE-2021-22569) | | CVE-2021-22569 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2021-22569](https://nvd.nist.gov/vuln/detail/CVE-2021-22569) | | CVE-2021-22570 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2021-22570](https://nvd.nist.gov/vuln/detail/CVE-2021-22570) | | CVE-2021-22570 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2021-22570](https://nvd.nist.gov/vuln/detail/CVE-2021-22570) | | CVE-2021-27568 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2021-27568](https://nvd.nist.gov/vuln/detail/CVE-2021-27568) | | CVE-2021-28170 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2021-28170](https://nvd.nist.gov/vuln/detail/CVE-2021-28170) | | CVE-2021-28170 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2021-28170](https://nvd.nist.gov/vuln/detail/CVE-2021-28170) | | CVE-2021-28170 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2021-28170](https://nvd.nist.gov/vuln/detail/CVE-2021-28170) | | CVE-2021-28170 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2021-28170](https://nvd.nist.gov/vuln/detail/CVE-2021-28170) | | CVE-2021-29425 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2021-29425](https://nvd.nist.gov/vuln/detail/CVE-2021-29425) | | CVE-2021-36373 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2021-36373](https://nvd.nist.gov/vuln/detail/CVE-2021-36373) | | CVE-2021-36374 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2021-36374](https://nvd.nist.gov/vuln/detail/CVE-2021-36374) | | CVE-2021-38153 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2021-38153](https://nvd.nist.gov/vuln/detail/CVE-2021-38153) | | CVE-2022-24329 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2022-24329](https://nvd.nist.gov/vuln/detail/CVE-2022-24329) | | CVE-2022-38749 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2022-38749](https://nvd.nist.gov/vuln/detail/CVE-2022-38749) | | CVE-2022-38750 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2022-38750](https://nvd.nist.gov/vuln/detail/CVE-2022-38750) | | CVE-2022-38751 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2022-38751](https://nvd.nist.gov/vuln/detail/CVE-2022-38751) | | CVE-2022-38752 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2022-38752](https://nvd.nist.gov/vuln/detail/CVE-2022-38752) | | CVE-2022-41854 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2022-41854](https://nvd.nist.gov/vuln/detail/CVE-2022-41854) | | CVE-2023-0833 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2023-0833](https://nvd.nist.gov/vuln/detail/CVE-2023-0833) | | CVE-2023-20863 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2023-20863](https://nvd.nist.gov/vuln/detail/CVE-2023-20863) | | CVE-2023-33201 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2023-33201](https://nvd.nist.gov/vuln/detail/CVE-2023-33201) | | CVE-2023-33201 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2023-33201](https://nvd.nist.gov/vuln/detail/CVE-2023-33201) | | CVE-2023-33201 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2023-33201](https://nvd.nist.gov/vuln/detail/CVE-2023-33201) | | CVE-2023-33201 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2023-33201](https://nvd.nist.gov/vuln/detail/CVE-2023-33201) | | CVE-2023-33201 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2023-33201](https://nvd.nist.gov/vuln/detail/CVE-2023-33201) | | CVE-2023-33202 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2023-33202](https://nvd.nist.gov/vuln/detail/CVE-2023-33202) | | CVE-2023-33202 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2023-33202](https://nvd.nist.gov/vuln/detail/CVE-2023-33202) | | CVE-2023-33202 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2023-33202](https://nvd.nist.gov/vuln/detail/CVE-2023-33202) | | CVE-2023-33202 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2023-33202](https://nvd.nist.gov/vuln/detail/CVE-2023-33202) | | CVE-2023-33202 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2023-33202](https://nvd.nist.gov/vuln/detail/CVE-2023-33202) | | CVE-2023-42503 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2023-42503](https://nvd.nist.gov/vuln/detail/CVE-2023-42503) | | CVE-2023-51074 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2023-51074](https://nvd.nist.gov/vuln/detail/CVE-2023-51074) | | CVE-2023-51074 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2023-51074](https://nvd.nist.gov/vuln/detail/CVE-2023-51074) | | CVE-2024-12798 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-12798](https://nvd.nist.gov/vuln/detail/CVE-2024-12798) | | CVE-2024-12798 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-12798](https://nvd.nist.gov/vuln/detail/CVE-2024-12798) | | CVE-2024-12798 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-12798](https://nvd.nist.gov/vuln/detail/CVE-2024-12798) | | CVE-2024-12798 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-12798](https://nvd.nist.gov/vuln/detail/CVE-2024-12798) | | CVE-2024-12798 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-12798](https://nvd.nist.gov/vuln/detail/CVE-2024-12798) | | CVE-2024-12798 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-12798](https://nvd.nist.gov/vuln/detail/CVE-2024-12798) | | CVE-2024-25710 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-25710](https://nvd.nist.gov/vuln/detail/CVE-2024-25710) | | CVE-2024-25710 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-25710](https://nvd.nist.gov/vuln/detail/CVE-2024-25710) | | CVE-2024-25710 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-25710](https://nvd.nist.gov/vuln/detail/CVE-2024-25710) | | CVE-2024-26308 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-26308](https://nvd.nist.gov/vuln/detail/CVE-2024-26308) | | CVE-2024-26308 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-26308](https://nvd.nist.gov/vuln/detail/CVE-2024-26308) | | CVE-2024-26308 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-26308](https://nvd.nist.gov/vuln/detail/CVE-2024-26308) | | CVE-2024-38820 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-38820](https://nvd.nist.gov/vuln/detail/CVE-2024-38820) | | CVE-2024-38820 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-38820](https://nvd.nist.gov/vuln/detail/CVE-2024-38820) | | CVE-2024-6763 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-6763](https://nvd.nist.gov/vuln/detail/CVE-2024-6763) | | CVE-2024-6763 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-6763](https://nvd.nist.gov/vuln/detail/CVE-2024-6763) | | CVE-2024-6763 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-6763](https://nvd.nist.gov/vuln/detail/CVE-2024-6763) | | CVE-2024-6763 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-6763](https://nvd.nist.gov/vuln/detail/CVE-2024-6763) | | CVE-2024-6763 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-6763](https://nvd.nist.gov/vuln/detail/CVE-2024-6763) | | CVE-2024-8184 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-8184](https://nvd.nist.gov/vuln/detail/CVE-2024-8184) | | CVE-2020-8908 | LOW | [https://nvd.nist.gov/vuln/detail/CVE-2020-8908](https://nvd.nist.gov/vuln/detail/CVE-2020-8908) | | CVE-2020-8908 | LOW | [https://nvd.nist.gov/vuln/detail/CVE-2020-8908](https://nvd.nist.gov/vuln/detail/CVE-2020-8908) | | CVE-2020-8908 | LOW | [https://nvd.nist.gov/vuln/detail/CVE-2020-8908](https://nvd.nist.gov/vuln/detail/CVE-2020-8908) | | CVE-2020-8908 | LOW | [https://nvd.nist.gov/vuln/detail/CVE-2020-8908](https://nvd.nist.gov/vuln/detail/CVE-2020-8908) | | CVE-2020-8908 | LOW | [https://nvd.nist.gov/vuln/detail/CVE-2020-8908](https://nvd.nist.gov/vuln/detail/CVE-2020-8908) | | CVE-2020-8908 | LOW | [https://nvd.nist.gov/vuln/detail/CVE-2020-8908](https://nvd.nist.gov/vuln/detail/CVE-2020-8908) | | CVE-2020-8908 | LOW | [https://nvd.nist.gov/vuln/detail/CVE-2020-8908](https://nvd.nist.gov/vuln/detail/CVE-2020-8908) | | CVE-2020-8908 | LOW | [https://nvd.nist.gov/vuln/detail/CVE-2020-8908](https://nvd.nist.gov/vuln/detail/CVE-2020-8908) | | CVE-2020-8908 | LOW | [https://nvd.nist.gov/vuln/detail/CVE-2020-8908](https://nvd.nist.gov/vuln/detail/CVE-2020-8908) | | CVE-2020-8908 | LOW | [https://nvd.nist.gov/vuln/detail/CVE-2020-8908](https://nvd.nist.gov/vuln/detail/CVE-2020-8908) | | CVE-2024-12801 | LOW | [https://nvd.nist.gov/vuln/detail/CVE-2024-12801](https://nvd.nist.gov/vuln/detail/CVE-2024-12801) | | CVE-2024-12801 | LOW | [https://nvd.nist.gov/vuln/detail/CVE-2024-12801](https://nvd.nist.gov/vuln/detail/CVE-2024-12801) | | CVE-2024-12801 | LOW | [https://nvd.nist.gov/vuln/detail/CVE-2024-12801](https://nvd.nist.gov/vuln/detail/CVE-2024-12801) | | CVE-2024-12801 | LOW | [https://nvd.nist.gov/vuln/detail/CVE-2024-12801](https://nvd.nist.gov/vuln/detail/CVE-2024-12801) | | GHSA-chfm-68vv-pvw5 | LOW | [https://nvd.nist.gov/vuln/detail/GHSA-chfm-68vv-pvw5](https://nvd.nist.gov/vuln/detail/GHSA-chfm-68vv-pvw5) | | CVE-2024-23454 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2024-23454](https://nvd.nist.gov/vuln/detail/CVE-2024-23454) | | CVE-2024-23944 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2024-23944](https://nvd.nist.gov/vuln/detail/CVE-2024-23944) | | CVE-2024-23945 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2024-23945](https://nvd.nist.gov/vuln/detail/CVE-2024-23945) | | CVE-2024-23945 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2024-23945](https://nvd.nist.gov/vuln/detail/CVE-2024-23945) | | CVE-2024-23953 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2024-23953](https://nvd.nist.gov/vuln/detail/CVE-2024-23953) | | CVE-2024-23953 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2024-23953](https://nvd.nist.gov/vuln/detail/CVE-2024-23953) | | CVE-2024-27137 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2024-27137](https://nvd.nist.gov/vuln/detail/CVE-2024-27137) | | CVE-2024-29025 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2024-29025](https://nvd.nist.gov/vuln/detail/CVE-2024-29025) | | CVE-2024-29131 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2024-29131](https://nvd.nist.gov/vuln/detail/CVE-2024-29131) | | CVE-2024-29133 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2024-29133](https://nvd.nist.gov/vuln/detail/CVE-2024-29133) | | CVE-2024-29857 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2024-29857](https://nvd.nist.gov/vuln/detail/CVE-2024-29857) | | CVE-2024-29857 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2024-29857](https://nvd.nist.gov/vuln/detail/CVE-2024-29857) | | CVE-2024-29857 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2024-29857](https://nvd.nist.gov/vuln/detail/CVE-2024-29857) | | CVE-2024-29857 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2024-29857](https://nvd.nist.gov/vuln/detail/CVE-2024-29857) | | CVE-2024-29869 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2024-29869](https://nvd.nist.gov/vuln/detail/CVE-2024-29869) | | CVE-2024-29869 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2024-29869](https://nvd.nist.gov/vuln/detail/CVE-2024-29869) | | CVE-2024-30171 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2024-30171](https://nvd.nist.gov/vuln/detail/CVE-2024-30171) | | CVE-2024-30171 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2024-30171](https://nvd.nist.gov/vuln/detail/CVE-2024-30171) | | CVE-2024-30171 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2024-30171](https://nvd.nist.gov/vuln/detail/CVE-2024-30171) | | CVE-2024-30171 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2024-30171](https://nvd.nist.gov/vuln/detail/CVE-2024-30171) | | CVE-2024-30172 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2024-30172](https://nvd.nist.gov/vuln/detail/CVE-2024-30172) | | CVE-2024-30172 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2024-30172](https://nvd.nist.gov/vuln/detail/CVE-2024-30172) | | CVE-2024-31141 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2024-31141](https://nvd.nist.gov/vuln/detail/CVE-2024-31141) | | CVE-2024-31141 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2024-31141](https://nvd.nist.gov/vuln/detail/CVE-2024-31141) | | CVE-2024-34447 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2024-34447](https://nvd.nist.gov/vuln/detail/CVE-2024-34447) | | CVE-2024-34447 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2024-34447](https://nvd.nist.gov/vuln/detail/CVE-2024-34447) | | CVE-2024-36114 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2024-36114](https://nvd.nist.gov/vuln/detail/CVE-2024-36114) | | CVE-2024-36114 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2024-36114](https://nvd.nist.gov/vuln/detail/CVE-2024-36114) | | CVE-2024-36114 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2024-36114](https://nvd.nist.gov/vuln/detail/CVE-2024-36114) | | CVE-2024-36114 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2024-36114](https://nvd.nist.gov/vuln/detail/CVE-2024-36114) | | CVE-2024-38808 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2024-38808](https://nvd.nist.gov/vuln/detail/CVE-2024-38808) | | CVE-2024-47535 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2024-47535](https://nvd.nist.gov/vuln/detail/CVE-2024-47535) | | CVE-2024-47535 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2024-47535](https://nvd.nist.gov/vuln/detail/CVE-2024-47535) | | CVE-2024-47535 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2024-47535](https://nvd.nist.gov/vuln/detail/CVE-2024-47535) | | CVE-2024-47554 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2024-47554](https://nvd.nist.gov/vuln/detail/CVE-2024-47554) | | CVE-2024-47554 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2024-47554](https://nvd.nist.gov/vuln/detail/CVE-2024-47554) | | CVE-2024-47554 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2024-47554](https://nvd.nist.gov/vuln/detail/CVE-2024-47554) | | CVE-2024-47554 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2024-47554](https://nvd.nist.gov/vuln/detail/CVE-2024-47554) | | CVE-2024-51504 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2024-51504](https://nvd.nist.gov/vuln/detail/CVE-2024-51504) | | CVE-2024-51504 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2024-51504](https://nvd.nist.gov/vuln/detail/CVE-2024-51504) | | CVE-2024-56128 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2024-56128](https://nvd.nist.gov/vuln/detail/CVE-2024-56128) | | CVE-2024-56128 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2024-56128](https://nvd.nist.gov/vuln/detail/CVE-2024-56128) | | CVE-2024-57699 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2024-57699](https://nvd.nist.gov/vuln/detail/CVE-2024-57699) | | CVE-2024-9823 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2024-9823](https://nvd.nist.gov/vuln/detail/CVE-2024-9823) | | CVE-2025-23015 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2025-23015](https://nvd.nist.gov/vuln/detail/CVE-2025-23015) | | CVE-2025-24860 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2025-24860](https://nvd.nist.gov/vuln/detail/CVE-2025-24860) | | CVE-2025-24970 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2025-24970](https://nvd.nist.gov/vuln/detail/CVE-2025-24970) | | CVE-2025-24970 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2025-24970](https://nvd.nist.gov/vuln/detail/CVE-2025-24970) | | CVE-2025-24970 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2025-24970](https://nvd.nist.gov/vuln/detail/CVE-2025-24970) | | CVE-2025-24970 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2025-24970](https://nvd.nist.gov/vuln/detail/CVE-2025-24970) | | CVE-2025-25193 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2025-25193](https://nvd.nist.gov/vuln/detail/CVE-2025-25193) | | CVE-2025-25193 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2025-25193](https://nvd.nist.gov/vuln/detail/CVE-2025-25193) | | CVE-2025-25193 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2025-25193](https://nvd.nist.gov/vuln/detail/CVE-2025-25193) | | CVE-2025-25193 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2025-25193](https://nvd.nist.gov/vuln/detail/CVE-2025-25193) | | CVE-2025-25193 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2025-25193](https://nvd.nist.gov/vuln/detail/CVE-2025-25193) | ## 7.8.0 release notes The following release notes include information for Amazon EMR release 7.8.0. **New features** + **Application upgrades** – Amazon EMR 7.8.0 application upgrades include AWS SDK for Java 2.29.52/1.12.780, Delta 3.3.0-amzn-0, Hadoop 3.4.1, Hudi 0.15.0-amzn-5, Iceberg 1.7.1-amzn-1, Spark 3.5.4, and Zookeeper 3.9.3. **Known issues** + **KMS encryption using the EMR-EC2 SecretAgent component** – There is an error that occurs when KMS encryption at rest is enabled for Lake Formation-enabled clusters in EMR versions 7.8 and 7.9. Lake Formation-enabled jobs will fail with a *NoSuchMethodError*. There are two workarounds available: + Temporarily disable KMS encryption at rest for affected clusters. + Contact AWS Support to request a Bootstrap Action artifact if KMS encryption must be maintained. **Note** **Fix update** – In August 2025 an update to address the above issue was deployed to all regions. There is no need to disable KMS encryption at rest. + **HBase table creation in Hive with Glue catalog** – From EMR-7.3.0 to EMR-7.10.0, there is a bug due to Hive Iceberg integration, which causes HBase table creation in Hive to fail when Glue Data Catalog is used as the metastore. Please reach out to the AWS support team if you encounter this issue. + **Tez UI** – From EMR-7.3 to EMR-7.10, there is a bug where accessing Tez UI through YARN Resource Manager Web UI fails when in-transit encryption is enabled. This occurs because YARN proxy attempts to use HTTP protocol while Tez UI requires HTTPS. Please modify the `tez.tez-ui.history-url.base` configuration in `tez-site` from `http` to `https` to resolve this issue. **Changes, enhancements, and resolved issues** + Starting with Spark 3.3.1 (supported in EMR versions 6.10 and above), all executors in a decommissioning host are set to a new `ExecutorState`, called *DECOMMISSIONING* state. The executors being decommissioned cannot be used by Yarn to allocate tasks and thus it will request for new executors, if needed, for the tasks being executed. Thus, if you disable Spark DRA while using EMR Managed Scaling, EMR Auto Scaling, or any custom scaling mechanism on EMR-EC2 clusters, then Yarn may request maximum permissible executors for each job. In order to avoid this issue, leave the `spark.dynamicAllocation.enabled` property set to `TRUE` (which is the default) when you are using the above combination of features. In addition, you can also set minimum and maximum executor constraints by setting values for `spark.dynamicAllocation.maxExecutors` and `spark.dynamicAllocation.minExecutors` properties for your Spark jobs, to restrict the number of executors allocated during the job’s execution. + [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-780-release.html) ## 7.8.0 default Java versions Amazon EMR release 7.8 and higher ships with Amazon Corretto 17 (built on OpenJDK) by default for applications that support Corretto 17 (JDK 17), with the exception of Apache Livy. The following table shows the default Java versions for applications in Amazon EMR 7.8.0. If you want to change the default JVM on your cluster, follow the instructions in [Configure applications to use a specific Java Virtual Machine](configuring-java8.md) for each application that runs on the cluster. You can only use one Java runtime version for a cluster. Amazon EMR doesn't support running different nodes or applications on different runtime versions on the same cluster. | Application | Java / Amazon Corretto version (default is bold) | | --- | --- | | Delta | 17, 11, 8 | | Flink | 17, 11, 8 | | HBase | 17, 11, 8 | | HCatalog | 17, 11, 8 | | Hadoop | 17, 11, 8 | | Hive | 17, 11, 8 | | Hudi | 17, 11, 8 | | Iceberg | 17, 11 | | Livy | 17, 11, 8 | | Oozie | 17, 11, 8 | | Phoenix | 17, 11, 8 | | PrestoDB | 8 | | Spark | 17, 11, 8 | | Spark RAPIDS | 17, 11, 8 | | Tez | 17, 11, 8 | | Trino | 21, 17 | | Zeppelin | 17, 11, 8 | | Pig | 17, 11, 8 | | Zookeeper | 17, 11, 8 | ## 7.8.0 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | adot-java-agent | 1.31.0 | A Java Agent that collects metrics from application daemons. | | delta | 3.3.0-amzn-0 | Delta lake is an open table format for huge analytic datasets | | emr-amazon-cloudwatch-agent | 1.300032.2-amzn-0 | An application that collects internal system-level metrics and custom application metrics from Amazon EC2 instances. | | emr-ddb | 5.6.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 3.17.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.21.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-notebook-env | 1.18.0 | Conda env for emr notebook which includes jupyter enterprise gateway | | emr-s3-dist-cp | 2.38.0 | Distributed copy application optimized for Amazon S3. | | emr-s3-select | 2.17.0 | EMR S3Select Connector | | emr-wal-cli | 1.3.3 | Cli used for emrwal list/deletion. | | emrfs | 2.69.0 | Amazon S3 connector for Hadoop ecosystem applications. | | flink-client | 1.20.0-amzn-2 | Apache Flink command line client scripts and applications. | | flink-jobmanager-config | 1.20.0-amzn-2 | Managing resources on EMR nodes for Apache Flink JobManager. | | hadoop-client | 3.4.1-amzn-0 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 3.4.1-amzn-0 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 3.4.1-amzn-0 | HDFS command-line client and library | | hadoop-hdfs-namenode | 3.4.1-amzn-0 | HDFS service for tracking file names and block locations. | | hadoop-hdfs-journalnode | 3.4.1-amzn-0 | HDFS service for managing the Hadoop filesystem journal on HA clusters. | | hadoop-httpfs-server | 3.4.1-amzn-0 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 3.4.1-amzn-0 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 3.4.1-amzn-0 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 3.4.1-amzn-0 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 3.4.1-amzn-0 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 3.4.1-amzn-0 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 2.6.1-amzn-2 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 2.6.1-amzn-2 | Service for serving one or more HBase regions. | | hbase-client | 2.6.1-amzn-2 | HBase command-line client. | | hbase-rest-server | 2.6.1-amzn-2 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 2.6.1-amzn-2 | Service providing a Thrift endpoint to HBase. | | hbase-operator-tools | 2.6.1-amzn-2 | Repair tool for Apache HBase clusters. | | hcatalog-client | 3.1.3-amzn-17 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 3.1.3-amzn-17 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 3.1.3-amzn-17 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 3.1.3-amzn-17 | Hive command line client. | | hive-hbase | 3.1.3-amzn-17 | Hive-hbase client. | | hive-metastore-server | 3.1.3-amzn-17 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server2 | 3.1.3-amzn-17 | Service for accepting Hive queries as web requests. | | hudi | 0.15.0-amzn-5 | Incremental processing framework to power data pipeline at low latency and high efficiency. | | hudi-presto | 0.15.0-amzn-5 | Bundle library for running Presto with Hudi. | | hudi-trino | 0.15.0-amzn-5 | Bundle library for running Trino with Hudi. | | hudi-spark | 0.15.0-amzn-5 | Bundle library for running Spark with Hudi. | | hue-server | 4.11.0 | Web application for analyzing data using Hadoop ecosystem applications | | iceberg | 1.7.1-amzn-1 | Apache Iceberg is an open table format for huge analytic datasets | | jupyterhub | 1.5.0 | Multi-user server for Jupyter notebooks | | livy-server | 0.8.0-incubating | REST interface for interacting with Apache Spark | | nginx | 1.12.1 | nginx [engine x] is an HTTP and reverse proxy server | | mariadb-server | 5.5.68\$1 | MariaDB database server. | | nvidia-cuda | 12.3.0 | Nvidia drivers and Cuda toolkit | | oozie-client | 5.2.1 | Oozie command-line client. | | oozie-server | 5.2.1 | Service for accepting Oozie workflow requests. | | opencv | 4.7.0 | Open Source Computer Vision Library. | | phoenix-library | 5.2.1 | The phoenix libraries for server and client | | phoenix-connectors | 6.0.0 | Apache Phoenix-Connectors for Spark-3 | | phoenix-query-server | 6.0.0 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.287-amzn-2 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.287-amzn-2 | Service for executing pieces of a query. | | presto-client | 0.287-amzn-2 | Presto command-line client which is installed on an HA cluster's stand-by masters where Presto server is not started. | | trino-coordinator | 467-amzn-1 | Service for accepting queries and managing query execution among trino-workers. | | trino-worker | 467-amzn-1 | Service for executing pieces of a query. | | trino-client | 467-amzn-1 | Trino command-line client which is installed on an HA cluster's stand-by masters where Trino server is not started. | | pig-client | 0.17.0 | Pig command-line client. | | r | 4.3.2 | The R Project for Statistical Computing | | ranger-kms-server | 2.0.0 | Apache Ranger Key Management System | | spark-client | 3.5.4-amzn-0 | Spark command-line clients. | | spark-history-server | 3.5.4-amzn-0 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 3.5.4-amzn-0 | In-memory execution engine for YARN. | | spark-yarn-slave | 3.5.4-amzn-0 | Apache Spark libraries needed by YARN slaves. | | spark-rapids | 24.12.0-amzn-0 | Nvidia Spark RAPIDS plugin that accelerates Apache Spark with GPUs. | | tensorflow | 2.16.1 | TensorFlow open source software library for high performance numerical computation. | | tez-on-yarn | 0.10.2-amzn-15 | The tez YARN application and libraries. | | tez-on-worker | 0.10.2-amzn-15 | The tez YARN application and libraries for worker nodes. | | zeppelin-server | 0.11.1 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.9.3-amzn-0 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.9.3-amzn-0 | ZooKeeper command line client. | ## 7.8.0 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). Reconfiguration actions occur when you specify a configuration for instance groups in a running cluster. Amazon EMR only initiates reconfiguration actions for the classifications that you modify. For more information, see [Reconfigure an instance group in a running cluster](emr-configure-apps-running-cluster.md). **emr-7.8.0 classifications** | Classifications | Description | Reconfiguration Actions | | --- | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | Restarts the ResourceManager service. | | container-executor | Change values in Hadoop YARN's container-executor.cfg file. | Not available. | | container-log4j | Change values in Hadoop YARN's container-log4j.properties file. | Not available. | | core-site | Change values in Hadoop's core-site.xml file. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts Hadoop KMS, Ranger KMS, HiveServer2, Hive MetaStore, Hadoop Httpfs, and MapReduce-HistoryServer. | | docker-conf | Change docker related settings. | Not available. | | emrfs-site | Change EMRFS settings. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts HBaseRegionserver, HBaseMaster, HBaseThrift, HBaseRest, HiveServer2, Hive MetaStore, Hadoop Httpfs, and MapReduce-HistoryServer. | | flink-conf | Change flink-conf.yaml settings. | Restarts Flink history server. | | flink-log4j | Change Flink log4j.properties settings. | Restarts Flink history server. | | flink-log4j-session | Change Flink log4j-session.properties settings for Kubernetes/Yarn session. | Restarts Flink history server. | | flink-log4j-cli | Change Flink log4j-cli.properties settings. | Restarts Flink history server. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts PhoenixQueryserver, HiveServer2, Hive MetaStore, and MapReduce-HistoryServer. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | Restarts the Hadoop HDFS services SecondaryNamenode, Datanode, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts Hadoop KMS, Hadoop Httpfs, and MapReduce-HistoryServer. | | hadoop-ssl-server | Change hadoop ssl server configuration | Not available. | | hadoop-ssl-client | Change hadoop ssl client configuration | Not available. | | hbase | Amazon EMR-curated settings for Apache HBase. | Custom EMR specific property. Sets emrfs-site and hbase-site configs. See those for their associated restarts. | | hbase-env | Change values in HBase's environment. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbase.properties file. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | Not available. | | hbase-site | Change values in HBase's hbase-site.xml file. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. Additionally restarts Phoenix QueryServer. | | hdfs-encryption-zones | Configure HDFS encryption zones. | This classification should not be reconfigured. | | hdfs-env | Change values in the HDFS environment. | Restarts Hadoop HDFS services Namenode, Datanode, and ZKFC. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Additionally restarts Hadoop Httpfs. | | hcatalog-env | Change values in HCatalog's environment. | Restarts Hive HCatalog Server. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | Restarts Hive HCatalog Server. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | Restarts Hive HCatalog Server. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | Restarts Hive WebHCat server. | | hcatalog-webhcat-log4j2 | Change values in HCatalog WebHCat's log4j2.properties. | Restarts Hive WebHCat server. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | Restarts Hive WebHCat server. | | hive | Amazon EMR-curated settings for Apache Hive. | Sets configurations to launch Hive LLAP service. | | hive-beeline-log4j2 | Change values in Hive's beeline-log4j2.properties file. | Not available. | | hive-parquet-logging | Change values in Hive's parquet-logging.properties file. | Not available. | | hive-env | Change values in the Hive environment. | Restarts HiveServer2, HiveMetastore, and Hive HCatalog-Server. Runs Hive schemaTool CLI commands to verify hive-metastore. | | hive-exec-log4j2 | Change values in Hive's hive-exec-log4j2.properties file. | Not available. | | hive-llap-daemon-log4j2 | Change values in Hive's llap-daemon-log4j2.properties file. | Not available. | | hive-log4j2 | Change values in Hive's hive-log4j2.properties file. | Not available. | | hive-site | Change values in Hive's hive-site.xml file | Restarts HiveServer2, HiveMetastore, and Hive HCatalog-Server. Runs Hive schemaTool CLI commands to verify hive-metastore. Also restarts Oozie and Zeppelin. | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | Not available. | | hue-ini | Change values in Hue's ini file | Restarts Hue. Also activates Hue config override CLI commands to pick up new configurations. | | httpfs-env | Change values in the HTTPFS environment. | Restarts Hadoop Httpfs service. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | Restarts Hadoop Httpfs service. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | Not available. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | Restarts Hadoop-KMS service. | | hadoop-kms-java-home | Change Hadoop's KMS java home | Not available. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | Not available. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | Restarts Hadoop-KMS and Ranger-KMS service. | | hudi-env | Change values in the Hudi environment. | Not available. | | hudi-defaults | Change values in Hudi's hudi-defaults.conf file. | Not available. | | iceberg-defaults | Change values in Iceberg's iceberg-defaults.conf file. | Not available. | | delta-defaults | Change values in Delta's delta-defaults.conf file. | Not available. | | jupyter-notebook-conf | Change values in Jupyter Notebook's jupyter\$1notebook\$1config.py file. | Not available. | | jupyter-hub-conf | Change values in JupyterHubs's jupyterhub\$1config.py file. | Not available. | | jupyter-s3-conf | Configure Jupyter Notebook S3 persistence. | Not available. | | jupyter-sparkmagic-conf | Change values in Sparkmagic's config.json file. | Not available. | | livy-conf | Change values in Livy's livy.conf file. | Restarts Livy Server. | | livy-env | Change values in the Livy environment. | Restarts Livy Server. | | livy-log4j2 | Change Livy log4j2.properties settings. | Restarts Livy Server. | | mapred-env | Change values in the MapReduce application's environment. | Restarts Hadoop MapReduce-HistoryServer. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | Restarts Hadoop MapReduce-HistoryServer. | | oozie-env | Change values in Oozie's environment. | Restarts Oozie. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | Restarts Oozie. | | oozie-site | Change values in Oozie's oozie-site.xml file. | Restarts Oozie. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | Not available. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | Not available. | | phoenix-log4j2 | Change values in Phoenix's log4j2.properties file. | Restarts Phoenix-QueryServer. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | Not available. | | pig-env | Change values in the Pig environment. | Not available. | | pig-properties | Change values in Pig's pig.properties file. | Restarts Oozie. | | pig-log4j | Change values in Pig's log4j.properties file. | Not available. | | presto-log | Change values in Presto's log.properties file. | Restarts Presto-Server (for PrestoDB) | | presto-config | Change values in Presto's config.properties file. | Restarts Presto-Server (for PrestoDB) | | presto-password-authenticator | Change values in Presto's password-authenticator.properties file. | Not available. | | presto-env | Change values in Presto's presto-env.sh file. | Restarts Presto-Server (for PrestoDB) | | presto-node | Change values in Presto's node.properties file. | Not available. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | Not available. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | Not available. | | presto-connector-hive | Change values in Presto's hive.properties file. | Restarts Presto-Server (for PrestoDB) | | presto-connector-jmx | Change values in Presto's jmx.properties file. | Not available. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | Not available. | | presto-connector-lakeformation | Change values in Presto's lakeformation.properties file. | Restarts Presto-Server (for PrestoDB) | | presto-connector-localfile | Change values in Presto's localfile.properties file. | Not available. | | presto-connector-memory | Change values in Presto's memory.properties file. | Not available. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | Not available. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | Not available. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | Not available. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | Not available. | | presto-connector-redis | Change values in Presto's redis.properties file. | Not available. | | presto-connector-redshift | Change values in Presto's redshift.properties file. | Not available. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | Not available. | | presto-connector-tpcds | Change values in Presto's tpcds.properties file. | Not available. | | trino-log | Change values in Trino's log.properties file. | Restarts Trino-Server (for Trino) | | trino-config | Change values in Trino's config.properties file. | Restarts Trino-Server (for Trino) | | trino-password-authenticator | Change values in Trino's password-authenticator.properties file. | Restarts Trino-Server (for Trino) | | trino-env | Change values in Trino's trino-env.sh file. | Restarts Trino-Server (for Trino) | | trino-node | Change values in Trino's node.properties file. | Not available. | | trino-connector-blackhole | Change values in Trino's blackhole.properties file. | Not available. | | trino-connector-cassandra | Change values in Trino's cassandra.properties file. | Not available. | | trino-connector-delta | Change values in Trino's delta.properties file. | Restarts Trino-Server (for Trino) | | trino-connector-hive | Change values in Trino's hive.properties file. | Restarts Trino-Server (for Trino) | | trino-exchange-manager | Change values in Trino's exchange-manager.properties file. | Restarts Trino-Server (for Trino) | | trino-connector-iceberg | Change values in Trino's iceberg.properties file. | Restarts Trino-Server (for Trino) | | trino-connector-hudi | Change values in Trino's hudi.properties file. | Restarts Trino-Server (for Trino) | | trino-connector-jmx | Change values in Trino's jmx.properties file. | Not available. | | trino-connector-kafka | Change values in Trino's kafka.properties file. | Not available. | | trino-connector-localfile | Change values in Trino's localfile.properties file. | Not available. | | trino-connector-memory | Change values in Trino's memory.properties file. | Not available. | | trino-connector-mongodb | Change values in Trino's mongodb.properties file. | Not available. | | trino-connector-mysql | Change values in Trino's mysql.properties file. | Not available. | | trino-connector-postgresql | Change values in Trino's postgresql.properties file. | Not available. | | trino-connector-raptor | Change values in Trino's raptor.properties file. | Not available. | | trino-connector-redis | Change values in Trino's redis.properties file. | Not available. | | trino-connector-redshift | Change values in Trino's redshift.properties file. | Not available. | | trino-connector-tpch | Change values in Trino's tpch.properties file. | Not available. | | trino-connector-tpcds | Change values in Trino's tpcds.properties file. | Not available. | | ranger-kms-dbks-site | Change values in dbks-site.xml file of Ranger KMS. | Restarts Ranger KMS Server. | | ranger-kms-site | Change values in ranger-kms-site.xml file of Ranger KMS. | Restarts Ranger KMS Server. | | ranger-kms-env | Change values in the Ranger KMS environment. | Restarts Ranger KMS Server. | | ranger-kms-logback | Change values in kms-logback.xml file of Ranger KMS. | Not available. | | ranger-kms-db-ca | Change values for CA file on S3 for MySQL SSL connection with Ranger KMS. | Not available. | | spark | Amazon EMR-curated settings for Apache Spark. | This property modifies spark-defaults. See actions there. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | Restarts Spark history server and Spark thrift server. | | spark-env | Change values in the Spark environment. | Restarts Spark history server and Spark thrift server. | | spark-hive-site | Change values in Spark's hive-site.xml file | Not available. | | spark-log4j2 | Change values in Spark's log4j2.properties file. | Restarts Spark history server and Spark thrift server. | | spark-metrics | Change values in Spark's metrics.properties file. | Restarts Spark history server and Spark thrift server. | | tez-site | Change values in Tez's tez-site.xml file. | Restart Oozie and HiveServer2. | | yarn-env | Change values in the YARN environment. | Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts MapReduce-HistoryServer. | | yarn-site | Change values in YARN's yarn-site.xml file. | Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts Livy Server and MapReduce-HistoryServer. | | zeppelin-env | Change values in the Zeppelin environment. | Restarts Zeppelin. | | zeppelin-site | Change configuration settings in zeppelin-site.xml. | Restarts Zeppelin. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | Restarts Zookeeper server. | | zookeeper-logback | Change values in ZooKeeper's logback.xml file. | Restarts Zookeeper server. | | emr-metrics | Change emr metric settings for this node. | Restarts the CloudWatchAgent service. | ## 7.8.0 change log **Change log for 7.8.0 release and release notes** | Date | Event | Description | | --- | --- | --- | | 2025-03-07 | Docs publication | Amazon EMR 7.8.0 release notes first published | | 2025-03-06 | Deployment complete | Amazon EMR 7.8.0 fully deployed to all [supported Regions](https://aws.amazon.com/about-aws/global-infrastructure/regional-product-services/) | | 2025-02-28 | Initial limited release | Amazon EMR 7.8.0 first deployed to initial commercial Regions | # Amazon EMR release 7.7.0 ## 7.7.0 supported lifecycle The following table describes the supported lifecycle dates for Amazon EMR release 7.7.0. | Support phase | Date | | --- | --- | | Initial release date | February 6, 2025 | | Standard support until | February 5, 2027 | | End of support | February 6, 2027 | | End of life | February 6, 2028 | ## 7.7.0 application versions This release includes the following applications: [https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/Install-CloudWatch-Agent.html](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/Install-CloudWatch-Agent.html), [https://delta.io/](https://delta.io/), [https://flink.apache.org/](https://flink.apache.org/), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [https://hudi.apache.org](https://hudi.apache.org), [http://gethue.com/](http://gethue.com/), [https://iceberg.apache.org/](https://iceberg.apache.org/), [https://jupyter-enterprise-gateway.readthedocs.io/en/latest/](https://jupyter-enterprise-gateway.readthedocs.io/en/latest/), [https://jupyterhub.readthedocs.io/en/latest/#](https://jupyterhub.readthedocs.io/en/latest/#), [https://livy.incubator.apache.org/](https://livy.incubator.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [https://www.tensorflow.org/](https://www.tensorflow.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://trino.io/](https://trino.io/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-7.7.0 | emr-7.6.0 | emr-7.5.0 | emr-7.4.0 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 2.29.44, 1.12.780 | 2.29.25, 1.12.779 | 2.28.8, 1.12.772 | 2.28.8, 1.12.772 | | Python | 3.9, 3.11 | 3.9, 3.11 | 3.9, 3.11 | 3.9, 3.11 | | Scala | 2.12.18 | 2.12.18 | 2.12.18 | 2.12.18 | | AmazonCloudWatchAgent | 1.300032.2-amzn-0 | 1.300032.2-amzn-0 | 1.300032.2-amzn-0 | 1.300032.2-amzn-0 | | Delta | 3.2.1-amzn-2 | 3.2.1-amzn-1 | 3.2.1-amzn-0 | 3.2.0-amzn-1 | | Flink | 1.20.0-amzn-1 | 1.20.0-amzn-0 | 1.19.1-amzn-1 | 1.19.1-amzn-0 | | Ganglia | - | - | - | - | | HBase | 2.6.1-amzn-1 | 2.6.1-amzn-0 | 2.5.10-amzn-0 | 2.5.5-amzn-0 | | HCatalog | 3.1.3-amzn-16 | 3.1.3-amzn-15 | 3.1.3-amzn-14 | 3.1.3-amzn-13 | | Hadoop | 3.4.0-amzn-3 | 3.4.0-amzn-2 | 3.4.0-amzn-1 | 3.4.0-amzn-0 | | Hive | 3.1.3-amzn-16 | 3.1.3-amzn-15 | 3.1.3-amzn-14 | 3.1.3-amzn-13 | | Hudi | 0.15.0-amzn-4 | 0.15.0-amzn-3 | 0.15.0-amzn-2 | 0.15.0-amzn-1 | | Hue | 4.11.0 | 4.11.0 | 4.11.0 | 4.11.0 | | Iceberg | 1.7.1-amzn-0 | 1.6.1-amzn-2 | 1.6.1-amzn-1 | 1.6.1-amzn-0 | | JupyterEnterpriseGateway | 2.6.0 | 2.6.0 | 2.6.0 | 2.6.0 | | JupyterHub | 1.5.0 | 1.5.0 | 1.5.0 | 1.5.0 | | Livy | 0.8.0-incubating | 0.8.0-incubating | 0.8.0-incubating | 0.8.0-incubating | | MXNet | - | - | - | - | | Mahout | - | - | - | - | | Oozie | 5.2.1 | 5.2.1 | 5.2.1 | 5.2.1 | | Phoenix | 5.2.1 | 5.2.0 | 5.2.0 | 5.2.0 | | Pig | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | | Presto | 0.287-amzn-1 | 0.287-amzn-1 | 0.287-amzn-0 | 0.287-amzn-0 | | Spark | 3.5.3-amzn-1 | 3.5.3-amzn-0 | 3.5.2-amzn-1 | 3.5.2-amzn-0 | | Sqoop | - | - | - | 1.4.7 | | TensorFlow | 2.16.1 | 2.16.1 | 2.16.1 | 2.16.1 | | Tez | 0.10.2-amzn-14 | 0.10.2-amzn-13 | 0.10.2-amzn-12 | 0.10.2-amzn-11 | | Trino (PrestoSQL) | 467-amzn-0 | 457-amzn-0 | 446-amzn-0 | 446-amzn-0 | | Zeppelin | 0.11.1 | 0.11.1 | 0.11.1 | 0.11.1 | | ZooKeeper | 3.9.2-amzn-1 | 3.9.2-amzn-0 | 3.9.2-amzn-0 | 3.9.2-amzn-0 | # 7.7.0 release components details See the following table for more information about the *Extras* packages in Amazon EMR 7.7.0. | Application | Application Type | Package | Package Type | Package Version | | --- | --- | --- | --- | --- | | AmazonCloudWatchAgent | Core | hadoop-hdfs-journalnode | Extras | 3.4.0-amzn-3-1.amzn2023 | | AmazonCloudWatchAgent | Core | ranger-kms | Extras | 2.0.0-1.amzn2023 | | AmazonCloudWatchAgent | Core | s3-dist-cp | Extras | 2.37.0-1.amzn2023 | | AmazonCloudWatchAgent | Core | zookeeper | Extras | 3.9.2.amzn.1-1.amzn2023 | | AmazonCloudWatchAgent | Core | zookeeper-server | Extras | 3.9.2.amzn.1-1.amzn2023 | | Flink | Core | hadoop-client | Extras | 3.4.0.amzn.3-1.amzn2023 | | Flink | Core | hadoop-hdfs | Extras | 3.4.0.amzn.3-1.amzn2023 | | Flink | Core | hadoop-hdfs-datanode | Extras | 3.4.0.amzn.3-1.amzn2023 | | Flink | Core | hadoop-hdfs-journalnode | Extras | 3.4.0.amzn.3-1.amzn2023 | | Flink | Core | hadoop-hdfs-namenode | Extras | 3.4.0.amzn.3-1.amzn2023 | | Flink | Core | hadoop-httpfs | Extras | 3.4.0.amzn.3-1.amzn2023 | | Flink | Core | hadoop-kms | Extras | 3.4.0.amzn.3-1.amzn2023 | | Flink | Core | hadoop-mapreduce | Extras | 3.4.0.amzn.3-1.amzn2023 | | Flink | Core | hadoop-mapreduce-historyserver | Extras | 3.4.0.amzn.3-1.amzn2023 | | Flink | Core | hadoop-yarn | Extras | 3.4.0.amzn.3-1.amzn2023 | | Flink | Core | hadoop-yarn-nodemanager | Extras | 3.4.0.amzn.3-1.amzn2023 | | Flink | Core | hadoop-yarn-proxyserver | Extras | 3.4.0.amzn.3-1.amzn2023 | | Flink | Core | hadoop-yarn-resourcemanager | Extras | 3.4.0.amzn.3-1.amzn2023 | | Flink | Core | hadoop-yarn-timelineserver | Extras | 3.4.0.amzn.3-1.amzn2023 | | Flink | Core | hudi | Extras | 0.15.0.amzn.4-1.amzn2023 | | Flink | Core | ranger-kms | Extras | 2.0.0-1.amzn2023 | | Flink | Core | hadoop | Extras | 3.4.0.amzn.3-1.amzn2023 | | HBase | Core | s3-dist-cp | Extras | 2.37.0-1.amzn2023 | | HCatalog | Core | hadoop-yarn-proxyserver | Extras | 3.4.0.amzn.3-1.amzn2023 | | HCatalog | Core | mariadb-connector-java | Extras | | | Hive | Core | hadoop-httpfs | Extras | 3.4.0.amzn.3-1.amzn2023 | | Hive | Core | hadoop-yarn-proxyserver | Extras | 3.4.0.amzn.3-1.amzn2023 | | Hive | Core | hive-hbase | Extras | 3.1.3.amzn.16-1.amzn2023 | | Hive | Core | mariadb-connector-java | Extras | 2.7.2-1 | | Hive | Core | s3-dist-cp | Extras | 2.37.0-1.amzn2023 | | Phoenix | Core | s3-dist-cp | Extras | 2.37.0-1.amzn2023 | | Spark | Core | spark-datanucleus | Extras | 3.5.3.amzn.1-1.amzn2023 | | Tez | Core | hadoop-mapreduce | Extras | 3.4.0.amzn.3-1.amzn2023 | | Tez | Core | hadoop-mapreduce-historyserver | Extras | 3.4.0.amzn.3-1.amzn2023 | | Tez | Core | hadoop-yarn-proxyserver | Extras | 3.4.0.amzn.3-1.amzn2023 | | Trino | Core | bigtop-utils | Extras | 1.2.0-1.amzn2023 | | Trino | Core | hadoop-client | Extras | 3.4.0.amzn.3-1.amzn2023 | | Trino | Core | hadoop-hdfs | Extras | 3.4.0.amzn.3-1.amzn2023 | | Trino | Core | hadoop-hdfs-datanode | Extras | 3.4.0.amzn.3-1.amzn2023 | | Trino | Core | hadoop-hdfs-journalnode | Extras | 3.4.0.amzn.3-1.amzn2023 | | Trino | Core | hadoop-hdfs-namenode | Extras | 3.4.0.amzn.3-1.amzn2023 | | Trino | Core | hadoop-kms | Extras | 3.4.0.amzn.3-1.amzn2023 | | Trino | Core | hadoop-mapreduce | Extras | 3.4.0.amzn.3-1.amzn2023 | | Trino | Core | hadoop-mapreduce-historyserver | Extras | 3.4.0.amzn.3-1.amzn2023 | | Trino | Core | hadoop-yarn | Extras | 3.4.0.amzn.3-1.amzn2023 | | Trino | Core | hadoop-yarn-nodemanager | Extras | 3.4.0.amzn.3-1.amzn2023 | | Trino | Core | hadoop-yarn-proxyserver | Extras | 3.4.0.amzn.3-1.amzn2023 | | Trino | Core | hadoop-yarn-resourcemanager | Extras | 3.4.0.amzn.3-1.amzn2023 | | Trino | Core | hadoop-yarn-timelineserver | Extras | 3.4.0.amzn.3-1.amzn2023 | | Trino | Core | hive | Extras | 3.1.3.amzn.16-1.amzn2023 | | Trino | Core | hive-hcatalog-server | Extras | 3.1.3.amzn.16-1.amzn2023 | | Trino | Core | mariadb-connector-java | Extras | 2.7.2-1 | | Trino | Core | ranger-kms | Extras | 2.0.0-1.amzn2023 | | Trino | Core | zookeeper | Extras | 3.9.2.amzn.1-1.amzn2023 | | Trino | Core | zookeeper-server | Extras | 3.9.2.amzn.1-1.amzn2023 | | Trino | Core | bigtop-groovy | Extras | 2.5.4-1.amzn2023 | | Trino | Core | bigtop-jsvc | Extras | 1.2.4-1.amzn2023 | | Trino | Core | hadoop | Extras | 3.4.0.amzn.3-1.amzn2023 | | Trino | Core | hadoop-lzo | Extras | 0.4.19-1.amzn2023 | | Trino | Core | hive-hcatalog | Extras | 3.1.3.amzn.16-1.amzn2023 | | Trino | Core | hive-jdbc | Extras | 3.1.3.amzn.16-1.amzn2023 | | Zookeeper | Core | emrfs | Extras | 3.9.2.amzn.1-1.amzn2023 | | Zookeeper | Core | hadoop-client | Extras | 3.9.2.amzn.1-1.amzn2023 | | Zookeeper | Core | hadoop-hdfs | Extras | 3.9.2.amzn.1-1.amzn2023 | | Zookeeper | Core | hadoop-hdfs-datanode | Extras | 3.9.2.amzn.1-1.amzn2023 | | Zookeeper | Core | hadoop-hdfs-journalnode | Extras | 3.9.2.amzn.1-1.amzn2023 | | Zookeeper | Core | hadoop-hdfs-namenode | Extras | 3.9.2.amzn.1-1.amzn2023 | | Zookeeper | Core | hadoop-httpfs | Extras | 3.9.2.amzn.1-1.amzn2023 | | Zookeeper | Core | hadoop-kms | Extras | 3.9.2.amzn.1-1.amzn2023 | | Zookeeper | Core | hadoop-mapreduce | Extras | 3.4.0.amzn.3-1.amzn2023 | | Zookeeper | Core | hadoop-mapreduce-historyserver | Extras | 3.4.0.amzn.3-1.amzn2023 | | Zookeeper | Core | hadoop-yarn | Extras | 3.4.0.amzn.3-1.amzn2023 | | Zookeeper | Core | hadoop-yarn-nodemanager | Extras | 3.4.0.amzn.3-1.amzn2023 | | Zookeeper | Core | hadoop-yarn-proxyserver | Extras | 3.4.0.amzn.3-1.amzn2023 | | Zookeeper | Core | hadoop-yarn-resourcemanager | Extras | 3.4.0.amzn.3-1.amzn2023 | | Zookeeper | Core | hadoop-yarn-timelineserver | Extras | 3.4.0.amzn.3-1.amzn2023 | | Zookeeper | Core | ranger-kms | Extras | 2.0.0-1.amzn2023 | # 7.7.0 common vulnerabilities and exposures The following table lists all CVEs that don't impact EMR clusters that run on recommended configurations of Amazon EMR 7.7.0. Amazon EMR is dependent on upstream open-source for availability of fixes and will provide the latest stable version as part of the Core Engine components within 90 days of Amazon EMR verifying the fixes. This table was updated on February 4, 2025. | CVE ID | Severity | CVE details URL | | --- | --- | --- | | CVE-2015-1832 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2015-1832](https://nvd.nist.gov/vuln/detail/CVE-2015-1832) | | CVE-2016-5018 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2016-5018](https://nvd.nist.gov/vuln/detail/CVE-2016-5018) | | CVE-2017-15095 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2017-15095](https://nvd.nist.gov/vuln/detail/CVE-2017-15095) | | CVE-2017-17485 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2017-17485](https://nvd.nist.gov/vuln/detail/CVE-2017-17485) | | CVE-2017-7525 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2017-7525](https://nvd.nist.gov/vuln/detail/CVE-2017-7525) | | CVE-2018-11307 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-11307](https://nvd.nist.gov/vuln/detail/CVE-2018-11307) | | CVE-2018-11307 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-11307](https://nvd.nist.gov/vuln/detail/CVE-2018-11307) | | CVE-2018-14718 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-14718](https://nvd.nist.gov/vuln/detail/CVE-2018-14718) | | CVE-2018-14718 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-14718](https://nvd.nist.gov/vuln/detail/CVE-2018-14718) | | CVE-2018-14719 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-14719](https://nvd.nist.gov/vuln/detail/CVE-2018-14719) | | CVE-2018-14719 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-14719](https://nvd.nist.gov/vuln/detail/CVE-2018-14719) | | CVE-2018-14720 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-14720](https://nvd.nist.gov/vuln/detail/CVE-2018-14720) | | CVE-2018-14720 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-14720](https://nvd.nist.gov/vuln/detail/CVE-2018-14720) | | CVE-2018-14721 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-14721](https://nvd.nist.gov/vuln/detail/CVE-2018-14721) | | CVE-2018-14721 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-14721](https://nvd.nist.gov/vuln/detail/CVE-2018-14721) | | CVE-2018-19360 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-19360](https://nvd.nist.gov/vuln/detail/CVE-2018-19360) | | CVE-2018-19360 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-19360](https://nvd.nist.gov/vuln/detail/CVE-2018-19360) | | CVE-2018-19361 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-19361](https://nvd.nist.gov/vuln/detail/CVE-2018-19361) | | CVE-2018-19361 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-19361](https://nvd.nist.gov/vuln/detail/CVE-2018-19361) | | CVE-2018-19362 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-19362](https://nvd.nist.gov/vuln/detail/CVE-2018-19362) | | CVE-2018-19362 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-19362](https://nvd.nist.gov/vuln/detail/CVE-2018-19362) | | CVE-2018-7489 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-7489](https://nvd.nist.gov/vuln/detail/CVE-2018-7489) | | CVE-2018-7489 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-7489](https://nvd.nist.gov/vuln/detail/CVE-2018-7489) | | CVE-2019-10202 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-10202](https://nvd.nist.gov/vuln/detail/CVE-2019-10202) | | CVE-2019-14379 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-14379](https://nvd.nist.gov/vuln/detail/CVE-2019-14379) | | CVE-2019-14379 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-14379](https://nvd.nist.gov/vuln/detail/CVE-2019-14379) | | CVE-2019-14540 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-14540](https://nvd.nist.gov/vuln/detail/CVE-2019-14540) | | CVE-2019-14540 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-14540](https://nvd.nist.gov/vuln/detail/CVE-2019-14540) | | CVE-2019-14892 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-14892](https://nvd.nist.gov/vuln/detail/CVE-2019-14892) | | CVE-2019-14892 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-14892](https://nvd.nist.gov/vuln/detail/CVE-2019-14892) | | CVE-2019-14893 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-14893](https://nvd.nist.gov/vuln/detail/CVE-2019-14893) | | CVE-2019-14893 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-14893](https://nvd.nist.gov/vuln/detail/CVE-2019-14893) | | CVE-2019-16335 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-16335](https://nvd.nist.gov/vuln/detail/CVE-2019-16335) | | CVE-2019-16335 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-16335](https://nvd.nist.gov/vuln/detail/CVE-2019-16335) | | CVE-2019-16942 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-16942](https://nvd.nist.gov/vuln/detail/CVE-2019-16942) | | CVE-2019-16942 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-16942](https://nvd.nist.gov/vuln/detail/CVE-2019-16942) | | CVE-2019-16943 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-16943](https://nvd.nist.gov/vuln/detail/CVE-2019-16943) | | CVE-2019-16943 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-16943](https://nvd.nist.gov/vuln/detail/CVE-2019-16943) | | CVE-2019-17267 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-17267](https://nvd.nist.gov/vuln/detail/CVE-2019-17267) | | CVE-2019-17267 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-17267](https://nvd.nist.gov/vuln/detail/CVE-2019-17267) | | CVE-2019-17531 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-17531](https://nvd.nist.gov/vuln/detail/CVE-2019-17531) | | CVE-2019-17531 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-17531](https://nvd.nist.gov/vuln/detail/CVE-2019-17531) | | CVE-2019-20330 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-20330](https://nvd.nist.gov/vuln/detail/CVE-2019-20330) | | CVE-2019-20330 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-20330](https://nvd.nist.gov/vuln/detail/CVE-2019-20330) | | CVE-2020-8840 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2020-8840](https://nvd.nist.gov/vuln/detail/CVE-2020-8840) | | CVE-2020-8840 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2020-8840](https://nvd.nist.gov/vuln/detail/CVE-2020-8840) | | CVE-2020-9546 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2020-9546](https://nvd.nist.gov/vuln/detail/CVE-2020-9546) | | CVE-2020-9546 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2020-9546](https://nvd.nist.gov/vuln/detail/CVE-2020-9546) | | CVE-2020-9547 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2020-9547](https://nvd.nist.gov/vuln/detail/CVE-2020-9547) | | CVE-2020-9547 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2020-9547](https://nvd.nist.gov/vuln/detail/CVE-2020-9547) | | CVE-2020-9548 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2020-9548](https://nvd.nist.gov/vuln/detail/CVE-2020-9548) | | CVE-2020-9548 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2020-9548](https://nvd.nist.gov/vuln/detail/CVE-2020-9548) | | CVE-2022-1471 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2022-1471](https://nvd.nist.gov/vuln/detail/CVE-2022-1471) | | CVE-2022-1471 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2022-1471](https://nvd.nist.gov/vuln/detail/CVE-2022-1471) | | CVE-2022-37865 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2022-37865](https://nvd.nist.gov/vuln/detail/CVE-2022-37865) | | CVE-2022-39135 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2022-39135](https://nvd.nist.gov/vuln/detail/CVE-2022-39135) | | CVE-2022-42889 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2022-42889](https://nvd.nist.gov/vuln/detail/CVE-2022-42889) | | CVE-2022-46337 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2022-46337](https://nvd.nist.gov/vuln/detail/CVE-2022-46337) | | CVE-2022-46337 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2022-46337](https://nvd.nist.gov/vuln/detail/CVE-2022-46337) | | CVE-2023-44981 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2023-44981](https://nvd.nist.gov/vuln/detail/CVE-2023-44981) | | CVE-2024-47561 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2024-47561](https://nvd.nist.gov/vuln/detail/CVE-2024-47561) | | CVE-2024-47561 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2024-47561](https://nvd.nist.gov/vuln/detail/CVE-2024-47561) | | CVE-2024-47561 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2024-47561](https://nvd.nist.gov/vuln/detail/CVE-2024-47561) | | CVE-2024-47561 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2024-47561](https://nvd.nist.gov/vuln/detail/CVE-2024-47561) | | CVE-2024-47561 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2024-47561](https://nvd.nist.gov/vuln/detail/CVE-2024-47561) | | CVE-2024-47561 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2024-47561](https://nvd.nist.gov/vuln/detail/CVE-2024-47561) | | CVE-2024-52046 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2024-52046](https://nvd.nist.gov/vuln/detail/CVE-2024-52046) | | CVE-2024-52046 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2024-52046](https://nvd.nist.gov/vuln/detail/CVE-2024-52046) | | CVE-2024-53990 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2024-53990](https://nvd.nist.gov/vuln/detail/CVE-2024-53990) | | CVE-2024-53990 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2024-53990](https://nvd.nist.gov/vuln/detail/CVE-2024-53990) | | CVE-2012-0881 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2012-0881](https://nvd.nist.gov/vuln/detail/CVE-2012-0881) | | CVE-2013-4002 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2013-4002](https://nvd.nist.gov/vuln/detail/CVE-2013-4002) | | CVE-2015-5237 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2015-5237](https://nvd.nist.gov/vuln/detail/CVE-2015-5237) | | CVE-2015-5237 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2015-5237](https://nvd.nist.gov/vuln/detail/CVE-2015-5237) | | CVE-2016-6796 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2016-6796](https://nvd.nist.gov/vuln/detail/CVE-2016-6796) | | CVE-2018-1000180 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2018-1000180](https://nvd.nist.gov/vuln/detail/CVE-2018-1000180) | | CVE-2018-12022 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2018-12022](https://nvd.nist.gov/vuln/detail/CVE-2018-12022) | | CVE-2018-12022 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2018-12022](https://nvd.nist.gov/vuln/detail/CVE-2018-12022) | | CVE-2018-12023 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2018-12023](https://nvd.nist.gov/vuln/detail/CVE-2018-12023) | | CVE-2018-12023 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2018-12023](https://nvd.nist.gov/vuln/detail/CVE-2018-12023) | | CVE-2018-5968 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2018-5968](https://nvd.nist.gov/vuln/detail/CVE-2018-5968) | | CVE-2019-0205 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2019-0205](https://nvd.nist.gov/vuln/detail/CVE-2019-0205) | | CVE-2019-0205 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2019-0205](https://nvd.nist.gov/vuln/detail/CVE-2019-0205) | | CVE-2019-10172 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2019-10172](https://nvd.nist.gov/vuln/detail/CVE-2019-10172) | | CVE-2019-12086 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2019-12086](https://nvd.nist.gov/vuln/detail/CVE-2019-12086) | | CVE-2019-12086 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2019-12086](https://nvd.nist.gov/vuln/detail/CVE-2019-12086) | | CVE-2019-14439 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2019-14439](https://nvd.nist.gov/vuln/detail/CVE-2019-14439) | | CVE-2019-14439 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2019-14439](https://nvd.nist.gov/vuln/detail/CVE-2019-14439) | | CVE-2020-10650 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-10650](https://nvd.nist.gov/vuln/detail/CVE-2020-10650) | | CVE-2020-10650 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-10650](https://nvd.nist.gov/vuln/detail/CVE-2020-10650) | | CVE-2020-10672 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-10672](https://nvd.nist.gov/vuln/detail/CVE-2020-10672) | | CVE-2020-10672 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-10672](https://nvd.nist.gov/vuln/detail/CVE-2020-10672) | | CVE-2020-10673 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-10673](https://nvd.nist.gov/vuln/detail/CVE-2020-10673) | | CVE-2020-10673 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-10673](https://nvd.nist.gov/vuln/detail/CVE-2020-10673) | | CVE-2020-10968 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-10968](https://nvd.nist.gov/vuln/detail/CVE-2020-10968) | | CVE-2020-10968 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-10968](https://nvd.nist.gov/vuln/detail/CVE-2020-10968) | | CVE-2020-10969 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-10969](https://nvd.nist.gov/vuln/detail/CVE-2020-10969) | | CVE-2020-10969 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-10969](https://nvd.nist.gov/vuln/detail/CVE-2020-10969) | | CVE-2020-11111 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-11111](https://nvd.nist.gov/vuln/detail/CVE-2020-11111) | | CVE-2020-11111 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-11111](https://nvd.nist.gov/vuln/detail/CVE-2020-11111) | | CVE-2020-11112 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-11112](https://nvd.nist.gov/vuln/detail/CVE-2020-11112) | | CVE-2020-11112 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-11112](https://nvd.nist.gov/vuln/detail/CVE-2020-11112) | | CVE-2020-11113 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-11113](https://nvd.nist.gov/vuln/detail/CVE-2020-11113) | | CVE-2020-11113 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-11113](https://nvd.nist.gov/vuln/detail/CVE-2020-11113) | | CVE-2020-11619 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-11619](https://nvd.nist.gov/vuln/detail/CVE-2020-11619) | | CVE-2020-11619 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-11619](https://nvd.nist.gov/vuln/detail/CVE-2020-11619) | | CVE-2020-11620 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-11620](https://nvd.nist.gov/vuln/detail/CVE-2020-11620) | | CVE-2020-11620 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-11620](https://nvd.nist.gov/vuln/detail/CVE-2020-11620) | | CVE-2020-13936 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-13936](https://nvd.nist.gov/vuln/detail/CVE-2020-13936) | | CVE-2020-13949 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-13949](https://nvd.nist.gov/vuln/detail/CVE-2020-13949) | | CVE-2020-13949 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-13949](https://nvd.nist.gov/vuln/detail/CVE-2020-13949) | | CVE-2020-13949 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-13949](https://nvd.nist.gov/vuln/detail/CVE-2020-13949) | | CVE-2020-14060 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-14060](https://nvd.nist.gov/vuln/detail/CVE-2020-14060) | | CVE-2020-14060 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-14060](https://nvd.nist.gov/vuln/detail/CVE-2020-14060) | | CVE-2020-14061 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-14061](https://nvd.nist.gov/vuln/detail/CVE-2020-14061) | | CVE-2020-14061 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-14061](https://nvd.nist.gov/vuln/detail/CVE-2020-14061) | | CVE-2020-14062 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-14062](https://nvd.nist.gov/vuln/detail/CVE-2020-14062) | | CVE-2020-14062 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-14062](https://nvd.nist.gov/vuln/detail/CVE-2020-14062) | | CVE-2020-14195 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-14195](https://nvd.nist.gov/vuln/detail/CVE-2020-14195) | | CVE-2020-14195 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-14195](https://nvd.nist.gov/vuln/detail/CVE-2020-14195) | | CVE-2020-24616 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-24616](https://nvd.nist.gov/vuln/detail/CVE-2020-24616) | | CVE-2020-24616 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-24616](https://nvd.nist.gov/vuln/detail/CVE-2020-24616) | | CVE-2020-24750 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-24750](https://nvd.nist.gov/vuln/detail/CVE-2020-24750) | | CVE-2020-24750 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-24750](https://nvd.nist.gov/vuln/detail/CVE-2020-24750) | | CVE-2020-25649 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-25649](https://nvd.nist.gov/vuln/detail/CVE-2020-25649) | | CVE-2020-28052 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-28052](https://nvd.nist.gov/vuln/detail/CVE-2020-28052) | | CVE-2020-35490 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-35490](https://nvd.nist.gov/vuln/detail/CVE-2020-35490) | | CVE-2020-35490 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-35490](https://nvd.nist.gov/vuln/detail/CVE-2020-35490) | | CVE-2020-35491 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-35491](https://nvd.nist.gov/vuln/detail/CVE-2020-35491) | | CVE-2020-35491 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-35491](https://nvd.nist.gov/vuln/detail/CVE-2020-35491) | | CVE-2020-35728 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-35728](https://nvd.nist.gov/vuln/detail/CVE-2020-35728) | | CVE-2020-35728 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-35728](https://nvd.nist.gov/vuln/detail/CVE-2020-35728) | | CVE-2020-36179 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36179](https://nvd.nist.gov/vuln/detail/CVE-2020-36179) | | CVE-2020-36179 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36179](https://nvd.nist.gov/vuln/detail/CVE-2020-36179) | | CVE-2020-36180 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36180](https://nvd.nist.gov/vuln/detail/CVE-2020-36180) | | CVE-2020-36180 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36180](https://nvd.nist.gov/vuln/detail/CVE-2020-36180) | | CVE-2020-36181 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36181](https://nvd.nist.gov/vuln/detail/CVE-2020-36181) | | CVE-2020-36181 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36181](https://nvd.nist.gov/vuln/detail/CVE-2020-36181) | | CVE-2020-36182 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36182](https://nvd.nist.gov/vuln/detail/CVE-2020-36182) | | CVE-2020-36182 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36182](https://nvd.nist.gov/vuln/detail/CVE-2020-36182) | | CVE-2020-36183 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36183](https://nvd.nist.gov/vuln/detail/CVE-2020-36183) | | CVE-2020-36183 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36183](https://nvd.nist.gov/vuln/detail/CVE-2020-36183) | | CVE-2020-36184 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36184](https://nvd.nist.gov/vuln/detail/CVE-2020-36184) | | CVE-2020-36184 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36184](https://nvd.nist.gov/vuln/detail/CVE-2020-36184) | | CVE-2020-36185 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36185](https://nvd.nist.gov/vuln/detail/CVE-2020-36185) | | CVE-2020-36185 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36185](https://nvd.nist.gov/vuln/detail/CVE-2020-36185) | | CVE-2020-36186 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36186](https://nvd.nist.gov/vuln/detail/CVE-2020-36186) | | CVE-2020-36186 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36186](https://nvd.nist.gov/vuln/detail/CVE-2020-36186) | | CVE-2020-36187 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36187](https://nvd.nist.gov/vuln/detail/CVE-2020-36187) | | CVE-2020-36187 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36187](https://nvd.nist.gov/vuln/detail/CVE-2020-36187) | | CVE-2020-36188 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36188](https://nvd.nist.gov/vuln/detail/CVE-2020-36188) | | CVE-2020-36188 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36188](https://nvd.nist.gov/vuln/detail/CVE-2020-36188) | | CVE-2020-36189 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36189](https://nvd.nist.gov/vuln/detail/CVE-2020-36189) | | CVE-2020-36189 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36189](https://nvd.nist.gov/vuln/detail/CVE-2020-36189) | | CVE-2020-36518 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36518](https://nvd.nist.gov/vuln/detail/CVE-2020-36518) | | CVE-2020-36518 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36518](https://nvd.nist.gov/vuln/detail/CVE-2020-36518) | | CVE-2021-20190 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-20190](https://nvd.nist.gov/vuln/detail/CVE-2021-20190) | | CVE-2021-20190 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-20190](https://nvd.nist.gov/vuln/detail/CVE-2021-20190) | | CVE-2021-31684 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-31684](https://nvd.nist.gov/vuln/detail/CVE-2021-31684) | | CVE-2021-31684 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-31684](https://nvd.nist.gov/vuln/detail/CVE-2021-31684) | | CVE-2021-34538 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-34538](https://nvd.nist.gov/vuln/detail/CVE-2021-34538) | | CVE-2021-34538 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-34538](https://nvd.nist.gov/vuln/detail/CVE-2021-34538) | | CVE-2021-34538 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-34538](https://nvd.nist.gov/vuln/detail/CVE-2021-34538) | | CVE-2021-43859 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-43859](https://nvd.nist.gov/vuln/detail/CVE-2021-43859) | | CVE-2022-1415 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-1415](https://nvd.nist.gov/vuln/detail/CVE-2022-1415) | | CVE-2022-1415 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-1415](https://nvd.nist.gov/vuln/detail/CVE-2022-1415) | | CVE-2022-25647 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-25647](https://nvd.nist.gov/vuln/detail/CVE-2022-25647) | | CVE-2022-25647 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-25647](https://nvd.nist.gov/vuln/detail/CVE-2022-25647) | | CVE-2022-25647 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-25647](https://nvd.nist.gov/vuln/detail/CVE-2022-25647) | | CVE-2022-25857 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-25857](https://nvd.nist.gov/vuln/detail/CVE-2022-25857) | | CVE-2022-3171 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-3171](https://nvd.nist.gov/vuln/detail/CVE-2022-3171) | | CVE-2022-3171 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-3171](https://nvd.nist.gov/vuln/detail/CVE-2022-3171) | | CVE-2022-3171 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-3171](https://nvd.nist.gov/vuln/detail/CVE-2022-3171) | | CVE-2022-3171 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-3171](https://nvd.nist.gov/vuln/detail/CVE-2022-3171) | | CVE-2022-3509 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-3509](https://nvd.nist.gov/vuln/detail/CVE-2022-3509) | | CVE-2022-3509 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-3509](https://nvd.nist.gov/vuln/detail/CVE-2022-3509) | | CVE-2022-3509 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-3509](https://nvd.nist.gov/vuln/detail/CVE-2022-3509) | | CVE-2022-3509 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-3509](https://nvd.nist.gov/vuln/detail/CVE-2022-3509) | | CVE-2022-3510 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-3510](https://nvd.nist.gov/vuln/detail/CVE-2022-3510) | | CVE-2022-3510 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-3510](https://nvd.nist.gov/vuln/detail/CVE-2022-3510) | | CVE-2022-3510 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-3510](https://nvd.nist.gov/vuln/detail/CVE-2022-3510) | | CVE-2022-3510 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-3510](https://nvd.nist.gov/vuln/detail/CVE-2022-3510) | | CVE-2022-36364 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-36364](https://nvd.nist.gov/vuln/detail/CVE-2022-36364) | | CVE-2022-37866 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-37866](https://nvd.nist.gov/vuln/detail/CVE-2022-37866) | | CVE-2022-40149 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-40149](https://nvd.nist.gov/vuln/detail/CVE-2022-40149) | | CVE-2022-40149 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-40149](https://nvd.nist.gov/vuln/detail/CVE-2022-40149) | | CVE-2022-40149 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-40149](https://nvd.nist.gov/vuln/detail/CVE-2022-40149) | | CVE-2022-40150 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-40150](https://nvd.nist.gov/vuln/detail/CVE-2022-40150) | | CVE-2022-40150 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-40150](https://nvd.nist.gov/vuln/detail/CVE-2022-40150) | | CVE-2022-40150 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-40150](https://nvd.nist.gov/vuln/detail/CVE-2022-40150) | | CVE-2022-40151 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-40151](https://nvd.nist.gov/vuln/detail/CVE-2022-40151) | | CVE-2022-40152 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-40152](https://nvd.nist.gov/vuln/detail/CVE-2022-40152) | | CVE-2022-41966 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-41966](https://nvd.nist.gov/vuln/detail/CVE-2022-41966) | | CVE-2022-42003 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-42003](https://nvd.nist.gov/vuln/detail/CVE-2022-42003) | | CVE-2022-42003 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-42003](https://nvd.nist.gov/vuln/detail/CVE-2022-42003) | | CVE-2022-42003 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-42003](https://nvd.nist.gov/vuln/detail/CVE-2022-42003) | | CVE-2022-42003 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-42003](https://nvd.nist.gov/vuln/detail/CVE-2022-42003) | | CVE-2022-42003 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-42003](https://nvd.nist.gov/vuln/detail/CVE-2022-42003) | | CVE-2022-42003 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-42003](https://nvd.nist.gov/vuln/detail/CVE-2022-42003) | | CVE-2022-42004 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-42004](https://nvd.nist.gov/vuln/detail/CVE-2022-42004) | | CVE-2022-42004 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-42004](https://nvd.nist.gov/vuln/detail/CVE-2022-42004) | | CVE-2022-42004 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-42004](https://nvd.nist.gov/vuln/detail/CVE-2022-42004) | | CVE-2022-42004 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-42004](https://nvd.nist.gov/vuln/detail/CVE-2022-42004) | | CVE-2022-42004 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-42004](https://nvd.nist.gov/vuln/detail/CVE-2022-42004) | | CVE-2022-45685 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-45685](https://nvd.nist.gov/vuln/detail/CVE-2022-45685) | | CVE-2022-45685 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-45685](https://nvd.nist.gov/vuln/detail/CVE-2022-45685) | | CVE-2022-45685 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-45685](https://nvd.nist.gov/vuln/detail/CVE-2022-45685) | | CVE-2022-45688 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-45688](https://nvd.nist.gov/vuln/detail/CVE-2022-45688) | | CVE-2022-45693 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-45693](https://nvd.nist.gov/vuln/detail/CVE-2022-45693) | | CVE-2022-45693 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-45693](https://nvd.nist.gov/vuln/detail/CVE-2022-45693) | | CVE-2022-45693 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-45693](https://nvd.nist.gov/vuln/detail/CVE-2022-45693) | | CVE-2022-46751 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-46751](https://nvd.nist.gov/vuln/detail/CVE-2022-46751) | | CVE-2022-46751 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-46751](https://nvd.nist.gov/vuln/detail/CVE-2022-46751) | | CVE-2023-1370 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-1370](https://nvd.nist.gov/vuln/detail/CVE-2023-1370) | | CVE-2023-1370 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-1370](https://nvd.nist.gov/vuln/detail/CVE-2023-1370) | | CVE-2023-1370 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-1370](https://nvd.nist.gov/vuln/detail/CVE-2023-1370) | | CVE-2023-1436 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-1436](https://nvd.nist.gov/vuln/detail/CVE-2023-1436) | | CVE-2023-1436 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-1436](https://nvd.nist.gov/vuln/detail/CVE-2023-1436) | | CVE-2023-1436 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-1436](https://nvd.nist.gov/vuln/detail/CVE-2023-1436) | | CVE-2023-20883 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-20883](https://nvd.nist.gov/vuln/detail/CVE-2023-20883) | | CVE-2023-25194 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-25194](https://nvd.nist.gov/vuln/detail/CVE-2023-25194) | | CVE-2023-25194 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-25194](https://nvd.nist.gov/vuln/detail/CVE-2023-25194) | | CVE-2023-2976 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-2976](https://nvd.nist.gov/vuln/detail/CVE-2023-2976) | | CVE-2023-2976 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-2976](https://nvd.nist.gov/vuln/detail/CVE-2023-2976) | | CVE-2023-2976 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-2976](https://nvd.nist.gov/vuln/detail/CVE-2023-2976) | | CVE-2023-2976 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-2976](https://nvd.nist.gov/vuln/detail/CVE-2023-2976) | | CVE-2023-2976 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-2976](https://nvd.nist.gov/vuln/detail/CVE-2023-2976) | | CVE-2023-2976 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-2976](https://nvd.nist.gov/vuln/detail/CVE-2023-2976) | | CVE-2023-2976 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-2976](https://nvd.nist.gov/vuln/detail/CVE-2023-2976) | | CVE-2023-2976 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-2976](https://nvd.nist.gov/vuln/detail/CVE-2023-2976) | | CVE-2023-2976 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-2976](https://nvd.nist.gov/vuln/detail/CVE-2023-2976) | | CVE-2023-2976 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-2976](https://nvd.nist.gov/vuln/detail/CVE-2023-2976) | | CVE-2023-2976 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-2976](https://nvd.nist.gov/vuln/detail/CVE-2023-2976) | | CVE-2023-2976 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-2976](https://nvd.nist.gov/vuln/detail/CVE-2023-2976) | | CVE-2023-34453 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-34453](https://nvd.nist.gov/vuln/detail/CVE-2023-34453) | | CVE-2023-34454 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-34454](https://nvd.nist.gov/vuln/detail/CVE-2023-34454) | | CVE-2023-34455 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-34455](https://nvd.nist.gov/vuln/detail/CVE-2023-34455) | | CVE-2023-3635 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-3635](https://nvd.nist.gov/vuln/detail/CVE-2023-3635) | | CVE-2023-3635 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-3635](https://nvd.nist.gov/vuln/detail/CVE-2023-3635) | | CVE-2023-36478 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-36478](https://nvd.nist.gov/vuln/detail/CVE-2023-36478) | | CVE-2023-39410 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-39410](https://nvd.nist.gov/vuln/detail/CVE-2023-39410) | | CVE-2023-39410 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-39410](https://nvd.nist.gov/vuln/detail/CVE-2023-39410) | | CVE-2023-39410 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-39410](https://nvd.nist.gov/vuln/detail/CVE-2023-39410) | | CVE-2023-39410 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-39410](https://nvd.nist.gov/vuln/detail/CVE-2023-39410) | | CVE-2023-39410 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-39410](https://nvd.nist.gov/vuln/detail/CVE-2023-39410) | | CVE-2023-43642 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-43642](https://nvd.nist.gov/vuln/detail/CVE-2023-43642) | | CVE-2023-50387 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-50387](https://nvd.nist.gov/vuln/detail/CVE-2023-50387) | | CVE-2023-5072 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-5072](https://nvd.nist.gov/vuln/detail/CVE-2023-5072) | | CVE-2023-5072 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-5072](https://nvd.nist.gov/vuln/detail/CVE-2023-5072) | | CVE-2023-52428 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-52428](https://nvd.nist.gov/vuln/detail/CVE-2023-52428) | | CVE-2023-6378 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-6378](https://nvd.nist.gov/vuln/detail/CVE-2023-6378) | | CVE-2023-6378 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-6378](https://nvd.nist.gov/vuln/detail/CVE-2023-6378) | | CVE-2023-6481 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-6481](https://nvd.nist.gov/vuln/detail/CVE-2023-6481) | | CVE-2023-6481 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-6481](https://nvd.nist.gov/vuln/detail/CVE-2023-6481) | | CVE-2024-21634 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-21634](https://nvd.nist.gov/vuln/detail/CVE-2024-21634) | | CVE-2024-35178 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-35178](https://nvd.nist.gov/vuln/detail/CVE-2024-35178) | | CVE-2024-7254 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-7254](https://nvd.nist.gov/vuln/detail/CVE-2024-7254) | | CVE-2024-7254 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-7254](https://nvd.nist.gov/vuln/detail/CVE-2024-7254) | | CVE-2024-7254 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-7254](https://nvd.nist.gov/vuln/detail/CVE-2024-7254) | | CVE-2024-7254 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-7254](https://nvd.nist.gov/vuln/detail/CVE-2024-7254) | | CVE-2024-7254 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-7254](https://nvd.nist.gov/vuln/detail/CVE-2024-7254) | | CVE-2024-7254 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-7254](https://nvd.nist.gov/vuln/detail/CVE-2024-7254) | | CVE-2024-7254 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-7254](https://nvd.nist.gov/vuln/detail/CVE-2024-7254) | | CVE-2024-7254 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-7254](https://nvd.nist.gov/vuln/detail/CVE-2024-7254) | | CVE-2024-7254 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-7254](https://nvd.nist.gov/vuln/detail/CVE-2024-7254) | | CVE-2009-2625 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2009-2625](https://nvd.nist.gov/vuln/detail/CVE-2009-2625) | | CVE-2012-5783 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2012-5783](https://nvd.nist.gov/vuln/detail/CVE-2012-5783) | | CVE-2012-6153 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2012-6153](https://nvd.nist.gov/vuln/detail/CVE-2012-6153) | | CVE-2017-10355 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2017-10355](https://nvd.nist.gov/vuln/detail/CVE-2017-10355) | | CVE-2018-10237 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2018-10237](https://nvd.nist.gov/vuln/detail/CVE-2018-10237) | | CVE-2018-10237 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2018-10237](https://nvd.nist.gov/vuln/detail/CVE-2018-10237) | | CVE-2018-10237 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2018-10237](https://nvd.nist.gov/vuln/detail/CVE-2018-10237) | | CVE-2018-10237 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2018-10237](https://nvd.nist.gov/vuln/detail/CVE-2018-10237) | | CVE-2018-10237 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2018-10237](https://nvd.nist.gov/vuln/detail/CVE-2018-10237) | | CVE-2018-10237 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2018-10237](https://nvd.nist.gov/vuln/detail/CVE-2018-10237) | | CVE-2018-11798 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2018-11798](https://nvd.nist.gov/vuln/detail/CVE-2018-11798) | | CVE-2018-1313 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2018-1313](https://nvd.nist.gov/vuln/detail/CVE-2018-1313) | | CVE-2018-2799 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2018-2799](https://nvd.nist.gov/vuln/detail/CVE-2018-2799) | | CVE-2019-10219 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2019-10219](https://nvd.nist.gov/vuln/detail/CVE-2019-10219) | | CVE-2019-12384 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2019-12384](https://nvd.nist.gov/vuln/detail/CVE-2019-12384) | | CVE-2019-12384 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2019-12384](https://nvd.nist.gov/vuln/detail/CVE-2019-12384) | | CVE-2019-12814 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2019-12814](https://nvd.nist.gov/vuln/detail/CVE-2019-12814) | | CVE-2019-12814 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2019-12814](https://nvd.nist.gov/vuln/detail/CVE-2019-12814) | | CVE-2020-10693 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2020-10693](https://nvd.nist.gov/vuln/detail/CVE-2020-10693) | | CVE-2020-13955 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2020-13955](https://nvd.nist.gov/vuln/detail/CVE-2020-13955) | | CVE-2020-13956 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2020-13956](https://nvd.nist.gov/vuln/detail/CVE-2020-13956) | | CVE-2020-13956 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2020-13956](https://nvd.nist.gov/vuln/detail/CVE-2020-13956) | | CVE-2020-14338 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2020-14338](https://nvd.nist.gov/vuln/detail/CVE-2020-14338) | | CVE-2020-15250 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2020-15250](https://nvd.nist.gov/vuln/detail/CVE-2020-15250) | | CVE-2020-15250 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2020-15250](https://nvd.nist.gov/vuln/detail/CVE-2020-15250) | | CVE-2020-15522 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2020-15522](https://nvd.nist.gov/vuln/detail/CVE-2020-15522) | | CVE-2020-15522 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2020-15522](https://nvd.nist.gov/vuln/detail/CVE-2020-15522) | | CVE-2020-17521 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2020-17521](https://nvd.nist.gov/vuln/detail/CVE-2020-17521) | | CVE-2020-17521 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2020-17521](https://nvd.nist.gov/vuln/detail/CVE-2020-17521) | | CVE-2020-1945 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2020-1945](https://nvd.nist.gov/vuln/detail/CVE-2020-1945) | | CVE-2020-26939 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2020-26939](https://nvd.nist.gov/vuln/detail/CVE-2020-26939) | | CVE-2020-26939 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2020-26939](https://nvd.nist.gov/vuln/detail/CVE-2020-26939) | | CVE-2020-29582 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2020-29582](https://nvd.nist.gov/vuln/detail/CVE-2020-29582) | | CVE-2021-21290 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2021-21290](https://nvd.nist.gov/vuln/detail/CVE-2021-21290) | | CVE-2021-22569 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2021-22569](https://nvd.nist.gov/vuln/detail/CVE-2021-22569) | | CVE-2021-22569 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2021-22569](https://nvd.nist.gov/vuln/detail/CVE-2021-22569) | | CVE-2021-22569 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2021-22569](https://nvd.nist.gov/vuln/detail/CVE-2021-22569) | | CVE-2021-22569 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2021-22569](https://nvd.nist.gov/vuln/detail/CVE-2021-22569) | | CVE-2021-22570 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2021-22570](https://nvd.nist.gov/vuln/detail/CVE-2021-22570) | | CVE-2021-22570 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2021-22570](https://nvd.nist.gov/vuln/detail/CVE-2021-22570) | | CVE-2021-22570 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2021-22570](https://nvd.nist.gov/vuln/detail/CVE-2021-22570) | | CVE-2021-27568 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2021-27568](https://nvd.nist.gov/vuln/detail/CVE-2021-27568) | | CVE-2021-28170 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2021-28170](https://nvd.nist.gov/vuln/detail/CVE-2021-28170) | | CVE-2021-28170 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2021-28170](https://nvd.nist.gov/vuln/detail/CVE-2021-28170) | | CVE-2021-28170 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2021-28170](https://nvd.nist.gov/vuln/detail/CVE-2021-28170) | | CVE-2021-28170 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2021-28170](https://nvd.nist.gov/vuln/detail/CVE-2021-28170) | | CVE-2021-29425 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2021-29425](https://nvd.nist.gov/vuln/detail/CVE-2021-29425) | | CVE-2021-29425 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2021-29425](https://nvd.nist.gov/vuln/detail/CVE-2021-29425) | | CVE-2021-36373 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2021-36373](https://nvd.nist.gov/vuln/detail/CVE-2021-36373) | | CVE-2021-36374 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2021-36374](https://nvd.nist.gov/vuln/detail/CVE-2021-36374) | | CVE-2021-37533 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2021-37533](https://nvd.nist.gov/vuln/detail/CVE-2021-37533) | | CVE-2021-38153 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2021-38153](https://nvd.nist.gov/vuln/detail/CVE-2021-38153) | | CVE-2022-23437 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2022-23437](https://nvd.nist.gov/vuln/detail/CVE-2022-23437) | | CVE-2022-24329 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2022-24329](https://nvd.nist.gov/vuln/detail/CVE-2022-24329) | | CVE-2022-38749 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2022-38749](https://nvd.nist.gov/vuln/detail/CVE-2022-38749) | | CVE-2022-38750 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2022-38750](https://nvd.nist.gov/vuln/detail/CVE-2022-38750) | | CVE-2022-38751 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2022-38751](https://nvd.nist.gov/vuln/detail/CVE-2022-38751) | | CVE-2022-38752 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2022-38752](https://nvd.nist.gov/vuln/detail/CVE-2022-38752) | | CVE-2022-41854 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2022-41854](https://nvd.nist.gov/vuln/detail/CVE-2022-41854) | | CVE-2023-0833 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2023-0833](https://nvd.nist.gov/vuln/detail/CVE-2023-0833) | | CVE-2023-20863 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2023-20863](https://nvd.nist.gov/vuln/detail/CVE-2023-20863) | | CVE-2023-26048 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2023-26048](https://nvd.nist.gov/vuln/detail/CVE-2023-26048) | | CVE-2023-26049 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2023-26049](https://nvd.nist.gov/vuln/detail/CVE-2023-26049) | | CVE-2023-33201 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2023-33201](https://nvd.nist.gov/vuln/detail/CVE-2023-33201) | | CVE-2023-33201 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2023-33201](https://nvd.nist.gov/vuln/detail/CVE-2023-33201) | | CVE-2023-33201 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2023-33201](https://nvd.nist.gov/vuln/detail/CVE-2023-33201) | | CVE-2023-33201 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2023-33201](https://nvd.nist.gov/vuln/detail/CVE-2023-33201) | | CVE-2023-33201 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2023-33201](https://nvd.nist.gov/vuln/detail/CVE-2023-33201) | | CVE-2023-33202 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2023-33202](https://nvd.nist.gov/vuln/detail/CVE-2023-33202) | | CVE-2023-33202 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2023-33202](https://nvd.nist.gov/vuln/detail/CVE-2023-33202) | | CVE-2023-33202 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2023-33202](https://nvd.nist.gov/vuln/detail/CVE-2023-33202) | | CVE-2023-33202 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2023-33202](https://nvd.nist.gov/vuln/detail/CVE-2023-33202) | | CVE-2023-33202 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2023-33202](https://nvd.nist.gov/vuln/detail/CVE-2023-33202) | | CVE-2023-39968 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2023-39968](https://nvd.nist.gov/vuln/detail/CVE-2023-39968) | | CVE-2023-40167 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2023-40167](https://nvd.nist.gov/vuln/detail/CVE-2023-40167) | | CVE-2023-40170 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2023-40170](https://nvd.nist.gov/vuln/detail/CVE-2023-40170) | | CVE-2023-42503 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2023-42503](https://nvd.nist.gov/vuln/detail/CVE-2023-42503) | | CVE-2023-49080 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2023-49080](https://nvd.nist.gov/vuln/detail/CVE-2023-49080) | | CVE-2023-51074 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2023-51074](https://nvd.nist.gov/vuln/detail/CVE-2023-51074) | | CVE-2023-51074 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2023-51074](https://nvd.nist.gov/vuln/detail/CVE-2023-51074) | | CVE-2024-12798 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-12798](https://nvd.nist.gov/vuln/detail/CVE-2024-12798) | | CVE-2024-12798 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-12798](https://nvd.nist.gov/vuln/detail/CVE-2024-12798) | | CVE-2024-12798 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-12798](https://nvd.nist.gov/vuln/detail/CVE-2024-12798) | | CVE-2024-12798 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-12798](https://nvd.nist.gov/vuln/detail/CVE-2024-12798) | | CVE-2024-12798 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-12798](https://nvd.nist.gov/vuln/detail/CVE-2024-12798) | | CVE-2024-12798 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-12798](https://nvd.nist.gov/vuln/detail/CVE-2024-12798) | | CVE-2024-12798 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-12798](https://nvd.nist.gov/vuln/detail/CVE-2024-12798) | | CVE-2024-25710 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-25710](https://nvd.nist.gov/vuln/detail/CVE-2024-25710) | | CVE-2024-25710 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-25710](https://nvd.nist.gov/vuln/detail/CVE-2024-25710) | | CVE-2024-25710 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-25710](https://nvd.nist.gov/vuln/detail/CVE-2024-25710) | | CVE-2024-25710 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-25710](https://nvd.nist.gov/vuln/detail/CVE-2024-25710) | | CVE-2024-26308 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-26308](https://nvd.nist.gov/vuln/detail/CVE-2024-26308) | | CVE-2024-26308 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-26308](https://nvd.nist.gov/vuln/detail/CVE-2024-26308) | | CVE-2024-26308 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-26308](https://nvd.nist.gov/vuln/detail/CVE-2024-26308) | | CVE-2024-26308 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-26308](https://nvd.nist.gov/vuln/detail/CVE-2024-26308) | | CVE-2024-38820 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-38820](https://nvd.nist.gov/vuln/detail/CVE-2024-38820) | | CVE-2024-38820 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-38820](https://nvd.nist.gov/vuln/detail/CVE-2024-38820) | | CVE-2024-43805 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-43805](https://nvd.nist.gov/vuln/detail/CVE-2024-43805) | | CVE-2024-6763 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-6763](https://nvd.nist.gov/vuln/detail/CVE-2024-6763) | | CVE-2024-6763 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-6763](https://nvd.nist.gov/vuln/detail/CVE-2024-6763) | | CVE-2024-6763 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-6763](https://nvd.nist.gov/vuln/detail/CVE-2024-6763) | | CVE-2024-6763 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-6763](https://nvd.nist.gov/vuln/detail/CVE-2024-6763) | | CVE-2024-6763 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-6763](https://nvd.nist.gov/vuln/detail/CVE-2024-6763) | | CVE-2024-6763 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-6763](https://nvd.nist.gov/vuln/detail/CVE-2024-6763) | | CVE-2024-8184 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-8184](https://nvd.nist.gov/vuln/detail/CVE-2024-8184) | | CVE-2024-8184 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-8184](https://nvd.nist.gov/vuln/detail/CVE-2024-8184) | | CVE-2020-8908 | LOW | [https://nvd.nist.gov/vuln/detail/CVE-2020-8908](https://nvd.nist.gov/vuln/detail/CVE-2020-8908) | | CVE-2020-8908 | LOW | [https://nvd.nist.gov/vuln/detail/CVE-2020-8908](https://nvd.nist.gov/vuln/detail/CVE-2020-8908) | | CVE-2020-8908 | LOW | [https://nvd.nist.gov/vuln/detail/CVE-2020-8908](https://nvd.nist.gov/vuln/detail/CVE-2020-8908) | | CVE-2020-8908 | LOW | [https://nvd.nist.gov/vuln/detail/CVE-2020-8908](https://nvd.nist.gov/vuln/detail/CVE-2020-8908) | | CVE-2020-8908 | LOW | [https://nvd.nist.gov/vuln/detail/CVE-2020-8908](https://nvd.nist.gov/vuln/detail/CVE-2020-8908) | | CVE-2020-8908 | LOW | [https://nvd.nist.gov/vuln/detail/CVE-2020-8908](https://nvd.nist.gov/vuln/detail/CVE-2020-8908) | | CVE-2020-8908 | LOW | [https://nvd.nist.gov/vuln/detail/CVE-2020-8908](https://nvd.nist.gov/vuln/detail/CVE-2020-8908) | | CVE-2020-8908 | LOW | [https://nvd.nist.gov/vuln/detail/CVE-2020-8908](https://nvd.nist.gov/vuln/detail/CVE-2020-8908) | | CVE-2020-8908 | LOW | [https://nvd.nist.gov/vuln/detail/CVE-2020-8908](https://nvd.nist.gov/vuln/detail/CVE-2020-8908) | | CVE-2020-8908 | LOW | [https://nvd.nist.gov/vuln/detail/CVE-2020-8908](https://nvd.nist.gov/vuln/detail/CVE-2020-8908) | | CVE-2020-8908 | LOW | [https://nvd.nist.gov/vuln/detail/CVE-2020-8908](https://nvd.nist.gov/vuln/detail/CVE-2020-8908) | | CVE-2020-8908 | LOW | [https://nvd.nist.gov/vuln/detail/CVE-2020-8908](https://nvd.nist.gov/vuln/detail/CVE-2020-8908) | | CVE-2022-2047 | LOW | [https://nvd.nist.gov/vuln/detail/CVE-2022-2047](https://nvd.nist.gov/vuln/detail/CVE-2022-2047) | | CVE-2024-12801 | LOW | [https://nvd.nist.gov/vuln/detail/CVE-2024-12801](https://nvd.nist.gov/vuln/detail/CVE-2024-12801) | | CVE-2024-12801 | LOW | [https://nvd.nist.gov/vuln/detail/CVE-2024-12801](https://nvd.nist.gov/vuln/detail/CVE-2024-12801) | | CVE-2024-12801 | LOW | [https://nvd.nist.gov/vuln/detail/CVE-2024-12801](https://nvd.nist.gov/vuln/detail/CVE-2024-12801) | | CVE-2024-12801 | LOW | [https://nvd.nist.gov/vuln/detail/CVE-2024-12801](https://nvd.nist.gov/vuln/detail/CVE-2024-12801) | | GHSA-58qw-p7qm-5rvh | LOW | [https://nvd.nist.gov/vuln/detail/GHSA-58qw-p7qm-5rvh](https://nvd.nist.gov/vuln/detail/GHSA-58qw-p7qm-5rvh) | | GHSA-chfm-68vv-pvw5 | LOW | [https://nvd.nist.gov/vuln/detail/GHSA-chfm-68vv-pvw5](https://nvd.nist.gov/vuln/detail/GHSA-chfm-68vv-pvw5) | | CVE-2023-50868 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2023-50868](https://nvd.nist.gov/vuln/detail/CVE-2023-50868) | | CVE-2024-23454 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2024-23454](https://nvd.nist.gov/vuln/detail/CVE-2024-23454) | | CVE-2024-23454 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2024-23454](https://nvd.nist.gov/vuln/detail/CVE-2024-23454) | | CVE-2024-23944 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2024-23944](https://nvd.nist.gov/vuln/detail/CVE-2024-23944) | | CVE-2024-23944 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2024-23944](https://nvd.nist.gov/vuln/detail/CVE-2024-23944) | | CVE-2024-23944 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2024-23944](https://nvd.nist.gov/vuln/detail/CVE-2024-23944) | | CVE-2024-23945 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2024-23945](https://nvd.nist.gov/vuln/detail/CVE-2024-23945) | | CVE-2024-23953 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2024-23953](https://nvd.nist.gov/vuln/detail/CVE-2024-23953) | | CVE-2024-25638 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2024-25638](https://nvd.nist.gov/vuln/detail/CVE-2024-25638) | | CVE-2024-27137 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2024-27137](https://nvd.nist.gov/vuln/detail/CVE-2024-27137) | | CVE-2024-29025 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2024-29025](https://nvd.nist.gov/vuln/detail/CVE-2024-29025) | | CVE-2024-29131 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2024-29131](https://nvd.nist.gov/vuln/detail/CVE-2024-29131) | | CVE-2024-29131 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2024-29131](https://nvd.nist.gov/vuln/detail/CVE-2024-29131) | | CVE-2024-29133 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2024-29133](https://nvd.nist.gov/vuln/detail/CVE-2024-29133) | | CVE-2024-29133 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2024-29133](https://nvd.nist.gov/vuln/detail/CVE-2024-29133) | | CVE-2024-29857 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2024-29857](https://nvd.nist.gov/vuln/detail/CVE-2024-29857) | | CVE-2024-29857 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2024-29857](https://nvd.nist.gov/vuln/detail/CVE-2024-29857) | | CVE-2024-29857 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2024-29857](https://nvd.nist.gov/vuln/detail/CVE-2024-29857) | | CVE-2024-29857 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2024-29857](https://nvd.nist.gov/vuln/detail/CVE-2024-29857) | | CVE-2024-29869 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2024-29869](https://nvd.nist.gov/vuln/detail/CVE-2024-29869) | | CVE-2024-30171 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2024-30171](https://nvd.nist.gov/vuln/detail/CVE-2024-30171) | | CVE-2024-30171 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2024-30171](https://nvd.nist.gov/vuln/detail/CVE-2024-30171) | | CVE-2024-30171 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2024-30171](https://nvd.nist.gov/vuln/detail/CVE-2024-30171) | | CVE-2024-30171 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2024-30171](https://nvd.nist.gov/vuln/detail/CVE-2024-30171) | | CVE-2024-30172 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2024-30172](https://nvd.nist.gov/vuln/detail/CVE-2024-30172) | | CVE-2024-30172 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2024-30172](https://nvd.nist.gov/vuln/detail/CVE-2024-30172) | | CVE-2024-31141 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2024-31141](https://nvd.nist.gov/vuln/detail/CVE-2024-31141) | | CVE-2024-31141 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2024-31141](https://nvd.nist.gov/vuln/detail/CVE-2024-31141) | | CVE-2024-34447 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2024-34447](https://nvd.nist.gov/vuln/detail/CVE-2024-34447) | | CVE-2024-34447 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2024-34447](https://nvd.nist.gov/vuln/detail/CVE-2024-34447) | | CVE-2024-36114 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2024-36114](https://nvd.nist.gov/vuln/detail/CVE-2024-36114) | | CVE-2024-36114 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2024-36114](https://nvd.nist.gov/vuln/detail/CVE-2024-36114) | | CVE-2024-36114 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2024-36114](https://nvd.nist.gov/vuln/detail/CVE-2024-36114) | | CVE-2024-36114 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2024-36114](https://nvd.nist.gov/vuln/detail/CVE-2024-36114) | | CVE-2024-38808 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2024-38808](https://nvd.nist.gov/vuln/detail/CVE-2024-38808) | | CVE-2024-43382 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2024-43382](https://nvd.nist.gov/vuln/detail/CVE-2024-43382) | | CVE-2024-47072 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2024-47072](https://nvd.nist.gov/vuln/detail/CVE-2024-47072) | | CVE-2024-47535 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2024-47535](https://nvd.nist.gov/vuln/detail/CVE-2024-47535) | | CVE-2024-47535 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2024-47535](https://nvd.nist.gov/vuln/detail/CVE-2024-47535) | | CVE-2024-47535 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2024-47535](https://nvd.nist.gov/vuln/detail/CVE-2024-47535) | | CVE-2024-47535 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2024-47535](https://nvd.nist.gov/vuln/detail/CVE-2024-47535) | | CVE-2024-47554 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2024-47554](https://nvd.nist.gov/vuln/detail/CVE-2024-47554) | | CVE-2024-47554 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2024-47554](https://nvd.nist.gov/vuln/detail/CVE-2024-47554) | | CVE-2024-47554 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2024-47554](https://nvd.nist.gov/vuln/detail/CVE-2024-47554) | | CVE-2024-47554 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2024-47554](https://nvd.nist.gov/vuln/detail/CVE-2024-47554) | | CVE-2024-47554 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2024-47554](https://nvd.nist.gov/vuln/detail/CVE-2024-47554) | | CVE-2024-47554 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2024-47554](https://nvd.nist.gov/vuln/detail/CVE-2024-47554) | | CVE-2024-51504 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2024-51504](https://nvd.nist.gov/vuln/detail/CVE-2024-51504) | | CVE-2024-51504 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2024-51504](https://nvd.nist.gov/vuln/detail/CVE-2024-51504) | | CVE-2024-51504 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2024-51504](https://nvd.nist.gov/vuln/detail/CVE-2024-51504) | | CVE-2024-56128 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2024-56128](https://nvd.nist.gov/vuln/detail/CVE-2024-56128) | | CVE-2024-56128 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2024-56128](https://nvd.nist.gov/vuln/detail/CVE-2024-56128) | | CVE-2024-9823 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2024-9823](https://nvd.nist.gov/vuln/detail/CVE-2024-9823) | | CVE-2025-23015 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2025-23015](https://nvd.nist.gov/vuln/detail/CVE-2025-23015) | | CVE-2025-24789 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2025-24789](https://nvd.nist.gov/vuln/detail/CVE-2025-24789) | | CVE-2025-24790 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2025-24790](https://nvd.nist.gov/vuln/detail/CVE-2025-24790) | | CVE-2025-24860 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2025-24860](https://nvd.nist.gov/vuln/detail/CVE-2025-24860) | ## 7.7.0 release notes The following release notes include information for Amazon EMR release 7.7.0. **New features** + **Application upgrades** – Amazon EMR 7.7.0 application upgrades include Delta 3.2.1-amzn-2, Hudi 0.15.0-amzn-4, Iceberg 1.7.1-amzn-0, and Trino 467. **Known issues** + **HBase table creation in Hive with Glue catalog** – From EMR-7.3.0 to EMR-7.10.0, there is a bug due to Hive Iceberg integration, which causes HBase table creation in Hive to fail when Glue Data Catalog is used as the metastore. Please reach out to the AWS support team if you encounter this issue. + **Tez UI** – From EMR-7.3 to EMR-7.10, there is a bug where accessing Tez UI through YARN Resource Manager Web UI fails when in-transit encryption is enabled. This occurs because YARN proxy attempts to use HTTP protocol while Tez UI requires HTTPS. Please modify the `tez.tez-ui.history-url.base` configuration in `tez-site` from `http` to `https` to resolve this issue. **Changes, enhancements, and resolved issues** + Starting with Spark 3.3.1 (supported in EMR versions 6.10 and above), all executors in a decommissioning host are set to a new `ExecutorState`, called *DECOMMISSIONING* state. The executors being decommissioned cannot be used by Yarn to allocate tasks and thus it will request for new executors, if needed, for the tasks being executed. Thus, if you disable Spark DRA while using EMR Managed Scaling, EMR Auto Scaling, or any custom scaling mechanism on EMR-EC2 clusters, then Yarn may request maximum permissible executors for each job. In order to avoid this issue, leave the `spark.dynamicAllocation.enabled` property set to `TRUE` (which is the default) when you are using the above combination of features. In addition, you can also set minimum and maximum executor constraints by setting values for `spark.dynamicAllocation.maxExecutors` and `spark.dynamicAllocation.minExecutors` properties for your Spark jobs, to restrict the number of executors allocated during the job’s execution. + The Iceberg version in use as of EMR 7.7.0 no longer supports Java 8. Additionally, Iceberg is excluded from the following Java 8 images: `emr-7.7.0-java8-latest` and `emr-7.7.0-spark-rapids-java8-latest`. + Lake Formation for FGAC with Amazon EMR on EKS – With Amazon EMR release 7.7 and higher, you can leverage AWS Lake Formation to apply fine-grained access controls on Data Catalog tables that are backed by Amazon S3. This capability lets you configure table, row, column, and cell-level access controls for read queries within your Amazon Amazon EMR on EKS Spark Jobs. For more information, see [Using Amazon Amazon EMR on EKS with AWS Lake Formation for fine-grained access control](https://docs.aws.amazon.com/emr/latest/EMR-on-EKS-DevelopmentGuide/security_iam_fgac-lf.html). + [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-770-release.html) ## 7.7.0 default Java versions Amazon EMR release 7.7 and higher ships with Amazon Corretto 17 (built on OpenJDK) by default for applications that support Corretto 17 (JDK 17), with the exception of Apache Livy. The following table shows the default Java versions for applications in Amazon EMR 7.7.0. If you want to change the default JVM on your cluster, follow the instructions in [Configure applications to use a specific Java Virtual Machine](configuring-java8.md) for each application that runs on the cluster. You can only use one Java runtime version for a cluster. Amazon EMR doesn't support running different nodes or applications on different runtime versions on the same cluster. | Application | Java / Amazon Corretto version (default is bold) | | --- | --- | | Delta | 17, 11, 8 | | Flink | 17, 11, 8 | | HBase | 17, 11, 8 | | HCatalog | 17, 11, 8 | | Hadoop | 17, 11, 8 | | Hive | 17, 11, 8 | | Hudi | 17, 11, 8 | | Iceberg | 17, 11 | | Livy | 17, 11, 8 | | Oozie | 17, 11, 8 | | Phoenix | 17, 11, 8 | | PrestoDB | 8 | | Spark | 17, 11, 8 | | Spark RAPIDS | 17, 11, 8 | | Tez | 17, 11, 8 | | Trino | 21, 17 | | Zeppelin | 17, 11, 8 | | Pig | 17, 11, 8 | | Zookeeper | 17, 11, 8 | ## 7.7.0 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | adot-java-agent | 1.31.0 | A Java Agent that collects metrics from application daemons. | | delta | 3.2.1-amzn-2 | Delta lake is an open table format for huge analytic datasets | | emr-amazon-cloudwatch-agent | 1.300032.2-amzn-0 | An application that collects internal system-level metrics and custom application metrics from Amazon EC2 instances. | | emr-ddb | 5.6.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 3.16.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.20.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-notebook-env | 1.18.0 | Conda env for emr notebook which includes jupyter enterprise gateway | | emr-s3-dist-cp | 2.37.0 | Distributed copy application optimized for Amazon S3. | | emr-s3-select | 2.16.0 | EMR S3Select Connector | | emr-wal-cli | 1.3.2 | Cli used for emrwal list/deletion. | | emrfs | 2.68.0 | Amazon S3 connector for Hadoop ecosystem applications. | | flink-client | 1.20.0-amzn-1 | Apache Flink command line client scripts and applications. | | flink-jobmanager-config | 1.20.0-amzn-1 | Managing resources on EMR nodes for Apache Flink JobManager. | | hadoop-client | 3.4.0-amzn-3 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 3.4.0-amzn-3 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 3.4.0-amzn-3 | HDFS command-line client and library | | hadoop-hdfs-namenode | 3.4.0-amzn-3 | HDFS service for tracking file names and block locations. | | hadoop-hdfs-journalnode | 3.4.0-amzn-3 | HDFS service for managing the Hadoop filesystem journal on HA clusters. | | hadoop-httpfs-server | 3.4.0-amzn-3 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 3.4.0-amzn-3 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 3.4.0-amzn-3 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 3.4.0-amzn-3 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 3.4.0-amzn-3 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 3.4.0-amzn-3 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 2.6.1-amzn-1 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 2.6.1-amzn-1 | Service for serving one or more HBase regions. | | hbase-client | 2.6.1-amzn-1 | HBase command-line client. | | hbase-rest-server | 2.6.1-amzn-1 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 2.6.1-amzn-1 | Service providing a Thrift endpoint to HBase. | | hbase-operator-tools | 2.6.1-amzn-1 | Repair tool for Apache HBase clusters. | | hcatalog-client | 3.1.3-amzn-16 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 3.1.3-amzn-16 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 3.1.3-amzn-16 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 3.1.3-amzn-16 | Hive command line client. | | hive-hbase | 3.1.3-amzn-16 | Hive-hbase client. | | hive-metastore-server | 3.1.3-amzn-16 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server2 | 3.1.3-amzn-16 | Service for accepting Hive queries as web requests. | | hudi | 0.15.0-amzn-4 | Incremental processing framework to power data pipeline at low latency and high efficiency. | | hudi-presto | 0.15.0-amzn-4 | Bundle library for running Presto with Hudi. | | hudi-trino | 0.15.0-amzn-4 | Bundle library for running Trino with Hudi. | | hudi-spark | 0.15.0-amzn-4 | Bundle library for running Spark with Hudi. | | hue-server | 4.11.0 | Web application for analyzing data using Hadoop ecosystem applications | | iceberg | 1.7.1-amzn-0 | Apache Iceberg is an open table format for huge analytic datasets | | jupyterhub | 1.5.0 | Multi-user server for Jupyter notebooks | | livy-server | 0.8.0-incubating | REST interface for interacting with Apache Spark | | nginx | 1.12.1 | nginx [engine x] is an HTTP and reverse proxy server | | mariadb-server | 5.5.68\$1 | MariaDB database server. | | nvidia-cuda | 12.3.0 | Nvidia drivers and Cuda toolkit | | oozie-client | 5.2.1 | Oozie command-line client. | | oozie-server | 5.2.1 | Service for accepting Oozie workflow requests. | | opencv | 4.7.0 | Open Source Computer Vision Library. | | phoenix-library | 5.2.1 | The phoenix libraries for server and client | | phoenix-connectors | 6.0.0 | Apache Phoenix-Connectors for Spark-3 | | phoenix-query-server | 6.0.0 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.287-amzn-1 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.287-amzn-1 | Service for executing pieces of a query. | | presto-client | 0.287-amzn-1 | Presto command-line client which is installed on an HA cluster's stand-by masters where Presto server is not started. | | trino-coordinator | 467-amzn-0 | Service for accepting queries and managing query execution among trino-workers. | | trino-worker | 467-amzn-0 | Service for executing pieces of a query. | | trino-client | 467-amzn-0 | Trino command-line client which is installed on an HA cluster's stand-by masters where Trino server is not started. | | pig-client | 0.17.0 | Pig command-line client. | | r | 4.3.2 | The R Project for Statistical Computing | | ranger-kms-server | 2.0.0 | Apache Ranger Key Management System | | spark-client | 3.5.3-amzn-1 | Spark command-line clients. | | spark-history-server | 3.5.3-amzn-1 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 3.5.3-amzn-1 | In-memory execution engine for YARN. | | spark-yarn-slave | 3.5.3-amzn-1 | Apache Spark libraries needed by YARN slaves. | | spark-rapids | 24.10.1-amzn-1 | Nvidia Spark RAPIDS plugin that accelerates Apache Spark with GPUs. | | tensorflow | 2.16.1 | TensorFlow open source software library for high performance numerical computation. | | tez-on-yarn | 0.10.2-amzn-14 | The tez YARN application and libraries. | | tez-on-worker | 0.10.2-amzn-14 | The tez YARN application and libraries for worker nodes. | | zeppelin-server | 0.11.1 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.9.2-amzn-1 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.9.2-amzn-1 | ZooKeeper command line client. | ## 7.7.0 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). Reconfiguration actions occur when you specify a configuration for instance groups in a running cluster. Amazon EMR only initiates reconfiguration actions for the classifications that you modify. For more information, see [Reconfigure an instance group in a running cluster](emr-configure-apps-running-cluster.md). **emr-7.7.0 classifications** | Classifications | Description | Reconfiguration Actions | | --- | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | Restarts the ResourceManager service. | | container-executor | Change values in Hadoop YARN's container-executor.cfg file. | Not available. | | container-log4j | Change values in Hadoop YARN's container-log4j.properties file. | Not available. | | core-site | Change values in Hadoop's core-site.xml file. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts Hadoop KMS, Ranger KMS, HiveServer2, Hive MetaStore, Hadoop Httpfs, and MapReduce-HistoryServer. | | docker-conf | Change docker related settings. | Not available. | | emrfs-site | Change EMRFS settings. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts HBaseRegionserver, HBaseMaster, HBaseThrift, HBaseRest, HiveServer2, Hive MetaStore, Hadoop Httpfs, and MapReduce-HistoryServer. | | flink-conf | Change flink-conf.yaml settings. | Restarts Flink history server. | | flink-log4j | Change Flink log4j.properties settings. | Restarts Flink history server. | | flink-log4j-session | Change Flink log4j-session.properties settings for Kubernetes/Yarn session. | Restarts Flink history server. | | flink-log4j-cli | Change Flink log4j-cli.properties settings. | Restarts Flink history server. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts PhoenixQueryserver, HiveServer2, Hive MetaStore, and MapReduce-HistoryServer. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | Restarts the Hadoop HDFS services SecondaryNamenode, Datanode, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts Hadoop KMS, Hadoop Httpfs, and MapReduce-HistoryServer. | | hadoop-ssl-server | Change hadoop ssl server configuration | Not available. | | hadoop-ssl-client | Change hadoop ssl client configuration | Not available. | | hbase | Amazon EMR-curated settings for Apache HBase. | Custom EMR specific property. Sets emrfs-site and hbase-site configs. See those for their associated restarts. | | hbase-env | Change values in HBase's environment. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbase.properties file. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | Not available. | | hbase-site | Change values in HBase's hbase-site.xml file. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. Additionally restarts Phoenix QueryServer. | | hdfs-encryption-zones | Configure HDFS encryption zones. | This classification should not be reconfigured. | | hdfs-env | Change values in the HDFS environment. | Restarts Hadoop HDFS services Namenode, Datanode, and ZKFC. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Additionally restarts Hadoop Httpfs. | | hcatalog-env | Change values in HCatalog's environment. | Restarts Hive HCatalog Server. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | Restarts Hive HCatalog Server. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | Restarts Hive HCatalog Server. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | Restarts Hive WebHCat server. | | hcatalog-webhcat-log4j2 | Change values in HCatalog WebHCat's log4j2.properties. | Restarts Hive WebHCat server. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | Restarts Hive WebHCat server. | | hive | Amazon EMR-curated settings for Apache Hive. | Sets configurations to launch Hive LLAP service. | | hive-beeline-log4j2 | Change values in Hive's beeline-log4j2.properties file. | Not available. | | hive-parquet-logging | Change values in Hive's parquet-logging.properties file. | Not available. | | hive-env | Change values in the Hive environment. | Restarts HiveServer2, HiveMetastore, and Hive HCatalog-Server. Runs Hive schemaTool CLI commands to verify hive-metastore. | | hive-exec-log4j2 | Change values in Hive's hive-exec-log4j2.properties file. | Not available. | | hive-llap-daemon-log4j2 | Change values in Hive's llap-daemon-log4j2.properties file. | Not available. | | hive-log4j2 | Change values in Hive's hive-log4j2.properties file. | Not available. | | hive-site | Change values in Hive's hive-site.xml file | Restarts HiveServer2, HiveMetastore, and Hive HCatalog-Server. Runs Hive schemaTool CLI commands to verify hive-metastore. Also restarts Oozie and Zeppelin. | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | Not available. | | hue-ini | Change values in Hue's ini file | Restarts Hue. Also activates Hue config override CLI commands to pick up new configurations. | | httpfs-env | Change values in the HTTPFS environment. | Restarts Hadoop Httpfs service. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | Restarts Hadoop Httpfs service. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | Not available. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | Restarts Hadoop-KMS service. | | hadoop-kms-java-home | Change Hadoop's KMS java home | Not available. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | Not available. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | Restarts Hadoop-KMS and Ranger-KMS service. | | hudi-env | Change values in the Hudi environment. | Not available. | | hudi-defaults | Change values in Hudi's hudi-defaults.conf file. | Not available. | | iceberg-defaults | Change values in Iceberg's iceberg-defaults.conf file. | Not available. | | delta-defaults | Change values in Delta's delta-defaults.conf file. | Not available. | | jupyter-notebook-conf | Change values in Jupyter Notebook's jupyter\$1notebook\$1config.py file. | Not available. | | jupyter-hub-conf | Change values in JupyterHubs's jupyterhub\$1config.py file. | Not available. | | jupyter-s3-conf | Configure Jupyter Notebook S3 persistence. | Not available. | | jupyter-sparkmagic-conf | Change values in Sparkmagic's config.json file. | Not available. | | livy-conf | Change values in Livy's livy.conf file. | Restarts Livy Server. | | livy-env | Change values in the Livy environment. | Restarts Livy Server. | | livy-log4j2 | Change Livy log4j2.properties settings. | Restarts Livy Server. | | mapred-env | Change values in the MapReduce application's environment. | Restarts Hadoop MapReduce-HistoryServer. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | Restarts Hadoop MapReduce-HistoryServer. | | oozie-env | Change values in Oozie's environment. | Restarts Oozie. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | Restarts Oozie. | | oozie-site | Change values in Oozie's oozie-site.xml file. | Restarts Oozie. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | Not available. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | Not available. | | phoenix-log4j2 | Change values in Phoenix's log4j2.properties file. | Restarts Phoenix-QueryServer. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | Not available. | | pig-env | Change values in the Pig environment. | Not available. | | pig-properties | Change values in Pig's pig.properties file. | Restarts Oozie. | | pig-log4j | Change values in Pig's log4j.properties file. | Not available. | | presto-log | Change values in Presto's log.properties file. | Restarts Presto-Server (for PrestoDB) | | presto-config | Change values in Presto's config.properties file. | Restarts Presto-Server (for PrestoDB) | | presto-password-authenticator | Change values in Presto's password-authenticator.properties file. | Not available. | | presto-env | Change values in Presto's presto-env.sh file. | Restarts Presto-Server (for PrestoDB) | | presto-node | Change values in Presto's node.properties file. | Not available. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | Not available. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | Not available. | | presto-connector-hive | Change values in Presto's hive.properties file. | Restarts Presto-Server (for PrestoDB) | | presto-connector-jmx | Change values in Presto's jmx.properties file. | Not available. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | Not available. | | presto-connector-lakeformation | Change values in Presto's lakeformation.properties file. | Restarts Presto-Server (for PrestoDB) | | presto-connector-localfile | Change values in Presto's localfile.properties file. | Not available. | | presto-connector-memory | Change values in Presto's memory.properties file. | Not available. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | Not available. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | Not available. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | Not available. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | Not available. | | presto-connector-redis | Change values in Presto's redis.properties file. | Not available. | | presto-connector-redshift | Change values in Presto's redshift.properties file. | Not available. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | Not available. | | presto-connector-tpcds | Change values in Presto's tpcds.properties file. | Not available. | | trino-log | Change values in Trino's log.properties file. | Restarts Trino-Server (for Trino) | | trino-config | Change values in Trino's config.properties file. | Restarts Trino-Server (for Trino) | | trino-password-authenticator | Change values in Trino's password-authenticator.properties file. | Restarts Trino-Server (for Trino) | | trino-env | Change values in Trino's trino-env.sh file. | Restarts Trino-Server (for Trino) | | trino-node | Change values in Trino's node.properties file. | Not available. | | trino-connector-blackhole | Change values in Trino's blackhole.properties file. | Not available. | | trino-connector-cassandra | Change values in Trino's cassandra.properties file. | Not available. | | trino-connector-delta | Change values in Trino's delta.properties file. | Restarts Trino-Server (for Trino) | | trino-connector-hive | Change values in Trino's hive.properties file. | Restarts Trino-Server (for Trino) | | trino-exchange-manager | Change values in Trino's exchange-manager.properties file. | Restarts Trino-Server (for Trino) | | trino-connector-iceberg | Change values in Trino's iceberg.properties file. | Restarts Trino-Server (for Trino) | | trino-connector-hudi | Change values in Trino's hudi.properties file. | Restarts Trino-Server (for Trino) | | trino-connector-jmx | Change values in Trino's jmx.properties file. | Not available. | | trino-connector-kafka | Change values in Trino's kafka.properties file. | Not available. | | trino-connector-localfile | Change values in Trino's localfile.properties file. | Not available. | | trino-connector-memory | Change values in Trino's memory.properties file. | Not available. | | trino-connector-mongodb | Change values in Trino's mongodb.properties file. | Not available. | | trino-connector-mysql | Change values in Trino's mysql.properties file. | Not available. | | trino-connector-postgresql | Change values in Trino's postgresql.properties file. | Not available. | | trino-connector-raptor | Change values in Trino's raptor.properties file. | Not available. | | trino-connector-redis | Change values in Trino's redis.properties file. | Not available. | | trino-connector-redshift | Change values in Trino's redshift.properties file. | Not available. | | trino-connector-tpch | Change values in Trino's tpch.properties file. | Not available. | | trino-connector-tpcds | Change values in Trino's tpcds.properties file. | Not available. | | ranger-kms-dbks-site | Change values in dbks-site.xml file of Ranger KMS. | Restarts Ranger KMS Server. | | ranger-kms-site | Change values in ranger-kms-site.xml file of Ranger KMS. | Restarts Ranger KMS Server. | | ranger-kms-env | Change values in the Ranger KMS environment. | Restarts Ranger KMS Server. | | ranger-kms-logback | Change values in kms-logback.xml file of Ranger KMS. | Not available. | | ranger-kms-db-ca | Change values for CA file on S3 for MySQL SSL connection with Ranger KMS. | Not available. | | spark | Amazon EMR-curated settings for Apache Spark. | This property modifies spark-defaults. See actions there. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | Restarts Spark history server and Spark thrift server. | | spark-env | Change values in the Spark environment. | Restarts Spark history server and Spark thrift server. | | spark-hive-site | Change values in Spark's hive-site.xml file | Not available. | | spark-log4j2 | Change values in Spark's log4j2.properties file. | Restarts Spark history server and Spark thrift server. | | spark-metrics | Change values in Spark's metrics.properties file. | Restarts Spark history server and Spark thrift server. | | tez-site | Change values in Tez's tez-site.xml file. | Restart Oozie and HiveServer2. | | yarn-env | Change values in the YARN environment. | Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts MapReduce-HistoryServer. | | yarn-site | Change values in YARN's yarn-site.xml file. | Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts Livy Server and MapReduce-HistoryServer. | | zeppelin-env | Change values in the Zeppelin environment. | Restarts Zeppelin. | | zeppelin-site | Change configuration settings in zeppelin-site.xml. | Restarts Zeppelin. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | Restarts Zookeeper server. | | zookeeper-logback | Change values in ZooKeeper's logback.xml file. | Restarts Zookeeper server. | | emr-metrics | Change emr metric settings for this node. | Restarts the CloudWatchAgent service. | ## 7.7.0 change log **Change log for 7.7.0 release and release notes** | Date | Event | Description | | --- | --- | --- | | 2025-02-07 | Docs publication | Amazon EMR 7.7.0 release notes first published | | 2025-02-06 | Deployment complete | Amazon EMR 7.7.0 fully deployed to all [supported Regions](https://aws.amazon.com/about-aws/global-infrastructure/regional-product-services/) | | 2025-01-30 | Initial limited release | Amazon EMR 7.7.0 first deployed to initial commercial Regions | # Amazon EMR release 7.6.0 ## 7.6.0 supported lifecycle The following table describes the supported lifecycle dates for Amazon EMR release 7.6.0. | Support phase | Date | | --- | --- | | Initial release date | January 10, 2025 | | Standard support until | January 09, 2027 | | End of support | January 10, 2027 | | End of life | January 10, 2028 | ## 7.6.0 application versions This release includes the following applications: [https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/Install-CloudWatch-Agent.html](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/Install-CloudWatch-Agent.html), [https://delta.io/](https://delta.io/), [https://flink.apache.org/](https://flink.apache.org/), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [https://hudi.apache.org](https://hudi.apache.org), [http://gethue.com/](http://gethue.com/), [https://iceberg.apache.org/](https://iceberg.apache.org/), [https://jupyter-enterprise-gateway.readthedocs.io/en/latest/](https://jupyter-enterprise-gateway.readthedocs.io/en/latest/), [https://jupyterhub.readthedocs.io/en/latest/#](https://jupyterhub.readthedocs.io/en/latest/#), [https://livy.incubator.apache.org/](https://livy.incubator.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [https://www.tensorflow.org/](https://www.tensorflow.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://trino.io/](https://trino.io/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-7.6.0 | emr-7.5.0 | emr-7.4.0 | emr-7.3.0 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 2.29.25, 1.12.779 | 2.28.8, 1.12.772 | 2.28.8, 1.12.772 | 2.25.70, 1.12.747 | | Python | 3.9, 3.11 | 3.9, 3.11 | 3.9, 3.11 | 3.9, 3.11 | | Scala | 2.12.18 | 2.12.18 | 2.12.18 | 2.12.18 | | AmazonCloudWatchAgent | 1.300032.2-amzn-0 | 1.300032.2-amzn-0 | 1.300032.2-amzn-0 | 1.300032.2-amzn-0 | | Delta | 3.2.1-amzn-1 | 3.2.1-amzn-0 | 3.2.0-amzn-1 | 3.2.0-amzn-0 | | Flink | 1.20.0-amzn-0 | 1.19.1-amzn-1 | 1.19.1-amzn-0 | 1.18.1-amzn-2 | | Ganglia | - | - | - | - | | HBase | 2.6.1-amzn-0 | 2.5.10-amzn-0 | 2.5.5-amzn-0 | 2.4.17-amzn-7 | | HCatalog | 3.1.3-amzn-15 | 3.1.3-amzn-14 | 3.1.3-amzn-13 | 3.1.3-amzn-12 | | Hadoop | 3.4.0-amzn-2 | 3.4.0-amzn-1 | 3.4.0-amzn-0 | 3.3.6-amzn-5 | | Hive | 3.1.3-amzn-15 | 3.1.3-amzn-14 | 3.1.3-amzn-13 | 3.1.3-amzn-12 | | Hudi | 0.15.0-amzn-3 | 0.15.0-amzn-2 | 0.15.0-amzn-1 | 0.15.0-amzn-0 | | Hue | 4.11.0 | 4.11.0 | 4.11.0 | 4.11.0 | | Iceberg | 1.6.1-amzn-2 | 1.6.1-amzn-1 | 1.6.1-amzn-0 | 1.5.2-amzn-0 | | JupyterEnterpriseGateway | 2.6.0 | 2.6.0 | 2.6.0 | 2.6.0 | | JupyterHub | 1.5.0 | 1.5.0 | 1.5.0 | 1.5.0 | | Livy | 0.8.0-incubating | 0.8.0-incubating | 0.8.0-incubating | 0.8.0-incubating | | MXNet | - | - | - | - | | Mahout | - | - | - | - | | Oozie | 5.2.1 | 5.2.1 | 5.2.1 | 5.2.1 | | Phoenix | 5.2.0 | 5.2.0 | 5.2.0 | 5.1.3 | | Pig | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | | Presto | 0.287-amzn-1 | 0.287-amzn-0 | 0.287-amzn-0 | 0.285-amzn-1 | | Spark | 3.5.3-amzn-0 | 3.5.2-amzn-1 | 3.5.2-amzn-0 | 3.5.1-amzn-1 | | Sqoop | - | - | 1.4.7 | 1.4.7 | | TensorFlow | 2.16.1 | 2.16.1 | 2.16.1 | 2.16.1 | | Tez | 0.10.2-amzn-13 | 0.10.2-amzn-12 | 0.10.2-amzn-11 | 0.10.2-amzn-10 | | Trino (PrestoSQL) | 457-amzn-0 | 446-amzn-0 | 446-amzn-0 | 442-amzn-0 | | Zeppelin | 0.11.1 | 0.11.1 | 0.11.1 | 0.11.1 | | ZooKeeper | 3.9.2-amzn-0 | 3.9.2-amzn-0 | 3.9.2-amzn-0 | 3.9.1-amzn-1 | # 7.6.0 release components details See the following table for more information about the *Extras* packages in Amazon EMR 7.6.0. | Application | Application Type | Package | Package Type | Package Version | | --- | --- | --- | --- | --- | | AmazonCloudWatchAgent | Core | hadoop-hdfs-journalnode | Extras | 3.4.0.amzn.2-1.amzn2023 | | AmazonCloudWatchAgent | Core | ranger-kms | Extras | 2.0.0-1.amzn2023 | | AmazonCloudWatchAgent | Core | s3-dist-cp | Extras | 2.36.0-1.amzn2023 | | AmazonCloudWatchAgent | Core | zookeeper | Extras | 3.9.2.amzn.0-1.amzn2023 | | AmazonCloudWatchAgent | Core | zookeeper-server | Extras | 3.9.2.amzn.0-1.amzn2023 | | Flink | Core | hadoop-client | Extras | 3.4.0.amzn.2-1.amzn2023 | | Flink | Core | hadoop-hdfs | Extras | 3.4.0.amzn.2-1.amzn2023 | | Flink | Core | hadoop-hdfs-datanode | Extras | 3.4.0.amzn.2-1.amzn2023 | | Flink | Core | hadoop-hdfs-journalnode | Extras | 3.4.0.amzn.2-1.amzn2023 | | Flink | Core | hadoop-hdfs-namenode | Extras | 3.4.0.amzn.2-1.amzn2023 | | Flink | Core | hadoop-httpfs | Extras | 3.4.0.amzn.2-1.amzn2023 | | Flink | Core | hadoop-kms | Extras | 3.4.0.amzn.2-1.amzn2023 | | Flink | Core | hadoop-mapreduce | Extras | 3.4.0.amzn.2-1.amzn2023 | | Flink | Core | hadoop-mapreduce-historyserver | Extras | 3.4.0.amzn.2-1.amzn2023 | | Flink | Core | hadoop-yarn | Extras | 3.4.0.amzn.2-1.amzn2023 | | Flink | Core | hadoop-yarn-nodemanager | Extras | 3.4.0.amzn.2-1.amzn2023 | | Flink | Core | hadoop-yarn-proxyserver | Extras | 3.4.0.amzn.2-1.amzn2023 | | Flink | Core | hadoop-yarn-resourcemanager | Extras | 3.4.0.amzn.2-1.amzn2023 | | Flink | Core | hadoop-yarn-timelineserver | Extras | 3.4.0.amzn.2-1.amzn2023 | | Flink | Core | hudi | Extras | 0.15.0.amzn.3-1.amzn2023 | | Flink | Core | ranger-kms | Extras | 2.0.0-1.amzn2023 | | Flink | Core | hadoop | Extras | 3.4.0.amzn.2-1.amzn2023 | | HBase | Core | s3-dist-cp | Extras | 2.36.0-1.amzn2023 | | HCatalog | Core | hadoop-yarn-proxyserver | Extras | 3.4.0.amzn.2-1.amzn2023 | | HCatalog | Core | mariadb-connector-java | Extras | | | Hive | Core | hadoop-httpfs | Extras | 3.4.0.amzn.2-1.amzn2023 | | Hive | Core | hadoop-yarn-proxyserver | Extras | 3.4.0.amzn.2-1.amzn2023 | | Hive | Core | hive-hbase | Extras | 3.1.3.amzn.15-1.amzn2023 | | Hive | Core | mariadb-connector-java | Extras | 2.7.2-1 | | Hive | Core | s3-dist-cp | Extras | 2.36.0-1.amzn2023 | | Phoenix | Core | s3-dist-cp | Extras | 2.36.0-1.amzn2023 | | Spark | Core | spark-datanucleus | Extras | 3.5.1.amzn.0-1.amzn2023 | | Tez | Core | hadoop-mapreduce | Extras | 3.4.0.amzn.2-1.amzn2023 | | Tez | Core | hadoop-mapreduce-historyserver | Extras | 3.4.0.amzn.2-1.amzn2023 | | Tez | Core | hadoop-yarn-proxyserver | Extras | 3.4.0.amzn.2-1.amzn2023 | | Trino | Core | bigtop-utils | Extras | 1.2.0-1.amzn2023 | | Trino | Core | hadoop-client | Extras | 3.4.0.amzn.2-1.amzn2023 | | Trino | Core | hadoop-hdfs | Extras | 3.4.0.amzn.2-1.amzn2023 | | Trino | Core | hadoop-hdfs-datanode | Extras | 3.4.0.amzn.2-1.amzn2023 | | Trino | Core | hadoop-hdfs-journalnode | Extras | 3.4.0.amzn.2-1.amzn2023 | | Trino | Core | hadoop-hdfs-namenode | Extras | 3.4.0.amzn.2-1.amzn2023 | | Trino | Core | hadoop-kms | Extras | 3.4.0.amzn.2-1.amzn2023 | | Trino | Core | hadoop-mapreduce | Extras | 3.4.0.amzn.2-1.amzn2023 | | Trino | Core | hadoop-mapreduce-historyserver | Extras | 3.4.0.amzn.2-1.amzn2023 | | Trino | Core | hadoop-yarn | Extras | 3.4.0.amzn.2-1.amzn2023 | | Trino | Core | hadoop-yarn-nodemanager | Extras | 3.4.0.amzn.2-1.amzn2023 | | Trino | Core | hadoop-yarn-proxyserver | Extras | 3.4.0.amzn.2-1.amzn2023 | | Trino | Core | hadoop-yarn-resourcemanager | Extras | 3.4.0.amzn.2-1.amzn2023 | | Trino | Core | hadoop-yarn-timelineserver | Extras | 3.4.0.amzn.2-1.amzn2023 | | Trino | Core | hive | Extras | 3.1.3.amzn.12-1.amzn2023 | | Trino | Core | hive-hcatalog-server | Extras | 3.1.3.amzn.12-1.amzn2023 | | Trino | Core | mariadb-connector-java | Extras | 2.7.2-1 | | Trino | Core | ranger-kms | Extras | 2.0.0-1.amzn2023 | | Trino | Core | zookeeper | Extras | 3.9.2.amzn.0-1.amzn2023 | | Trino | Core | zookeeper-server | Extras | 3.9.2.amzn.0-1.amzn2023 | | Trino | Core | bigtop-groovy | Extras | 2.5.4-1.amzn2023 | | Trino | Core | bigtop-jsvc | Extras | 1.2.4-1.amzn2023 | | Trino | Core | hadoop | Extras | 3.4.0.amzn.2-1.amzn2023 | | Trino | Core | hadoop-lzo | Extras | 0.4.19-1.amzn2023 | | Trino | Core | hive-hcatalog | Extras | 3.1.3.amzn.15-1.amzn2023 | | Trino | Core | hive-jdbc | Extras | 3.1.3.amzn.12-1.amzn2023 | | Zookeeper | Core | emrfs | Extras | 3.9.2.amzn.0-1.amzn2023 | | Zookeeper | Core | hadoop-client | Extras | 3.9.2.amzn.0-1.amzn2023 | | Zookeeper | Core | hadoop-hdfs | Extras | 3.9.2.amzn.0-1.amzn2023 | | Zookeeper | Core | hadoop-hdfs-datanode | Extras | 3.9.2.amzn.0-1.amzn2023 | | Zookeeper | Core | hadoop-hdfs-journalnode | Extras | 3.9.2.amzn.0-1.amzn2023 | | Zookeeper | Core | hadoop-hdfs-namenode | Extras | 3.9.2.amzn.0-1.amzn2023 | | Zookeeper | Core | hadoop-httpfs | Extras | 3.9.2.amzn.0-1.amzn2023 | | Zookeeper | Core | hadoop-kms | Extras | 3.9.2.amzn.0-1.amzn2023 | | Zookeeper | Core | hadoop-mapreduce | Extras | 3.4.0.amzn.2-1.amzn2023 | | Zookeeper | Core | hadoop-mapreduce-historyserver | Extras | 3.4.0.amzn.2-1.amzn2023 | | Zookeeper | Core | hadoop-yarn | Extras | 3.4.0.amzn.2-1.amzn2023 | | Zookeeper | Core | hadoop-yarn-nodemanager | Extras | 3.4.0.amzn.2-1.amzn2023 | | Zookeeper | Core | hadoop-yarn-proxyserver | Extras | 3.4.0.amzn.2-1.amzn2023 | | Zookeeper | Core | hadoop-yarn-resourcemanager | Extras | 3.4.0.amzn.2-1.amzn2023 | | Zookeeper | Core | hadoop-yarn-timelineserver | Extras | 3.4.0.amzn.2-1.amzn2023 | | Zookeeper | Core | ranger-kms | Extras | 2.0.0-1.amzn2023 | # 7.6.0 common vulnerabilities and exposures The following table lists all CVEs that don't impact EMR clusters that run on recommended configurations of Amazon EMR 7.6.0. Amazon EMR is dependent on upstream open-source for availability of fixes and will provide the latest stable version as part of the Core Engine components within 90 days of Amazon EMR verifying the fixes. This table was updated on January 9, 2025. | CVE ID | Severity | CVE details URL | | --- | --- | --- | | CVE-2013-7285 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2013-7285](https://nvd.nist.gov/vuln/detail/CVE-2013-7285) | | CVE-2015-1832 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2015-1832](https://nvd.nist.gov/vuln/detail/CVE-2015-1832) | | CVE-2016-5018 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2016-5018](https://nvd.nist.gov/vuln/detail/CVE-2016-5018) | | CVE-2017-15095 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2017-15095](https://nvd.nist.gov/vuln/detail/CVE-2017-15095) | | CVE-2017-17485 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2017-17485](https://nvd.nist.gov/vuln/detail/CVE-2017-17485) | | CVE-2017-7525 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2017-7525](https://nvd.nist.gov/vuln/detail/CVE-2017-7525) | | CVE-2018-11307 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-11307](https://nvd.nist.gov/vuln/detail/CVE-2018-11307) | | CVE-2018-14718 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-14718](https://nvd.nist.gov/vuln/detail/CVE-2018-14718) | | CVE-2018-14719 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-14719](https://nvd.nist.gov/vuln/detail/CVE-2018-14719) | | CVE-2018-14720 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-14720](https://nvd.nist.gov/vuln/detail/CVE-2018-14720) | | CVE-2018-14721 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-14721](https://nvd.nist.gov/vuln/detail/CVE-2018-14721) | | CVE-2018-19360 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-19360](https://nvd.nist.gov/vuln/detail/CVE-2018-19360) | | CVE-2018-19361 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-19361](https://nvd.nist.gov/vuln/detail/CVE-2018-19361) | | CVE-2018-19362 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-19362](https://nvd.nist.gov/vuln/detail/CVE-2018-19362) | | CVE-2018-7489 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-7489](https://nvd.nist.gov/vuln/detail/CVE-2018-7489) | | CVE-2019-10173 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-10173](https://nvd.nist.gov/vuln/detail/CVE-2019-10173) | | CVE-2019-10202 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-10202](https://nvd.nist.gov/vuln/detail/CVE-2019-10202) | | CVE-2019-14379 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-14379](https://nvd.nist.gov/vuln/detail/CVE-2019-14379) | | CVE-2019-14540 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-14540](https://nvd.nist.gov/vuln/detail/CVE-2019-14540) | | CVE-2019-14892 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-14892](https://nvd.nist.gov/vuln/detail/CVE-2019-14892) | | CVE-2019-14893 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-14893](https://nvd.nist.gov/vuln/detail/CVE-2019-14893) | | CVE-2019-16335 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-16335](https://nvd.nist.gov/vuln/detail/CVE-2019-16335) | | CVE-2019-16942 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-16942](https://nvd.nist.gov/vuln/detail/CVE-2019-16942) | | CVE-2019-16943 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-16943](https://nvd.nist.gov/vuln/detail/CVE-2019-16943) | | CVE-2019-17267 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-17267](https://nvd.nist.gov/vuln/detail/CVE-2019-17267) | | CVE-2019-17531 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-17531](https://nvd.nist.gov/vuln/detail/CVE-2019-17531) | | CVE-2019-20330 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-20330](https://nvd.nist.gov/vuln/detail/CVE-2019-20330) | | CVE-2019-20444 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-20444](https://nvd.nist.gov/vuln/detail/CVE-2019-20444) | | CVE-2019-20445 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-20445](https://nvd.nist.gov/vuln/detail/CVE-2019-20445) | | CVE-2020-8840 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2020-8840](https://nvd.nist.gov/vuln/detail/CVE-2020-8840) | | CVE-2020-9546 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2020-9546](https://nvd.nist.gov/vuln/detail/CVE-2020-9546) | | CVE-2020-9547 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2020-9547](https://nvd.nist.gov/vuln/detail/CVE-2020-9547) | | CVE-2020-9548 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2020-9548](https://nvd.nist.gov/vuln/detail/CVE-2020-9548) | | CVE-2021-21342 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2021-21342](https://nvd.nist.gov/vuln/detail/CVE-2021-21342) | | CVE-2021-21344 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2021-21344](https://nvd.nist.gov/vuln/detail/CVE-2021-21344) | | CVE-2021-21345 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2021-21345](https://nvd.nist.gov/vuln/detail/CVE-2021-21345) | | CVE-2021-21346 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2021-21346](https://nvd.nist.gov/vuln/detail/CVE-2021-21346) | | CVE-2021-21347 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2021-21347](https://nvd.nist.gov/vuln/detail/CVE-2021-21347) | | CVE-2021-21350 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2021-21350](https://nvd.nist.gov/vuln/detail/CVE-2021-21350) | | CVE-2021-21351 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2021-21351](https://nvd.nist.gov/vuln/detail/CVE-2021-21351) | | CVE-2021-41411 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2021-41411](https://nvd.nist.gov/vuln/detail/CVE-2021-41411) | | CVE-2022-1471 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2022-1471](https://nvd.nist.gov/vuln/detail/CVE-2022-1471) | | CVE-2022-37865 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2022-37865](https://nvd.nist.gov/vuln/detail/CVE-2022-37865) | | CVE-2022-39135 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2022-39135](https://nvd.nist.gov/vuln/detail/CVE-2022-39135) | | CVE-2022-42889 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2022-42889](https://nvd.nist.gov/vuln/detail/CVE-2022-42889) | | CVE-2022-46337 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2022-46337](https://nvd.nist.gov/vuln/detail/CVE-2022-46337) | | CVE-2024-47561 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2024-47561](https://nvd.nist.gov/vuln/detail/CVE-2024-47561) | | CVE-2024-52046 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2024-52046](https://nvd.nist.gov/vuln/detail/CVE-2024-52046) | | CVE-2024-53990 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2024-53990](https://nvd.nist.gov/vuln/detail/CVE-2024-53990) | | CVE-2012-0881 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2012-0881](https://nvd.nist.gov/vuln/detail/CVE-2012-0881) | | CVE-2013-4002 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2013-4002](https://nvd.nist.gov/vuln/detail/CVE-2013-4002) | | CVE-2014-8125 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2014-8125](https://nvd.nist.gov/vuln/detail/CVE-2014-8125) | | CVE-2015-5237 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2015-5237](https://nvd.nist.gov/vuln/detail/CVE-2015-5237) | | CVE-2016-3674 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2016-3674](https://nvd.nist.gov/vuln/detail/CVE-2016-3674) | | CVE-2016-6796 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2016-6796](https://nvd.nist.gov/vuln/detail/CVE-2016-6796) | | CVE-2017-7957 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2017-7957](https://nvd.nist.gov/vuln/detail/CVE-2017-7957) | | CVE-2018-1000180 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2018-1000180](https://nvd.nist.gov/vuln/detail/CVE-2018-1000180) | | CVE-2018-12022 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2018-12022](https://nvd.nist.gov/vuln/detail/CVE-2018-12022) | | CVE-2018-12023 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2018-12023](https://nvd.nist.gov/vuln/detail/CVE-2018-12023) | | CVE-2018-5968 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2018-5968](https://nvd.nist.gov/vuln/detail/CVE-2018-5968) | | CVE-2019-0205 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2019-0205](https://nvd.nist.gov/vuln/detail/CVE-2019-0205) | | CVE-2019-10172 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2019-10172](https://nvd.nist.gov/vuln/detail/CVE-2019-10172) | | CVE-2019-12086 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2019-12086](https://nvd.nist.gov/vuln/detail/CVE-2019-12086) | | CVE-2019-14439 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2019-14439](https://nvd.nist.gov/vuln/detail/CVE-2019-14439) | | CVE-2020-10650 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-10650](https://nvd.nist.gov/vuln/detail/CVE-2020-10650) | | CVE-2020-10672 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-10672](https://nvd.nist.gov/vuln/detail/CVE-2020-10672) | | CVE-2020-10673 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-10673](https://nvd.nist.gov/vuln/detail/CVE-2020-10673) | | CVE-2020-10968 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-10968](https://nvd.nist.gov/vuln/detail/CVE-2020-10968) | | CVE-2020-10969 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-10969](https://nvd.nist.gov/vuln/detail/CVE-2020-10969) | | CVE-2020-11111 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-11111](https://nvd.nist.gov/vuln/detail/CVE-2020-11111) | | CVE-2020-11112 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-11112](https://nvd.nist.gov/vuln/detail/CVE-2020-11112) | | CVE-2020-11113 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-11113](https://nvd.nist.gov/vuln/detail/CVE-2020-11113) | | CVE-2020-11619 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-11619](https://nvd.nist.gov/vuln/detail/CVE-2020-11619) | | CVE-2020-11620 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-11620](https://nvd.nist.gov/vuln/detail/CVE-2020-11620) | | CVE-2020-13936 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-13936](https://nvd.nist.gov/vuln/detail/CVE-2020-13936) | | CVE-2020-13949 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-13949](https://nvd.nist.gov/vuln/detail/CVE-2020-13949) | | CVE-2020-14060 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-14060](https://nvd.nist.gov/vuln/detail/CVE-2020-14060) | | CVE-2020-14061 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-14061](https://nvd.nist.gov/vuln/detail/CVE-2020-14061) | | CVE-2020-14062 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-14062](https://nvd.nist.gov/vuln/detail/CVE-2020-14062) | | CVE-2020-14195 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-14195](https://nvd.nist.gov/vuln/detail/CVE-2020-14195) | | CVE-2020-24616 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-24616](https://nvd.nist.gov/vuln/detail/CVE-2020-24616) | | CVE-2020-24750 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-24750](https://nvd.nist.gov/vuln/detail/CVE-2020-24750) | | CVE-2020-25649 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-25649](https://nvd.nist.gov/vuln/detail/CVE-2020-25649) | | CVE-2020-26217 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-26217](https://nvd.nist.gov/vuln/detail/CVE-2020-26217) | | CVE-2020-26258 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-26258](https://nvd.nist.gov/vuln/detail/CVE-2020-26258) | | CVE-2020-28052 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-28052](https://nvd.nist.gov/vuln/detail/CVE-2020-28052) | | CVE-2020-35490 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-35490](https://nvd.nist.gov/vuln/detail/CVE-2020-35490) | | CVE-2020-35491 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-35491](https://nvd.nist.gov/vuln/detail/CVE-2020-35491) | | CVE-2020-35728 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-35728](https://nvd.nist.gov/vuln/detail/CVE-2020-35728) | | CVE-2020-36179 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36179](https://nvd.nist.gov/vuln/detail/CVE-2020-36179) | | CVE-2020-36180 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36180](https://nvd.nist.gov/vuln/detail/CVE-2020-36180) | | CVE-2020-36181 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36181](https://nvd.nist.gov/vuln/detail/CVE-2020-36181) | | CVE-2020-36182 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36182](https://nvd.nist.gov/vuln/detail/CVE-2020-36182) | | CVE-2020-36183 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36183](https://nvd.nist.gov/vuln/detail/CVE-2020-36183) | | CVE-2020-36184 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36184](https://nvd.nist.gov/vuln/detail/CVE-2020-36184) | | CVE-2020-36185 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36185](https://nvd.nist.gov/vuln/detail/CVE-2020-36185) | | CVE-2020-36186 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36186](https://nvd.nist.gov/vuln/detail/CVE-2020-36186) | | CVE-2020-36187 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36187](https://nvd.nist.gov/vuln/detail/CVE-2020-36187) | | CVE-2020-36188 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36188](https://nvd.nist.gov/vuln/detail/CVE-2020-36188) | | CVE-2020-36189 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36189](https://nvd.nist.gov/vuln/detail/CVE-2020-36189) | | CVE-2020-36518 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36518](https://nvd.nist.gov/vuln/detail/CVE-2020-36518) | | CVE-2021-20190 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-20190](https://nvd.nist.gov/vuln/detail/CVE-2021-20190) | | CVE-2021-21341 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-21341](https://nvd.nist.gov/vuln/detail/CVE-2021-21341) | | CVE-2021-21343 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-21343](https://nvd.nist.gov/vuln/detail/CVE-2021-21343) | | CVE-2021-21348 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-21348](https://nvd.nist.gov/vuln/detail/CVE-2021-21348) | | CVE-2021-21349 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-21349](https://nvd.nist.gov/vuln/detail/CVE-2021-21349) | | CVE-2021-29505 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-29505](https://nvd.nist.gov/vuln/detail/CVE-2021-29505) | | CVE-2021-31684 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-31684](https://nvd.nist.gov/vuln/detail/CVE-2021-31684) | | CVE-2021-34538 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-34538](https://nvd.nist.gov/vuln/detail/CVE-2021-34538) | | CVE-2021-37136 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-37136](https://nvd.nist.gov/vuln/detail/CVE-2021-37136) | | CVE-2021-39139 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-39139](https://nvd.nist.gov/vuln/detail/CVE-2021-39139) | | CVE-2021-39141 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-39141](https://nvd.nist.gov/vuln/detail/CVE-2021-39141) | | CVE-2021-39144 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-39144](https://nvd.nist.gov/vuln/detail/CVE-2021-39144) | | CVE-2021-39145 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-39145](https://nvd.nist.gov/vuln/detail/CVE-2021-39145) | | CVE-2021-39146 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-39146](https://nvd.nist.gov/vuln/detail/CVE-2021-39146) | | CVE-2021-39147 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-39147](https://nvd.nist.gov/vuln/detail/CVE-2021-39147) | | CVE-2021-39148 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-39148](https://nvd.nist.gov/vuln/detail/CVE-2021-39148) | | CVE-2021-39149 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-39149](https://nvd.nist.gov/vuln/detail/CVE-2021-39149) | | CVE-2021-39150 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-39150](https://nvd.nist.gov/vuln/detail/CVE-2021-39150) | | CVE-2021-39151 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-39151](https://nvd.nist.gov/vuln/detail/CVE-2021-39151) | | CVE-2021-39152 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-39152](https://nvd.nist.gov/vuln/detail/CVE-2021-39152) | | CVE-2021-39153 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-39153](https://nvd.nist.gov/vuln/detail/CVE-2021-39153) | | CVE-2021-39154 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-39154](https://nvd.nist.gov/vuln/detail/CVE-2021-39154) | | CVE-2021-43859 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-43859](https://nvd.nist.gov/vuln/detail/CVE-2021-43859) | | CVE-2022-1415 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-1415](https://nvd.nist.gov/vuln/detail/CVE-2022-1415) | | CVE-2022-25647 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-25647](https://nvd.nist.gov/vuln/detail/CVE-2022-25647) | | CVE-2022-25857 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-25857](https://nvd.nist.gov/vuln/detail/CVE-2022-25857) | | CVE-2022-3171 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-3171](https://nvd.nist.gov/vuln/detail/CVE-2022-3171) | | CVE-2022-3509 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-3509](https://nvd.nist.gov/vuln/detail/CVE-2022-3509) | | CVE-2022-3510 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-3510](https://nvd.nist.gov/vuln/detail/CVE-2022-3510) | | CVE-2022-36364 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-36364](https://nvd.nist.gov/vuln/detail/CVE-2022-36364) | | CVE-2022-37866 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-37866](https://nvd.nist.gov/vuln/detail/CVE-2022-37866) | | CVE-2022-40149 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-40149](https://nvd.nist.gov/vuln/detail/CVE-2022-40149) | | CVE-2022-40150 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-40150](https://nvd.nist.gov/vuln/detail/CVE-2022-40150) | | CVE-2022-40151 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-40151](https://nvd.nist.gov/vuln/detail/CVE-2022-40151) | | CVE-2022-40152 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-40152](https://nvd.nist.gov/vuln/detail/CVE-2022-40152) | | CVE-2022-41966 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-41966](https://nvd.nist.gov/vuln/detail/CVE-2022-41966) | | CVE-2022-42003 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-42003](https://nvd.nist.gov/vuln/detail/CVE-2022-42003) | | CVE-2022-42004 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-42004](https://nvd.nist.gov/vuln/detail/CVE-2022-42004) | | CVE-2022-45685 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-45685](https://nvd.nist.gov/vuln/detail/CVE-2022-45685) | | CVE-2022-45688 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-45688](https://nvd.nist.gov/vuln/detail/CVE-2022-45688) | | CVE-2022-45693 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-45693](https://nvd.nist.gov/vuln/detail/CVE-2022-45693) | | CVE-2022-46751 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-46751](https://nvd.nist.gov/vuln/detail/CVE-2022-46751) | | CVE-2023-1370 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-1370](https://nvd.nist.gov/vuln/detail/CVE-2023-1370) | | CVE-2023-1436 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-1436](https://nvd.nist.gov/vuln/detail/CVE-2023-1436) | | CVE-2023-20883 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-20883](https://nvd.nist.gov/vuln/detail/CVE-2023-20883) | | CVE-2023-25194 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-25194](https://nvd.nist.gov/vuln/detail/CVE-2023-25194) | | CVE-2023-2976 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-2976](https://nvd.nist.gov/vuln/detail/CVE-2023-2976) | | CVE-2023-34453 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-34453](https://nvd.nist.gov/vuln/detail/CVE-2023-34453) | | CVE-2023-34454 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-34454](https://nvd.nist.gov/vuln/detail/CVE-2023-34454) | | CVE-2023-34455 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-34455](https://nvd.nist.gov/vuln/detail/CVE-2023-34455) | | CVE-2023-3635 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-3635](https://nvd.nist.gov/vuln/detail/CVE-2023-3635) | | CVE-2023-39410 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-39410](https://nvd.nist.gov/vuln/detail/CVE-2023-39410) | | CVE-2023-43642 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-43642](https://nvd.nist.gov/vuln/detail/CVE-2023-43642) | | CVE-2023-44487 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-44487](https://nvd.nist.gov/vuln/detail/CVE-2023-44487) | | CVE-2023-50387 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-50387](https://nvd.nist.gov/vuln/detail/CVE-2023-50387) | | CVE-2023-5072 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-5072](https://nvd.nist.gov/vuln/detail/CVE-2023-5072) | | CVE-2023-52428 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-52428](https://nvd.nist.gov/vuln/detail/CVE-2023-52428) | | CVE-2023-6378 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-6378](https://nvd.nist.gov/vuln/detail/CVE-2023-6378) | | CVE-2023-6481 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-6481](https://nvd.nist.gov/vuln/detail/CVE-2023-6481) | | CVE-2024-21634 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-21634](https://nvd.nist.gov/vuln/detail/CVE-2024-21634) | | CVE-2024-7254 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-7254](https://nvd.nist.gov/vuln/detail/CVE-2024-7254) | | CVE-2009-2625 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2009-2625](https://nvd.nist.gov/vuln/detail/CVE-2009-2625) | | CVE-2012-5783 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2012-5783](https://nvd.nist.gov/vuln/detail/CVE-2012-5783) | | CVE-2012-6153 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2012-6153](https://nvd.nist.gov/vuln/detail/CVE-2012-6153) | | CVE-2017-10355 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2017-10355](https://nvd.nist.gov/vuln/detail/CVE-2017-10355) | | CVE-2018-10237 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2018-10237](https://nvd.nist.gov/vuln/detail/CVE-2018-10237) | | CVE-2018-11798 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2018-11798](https://nvd.nist.gov/vuln/detail/CVE-2018-11798) | | CVE-2018-1313 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2018-1313](https://nvd.nist.gov/vuln/detail/CVE-2018-1313) | | CVE-2018-2799 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2018-2799](https://nvd.nist.gov/vuln/detail/CVE-2018-2799) | | CVE-2019-10219 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2019-10219](https://nvd.nist.gov/vuln/detail/CVE-2019-10219) | | CVE-2019-12384 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2019-12384](https://nvd.nist.gov/vuln/detail/CVE-2019-12384) | | CVE-2019-12814 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2019-12814](https://nvd.nist.gov/vuln/detail/CVE-2019-12814) | | CVE-2020-10693 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2020-10693](https://nvd.nist.gov/vuln/detail/CVE-2020-10693) | | CVE-2020-13955 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2020-13955](https://nvd.nist.gov/vuln/detail/CVE-2020-13955) | | CVE-2020-13956 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2020-13956](https://nvd.nist.gov/vuln/detail/CVE-2020-13956) | | CVE-2020-14338 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2020-14338](https://nvd.nist.gov/vuln/detail/CVE-2020-14338) | | CVE-2020-15250 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2020-15250](https://nvd.nist.gov/vuln/detail/CVE-2020-15250) | | CVE-2020-15522 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2020-15522](https://nvd.nist.gov/vuln/detail/CVE-2020-15522) | | CVE-2020-17521 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2020-17521](https://nvd.nist.gov/vuln/detail/CVE-2020-17521) | | CVE-2020-1945 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2020-1945](https://nvd.nist.gov/vuln/detail/CVE-2020-1945) | | CVE-2020-26939 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2020-26939](https://nvd.nist.gov/vuln/detail/CVE-2020-26939) | | CVE-2020-29582 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2020-29582](https://nvd.nist.gov/vuln/detail/CVE-2020-29582) | | CVE-2021-21290 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2021-21290](https://nvd.nist.gov/vuln/detail/CVE-2021-21290) | | CVE-2021-21295 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2021-21295](https://nvd.nist.gov/vuln/detail/CVE-2021-21295) | | CVE-2021-22569 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2021-22569](https://nvd.nist.gov/vuln/detail/CVE-2021-22569) | | CVE-2021-22570 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2021-22570](https://nvd.nist.gov/vuln/detail/CVE-2021-22570) | | CVE-2021-27568 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2021-27568](https://nvd.nist.gov/vuln/detail/CVE-2021-27568) | | CVE-2021-28170 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2021-28170](https://nvd.nist.gov/vuln/detail/CVE-2021-28170) | | CVE-2021-29425 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2021-29425](https://nvd.nist.gov/vuln/detail/CVE-2021-29425) | | CVE-2021-36373 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2021-36373](https://nvd.nist.gov/vuln/detail/CVE-2021-36373) | | CVE-2021-36374 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2021-36374](https://nvd.nist.gov/vuln/detail/CVE-2021-36374) | | CVE-2021-37533 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2021-37533](https://nvd.nist.gov/vuln/detail/CVE-2021-37533) | | CVE-2021-38153 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2021-38153](https://nvd.nist.gov/vuln/detail/CVE-2021-38153) | | CVE-2022-23437 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2022-23437](https://nvd.nist.gov/vuln/detail/CVE-2022-23437) | | CVE-2022-24329 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2022-24329](https://nvd.nist.gov/vuln/detail/CVE-2022-24329) | | CVE-2022-38749 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2022-38749](https://nvd.nist.gov/vuln/detail/CVE-2022-38749) | | CVE-2022-38750 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2022-38750](https://nvd.nist.gov/vuln/detail/CVE-2022-38750) | | CVE-2022-38751 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2022-38751](https://nvd.nist.gov/vuln/detail/CVE-2022-38751) | | CVE-2022-38752 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2022-38752](https://nvd.nist.gov/vuln/detail/CVE-2022-38752) | | CVE-2022-41854 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2022-41854](https://nvd.nist.gov/vuln/detail/CVE-2022-41854) | | CVE-2023-0833 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2023-0833](https://nvd.nist.gov/vuln/detail/CVE-2023-0833) | | CVE-2023-20863 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2023-20863](https://nvd.nist.gov/vuln/detail/CVE-2023-20863) | | CVE-2023-26049 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2023-26049](https://nvd.nist.gov/vuln/detail/CVE-2023-26049) | | CVE-2023-33201 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2023-33201](https://nvd.nist.gov/vuln/detail/CVE-2023-33201) | | CVE-2023-33202 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2023-33202](https://nvd.nist.gov/vuln/detail/CVE-2023-33202) | | CVE-2023-34462 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2023-34462](https://nvd.nist.gov/vuln/detail/CVE-2023-34462) | | CVE-2023-42503 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2023-42503](https://nvd.nist.gov/vuln/detail/CVE-2023-42503) | | CVE-2023-51074 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2023-51074](https://nvd.nist.gov/vuln/detail/CVE-2023-51074) | | CVE-2024-12798 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-12798](https://nvd.nist.gov/vuln/detail/CVE-2024-12798) | | CVE-2024-25710 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-25710](https://nvd.nist.gov/vuln/detail/CVE-2024-25710) | | CVE-2024-26308 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-26308](https://nvd.nist.gov/vuln/detail/CVE-2024-26308) | | CVE-2024-38820 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-38820](https://nvd.nist.gov/vuln/detail/CVE-2024-38820) | | CVE-2024-6763 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-6763](https://nvd.nist.gov/vuln/detail/CVE-2024-6763) | | CVE-2024-8184 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-8184](https://nvd.nist.gov/vuln/detail/CVE-2024-8184) | | CVE-2020-8908 | LOW | [https://nvd.nist.gov/vuln/detail/CVE-2020-8908](https://nvd.nist.gov/vuln/detail/CVE-2020-8908) | | CVE-2024-12801 | LOW | [https://nvd.nist.gov/vuln/detail/CVE-2024-12801](https://nvd.nist.gov/vuln/detail/CVE-2024-12801) | | CVE-2023-50868 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2023-50868](https://nvd.nist.gov/vuln/detail/CVE-2023-50868) | | CVE-2024-23454 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2024-23454](https://nvd.nist.gov/vuln/detail/CVE-2024-23454) | | CVE-2024-23944 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2024-23944](https://nvd.nist.gov/vuln/detail/CVE-2024-23944) | | CVE-2024-23945 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2024-23945](https://nvd.nist.gov/vuln/detail/CVE-2024-23945) | | CVE-2024-25638 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2024-25638](https://nvd.nist.gov/vuln/detail/CVE-2024-25638) | | CVE-2024-29025 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2024-29025](https://nvd.nist.gov/vuln/detail/CVE-2024-29025) | | CVE-2024-29131 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2024-29131](https://nvd.nist.gov/vuln/detail/CVE-2024-29131) | | CVE-2024-29133 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2024-29133](https://nvd.nist.gov/vuln/detail/CVE-2024-29133) | | CVE-2024-29857 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2024-29857](https://nvd.nist.gov/vuln/detail/CVE-2024-29857) | | CVE-2024-30171 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2024-30171](https://nvd.nist.gov/vuln/detail/CVE-2024-30171) | | CVE-2024-31141 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2024-31141](https://nvd.nist.gov/vuln/detail/CVE-2024-31141) | | CVE-2024-36114 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2024-36114](https://nvd.nist.gov/vuln/detail/CVE-2024-36114) | | CVE-2024-38808 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2024-38808](https://nvd.nist.gov/vuln/detail/CVE-2024-38808) | | CVE-2024-43382 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2024-43382](https://nvd.nist.gov/vuln/detail/CVE-2024-43382) | | CVE-2024-47072 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2024-47072](https://nvd.nist.gov/vuln/detail/CVE-2024-47072) | | CVE-2024-47535 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2024-47535](https://nvd.nist.gov/vuln/detail/CVE-2024-47535) | | CVE-2024-47554 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2024-47554](https://nvd.nist.gov/vuln/detail/CVE-2024-47554) | | CVE-2024-51504 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2024-51504](https://nvd.nist.gov/vuln/detail/CVE-2024-51504) | | CVE-2024-56128 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2024-56128](https://nvd.nist.gov/vuln/detail/CVE-2024-56128) | | CVE-2024-9823 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2024-9823](https://nvd.nist.gov/vuln/detail/CVE-2024-9823) | ## 7.6.0 release notes The following release notes include information for Amazon EMR release 7.6.0. **New features** + **Application upgrades** – Amazon EMR 7.6.0 application upgrades include HBase 2.6.1, Flink 1.20.0, Spark 3.5.3, and Trino 457. **Known issues** + When managed scaling is enabled, the YARN ResourceManager (RM) experiences a critical deadlock issue, causing it to become unresponsive, when transitioning from the **DECOMMISSIONING** to **DECOMMISSIONED** state with simultaneous operations. This happens particularly in large-cluster environments. When it occurs, both node-state transitions and application-state transitions fail to process, effectively paralyzing the core functionality of the ResourceManager. As a workaround, restart the RM JVM using the following command on the primary node: ``` sudo systemctl restart hadoop-yarn-resourcemanager ``` + When an EMR cluster is started with YARN schedulers other than the `CapacityScheduler` or `FairScheduler` (for example, the FIFO Scheduler), the `YARNMemoryAvailablePercentage` metric is not pushed to CloudWatch. This issue impacts down-scaling in managed-scaling enabled clusters. It might also affect custom managed scaling if it depends on the `YARNMemoryAvailablePercentage` metric. As a workaround, use the `CapacityScheduler` or `FairScheduler`. + **HBase table creation in Hive with Glue catalog** – From EMR-7.3.0 to EMR-7.10.0, there is a bug due to Hive Iceberg integration, which causes HBase table creation in Hive to fail when Glue Data Catalog is used as the metastore. Please reach out to the AWS support team if you encounter this issue. + **Tez UI** – From EMR-7.3 to EMR-7.10, there is a bug where accessing Tez UI through YARN Resource Manager Web UI fails when in-transit encryption is enabled. This occurs because YARN proxy attempts to use HTTP protocol while Tez UI requires HTTPS. Please modify the `tez.tez-ui.history-url.base` configuration in `tez-site` from `http` to `https` to resolve this issue. **Changes, enhancements, and resolved issues** + Starting with EMR 7.6.0, the step cancellation process now ensures that all Hadoop YARN applications created by a step are terminated when that step is cancelled. This provides more reliable cleanup for applications like Spark jobs running in cluster mode. + Starting with Spark 3.3.1 (supported in EMR versions 6.10 and above), all executors in a decommissioning host are set to a new `ExecutorState`, called *DECOMMISSIONING* state. The executors being decommissioned cannot be used by Yarn to allocate tasks and thus it will request for new executors, if needed, for the tasks being executed. Thus, if you disable Spark DRA while using EMR Managed Scaling, EMR Auto Scaling, or any custom scaling mechanism on EMR-EC2 clusters, then Yarn may request maximum permissible executors for each job. In order to avoid this issue, leave the `spark.dynamicAllocation.enabled` property set to `TRUE` (which is the default) when you are using the above combination of features. In addition, you can also set minimum and maximum executor constraints by setting values for `spark.dynamicAllocation.maxExecutors` and `spark.dynamicAllocation.minExecutors` properties for your Spark jobs, to restrict the number of executors allocated during the job’s execution. + CloudWatch agent is supported on Runtime Role Clusters for EMR 7.6 and above. For more information, see [Amazon CloudWatch agent](https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-AmazonCloudWatchAgent.html). + Starting with Amazon Elastic Map Reduce (EMR) release version 7.6.0, the S3A filesystem connector now supports Amazon S3 client-side encryption. + Starting with EMR 7.6.0, TLS is supported on HMaster and RegionServer endpoints. For more information, see the following table regarding TLS support with HBase: [HBase](https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-encryption-support-matrix.html#emr-encryption-support-matrix-hbase). + [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-760-release.html) ## 7.6.0 default Java versions Amazon EMR release 7.6 and higher ships with Amazon Corretto 17 (built on OpenJDK) by default for applications that support Corretto 17 (JDK 17), with the exception of Apache Livy. The following table shows the default Java versions for applications in Amazon EMR 7.6.0. If you want to change the default JVM on your cluster, follow the instructions in [Configure applications to use a specific Java Virtual Machine](configuring-java8.md) for each application that runs on the cluster. You can only use one Java runtime version for a cluster. Amazon EMR doesn't support running different nodes or applications on different runtime versions on the same cluster. | Application | Java / Amazon Corretto version (default is bold) | | --- | --- | | Delta | 17, 11, 8 | | Flink | 17, 11, 8 | | HBase | 17, 11, 8 | | HCatalog | 17, 11, 8 | | Hadoop | 17, 11, 8 | | Hive | 17, 11, 8 | | Hudi | 17, 11, 8 | | Iceberg | 17, 11, 8 | | Livy | 17, 11, 8 | | Oozie | 17, 11, 8 | | Phoenix | 17, 11, 8 | | PrestoDB | 8 | | Spark | 17, 11, 8 | | Spark RAPIDS | 17, 11, 8 | | Tez | 17, 11, 8 | | Trino | 21, 17 | | Zeppelin | 17, 11, 8 | | Pig | 17, 11, 8 | | Zookeeper | 17, 11, 8 | ## 7.6.0 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | adot-java-agent | 1.31.0 | A Java Agent that collects metrics from application daemons. | | delta | 3.2.1-amzn-1 | Delta lake is an open table format for huge analytic datasets | | emr-amazon-cloudwatch-agent | 1.300032.2-amzn-0 | An application that collects internal system-level metrics and custom application metrics from Amazon EC2 instances. | | emr-ddb | 5.6.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 3.15.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.19.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-notebook-env | 1.18.0 | Conda env for emr notebook which includes jupyter enterprise gateway | | emr-s3-dist-cp | 2.36.0 | Distributed copy application optimized for Amazon S3. | | emr-s3-select | 2.15.0 | EMR S3Select Connector | | emr-wal-cli | 1.3.1 | Cli used for emrwal list/deletion. | | emrfs | 2.67.0 | Amazon S3 connector for Hadoop ecosystem applications. | | flink-client | 1.20.0-amzn-0 | Apache Flink command line client scripts and applications. | | flink-jobmanager-config | 1.20.0-amzn-0 | Managing resources on EMR nodes for Apache Flink JobManager. | | hadoop-client | 3.4.0-amzn-2 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 3.4.0-amzn-2 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 3.4.0-amzn-2 | HDFS command-line client and library | | hadoop-hdfs-namenode | 3.4.0-amzn-2 | HDFS service for tracking file names and block locations. | | hadoop-hdfs-journalnode | 3.4.0-amzn-2 | HDFS service for managing the Hadoop filesystem journal on HA clusters. | | hadoop-httpfs-server | 3.4.0-amzn-2 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 3.4.0-amzn-2 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 3.4.0-amzn-2 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 3.4.0-amzn-2 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 3.4.0-amzn-2 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 3.4.0-amzn-2 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 2.6.1-amzn-0 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 2.6.1-amzn-0 | Service for serving one or more HBase regions. | | hbase-client | 2.6.1-amzn-0 | HBase command-line client. | | hbase-rest-server | 2.6.1-amzn-0 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 2.6.1-amzn-0 | Service providing a Thrift endpoint to HBase. | | hbase-operator-tools | 2.6.1-amzn-0 | Repair tool for Apache HBase clusters. | | hcatalog-client | 3.1.3-amzn-15 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 3.1.3-amzn-15 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 3.1.3-amzn-15 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 3.1.3-amzn-15 | Hive command line client. | | hive-hbase | 3.1.3-amzn-15 | Hive-hbase client. | | hive-metastore-server | 3.1.3-amzn-15 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server2 | 3.1.3-amzn-15 | Service for accepting Hive queries as web requests. | | hudi | 0.15.0-amzn-3 | Incremental processing framework to power data pipeline at low latency and high efficiency. | | hudi-presto | 0.15.0-amzn-3 | Bundle library for running Presto with Hudi. | | hudi-trino | 0.15.0-amzn-3 | Bundle library for running Trino with Hudi. | | hudi-spark | 0.15.0-amzn-3 | Bundle library for running Spark with Hudi. | | hue-server | 4.11.0 | Web application for analyzing data using Hadoop ecosystem applications | | iceberg | 1.6.1-amzn-2 | Apache Iceberg is an open table format for huge analytic datasets | | jupyterhub | 1.5.0 | Multi-user server for Jupyter notebooks | | livy-server | 0.8.0-incubating | REST interface for interacting with Apache Spark | | nginx | 1.12.1 | nginx [engine x] is an HTTP and reverse proxy server | | mariadb-server | 5.5.68\$1 | MariaDB database server. | | nvidia-cuda | 12.3.0 | Nvidia drivers and Cuda toolkit | | oozie-client | 5.2.1 | Oozie command-line client. | | oozie-server | 5.2.1 | Service for accepting Oozie workflow requests. | | opencv | 4.7.0 | Open Source Computer Vision Library. | | phoenix-library | 5.2.0 | The phoenix libraries for server and client | | phoenix-connectors | 6.0.0 | Apache Phoenix-Connectors for Spark-3 | | phoenix-query-server | 6.0.0 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.287-amzn-1 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.287-amzn-1 | Service for executing pieces of a query. | | presto-client | 0.287-amzn-1 | Presto command-line client which is installed on an HA cluster's stand-by masters where Presto server is not started. | | trino-coordinator | 457-amzn-0 | Service for accepting queries and managing query execution among trino-workers. | | trino-worker | 457-amzn-0 | Service for executing pieces of a query. | | trino-client | 457-amzn-0 | Trino command-line client which is installed on an HA cluster's stand-by masters where Trino server is not started. | | pig-client | 0.17.0 | Pig command-line client. | | r | 4.3.2 | The R Project for Statistical Computing | | ranger-kms-server | 2.0.0 | Apache Ranger Key Management System | | spark-client | 3.5.3-amzn-0 | Spark command-line clients. | | spark-history-server | 3.5.3-amzn-0 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 3.5.3-amzn-0 | In-memory execution engine for YARN. | | spark-yarn-slave | 3.5.3-amzn-0 | Apache Spark libraries needed by YARN slaves. | | spark-rapids | 24.10.1-amzn-0 | Nvidia Spark RAPIDS plugin that accelerates Apache Spark with GPUs. | | tensorflow | 2.16.1 | TensorFlow open source software library for high performance numerical computation. | | tez-on-yarn | 0.10.2-amzn-13 | The tez YARN application and libraries. | | tez-on-worker | 0.10.2-amzn-13 | The tez YARN application and libraries for worker nodes. | | zeppelin-server | 0.11.1 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.9.2-amzn-0 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.9.2-amzn-0 | ZooKeeper command line client. | ## 7.6.0 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). Reconfiguration actions occur when you specify a configuration for instance groups in a running cluster. Amazon EMR only initiates reconfiguration actions for the classifications that you modify. For more information, see [Reconfigure an instance group in a running cluster](emr-configure-apps-running-cluster.md). **emr-7.6.0 classifications** | Classifications | Description | Reconfiguration Actions | | --- | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | Restarts the ResourceManager service. | | container-executor | Change values in Hadoop YARN's container-executor.cfg file. | Not available. | | container-log4j | Change values in Hadoop YARN's container-log4j.properties file. | Not available. | | core-site | Change values in Hadoop's core-site.xml file. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts Hadoop KMS, Ranger KMS, HiveServer2, Hive MetaStore, Hadoop Httpfs, and MapReduce-HistoryServer. | | docker-conf | Change docker related settings. | Not available. | | emrfs-site | Change EMRFS settings. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts HBaseRegionserver, HBaseMaster, HBaseThrift, HBaseRest, HiveServer2, Hive MetaStore, Hadoop Httpfs, and MapReduce-HistoryServer. | | flink-conf | Change flink-conf.yaml settings. | Restarts Flink history server. | | flink-log4j | Change Flink log4j.properties settings. | Restarts Flink history server. | | flink-log4j-session | Change Flink log4j-session.properties settings for Kubernetes/Yarn session. | Restarts Flink history server. | | flink-log4j-cli | Change Flink log4j-cli.properties settings. | Restarts Flink history server. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts PhoenixQueryserver, HiveServer2, Hive MetaStore, and MapReduce-HistoryServer. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | Restarts the Hadoop HDFS services SecondaryNamenode, Datanode, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts Hadoop KMS, Hadoop Httpfs, and MapReduce-HistoryServer. | | hadoop-ssl-server | Change hadoop ssl server configuration | Not available. | | hadoop-ssl-client | Change hadoop ssl client configuration | Not available. | | hbase | Amazon EMR-curated settings for Apache HBase. | Custom EMR specific property. Sets emrfs-site and hbase-site configs. See those for their associated restarts. | | hbase-env | Change values in HBase's environment. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbase.properties file. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | Not available. | | hbase-site | Change values in HBase's hbase-site.xml file. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. Additionally restarts Phoenix QueryServer. | | hdfs-encryption-zones | Configure HDFS encryption zones. | This classification should not be reconfigured. | | hdfs-env | Change values in the HDFS environment. | Restarts Hadoop HDFS services Namenode, Datanode, and ZKFC. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Additionally restarts Hadoop Httpfs. | | hcatalog-env | Change values in HCatalog's environment. | Restarts Hive HCatalog Server. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | Restarts Hive HCatalog Server. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | Restarts Hive HCatalog Server. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | Restarts Hive WebHCat server. | | hcatalog-webhcat-log4j2 | Change values in HCatalog WebHCat's log4j2.properties. | Restarts Hive WebHCat server. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | Restarts Hive WebHCat server. | | hive | Amazon EMR-curated settings for Apache Hive. | Sets configurations to launch Hive LLAP service. | | hive-beeline-log4j2 | Change values in Hive's beeline-log4j2.properties file. | Not available. | | hive-parquet-logging | Change values in Hive's parquet-logging.properties file. | Not available. | | hive-env | Change values in the Hive environment. | Restarts HiveServer2, HiveMetastore, and Hive HCatalog-Server. Runs Hive schemaTool CLI commands to verify hive-metastore. | | hive-exec-log4j2 | Change values in Hive's hive-exec-log4j2.properties file. | Not available. | | hive-llap-daemon-log4j2 | Change values in Hive's llap-daemon-log4j2.properties file. | Not available. | | hive-log4j2 | Change values in Hive's hive-log4j2.properties file. | Not available. | | hive-site | Change values in Hive's hive-site.xml file | Restarts HiveServer2, HiveMetastore, and Hive HCatalog-Server. Runs Hive schemaTool CLI commands to verify hive-metastore. Also restarts Oozie and Zeppelin. | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | Not available. | | hue-ini | Change values in Hue's ini file | Restarts Hue. Also activates Hue config override CLI commands to pick up new configurations. | | httpfs-env | Change values in the HTTPFS environment. | Restarts Hadoop Httpfs service. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | Restarts Hadoop Httpfs service. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | Not available. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | Restarts Hadoop-KMS service. | | hadoop-kms-java-home | Change Hadoop's KMS java home | Not available. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | Not available. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | Restarts Hadoop-KMS and Ranger-KMS service. | | hudi-env | Change values in the Hudi environment. | Not available. | | hudi-defaults | Change values in Hudi's hudi-defaults.conf file. | Not available. | | iceberg-defaults | Change values in Iceberg's iceberg-defaults.conf file. | Not available. | | delta-defaults | Change values in Delta's delta-defaults.conf file. | Not available. | | jupyter-notebook-conf | Change values in Jupyter Notebook's jupyter\$1notebook\$1config.py file. | Not available. | | jupyter-hub-conf | Change values in JupyterHubs's jupyterhub\$1config.py file. | Not available. | | jupyter-s3-conf | Configure Jupyter Notebook S3 persistence. | Not available. | | jupyter-sparkmagic-conf | Change values in Sparkmagic's config.json file. | Not available. | | livy-conf | Change values in Livy's livy.conf file. | Restarts Livy Server. | | livy-env | Change values in the Livy environment. | Restarts Livy Server. | | livy-log4j2 | Change Livy log4j2.properties settings. | Restarts Livy Server. | | mapred-env | Change values in the MapReduce application's environment. | Restarts Hadoop MapReduce-HistoryServer. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | Restarts Hadoop MapReduce-HistoryServer. | | oozie-env | Change values in Oozie's environment. | Restarts Oozie. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | Restarts Oozie. | | oozie-site | Change values in Oozie's oozie-site.xml file. | Restarts Oozie. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | Not available. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | Not available. | | phoenix-log4j2 | Change values in Phoenix's log4j2.properties file. | Restarts Phoenix-QueryServer. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | Not available. | | pig-env | Change values in the Pig environment. | Not available. | | pig-properties | Change values in Pig's pig.properties file. | Restarts Oozie. | | pig-log4j | Change values in Pig's log4j.properties file. | Not available. | | presto-log | Change values in Presto's log.properties file. | Restarts Presto-Server (for PrestoDB) | | presto-config | Change values in Presto's config.properties file. | Restarts Presto-Server (for PrestoDB) | | presto-password-authenticator | Change values in Presto's password-authenticator.properties file. | Not available. | | presto-env | Change values in Presto's presto-env.sh file. | Restarts Presto-Server (for PrestoDB) | | presto-node | Change values in Presto's node.properties file. | Not available. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | Not available. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | Not available. | | presto-connector-hive | Change values in Presto's hive.properties file. | Restarts Presto-Server (for PrestoDB) | | presto-connector-jmx | Change values in Presto's jmx.properties file. | Not available. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | Not available. | | presto-connector-lakeformation | Change values in Presto's lakeformation.properties file. | Restarts Presto-Server (for PrestoDB) | | presto-connector-localfile | Change values in Presto's localfile.properties file. | Not available. | | presto-connector-memory | Change values in Presto's memory.properties file. | Not available. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | Not available. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | Not available. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | Not available. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | Not available. | | presto-connector-redis | Change values in Presto's redis.properties file. | Not available. | | presto-connector-redshift | Change values in Presto's redshift.properties file. | Not available. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | Not available. | | presto-connector-tpcds | Change values in Presto's tpcds.properties file. | Not available. | | trino-log | Change values in Trino's log.properties file. | Restarts Trino-Server (for Trino) | | trino-config | Change values in Trino's config.properties file. | Restarts Trino-Server (for Trino) | | trino-password-authenticator | Change values in Trino's password-authenticator.properties file. | Restarts Trino-Server (for Trino) | | trino-env | Change values in Trino's trino-env.sh file. | Restarts Trino-Server (for Trino) | | trino-node | Change values in Trino's node.properties file. | Not available. | | trino-connector-blackhole | Change values in Trino's blackhole.properties file. | Not available. | | trino-connector-cassandra | Change values in Trino's cassandra.properties file. | Not available. | | trino-connector-delta | Change values in Trino's delta.properties file. | Restarts Trino-Server (for Trino) | | trino-connector-hive | Change values in Trino's hive.properties file. | Restarts Trino-Server (for Trino) | | trino-exchange-manager | Change values in Trino's exchange-manager.properties file. | Restarts Trino-Server (for Trino) | | trino-connector-iceberg | Change values in Trino's iceberg.properties file. | Restarts Trino-Server (for Trino) | | trino-connector-hudi | Change values in Trino's hudi.properties file. | Restarts Trino-Server (for Trino) | | trino-connector-jmx | Change values in Trino's jmx.properties file. | Not available. | | trino-connector-kafka | Change values in Trino's kafka.properties file. | Not available. | | trino-connector-localfile | Change values in Trino's localfile.properties file. | Not available. | | trino-connector-memory | Change values in Trino's memory.properties file. | Not available. | | trino-connector-mongodb | Change values in Trino's mongodb.properties file. | Not available. | | trino-connector-mysql | Change values in Trino's mysql.properties file. | Not available. | | trino-connector-postgresql | Change values in Trino's postgresql.properties file. | Not available. | | trino-connector-raptor | Change values in Trino's raptor.properties file. | Not available. | | trino-connector-redis | Change values in Trino's redis.properties file. | Not available. | | trino-connector-redshift | Change values in Trino's redshift.properties file. | Not available. | | trino-connector-tpch | Change values in Trino's tpch.properties file. | Not available. | | trino-connector-tpcds | Change values in Trino's tpcds.properties file. | Not available. | | ranger-kms-dbks-site | Change values in dbks-site.xml file of Ranger KMS. | Restarts Ranger KMS Server. | | ranger-kms-site | Change values in ranger-kms-site.xml file of Ranger KMS. | Restarts Ranger KMS Server. | | ranger-kms-env | Change values in the Ranger KMS environment. | Restarts Ranger KMS Server. | | ranger-kms-logback | Change values in kms-logback.xml file of Ranger KMS. | Not available. | | ranger-kms-db-ca | Change values for CA file on S3 for MySQL SSL connection with Ranger KMS. | Not available. | | spark | Amazon EMR-curated settings for Apache Spark. | This property modifies spark-defaults. See actions there. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | Restarts Spark history server and Spark thrift server. | | spark-env | Change values in the Spark environment. | Restarts Spark history server and Spark thrift server. | | spark-hive-site | Change values in Spark's hive-site.xml file | Not available. | | spark-log4j2 | Change values in Spark's log4j2.properties file. | Restarts Spark history server and Spark thrift server. | | spark-metrics | Change values in Spark's metrics.properties file. | Restarts Spark history server and Spark thrift server. | | tez-site | Change values in Tez's tez-site.xml file. | Restart Oozie and HiveServer2. | | yarn-env | Change values in the YARN environment. | Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts MapReduce-HistoryServer. | | yarn-site | Change values in YARN's yarn-site.xml file. | Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts Livy Server and MapReduce-HistoryServer. | | zeppelin-env | Change values in the Zeppelin environment. | Restarts Zeppelin. | | zeppelin-site | Change configuration settings in zeppelin-site.xml. | Restarts Zeppelin. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | Restarts Zookeeper server. | | zookeeper-logback | Change values in ZooKeeper's logback.xml file. | Restarts Zookeeper server. | | emr-metrics | Change emr metric settings for this node. | Restarts the CloudWatchAgent service. | ## 7.6.0 change log **Change log for 7.6.0 release and release notes** | Date | Event | Description | | --- | --- | --- | | 2025-01-13 | Docs publication | Amazon EMR 7.6.0 release notes first published | | 2025-01-10 | Deployment complete | Amazon EMR 7.6.0 fully deployed to all [supported Regions](https://aws.amazon.com/about-aws/global-infrastructure/regional-product-services/) | | 2024-12-26 | Initial release | Amazon EMR 7.6.0 first deployed to initial commercial Regions | # Amazon EMR release 7.5.0 ## 7.5.0 supported lifecycle The following table describes the supported lifecycle dates for Amazon EMR release 7.5.0. | Support phase | Date | | --- | --- | | Initial release date | November 21, 2024 | | Standard support until | November 20, 2026 | | End of support | November 21, 2026 | | End of life | November 21, 2027 | ## 7.5.0 application versions This release includes the following applications: [https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/Install-CloudWatch-Agent.html](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/Install-CloudWatch-Agent.html), [https://delta.io/](https://delta.io/), [https://flink.apache.org/](https://flink.apache.org/), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [https://hudi.apache.org](https://hudi.apache.org), [http://gethue.com/](http://gethue.com/), [https://iceberg.apache.org/](https://iceberg.apache.org/), [https://jupyter-enterprise-gateway.readthedocs.io/en/latest/](https://jupyter-enterprise-gateway.readthedocs.io/en/latest/), [https://jupyterhub.readthedocs.io/en/latest/#](https://jupyterhub.readthedocs.io/en/latest/#), [https://livy.incubator.apache.org/](https://livy.incubator.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [https://www.tensorflow.org/](https://www.tensorflow.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://trino.io/](https://trino.io/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-7.5.0 | emr-7.4.0 | emr-7.3.0 | emr-7.2.0 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 2.28.8, 1.12.772 | 2.28.8, 1.12.772 | 2.25.70, 1.12.747 | 2.23.18, 1.12.705 | | Python | 3.9, 3.11 | 3.9, 3.11 | 3.9, 3.11 | 3.9, 3.11 | | Scala | 2.12.18 | 2.12.18 | 2.12.18 | 2.12.17 | | AmazonCloudWatchAgent | 1.300032.2-amzn-0 | 1.300032.2-amzn-0 | 1.300032.2-amzn-0 | 1.300032.2-amzn-0 | | Delta | 3.2.1-amzn-0 | 3.2.0-amzn-1 | 3.2.0-amzn-0 | 3.1.0 | | Flink | 1.19.1-amzn-1 | 1.19.1-amzn-0 | 1.18.1-amzn-2 | 1.18.1-amzn-1 | | Ganglia | - | - | - | - | | HBase | 2.5.10-amzn-0 | 2.5.5-amzn-0 | 2.4.17-amzn-7 | 2.4.17-amzn-6 | | HCatalog | 3.1.3-amzn-14 | 3.1.3-amzn-13 | 3.1.3-amzn-12 | 3.1.3-amzn-11 | | Hadoop | 3.4.0-amzn-1 | 3.4.0-amzn-0 | 3.3.6-amzn-5 | 3.3.6-amzn-4 | | Hive | 3.1.3-amzn-14 | 3.1.3-amzn-13 | 3.1.3-amzn-12 | 3.1.3-amzn-11 | | Hudi | 0.15.0-amzn-2 | 0.15.0-amzn-1 | 0.15.0-amzn-0 | 0.14.1-amzn-1 | | Hue | 4.11.0 | 4.11.0 | 4.11.0 | 4.11.0 | | Iceberg | 1.6.1-amzn-1 | 1.6.1-amzn-0 | 1.5.2-amzn-0 | 1.5.0-amzn-0 | | JupyterEnterpriseGateway | 2.6.0 | 2.6.0 | 2.6.0 | 2.6.0 | | JupyterHub | 1.5.0 | 1.5.0 | 1.5.0 | 1.5.0 | | Livy | 0.8.0-incubating | 0.8.0-incubating | 0.8.0-incubating | 0.8.0-incubating | | MXNet | - | - | - | 1.9.1 | | Mahout | - | - | - | - | | Oozie | 5.2.1 | 5.2.1 | 5.2.1 | 5.2.1 | | Phoenix | 5.2.0 | 5.2.0 | 5.1.3 | 5.1.3 | | Pig | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | | Presto | 0.287-amzn-0 | 0.287-amzn-0 | 0.285-amzn-1 | 0.285-amzn-0 | | Spark | 3.5.2-amzn-1 | 3.5.2-amzn-0 | 3.5.1-amzn-1 | 3.5.1-amzn-0 | | Sqoop | - | 1.4.7 | 1.4.7 | 1.4.7 | | TensorFlow | 2.16.1 | 2.16.1 | 2.16.1 | 2.11.0 | | Tez | 0.10.2-amzn-12 | 0.10.2-amzn-11 | 0.10.2-amzn-10 | 0.10.2-amzn-9 | | Trino (PrestoSQL) | 446-amzn-0 | 446-amzn-0 | 442-amzn-0 | 436-amzn-0 | | Zeppelin | 0.11.1 | 0.11.1 | 0.11.1 | 0.10.1 | | ZooKeeper | 3.9.2-amzn-0 | 3.9.2-amzn-0 | 3.9.1-amzn-1 | 3.9.1-amzn-1 | # 7.5.0 release components details See the following table for more information about the *Extras* packages in Amazon EMR 7.5.0. | Application | Application Type | Package | Package Type | Package Version | | --- | --- | --- | --- | --- | | AmazonCloudWatchAgent | Core | hadoop-hdfs-journalnode | Extras | 3.3.6.amzn.5-1.amzn2023 | | AmazonCloudWatchAgent | Core | ranger-kms | Extras | 2.0.0-1.amzn2023 | | AmazonCloudWatchAgent | Core | s3-dist-cp | Extras | 2.33.0-1.amzn2023 | | AmazonCloudWatchAgent | Core | zookeeper | Extras | 3.9.1.amzn.1-1.amzn2023 | | AmazonCloudWatchAgent | Core | zookeeper-server | Extras | 3.9.1.amzn.1-1.amzn2023 | | Flink | Core | hadoop-client | Extras | 3.3.6.amzn.5-1.amzn2023 | | Flink | Core | hadoop-hdfs | Extras | 3.3.6.amzn.5-1.amzn2023 | | Flink | Core | hadoop-hdfs-datanode | Extras | 3.3.6.amzn.5-1.amzn2023 | | Flink | Core | hadoop-hdfs-journalnode | Extras | 3.3.6.amzn.5-1.amzn2023 | | Flink | Core | hadoop-hdfs-namenode | Extras | 3.3.6.amzn.5-1.amzn2023 | | Flink | Core | hadoop-httpfs | Extras | 3.3.6.amzn.5-1.amzn2023 | | Flink | Core | hadoop-kms | Extras | 3.3.6.amzn.5-1.amzn2023 | | Flink | Core | hadoop-mapreduce | Extras | 3.3.6.amzn.5-1.amzn2023 | | Flink | Core | hadoop-mapreduce-historyserver | Extras | 3.3.6.amzn.5-1.amzn2023 | | Flink | Core | hadoop-yarn | Extras | 3.3.6.amzn.5-1.amzn2023 | | Flink | Core | hadoop-yarn-nodemanager | Extras | 3.3.6.amzn.5-1.amzn2023 | | Flink | Core | hadoop-yarn-proxyserver | Extras | 3.3.6.amzn.5-1.amzn2023 | | Flink | Core | hadoop-yarn-resourcemanager | Extras | 3.3.6.amzn.5-1.amzn2023 | | Flink | Core | hadoop-yarn-timelineserver | Extras | 3.3.6.amzn.5-1.amzn2023 | | Flink | Core | hudi | Extras | 0.15.0.amzn.0-1.amzn2023 | | Flink | Core | ranger-kms | Extras | 2.0.0-1.amzn2023 | | Flink | Core | hadoop | Extras | 3.3.6.amzn.5-1.amzn2023 | | HBase | Core | s3-dist-cp | Extras | 2.33.0-1.amzn2023 | | HCatalog | Core | hadoop-yarn-proxyserver | Extras | 3.3.6.amzn.5-1.amzn2023 | | HCatalog | Core | mariadb-connector-java | Extras | | | Hive | Core | hadoop-httpfs | Extras | 3.3.6.amzn.5-1.amzn2023 | | Hive | Core | hadoop-yarn-proxyserver | Extras | 3.3.6.amzn.5-1.amzn2023 | | Hive | Core | hive-hbase | Extras | 3.1.3.amzn.12-1.amzn2023 | | Hive | Core | mariadb-connector-java | Extras | 2.7.2-1 | | Hive | Core | s3-dist-cp | Extras | 2.33.0-1.amzn2023 | | Phoenix | Core | s3-dist-cp | Extras | 2.33.0-1.amzn2023 | | Spark | Core | spark-datanucleus | Extras | 3.5.1.amzn.0-1.amzn2023 | | Tez | Core | hadoop-mapreduce | Extras | 3.3.6.amzn.5-1.amzn2023 | | Tez | Core | hadoop-mapreduce-historyserver | Extras | 3.3.6.amzn.5-1.amzn2023 | | Tez | Core | hadoop-yarn-proxyserver | Extras | 3.3.6.amzn.5-1.amzn2023 | | Trino | Core | bigtop-utils | Extras | 1.2.0-1.amzn2023 | | Trino | Core | hadoop-client | Extras | 3.3.6.amzn.5-1.amzn2023 | | Trino | Core | hadoop-hdfs | Extras | 3.3.6.amzn.5-1.amzn2023 | | Trino | Core | hadoop-hdfs-datanode | Extras | 3.3.6.amzn.5-1.amzn2023 | | Trino | Core | hadoop-hdfs-journalnode | Extras | 3.3.6.amzn.5-1.amzn2023 | | Trino | Core | hadoop-hdfs-namenode | Extras | 3.3.6.amzn.5-1.amzn2023 | | Trino | Core | hadoop-kms | Extras | 3.3.6.amzn.5-1.amzn2023 | | Trino | Core | hadoop-mapreduce | Extras | 3.3.6.amzn.5-1.amzn2023 | | Trino | Core | hadoop-mapreduce-historyserver | Extras | 3.3.6.amzn.5-1.amzn2023 | | Trino | Core | hadoop-yarn | Extras | 3.3.6.amzn.5-1.amzn2023 | | Trino | Core | hadoop-yarn-nodemanager | Extras | 3.3.6.amzn.5-1.amzn2023 | | Trino | Core | hadoop-yarn-proxyserver | Extras | 3.3.6.amzn.5-1.amzn2023 | | Trino | Core | hadoop-yarn-resourcemanager | Extras | 3.3.6.amzn.5-1.amzn2023 | | Trino | Core | hadoop-yarn-timelineserver | Extras | 3.3.6.amzn.5-1.amzn2023 | | Trino | Core | hive | Extras | 3.1.3.amzn.12-1.amzn2023 | | Trino | Core | hive-hcatalog-server | Extras | 3.1.3.amzn.12-1.amzn2023 | | Trino | Core | mariadb-connector-java | Extras | 2.7.2-1 | | Trino | Core | ranger-kms | Extras | 2.0.0-1.amzn2023 | | Trino | Core | zookeeper | Extras | 3.9.1.amzn.1-1.amzn2023 | | Trino | Core | zookeeper-server | Extras | 3.9.1.amzn.1-1.amzn2023 | | Trino | Core | bigtop-groovy | Extras | 2.5.4-1.amzn2023 | | Trino | Core | bigtop-jsvc | Extras | 1.2.4-1.amzn2023 | | Trino | Core | hadoop | Extras | 3.3.6.amzn.5-1.amzn2023 | | Trino | Core | hadoop-lzo | Extras | 0.4.19-1.amzn2023 | | Trino | Core | hive-hcatalog | Extras | 3.1.3.amzn.12-1.amzn2023 | | Trino | Core | hive-jdbc | Extras | 3.1.3.amzn.12-1.amzn2023 | | Zookeeper | Core | emrfs | Extras | 2.64.0-1.amzn2023 | | Zookeeper | Core | hadoop-client | Extras | 3.3.6.amzn.5-1.amzn2023 | | Zookeeper | Core | hadoop-hdfs | Extras | 3.3.6.amzn.5-1.amzn2023 | | Zookeeper | Core | hadoop-hdfs-datanode | Extras | 3.3.6.amzn.5-1.amzn2023 | | Zookeeper | Core | hadoop-hdfs-journalnode | Extras | 3.3.6.amzn.5-1.amzn2023 | | Zookeeper | Core | hadoop-hdfs-namenode | Extras | 3.3.6.amzn.5-1.amzn2023 | | Zookeeper | Core | hadoop-httpfs | Extras | 3.3.6.amzn.5-1.amzn2023 | | Zookeeper | Core | hadoop-kms | Extras | 3.3.6.amzn.5-1.amzn2023 | | Zookeeper | Core | hadoop-mapreduce | Extras | 3.3.6.amzn.5-1.amzn2023 | | Zookeeper | Core | hadoop-mapreduce-historyserver | Extras | 3.3.6.amzn.5-1.amzn2023 | | Zookeeper | Core | hadoop-yarn | Extras | 3.3.6.amzn.5-1.amzn2023 | | Zookeeper | Core | hadoop-yarn-nodemanager | Extras | 3.3.6.amzn.5-1.amzn2023 | | Zookeeper | Core | hadoop-yarn-proxyserver | Extras | 3.3.6.amzn.5-1.amzn2023 | | Zookeeper | Core | hadoop-yarn-resourcemanager | Extras | 3.3.6.amzn.5-1.amzn2023 | | Zookeeper | Core | hadoop-yarn-timelineserver | Extras | 3.3.6.amzn.5-1.amzn2023 | | Zookeeper | Core | ranger-kms | Extras | 2.0.0-1.amzn2023 | # 7.5.0 common vulnerabilities and exposures The following table lists all CVEs that don't impact EMR clusters that run on recommended configurations of Amazon EMR 7.5.0. Amazon EMR is dependent on upstream open-source for availability of fixes and will provide the latest stable version as part of the Core Engine components within 90 days of Amazon EMR verifying the fixes. This table was updated on November 23, 2021. | CVE ID | Severity | CVE details URL | | --- | --- | --- | | CVE-2013-4002 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2013-4002](https://nvd.nist.gov/vuln/detail/CVE-2013-4002) | | CVE-2015-1832 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2015-1832](https://nvd.nist.gov/vuln/detail/CVE-2015-1832) | | CVE-2018-12022 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-12022](https://nvd.nist.gov/vuln/detail/CVE-2018-12022) | | CVE-2018-14718 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-14718](https://nvd.nist.gov/vuln/detail/CVE-2018-14718) | | CVE-2019-12384 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-12384](https://nvd.nist.gov/vuln/detail/CVE-2019-12384) | | CVE-2019-14379 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-14379](https://nvd.nist.gov/vuln/detail/CVE-2019-14379) | | CVE-2019-14439 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-14439](https://nvd.nist.gov/vuln/detail/CVE-2019-14439) | | CVE-2020-10672 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2020-10672](https://nvd.nist.gov/vuln/detail/CVE-2020-10672) | | CVE-2020-10673 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2020-10673](https://nvd.nist.gov/vuln/detail/CVE-2020-10673) | | CVE-2020-11112 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2020-11112](https://nvd.nist.gov/vuln/detail/CVE-2020-11112) | | CVE-2020-13949 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2020-13949](https://nvd.nist.gov/vuln/detail/CVE-2020-13949) | | CVE-2020-14195 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2020-14195](https://nvd.nist.gov/vuln/detail/CVE-2020-14195) | | CVE-2020-17521 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2020-17521](https://nvd.nist.gov/vuln/detail/CVE-2020-17521) | | CVE-2020-24616 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2020-24616](https://nvd.nist.gov/vuln/detail/CVE-2020-24616) | | CVE-2020-24750 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2020-24750](https://nvd.nist.gov/vuln/detail/CVE-2020-24750) | | CVE-2020-25649 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2020-25649](https://nvd.nist.gov/vuln/detail/CVE-2020-25649) | | CVE-2020-36183 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2020-36183](https://nvd.nist.gov/vuln/detail/CVE-2020-36183) | | CVE-2020-36186 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2020-36186](https://nvd.nist.gov/vuln/detail/CVE-2020-36186) | | CVE-2020-36188 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2020-36188](https://nvd.nist.gov/vuln/detail/CVE-2020-36188) | | CVE-2020-36189 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2020-36189](https://nvd.nist.gov/vuln/detail/CVE-2020-36189) | | CVE-2020-8908 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2020-8908](https://nvd.nist.gov/vuln/detail/CVE-2020-8908) | | CVE-2021-37137 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2021-37137](https://nvd.nist.gov/vuln/detail/CVE-2021-37137) | | CVE-2021-38153 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2021-38153](https://nvd.nist.gov/vuln/detail/CVE-2021-38153) | | CVE-2022-25857 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2022-25857](https://nvd.nist.gov/vuln/detail/CVE-2022-25857) | | CVE-2022-3509 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2022-3509](https://nvd.nist.gov/vuln/detail/CVE-2022-3509) | | CVE-2022-37866 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2022-37866](https://nvd.nist.gov/vuln/detail/CVE-2022-37866) | | CVE-2022-41966 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2022-41966](https://nvd.nist.gov/vuln/detail/CVE-2022-41966) | | CVE-2022-45688 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2022-45688](https://nvd.nist.gov/vuln/detail/CVE-2022-45688) | | CVE-2023-34455 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2023-34455](https://nvd.nist.gov/vuln/detail/CVE-2023-34455) | | CVE-2023-3635 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2023-3635](https://nvd.nist.gov/vuln/detail/CVE-2023-3635) | | CVE-2023-38647 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2023-38647](https://nvd.nist.gov/vuln/detail/CVE-2023-38647) | | CVE-2023-40167 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2023-40167](https://nvd.nist.gov/vuln/detail/CVE-2023-40167) | | CVE-2023-43642 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2023-43642](https://nvd.nist.gov/vuln/detail/CVE-2023-43642) | | CVE-2023-44981 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2023-44981](https://nvd.nist.gov/vuln/detail/CVE-2023-44981) | | CVE-2023-50387 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2023-50387](https://nvd.nist.gov/vuln/detail/CVE-2023-50387) | | CVE-2024-23454 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2024-23454](https://nvd.nist.gov/vuln/detail/CVE-2024-23454) | | CVE-2024-29025 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2024-29025](https://nvd.nist.gov/vuln/detail/CVE-2024-29025) | | CVE-2024-36124 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2024-36124](https://nvd.nist.gov/vuln/detail/CVE-2024-36124) | | CVE-2024-8184 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2024-8184](https://nvd.nist.gov/vuln/detail/CVE-2024-8184) | | CVE-2012-0881 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2012-0881](https://nvd.nist.gov/vuln/detail/CVE-2012-0881) | | CVE-2012-5783 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2012-5783](https://nvd.nist.gov/vuln/detail/CVE-2012-5783) | | CVE-2015-5237 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2015-5237](https://nvd.nist.gov/vuln/detail/CVE-2015-5237) | | CVE-2016-5018 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2016-5018](https://nvd.nist.gov/vuln/detail/CVE-2016-5018) | | CVE-2016-6796 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2016-6796](https://nvd.nist.gov/vuln/detail/CVE-2016-6796) | | CVE-2017-15095 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2017-15095](https://nvd.nist.gov/vuln/detail/CVE-2017-15095) | | CVE-2018-1000180 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2018-1000180](https://nvd.nist.gov/vuln/detail/CVE-2018-1000180) | | CVE-2018-10054 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2018-10054](https://nvd.nist.gov/vuln/detail/CVE-2018-10054) | | CVE-2018-10237 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2018-10237](https://nvd.nist.gov/vuln/detail/CVE-2018-10237) | | CVE-2018-11798 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2018-11798](https://nvd.nist.gov/vuln/detail/CVE-2018-11798) | | CVE-2018-12023 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2018-12023](https://nvd.nist.gov/vuln/detail/CVE-2018-12023) | | CVE-2018-14721 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2018-14721](https://nvd.nist.gov/vuln/detail/CVE-2018-14721) | | CVE-2018-19361 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2018-19361](https://nvd.nist.gov/vuln/detail/CVE-2018-19361) | | CVE-2018-5968 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2018-5968](https://nvd.nist.gov/vuln/detail/CVE-2018-5968) | | CVE-2018-7489 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2018-7489](https://nvd.nist.gov/vuln/detail/CVE-2018-7489) | | CVE-2019-0205 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2019-0205](https://nvd.nist.gov/vuln/detail/CVE-2019-0205) | | CVE-2019-12086 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2019-12086](https://nvd.nist.gov/vuln/detail/CVE-2019-12086) | | CVE-2019-12814 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2019-12814](https://nvd.nist.gov/vuln/detail/CVE-2019-12814) | | CVE-2019-14540 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2019-14540](https://nvd.nist.gov/vuln/detail/CVE-2019-14540) | | CVE-2019-14892 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2019-14892](https://nvd.nist.gov/vuln/detail/CVE-2019-14892) | | CVE-2019-14893 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2019-14893](https://nvd.nist.gov/vuln/detail/CVE-2019-14893) | | CVE-2019-16869 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2019-16869](https://nvd.nist.gov/vuln/detail/CVE-2019-16869) | | CVE-2019-16942 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2019-16942](https://nvd.nist.gov/vuln/detail/CVE-2019-16942) | | CVE-2019-17267 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2019-17267](https://nvd.nist.gov/vuln/detail/CVE-2019-17267) | | CVE-2019-17531 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2019-17531](https://nvd.nist.gov/vuln/detail/CVE-2019-17531) | | CVE-2019-20330 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2019-20330](https://nvd.nist.gov/vuln/detail/CVE-2019-20330) | | CVE-2019-20444 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2019-20444](https://nvd.nist.gov/vuln/detail/CVE-2019-20444) | | CVE-2019-20445 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2019-20445](https://nvd.nist.gov/vuln/detail/CVE-2019-20445) | | CVE-2020-10969 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-10969](https://nvd.nist.gov/vuln/detail/CVE-2020-10969) | | CVE-2020-11111 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-11111](https://nvd.nist.gov/vuln/detail/CVE-2020-11111) | | CVE-2020-11619 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-11619](https://nvd.nist.gov/vuln/detail/CVE-2020-11619) | | CVE-2020-11620 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-11620](https://nvd.nist.gov/vuln/detail/CVE-2020-11620) | | CVE-2020-13955 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-13955](https://nvd.nist.gov/vuln/detail/CVE-2020-13955) | | CVE-2020-13956 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-13956](https://nvd.nist.gov/vuln/detail/CVE-2020-13956) | | CVE-2020-14060 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-14060](https://nvd.nist.gov/vuln/detail/CVE-2020-14060) | | CVE-2020-14061 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-14061](https://nvd.nist.gov/vuln/detail/CVE-2020-14061) | | CVE-2020-15250 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-15250](https://nvd.nist.gov/vuln/detail/CVE-2020-15250) | | CVE-2020-26939 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-26939](https://nvd.nist.gov/vuln/detail/CVE-2020-26939) | | CVE-2020-35491 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-35491](https://nvd.nist.gov/vuln/detail/CVE-2020-35491) | | CVE-2020-35728 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-35728](https://nvd.nist.gov/vuln/detail/CVE-2020-35728) | | CVE-2020-36179 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36179](https://nvd.nist.gov/vuln/detail/CVE-2020-36179) | | CVE-2020-36181 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36181](https://nvd.nist.gov/vuln/detail/CVE-2020-36181) | | CVE-2020-36182 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36182](https://nvd.nist.gov/vuln/detail/CVE-2020-36182) | | CVE-2020-36184 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36184](https://nvd.nist.gov/vuln/detail/CVE-2020-36184) | | CVE-2020-36185 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36185](https://nvd.nist.gov/vuln/detail/CVE-2020-36185) | | CVE-2020-36187 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36187](https://nvd.nist.gov/vuln/detail/CVE-2020-36187) | | CVE-2020-36518 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36518](https://nvd.nist.gov/vuln/detail/CVE-2020-36518) | | CVE-2020-8840 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-8840](https://nvd.nist.gov/vuln/detail/CVE-2020-8840) | | CVE-2020-9546 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-9546](https://nvd.nist.gov/vuln/detail/CVE-2020-9546) | | CVE-2020-9547 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-9547](https://nvd.nist.gov/vuln/detail/CVE-2020-9547) | | CVE-2020-9548 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-9548](https://nvd.nist.gov/vuln/detail/CVE-2020-9548) | | CVE-2021-20190 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-20190](https://nvd.nist.gov/vuln/detail/CVE-2021-20190) | | CVE-2021-21290 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-21290](https://nvd.nist.gov/vuln/detail/CVE-2021-21290) | | CVE-2021-22569 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-22569](https://nvd.nist.gov/vuln/detail/CVE-2021-22569) | | CVE-2021-28170 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-28170](https://nvd.nist.gov/vuln/detail/CVE-2021-28170) | | CVE-2021-29425 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-29425](https://nvd.nist.gov/vuln/detail/CVE-2021-29425) | | CVE-2021-31684 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-31684](https://nvd.nist.gov/vuln/detail/CVE-2021-31684) | | CVE-2021-34538 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-34538](https://nvd.nist.gov/vuln/detail/CVE-2021-34538) | | CVE-2021-36374 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-36374](https://nvd.nist.gov/vuln/detail/CVE-2021-36374) | | CVE-2021-37533 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-37533](https://nvd.nist.gov/vuln/detail/CVE-2021-37533) | | CVE-2021-43859 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-43859](https://nvd.nist.gov/vuln/detail/CVE-2021-43859) | | CVE-2022-1415 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-1415](https://nvd.nist.gov/vuln/detail/CVE-2022-1415) | | CVE-2022-1471 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-1471](https://nvd.nist.gov/vuln/detail/CVE-2022-1471) | | CVE-2022-3171 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-3171](https://nvd.nist.gov/vuln/detail/CVE-2022-3171) | | CVE-2022-3510 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-3510](https://nvd.nist.gov/vuln/detail/CVE-2022-3510) | | CVE-2022-36364 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-36364](https://nvd.nist.gov/vuln/detail/CVE-2022-36364) | | CVE-2022-37865 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-37865](https://nvd.nist.gov/vuln/detail/CVE-2022-37865) | | CVE-2022-38749 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-38749](https://nvd.nist.gov/vuln/detail/CVE-2022-38749) | | CVE-2022-38751 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-38751](https://nvd.nist.gov/vuln/detail/CVE-2022-38751) | | CVE-2022-40150 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-40150](https://nvd.nist.gov/vuln/detail/CVE-2022-40150) | | CVE-2022-40151 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-40151](https://nvd.nist.gov/vuln/detail/CVE-2022-40151) | | CVE-2022-40152 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-40152](https://nvd.nist.gov/vuln/detail/CVE-2022-40152) | | CVE-2022-41854 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-41854](https://nvd.nist.gov/vuln/detail/CVE-2022-41854) | | CVE-2022-42003 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-42003](https://nvd.nist.gov/vuln/detail/CVE-2022-42003) | | CVE-2022-42889 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-42889](https://nvd.nist.gov/vuln/detail/CVE-2022-42889) | | CVE-2022-45685 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-45685](https://nvd.nist.gov/vuln/detail/CVE-2022-45685) | | CVE-2022-45693 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-45693](https://nvd.nist.gov/vuln/detail/CVE-2022-45693) | | CVE-2022-46337 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-46337](https://nvd.nist.gov/vuln/detail/CVE-2022-46337) | | CVE-2023-0833 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-0833](https://nvd.nist.gov/vuln/detail/CVE-2023-0833) | | CVE-2023-1370 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-1370](https://nvd.nist.gov/vuln/detail/CVE-2023-1370) | | CVE-2023-1436 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-1436](https://nvd.nist.gov/vuln/detail/CVE-2023-1436) | | CVE-2023-20863 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-20863](https://nvd.nist.gov/vuln/detail/CVE-2023-20863) | | CVE-2023-25194 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-25194](https://nvd.nist.gov/vuln/detail/CVE-2023-25194) | | CVE-2023-33201 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-33201](https://nvd.nist.gov/vuln/detail/CVE-2023-33201) | | CVE-2023-33202 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-33202](https://nvd.nist.gov/vuln/detail/CVE-2023-33202) | | CVE-2023-34453 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-34453](https://nvd.nist.gov/vuln/detail/CVE-2023-34453) | | CVE-2023-34454 | high | [https://nvd.nist.gov/vuln/detail/CVE-2023-34454](https://nvd.nist.gov/vuln/detail/CVE-2023-34454) | | CVE-2023-39410 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-39410](https://nvd.nist.gov/vuln/detail/CVE-2023-39410) | | CVE-2023-42503 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-42503](https://nvd.nist.gov/vuln/detail/CVE-2023-42503) | | CVE-2023-5072 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-5072](https://nvd.nist.gov/vuln/detail/CVE-2023-5072) | | CVE-2023-50868 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-50868](https://nvd.nist.gov/vuln/detail/CVE-2023-50868) | | CVE-2023-52428 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-52428](https://nvd.nist.gov/vuln/detail/CVE-2023-52428) | | CVE-2023-6378 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-6378](https://nvd.nist.gov/vuln/detail/CVE-2023-6378) | | CVE-2023-6481 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-6481](https://nvd.nist.gov/vuln/detail/CVE-2023-6481) | | CVE-2024-21634 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-21634](https://nvd.nist.gov/vuln/detail/CVE-2024-21634) | | CVE-2024-23944 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-23944](https://nvd.nist.gov/vuln/detail/CVE-2024-23944) | | CVE-2024-29131 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-29131](https://nvd.nist.gov/vuln/detail/CVE-2024-29131) | | CVE-2024-30171 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-30171](https://nvd.nist.gov/vuln/detail/CVE-2024-30171) | | CVE-2024-30172 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-30172](https://nvd.nist.gov/vuln/detail/CVE-2024-30172) | | CVE-2024-34447 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-34447](https://nvd.nist.gov/vuln/detail/CVE-2024-34447) | | CVE-2024-35255 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-35255](https://nvd.nist.gov/vuln/detail/CVE-2024-35255) | | CVE-2024-36114 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-36114](https://nvd.nist.gov/vuln/detail/CVE-2024-36114) | | CVE-2024-38808 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-38808](https://nvd.nist.gov/vuln/detail/CVE-2024-38808) | | CVE-2024-39676 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-39676](https://nvd.nist.gov/vuln/detail/CVE-2024-39676) | | CVE-2024-51504 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-51504](https://nvd.nist.gov/vuln/detail/CVE-2024-51504) | | CVE-2024-7254 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-7254](https://nvd.nist.gov/vuln/detail/CVE-2024-7254) | | CVE-2017-17485 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2017-17485](https://nvd.nist.gov/vuln/detail/CVE-2017-17485) | | CVE-2018-14719 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2018-14719](https://nvd.nist.gov/vuln/detail/CVE-2018-14719) | | CVE-2018-14720 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2018-14720](https://nvd.nist.gov/vuln/detail/CVE-2018-14720) | | CVE-2018-19362 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2018-19362](https://nvd.nist.gov/vuln/detail/CVE-2018-19362) | | CVE-2020-10650 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2020-10650](https://nvd.nist.gov/vuln/detail/CVE-2020-10650) | | CVE-2020-10968 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2020-10968](https://nvd.nist.gov/vuln/detail/CVE-2020-10968) | | CVE-2020-13936 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2020-13936](https://nvd.nist.gov/vuln/detail/CVE-2020-13936) | | CVE-2020-14062 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2020-14062](https://nvd.nist.gov/vuln/detail/CVE-2020-14062) | | CVE-2020-15522 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2020-15522](https://nvd.nist.gov/vuln/detail/CVE-2020-15522) | | CVE-2020-1945 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2020-1945](https://nvd.nist.gov/vuln/detail/CVE-2020-1945) | | CVE-2020-28052 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2020-28052](https://nvd.nist.gov/vuln/detail/CVE-2020-28052) | | CVE-2020-35490 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2020-35490](https://nvd.nist.gov/vuln/detail/CVE-2020-35490) | | CVE-2021-36373 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2021-36373](https://nvd.nist.gov/vuln/detail/CVE-2021-36373) | | CVE-2022-25647 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2022-25647](https://nvd.nist.gov/vuln/detail/CVE-2022-25647) | | CVE-2022-38750 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2022-38750](https://nvd.nist.gov/vuln/detail/CVE-2022-38750) | | CVE-2022-38752 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2022-38752](https://nvd.nist.gov/vuln/detail/CVE-2022-38752) | | CVE-2022-42004 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2022-42004](https://nvd.nist.gov/vuln/detail/CVE-2022-42004) | | CVE-2022-46751 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2022-46751](https://nvd.nist.gov/vuln/detail/CVE-2022-46751) | | CVE-2023-20883 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2023-20883](https://nvd.nist.gov/vuln/detail/CVE-2023-20883) | | CVE-2023-26048 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2023-26048](https://nvd.nist.gov/vuln/detail/CVE-2023-26048) | | CVE-2023-2976 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2023-2976](https://nvd.nist.gov/vuln/detail/CVE-2023-2976) | | CVE-2023-36478 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2023-36478](https://nvd.nist.gov/vuln/detail/CVE-2023-36478) | | CVE-2023-51074 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2023-51074](https://nvd.nist.gov/vuln/detail/CVE-2023-51074) | | CVE-2024-25638 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-25638](https://nvd.nist.gov/vuln/detail/CVE-2024-25638) | | CVE-2024-25710 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-25710](https://nvd.nist.gov/vuln/detail/CVE-2024-25710) | | CVE-2024-26308 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-26308](https://nvd.nist.gov/vuln/detail/CVE-2024-26308) | | CVE-2024-29133 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-29133](https://nvd.nist.gov/vuln/detail/CVE-2024-29133) | | CVE-2024-47554 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-47554](https://nvd.nist.gov/vuln/detail/CVE-2024-47554) | | CVE-2024-47561 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-47561](https://nvd.nist.gov/vuln/detail/CVE-2024-47561) | | CVE-2024-6763 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-6763](https://nvd.nist.gov/vuln/detail/CVE-2024-6763) | | CVE-2012-6153 | LOW | [https://nvd.nist.gov/vuln/detail/CVE-2012-6153](https://nvd.nist.gov/vuln/detail/CVE-2012-6153) | | CVE-2020-11113 | LOW | [https://nvd.nist.gov/vuln/detail/CVE-2020-11113](https://nvd.nist.gov/vuln/detail/CVE-2020-11113) | | CVE-2021-22570 | LOW | [https://nvd.nist.gov/vuln/detail/CVE-2021-22570](https://nvd.nist.gov/vuln/detail/CVE-2021-22570) | | CVE-2021-27568 | LOW | [https://nvd.nist.gov/vuln/detail/CVE-2021-27568](https://nvd.nist.gov/vuln/detail/CVE-2021-27568) | | CVE-2024-29857 | LOW | [https://nvd.nist.gov/vuln/detail/CVE-2024-29857](https://nvd.nist.gov/vuln/detail/CVE-2024-29857) | | CVE-2017-7525 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2017-7525](https://nvd.nist.gov/vuln/detail/CVE-2017-7525) | | CVE-2018-11307 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2018-11307](https://nvd.nist.gov/vuln/detail/CVE-2018-11307) | | CVE-2018-1313 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2018-1313](https://nvd.nist.gov/vuln/detail/CVE-2018-1313) | | CVE-2018-19360 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2018-19360](https://nvd.nist.gov/vuln/detail/CVE-2018-19360) | | CVE-2019-16335 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2019-16335](https://nvd.nist.gov/vuln/detail/CVE-2019-16335) | | CVE-2019-16943 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2019-16943](https://nvd.nist.gov/vuln/detail/CVE-2019-16943) | | CVE-2020-36180 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2020-36180](https://nvd.nist.gov/vuln/detail/CVE-2020-36180) | | CVE-2021-37136 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2021-37136](https://nvd.nist.gov/vuln/detail/CVE-2021-37136) | | CVE-2022-24329 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2022-24329](https://nvd.nist.gov/vuln/detail/CVE-2022-24329) | | CVE-2022-39135 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2022-39135](https://nvd.nist.gov/vuln/detail/CVE-2022-39135) | | CVE-2022-40149 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2022-40149](https://nvd.nist.gov/vuln/detail/CVE-2022-40149) | | CVE-2023-26049 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2023-26049](https://nvd.nist.gov/vuln/detail/CVE-2023-26049) | ## 7.5.0 release notes The following release notes include information for Amazon EMR release 7.5.0. **New features** + **Application upgrades** – Amazon EMR 7.5.0 application upgrades include Delta 3.2.1-amzn-0, HBase 2.5.10, Hudi 0.15.0-amzn-2, and Iceberg 1.6.1-amzn-1. + EMR write-ahead logs (WAL) support cross cluster replication. For more information, see [EMR WAL cross-cluster replication](https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-hbase-wal-cross-cluster.html). **Known issues** + Apache Sqoop has been removed from Amazon EMR Release 7.5 and later releases. + When managed scaling is enabled, the YARN ResourceManager (RM) experiences a critical deadlock issue, causing it to become unresponsive, when transitioning from the **DECOMMISSIONING** to **DECOMMISSIONED** state with simultaneous operations. This happens particularly in large-cluster environments. When it occurs, both node-state transitions and application-state transitions fail to process, effectively paralyzing the core functionality of the ResourceManager. As a workaround, restart the RM JVM using the following command on the primary node: ``` sudo systemctl restart hadoop-yarn-resourcemanager ``` + When an EMR cluster is started with YARN schedulers other than the `CapacityScheduler` or `FairScheduler` (for example, the FIFO Scheduler), the `YARNMemoryAvailablePercentage` metric is not pushed to CloudWatch. This issue impacts down-scaling in managed-scaling enabled clusters. It might also affect custom managed scaling if it depends on the `YARNMemoryAvailablePercentage` metric. As a workaround, use the `CapacityScheduler` or `FairScheduler`. + **HBase table creation in Hive with Glue catalog** – From EMR-7.3.0 to EMR-7.10.0, there is a bug due to Hive Iceberg integration, which causes HBase table creation in Hive to fail when Glue Data Catalog is used as the metastore. Please reach out to the AWS support team if you encounter this issue. + **Tez UI** – From EMR-7.3 to EMR-7.10, there is a bug where accessing Tez UI through YARN Resource Manager Web UI fails when in-transit encryption is enabled. This occurs because YARN proxy attempts to use HTTP protocol while Tez UI requires HTTPS. Please modify the `tez.tez-ui.history-url.base` configuration in `tez-site` from `http` to `https` to resolve this issue. **Changes, enhancements, and resolved issues** + Starting with Spark 3.3.1 (supported in EMR versions 6.10 and above), all executors in a decommissioning host are set to a new `ExecutorState`, called *DECOMMISSIONING* state. The executors being decommissioned cannot be used by Yarn to allocate tasks and thus it will request for new executors, if needed, for the tasks being executed. Thus, if you disable Spark DRA while using EMR Managed Scaling, EMR Auto Scaling, or any custom scaling mechanism on EMR-EC2 clusters, then Yarn may request maximum permissible executors for each job. In order to avoid this issue, leave the `spark.dynamicAllocation.enabled` property set to `TRUE` (which is the default) when you are using the above combination of features. In addition, you can also set minimum and maximum executor constraints by setting values for `spark.dynamicAllocation.maxExecutors` and `spark.dynamicAllocation.minExecutors` properties for your Spark jobs, to restrict the number of executors allocated during the job’s execution. + Amazon EMR uses different builds of the TensorFlow library, depending on the instance types that you choose for your cluster. Amazon EMR also supports TensorFlow for clusters with **aarch64** instance types (Graviton instances) from EMR-7.5.0 and above. The following table lists builds by instance type for EMR-7.5.0: [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-750-release.html) + An AWS Lake Formation with FGAC (Fine-Grained Access Control) enabled Amazon EMR cluster can now support up to 256 concurrent steps with the following hdfs-site.xml configuration: ``` [ { "Classification": "hdfs-site", "Properties": { "dfs.datanode.handler.count": "40", "dfs.datanode.max.transfer.threads": "16384", "dfs.namenode.handler.count": "64", "dfs.replication": "1" } }, { "Classification": "hdfs-env", "Configurations": [ { "Classification": "export", "Properties": { "HADOOP_DATANODE_HEAPSIZE": "5120" } } ], "Properties": {} } ] ``` + [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-750-release.html) ## 7.5.0 default Java versions Amazon EMR release 7.5 and higher ships with Amazon Corretto 17 (built on OpenJDK) by default for applications that support Corretto 17 (JDK 17), with the exception of Apache Livy. The following table shows the default Java versions for applications in Amazon EMR 7.5.0. If you want to change the default JVM on your cluster, follow the instructions in [Configure applications to use a specific Java Virtual Machine](configuring-java8.md) for each application that runs on the cluster. You can only use one Java runtime version for a cluster. Amazon EMR doesn't support running different nodes or applications on different runtime versions on the same cluster. | Application | Java / Amazon Corretto version (default is bold) | | --- | --- | | Delta | 17, 11, 8 | | Flink | 17, 11, 8 | | HBase | 17, 11, 8 | | HCatalog | 17, 11, 8 | | Hadoop | 17, 11, 8 | | Hive | 17, 11, 8 | | Hudi | 17, 11, 8 | | Iceberg | 17, 11, 8 | | Livy | 17, 11, 8 | | Oozie | 17, 11, 8 | | Phoenix | 17, 11, 8 | | PrestoDB | 8 | | Spark | 17, 11, 8 | | Spark RAPIDS | 17, 11, 8 | | Tez | 17, 11, 8 | | Trino | 21, 17 | | Zeppelin | 17, 11, 8 | | Pig | 8 | | Zookeeper | 17, 11, 8 | ## 7.5.0 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | adot-java-agent | 1.31.0 | A Java Agent that collects metrics from application daemons. | | delta | 3.2.1-amzn-0 | Delta lake is an open table format for huge analytic datasets | | emr-amazon-cloudwatch-agent | 1.300032.2-amzn-0 | An application that collects internal system-level metrics and custom application metrics from Amazon EC2 instances. | | emr-ddb | 5.6.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 3.14.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.18.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-notebook-env | 1.16.0 | Conda env for emr notebook which includes jupyter enterprise gateway | | emr-s3-dist-cp | 2.35.0 | Distributed copy application optimized for Amazon S3. | | emr-s3-select | 2.14.0 | EMR S3Select Connector | | emr-wal-cli | 1.3.1 | Cli used for emrwal list/deletion. | | emrfs | 2.66.0 | Amazon S3 connector for Hadoop ecosystem applications. | | flink-client | 1.19.1-amzn-1 | Apache Flink command line client scripts and applications. | | flink-jobmanager-config | 1.19.1-amzn-1 | Managing resources on EMR nodes for Apache Flink JobManager. | | hadoop-client | 3.4.0-amzn-1 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 3.4.0-amzn-1 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 3.4.0-amzn-1 | HDFS command-line client and library | | hadoop-hdfs-namenode | 3.4.0-amzn-1 | HDFS service for tracking file names and block locations. | | hadoop-hdfs-journalnode | 3.4.0-amzn-1 | HDFS service for managing the Hadoop filesystem journal on HA clusters. | | hadoop-httpfs-server | 3.4.0-amzn-1 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 3.4.0-amzn-1 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 3.4.0-amzn-1 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 3.4.0-amzn-1 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 3.4.0-amzn-1 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 3.4.0-amzn-1 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 2.5.10-amzn-0 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 2.5.10-amzn-0 | Service for serving one or more HBase regions. | | hbase-client | 2.5.10-amzn-0 | HBase command-line client. | | hbase-rest-server | 2.5.10-amzn-0 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 2.5.10-amzn-0 | Service providing a Thrift endpoint to HBase. | | hbase-operator-tools | 2.5.10-amzn-0 | Repair tool for Apache HBase clusters. | | hcatalog-client | 3.1.3-amzn-14 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 3.1.3-amzn-14 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 3.1.3-amzn-14 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 3.1.3-amzn-14 | Hive command line client. | | hive-hbase | 3.1.3-amzn-14 | Hive-hbase client. | | hive-metastore-server | 3.1.3-amzn-14 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server2 | 3.1.3-amzn-14 | Service for accepting Hive queries as web requests. | | hudi | 0.15.0-amzn-2 | Incremental processing framework to power data pipeline at low latency and high efficiency. | | hudi-presto | 0.15.0-amzn-2 | Bundle library for running Presto with Hudi. | | hudi-trino | 0.15.0-amzn-2 | Bundle library for running Trino with Hudi. | | hudi-spark | 0.15.0-amzn-2 | Bundle library for running Spark with Hudi. | | hue-server | 4.11.0 | Web application for analyzing data using Hadoop ecosystem applications | | iceberg | 1.6.1-amzn-1 | Apache Iceberg is an open table format for huge analytic datasets | | jupyterhub | 1.5.0 | Multi-user server for Jupyter notebooks | | livy-server | 0.8.0-incubating | REST interface for interacting with Apache Spark | | nginx | 1.12.1 | nginx [engine x] is an HTTP and reverse proxy server | | mariadb-server | 5.5.68\$1 | MariaDB database server. | | nvidia-cuda | 12.3.0 | Nvidia drivers and Cuda toolkit | | oozie-client | 5.2.1 | Oozie command-line client. | | oozie-server | 5.2.1 | Service for accepting Oozie workflow requests. | | opencv | 4.7.0 | Open Source Computer Vision Library. | | phoenix-library | 5.2.0 | The phoenix libraries for server and client | | phoenix-connectors | 6.0.0 | Apache Phoenix-Connectors for Spark-3 | | phoenix-query-server | 6.0.0 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.287-amzn-0 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.287-amzn-0 | Service for executing pieces of a query. | | presto-client | 0.287-amzn-0 | Presto command-line client which is installed on an HA cluster's stand-by masters where Presto server is not started. | | trino-coordinator | 446-amzn-0 | Service for accepting queries and managing query execution among trino-workers. | | trino-worker | 446-amzn-0 | Service for executing pieces of a query. | | trino-client | 446-amzn-0 | Trino command-line client which is installed on an HA cluster's stand-by masters where Trino server is not started. | | pig-client | 0.17.0 | Pig command-line client. | | r | 4.3.2 | The R Project for Statistical Computing | | ranger-kms-server | 2.0.0 | Apache Ranger Key Management System | | spark-client | 3.5.2-amzn-1 | Spark command-line clients. | | spark-history-server | 3.5.2-amzn-1 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 3.5.2-amzn-1 | In-memory execution engine for YARN. | | spark-yarn-slave | 3.5.2-amzn-1 | Apache Spark libraries needed by YARN slaves. | | spark-rapids | 24.08.1-amzn-1 | Nvidia Spark RAPIDS plugin that accelerates Apache Spark with GPUs. | | tensorflow | 2.16.1 | TensorFlow open source software library for high performance numerical computation. | | tez-on-yarn | 0.10.2-amzn-12 | The tez YARN application and libraries. | | tez-on-worker | 0.10.2-amzn-12 | The tez YARN application and libraries for worker nodes. | | zeppelin-server | 0.11.1 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.9.2-amzn-0 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.9.2-amzn-0 | ZooKeeper command line client. | ## 7.5.0 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). Reconfiguration actions occur when you specify a configuration for instance groups in a running cluster. Amazon EMR only initiates reconfiguration actions for the classifications that you modify. For more information, see [Reconfigure an instance group in a running cluster](emr-configure-apps-running-cluster.md). **emr-7.5.0 classifications** | Classifications | Description | Reconfiguration Actions | | --- | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | Restarts the ResourceManager service. | | container-executor | Change values in Hadoop YARN's container-executor.cfg file. | Not available. | | container-log4j | Change values in Hadoop YARN's container-log4j.properties file. | Not available. | | core-site | Change values in Hadoop's core-site.xml file. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts Hadoop KMS, Ranger KMS, HiveServer2, Hive MetaStore, Hadoop Httpfs, and MapReduce-HistoryServer. | | docker-conf | Change docker related settings. | Not available. | | emrfs-site | Change EMRFS settings. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts HBaseRegionserver, HBaseMaster, HBaseThrift, HBaseRest, HiveServer2, Hive MetaStore, Hadoop Httpfs, and MapReduce-HistoryServer. | | flink-conf | Change flink-conf.yaml settings. | Restarts Flink history server. | | flink-log4j | Change Flink log4j.properties settings. | Restarts Flink history server. | | flink-log4j-session | Change Flink log4j-session.properties settings for Kubernetes/Yarn session. | Restarts Flink history server. | | flink-log4j-cli | Change Flink log4j-cli.properties settings. | Restarts Flink history server. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts PhoenixQueryserver, HiveServer2, Hive MetaStore, and MapReduce-HistoryServer. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | Restarts the Hadoop HDFS services SecondaryNamenode, Datanode, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts Hadoop KMS, Hadoop Httpfs, and MapReduce-HistoryServer. | | hadoop-ssl-server | Change hadoop ssl server configuration | Not available. | | hadoop-ssl-client | Change hadoop ssl client configuration | Not available. | | hbase | Amazon EMR-curated settings for Apache HBase. | Custom EMR specific property. Sets emrfs-site and hbase-site configs. See those for their associated restarts. | | hbase-env | Change values in HBase's environment. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbase.properties file. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | Not available. | | hbase-site | Change values in HBase's hbase-site.xml file. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. Additionally restarts Phoenix QueryServer. | | hdfs-encryption-zones | Configure HDFS encryption zones. | This classification should not be reconfigured. | | hdfs-env | Change values in the HDFS environment. | Restarts Hadoop HDFS services Namenode, Datanode, and ZKFC. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Additionally restarts Hadoop Httpfs. | | hcatalog-env | Change values in HCatalog's environment. | Restarts Hive HCatalog Server. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | Restarts Hive HCatalog Server. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | Restarts Hive HCatalog Server. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | Restarts Hive WebHCat server. | | hcatalog-webhcat-log4j2 | Change values in HCatalog WebHCat's log4j2.properties. | Restarts Hive WebHCat server. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | Restarts Hive WebHCat server. | | hive | Amazon EMR-curated settings for Apache Hive. | Sets configurations to launch Hive LLAP service. | | hive-beeline-log4j2 | Change values in Hive's beeline-log4j2.properties file. | Not available. | | hive-parquet-logging | Change values in Hive's parquet-logging.properties file. | Not available. | | hive-env | Change values in the Hive environment. | Restarts HiveServer2, HiveMetastore, and Hive HCatalog-Server. Runs Hive schemaTool CLI commands to verify hive-metastore. | | hive-exec-log4j2 | Change values in Hive's hive-exec-log4j2.properties file. | Not available. | | hive-llap-daemon-log4j2 | Change values in Hive's llap-daemon-log4j2.properties file. | Not available. | | hive-log4j2 | Change values in Hive's hive-log4j2.properties file. | Not available. | | hive-site | Change values in Hive's hive-site.xml file | Restarts HiveServer2, HiveMetastore, and Hive HCatalog-Server. Runs Hive schemaTool CLI commands to verify hive-metastore. Also restarts Oozie and Zeppelin. | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | Not available. | | hue-ini | Change values in Hue's ini file | Restarts Hue. Also activates Hue config override CLI commands to pick up new configurations. | | httpfs-env | Change values in the HTTPFS environment. | Restarts Hadoop Httpfs service. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | Restarts Hadoop Httpfs service. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | Not available. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | Restarts Hadoop-KMS service. | | hadoop-kms-java-home | Change Hadoop's KMS java home | Not available. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | Not available. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | Restarts Hadoop-KMS and Ranger-KMS service. | | hudi-env | Change values in the Hudi environment. | Not available. | | hudi-defaults | Change values in Hudi's hudi-defaults.conf file. | Not available. | | iceberg-defaults | Change values in Iceberg's iceberg-defaults.conf file. | Not available. | | delta-defaults | Change values in Delta's delta-defaults.conf file. | Not available. | | jupyter-notebook-conf | Change values in Jupyter Notebook's jupyter\$1notebook\$1config.py file. | Not available. | | jupyter-hub-conf | Change values in JupyterHubs's jupyterhub\$1config.py file. | Not available. | | jupyter-s3-conf | Configure Jupyter Notebook S3 persistence. | Not available. | | jupyter-sparkmagic-conf | Change values in Sparkmagic's config.json file. | Not available. | | livy-conf | Change values in Livy's livy.conf file. | Restarts Livy Server. | | livy-env | Change values in the Livy environment. | Restarts Livy Server. | | livy-log4j2 | Change Livy log4j2.properties settings. | Restarts Livy Server. | | mapred-env | Change values in the MapReduce application's environment. | Restarts Hadoop MapReduce-HistoryServer. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | Restarts Hadoop MapReduce-HistoryServer. | | oozie-env | Change values in Oozie's environment. | Restarts Oozie. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | Restarts Oozie. | | oozie-site | Change values in Oozie's oozie-site.xml file. | Restarts Oozie. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | Not available. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | Not available. | | phoenix-log4j2 | Change values in Phoenix's log4j2.properties file. | Restarts Phoenix-QueryServer. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | Not available. | | pig-env | Change values in the Pig environment. | Not available. | | pig-properties | Change values in Pig's pig.properties file. | Restarts Oozie. | | pig-log4j | Change values in Pig's log4j.properties file. | Not available. | | presto-log | Change values in Presto's log.properties file. | Restarts Presto-Server (for PrestoDB) | | presto-config | Change values in Presto's config.properties file. | Restarts Presto-Server (for PrestoDB) | | presto-password-authenticator | Change values in Presto's password-authenticator.properties file. | Not available. | | presto-env | Change values in Presto's presto-env.sh file. | Restarts Presto-Server (for PrestoDB) | | presto-node | Change values in Presto's node.properties file. | Not available. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | Not available. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | Not available. | | presto-connector-hive | Change values in Presto's hive.properties file. | Restarts Presto-Server (for PrestoDB) | | presto-connector-jmx | Change values in Presto's jmx.properties file. | Not available. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | Not available. | | presto-connector-lakeformation | Change values in Presto's lakeformation.properties file. | Restarts Presto-Server (for PrestoDB) | | presto-connector-localfile | Change values in Presto's localfile.properties file. | Not available. | | presto-connector-memory | Change values in Presto's memory.properties file. | Not available. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | Not available. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | Not available. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | Not available. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | Not available. | | presto-connector-redis | Change values in Presto's redis.properties file. | Not available. | | presto-connector-redshift | Change values in Presto's redshift.properties file. | Not available. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | Not available. | | presto-connector-tpcds | Change values in Presto's tpcds.properties file. | Not available. | | trino-log | Change values in Trino's log.properties file. | Restarts Trino-Server (for Trino) | | trino-config | Change values in Trino's config.properties file. | Restarts Trino-Server (for Trino) | | trino-password-authenticator | Change values in Trino's password-authenticator.properties file. | Restarts Trino-Server (for Trino) | | trino-env | Change values in Trino's trino-env.sh file. | Restarts Trino-Server (for Trino) | | trino-node | Change values in Trino's node.properties file. | Not available. | | trino-connector-blackhole | Change values in Trino's blackhole.properties file. | Not available. | | trino-connector-cassandra | Change values in Trino's cassandra.properties file. | Not available. | | trino-connector-delta | Change values in Trino's delta.properties file. | Restarts Trino-Server (for Trino) | | trino-connector-hive | Change values in Trino's hive.properties file. | Restarts Trino-Server (for Trino) | | trino-exchange-manager | Change values in Trino's exchange-manager.properties file. | Restarts Trino-Server (for Trino) | | trino-connector-iceberg | Change values in Trino's iceberg.properties file. | Restarts Trino-Server (for Trino) | | trino-connector-hudi | Change values in Trino's hudi.properties file. | Restarts Trino-Server (for Trino) | | trino-connector-jmx | Change values in Trino's jmx.properties file. | Not available. | | trino-connector-kafka | Change values in Trino's kafka.properties file. | Not available. | | trino-connector-localfile | Change values in Trino's localfile.properties file. | Not available. | | trino-connector-memory | Change values in Trino's memory.properties file. | Not available. | | trino-connector-mongodb | Change values in Trino's mongodb.properties file. | Not available. | | trino-connector-mysql | Change values in Trino's mysql.properties file. | Not available. | | trino-connector-postgresql | Change values in Trino's postgresql.properties file. | Not available. | | trino-connector-raptor | Change values in Trino's raptor.properties file. | Not available. | | trino-connector-redis | Change values in Trino's redis.properties file. | Not available. | | trino-connector-redshift | Change values in Trino's redshift.properties file. | Not available. | | trino-connector-tpch | Change values in Trino's tpch.properties file. | Not available. | | trino-connector-tpcds | Change values in Trino's tpcds.properties file. | Not available. | | ranger-kms-dbks-site | Change values in dbks-site.xml file of Ranger KMS. | Restarts Ranger KMS Server. | | ranger-kms-site | Change values in ranger-kms-site.xml file of Ranger KMS. | Restarts Ranger KMS Server. | | ranger-kms-env | Change values in the Ranger KMS environment. | Restarts Ranger KMS Server. | | ranger-kms-logback | Change values in kms-logback.xml file of Ranger KMS. | Not available. | | ranger-kms-db-ca | Change values for CA file on S3 for MySQL SSL connection with Ranger KMS. | Not available. | | spark | Amazon EMR-curated settings for Apache Spark. | This property modifies spark-defaults. See actions there. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | Restarts Spark history server and Spark thrift server. | | spark-env | Change values in the Spark environment. | Restarts Spark history server and Spark thrift server. | | spark-hive-site | Change values in Spark's hive-site.xml file | Not available. | | spark-log4j2 | Change values in Spark's log4j2.properties file. | Restarts Spark history server and Spark thrift server. | | spark-metrics | Change values in Spark's metrics.properties file. | Restarts Spark history server and Spark thrift server. | | tez-site | Change values in Tez's tez-site.xml file. | Restart Oozie and HiveServer2. | | yarn-env | Change values in the YARN environment. | Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts MapReduce-HistoryServer. | | yarn-site | Change values in YARN's yarn-site.xml file. | Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts Livy Server and MapReduce-HistoryServer. | | zeppelin-env | Change values in the Zeppelin environment. | Restarts Zeppelin. | | zeppelin-site | Change configuration settings in zeppelin-site.xml. | Restarts Zeppelin. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | Restarts Zookeeper server. | | zookeeper-logback | Change values in ZooKeeper's logback.xml file. | Restarts Zookeeper server. | | emr-metrics | Change emr metric settings for this node. | Restarts the CloudWatchAgent service. | ## 7.5.0 change log **Change log for 7.5.0 release and release notes** | Date | Event | Description | | --- | --- | --- | | 2024-11-21 | Docs publication | Amazon EMR 7.5.0 release notes first published | | 2024-11-21 | Deployment complete | Amazon EMR 7.5.0 fully deployed to all [supported Regions](https://aws.amazon.com/about-aws/global-infrastructure/regional-product-services/) | | 2024-11-13 | Initial release | Amazon EMR 7.5.0 first deployed to initial commercial Regions | # Amazon EMR release 7.4.0 ## 7.4.0 supported lifecycle The following table describes the supported lifecycle dates for Amazon EMR release 7.4.0. | Support phase | Date | | --- | --- | | Initial release date | November 13, 2024 | | Standard support until | November 12, 2026 | | End of support | November 13, 2026 | | End of life | November 13, 2027 | ## 7.4.0 application versions This release includes the following applications: [https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/Install-CloudWatch-Agent.html](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/Install-CloudWatch-Agent.html), [https://delta.io/](https://delta.io/), [https://flink.apache.org/](https://flink.apache.org/), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [https://hudi.apache.org](https://hudi.apache.org), [http://gethue.com/](http://gethue.com/), [https://iceberg.apache.org/](https://iceberg.apache.org/), [https://jupyter-enterprise-gateway.readthedocs.io/en/latest/](https://jupyter-enterprise-gateway.readthedocs.io/en/latest/), [https://jupyterhub.readthedocs.io/en/latest/#](https://jupyterhub.readthedocs.io/en/latest/#), [https://livy.incubator.apache.org/](https://livy.incubator.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://www.tensorflow.org/](https://www.tensorflow.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://trino.io/](https://trino.io/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-7.4.0 | emr-7.3.0 | emr-7.2.0 | emr-7.1.0 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 2.28.8, 1.12.772 | 2.25.70, 1.12.747 | 2.23.18, 1.12.705 | 2.23.18, 1.12.656 | | Python | 3.9, 3.11 | 3.9, 3.11 | 3.9, 3.11 | 3.9, 3.11 | | Scala | 2.12.18 | 2.12.18 | 2.12.17 | 2.12.17 | | AmazonCloudWatchAgent | 1.300032.2-amzn-0 | 1.300032.2-amzn-0 | 1.300032.2-amzn-0 | 1.300032.2-amzn-0 | | Delta | 3.2.0-amzn-1 | 3.2.0-amzn-0 | 3.1.0 | 3.0.0 | | Flink | 1.19.1-amzn-0 | 1.18.1-amzn-2 | 1.18.1-amzn-1 | 1.18.1-amzn-0 | | Ganglia | - | - | - | - | | HBase | 2.5.5-amzn-0 | 2.4.17-amzn-7 | 2.4.17-amzn-6 | 2.4.17-amzn-5 | | HCatalog | 3.1.3-amzn-13 | 3.1.3-amzn-12 | 3.1.3-amzn-11 | 3.1.3-amzn-10 | | Hadoop | 3.4.0-amzn-0 | 3.3.6-amzn-5 | 3.3.6-amzn-4 | 3.3.6-amzn-3 | | Hive | 3.1.3-amzn-13 | 3.1.3-amzn-12 | 3.1.3-amzn-11 | 3.1.3-amzn-10 | | Hudi | 0.15.0-amzn-1 | 0.15.0-amzn-0 | 0.14.1-amzn-1 | 0.14.1-amzn-0 | | Hue | 4.11.0 | 4.11.0 | 4.11.0 | 4.11.0 | | Iceberg | 1.6.1-amzn-0 | 1.5.2-amzn-0 | 1.5.0-amzn-0 | 1.4.3-amzn-0 | | JupyterEnterpriseGateway | 2.6.0 | 2.6.0 | 2.6.0 | 2.6.0 | | JupyterHub | 1.5.0 | 1.5.0 | 1.5.0 | 1.5.0 | | Livy | 0.8.0-incubating | 0.8.0-incubating | 0.8.0-incubating | 0.8.0-incubating | | MXNet | - | - | 1.9.1 | 1.9.1 | | Mahout | - | - | - | - | | Oozie | 5.2.1 | 5.2.1 | 5.2.1 | 5.2.1 | | Phoenix | 5.2.0 | 5.1.3 | 5.1.3 | 5.1.3 | | Pig | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | | Presto | 0.287-amzn-0 | 0.285-amzn-1 | 0.285-amzn-0 | 0.284-amzn-0 | | Spark | 3.5.2-amzn-0 | 3.5.1-amzn-1 | 3.5.1-amzn-0 | 3.5.0-amzn-1 | | Sqoop | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | | TensorFlow | 2.16.1 | 2.16.1 | 2.11.0 | 2.11.0 | | Tez | 0.10.2-amzn-11 | 0.10.2-amzn-10 | 0.10.2-amzn-9 | 0.10.2-amzn-8 | | Trino (PrestoSQL) | 446-amzn-0 | 442-amzn-0 | 436-amzn-0 | 435-amzn-0 | | Zeppelin | 0.11.1 | 0.11.1 | 0.10.1 | 0.10.1 | | ZooKeeper | 3.9.2-amzn-0 | 3.9.1-amzn-1 | 3.9.1-amzn-1 | 3.9.1-amzn-0 | # 7.4.0 release components details See the following table for more information about the *Extras* packages in Amazon EMR 7.4.0. | Application | Application Type | Package | Package Type | Package Version | | --- | --- | --- | --- | --- | | AmazonCloudWatchAgent | Core | hadoop-hdfs-journalnode | Extras | 3.3.6.amzn.5-1.amzn2023 | | AmazonCloudWatchAgent | Core | ranger-kms | Extras | 2.0.0-1.amzn2023 | | AmazonCloudWatchAgent | Core | s3-dist-cp | Extras | 2.33.0-1.amzn2023 | | AmazonCloudWatchAgent | Core | zookeeper | Extras | 3.9.1.amzn.1-1.amzn2023 | | AmazonCloudWatchAgent | Core | zookeeper-server | Extras | 3.9.1.amzn.1-1.amzn2023 | | Flink | Core | hadoop-client | Extras | 3.3.6.amzn.5-1.amzn2023 | | Flink | Core | hadoop-hdfs | Extras | 3.3.6.amzn.5-1.amzn2023 | | Flink | Core | hadoop-hdfs-datanode | Extras | 3.3.6.amzn.5-1.amzn2023 | | Flink | Core | hadoop-hdfs-journalnode | Extras | 3.3.6.amzn.5-1.amzn2023 | | Flink | Core | hadoop-hdfs-namenode | Extras | 3.3.6.amzn.5-1.amzn2023 | | Flink | Core | hadoop-httpfs | Extras | 3.3.6.amzn.5-1.amzn2023 | | Flink | Core | hadoop-kms | Extras | 3.3.6.amzn.5-1.amzn2023 | | Flink | Core | hadoop-mapreduce | Extras | 3.3.6.amzn.5-1.amzn2023 | | Flink | Core | hadoop-mapreduce-historyserver | Extras | 3.3.6.amzn.5-1.amzn2023 | | Flink | Core | hadoop-yarn | Extras | 3.3.6.amzn.5-1.amzn2023 | | Flink | Core | hadoop-yarn-nodemanager | Extras | 3.3.6.amzn.5-1.amzn2023 | | Flink | Core | hadoop-yarn-proxyserver | Extras | 3.3.6.amzn.5-1.amzn2023 | | Flink | Core | hadoop-yarn-resourcemanager | Extras | 3.3.6.amzn.5-1.amzn2023 | | Flink | Core | hadoop-yarn-timelineserver | Extras | 3.3.6.amzn.5-1.amzn2023 | | Flink | Core | hudi | Extras | 0.15.0.amzn.0-1.amzn2023 | | Flink | Core | ranger-kms | Extras | 2.0.0-1.amzn2023 | | Flink | Core | hadoop | Extras | 3.3.6.amzn.5-1.amzn2023 | | HBase | Core | s3-dist-cp | Extras | 2.33.0-1.amzn2023 | | HCatalog | Core | hadoop-yarn-proxyserver | Extras | 3.3.6.amzn.5-1.amzn2023 | | HCatalog | Core | mariadb-connector-java | Extras | | | Hive | Core | hadoop-httpfs | Extras | 3.3.6.amzn.5-1.amzn2023 | | Hive | Core | hadoop-yarn-proxyserver | Extras | 3.3.6.amzn.5-1.amzn2023 | | Hive | Core | hive-hbase | Extras | 3.1.3.amzn.12-1.amzn2023 | | Hive | Core | mariadb-connector-java | Extras | 2.7.2-1 | | Hive | Core | s3-dist-cp | Extras | 2.33.0-1.amzn2023 | | Phoenix | Core | s3-dist-cp | Extras | 2.33.0-1.amzn2023 | | Spark | Core | spark-datanucleus | Extras | 3.5.1.amzn.0-1.amzn2023 | | Tez | Core | hadoop-mapreduce | Extras | 3.3.6.amzn.5-1.amzn2023 | | Tez | Core | hadoop-mapreduce-historyserver | Extras | 3.3.6.amzn.5-1.amzn2023 | | Tez | Core | hadoop-yarn-proxyserver | Extras | 3.3.6.amzn.5-1.amzn2023 | | Trino | Core | bigtop-utils | Extras | 1.2.0-1.amzn2023 | | Trino | Core | hadoop-client | Extras | 3.3.6.amzn.5-1.amzn2023 | | Trino | Core | hadoop-hdfs | Extras | 3.3.6.amzn.5-1.amzn2023 | | Trino | Core | hadoop-hdfs-datanode | Extras | 3.3.6.amzn.5-1.amzn2023 | | Trino | Core | hadoop-hdfs-journalnode | Extras | 3.3.6.amzn.5-1.amzn2023 | | Trino | Core | hadoop-hdfs-namenode | Extras | 3.3.6.amzn.5-1.amzn2023 | | Trino | Core | hadoop-kms | Extras | 3.3.6.amzn.5-1.amzn2023 | | Trino | Core | hadoop-mapreduce | Extras | 3.3.6.amzn.5-1.amzn2023 | | Trino | Core | hadoop-mapreduce-historyserver | Extras | 3.3.6.amzn.5-1.amzn2023 | | Trino | Core | hadoop-yarn | Extras | 3.3.6.amzn.5-1.amzn2023 | | Trino | Core | hadoop-yarn-nodemanager | Extras | 3.3.6.amzn.5-1.amzn2023 | | Trino | Core | hadoop-yarn-proxyserver | Extras | 3.3.6.amzn.5-1.amzn2023 | | Trino | Core | hadoop-yarn-resourcemanager | Extras | 3.3.6.amzn.5-1.amzn2023 | | Trino | Core | hadoop-yarn-timelineserver | Extras | 3.3.6.amzn.5-1.amzn2023 | | Trino | Core | hive | Extras | 3.1.3.amzn.12-1.amzn2023 | | Trino | Core | hive-hcatalog-server | Extras | 3.1.3.amzn.12-1.amzn2023 | | Trino | Core | mariadb-connector-java | Extras | 2.7.2-1 | | Trino | Core | ranger-kms | Extras | 2.0.0-1.amzn2023 | | Trino | Core | zookeeper | Extras | 3.9.1.amzn.1-1.amzn2023 | | Trino | Core | zookeeper-server | Extras | 3.9.1.amzn.1-1.amzn2023 | | Trino | Core | bigtop-groovy | Extras | 2.5.4-1.amzn2023 | | Trino | Core | bigtop-jsvc | Extras | 1.2.4-1.amzn2023 | | Trino | Core | hadoop | Extras | 3.3.6.amzn.5-1.amzn2023 | | Trino | Core | hadoop-lzo | Extras | 0.4.19-1.amzn2023 | | Trino | Core | hive-hcatalog | Extras | 3.1.3.amzn.12-1.amzn2023 | | Trino | Core | hive-jdbc | Extras | 3.1.3.amzn.12-1.amzn2023 | | Zookeeper | Core | emrfs | Extras | 2.64.0-1.amzn2023 | | Zookeeper | Core | hadoop-client | Extras | 3.3.6.amzn.5-1.amzn2023 | | Zookeeper | Core | hadoop-hdfs | Extras | 3.3.6.amzn.5-1.amzn2023 | | Zookeeper | Core | hadoop-hdfs-datanode | Extras | 3.3.6.amzn.5-1.amzn2023 | | Zookeeper | Core | hadoop-hdfs-journalnode | Extras | 3.3.6.amzn.5-1.amzn2023 | | Zookeeper | Core | hadoop-hdfs-namenode | Extras | 3.3.6.amzn.5-1.amzn2023 | | Zookeeper | Core | hadoop-httpfs | Extras | 3.3.6.amzn.5-1.amzn2023 | | Zookeeper | Core | hadoop-kms | Extras | 3.3.6.amzn.5-1.amzn2023 | | Zookeeper | Core | hadoop-mapreduce | Extras | 3.3.6.amzn.5-1.amzn2023 | | Zookeeper | Core | hadoop-mapreduce-historyserver | Extras | 3.3.6.amzn.5-1.amzn2023 | | Zookeeper | Core | hadoop-yarn | Extras | 3.3.6.amzn.5-1.amzn2023 | | Zookeeper | Core | hadoop-yarn-nodemanager | Extras | 3.3.6.amzn.5-1.amzn2023 | | Zookeeper | Core | hadoop-yarn-proxyserver | Extras | 3.3.6.amzn.5-1.amzn2023 | | Zookeeper | Core | hadoop-yarn-resourcemanager | Extras | 3.3.6.amzn.5-1.amzn2023 | | Zookeeper | Core | hadoop-yarn-timelineserver | Extras | 3.3.6.amzn.5-1.amzn2023 | | Zookeeper | Core | ranger-kms | Extras | 2.0.0-1.amzn2023 | # 7.4.0 common vulnerabilities and exposures The following table lists all CVEs that don't impact EMR clusters that run on recommended configurations of Amazon EMR 7.4.0. Amazon EMR is dependent on upstream open-source for availability of fixes and will provide the latest stable version as part of the Core Engine components within 90 days of Amazon EMR verifying the fixes. This table was updated on November 14, 2024. | CVE ID | Severity | CVE details URL | | --- | --- | --- | | CVE-2013-4002 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2013-4002](https://nvd.nist.gov/vuln/detail/CVE-2013-4002) | | CVE-2015-1832 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2015-1832](https://nvd.nist.gov/vuln/detail/CVE-2015-1832) | | CVE-2017-7525 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2017-7525](https://nvd.nist.gov/vuln/detail/CVE-2017-7525) | | CVE-2018-11307 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-11307](https://nvd.nist.gov/vuln/detail/CVE-2018-11307) | | CVE-2018-12022 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-12022](https://nvd.nist.gov/vuln/detail/CVE-2018-12022) | | CVE-2018-14718 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-14718](https://nvd.nist.gov/vuln/detail/CVE-2018-14718) | | CVE-2018-19360 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-19360](https://nvd.nist.gov/vuln/detail/CVE-2018-19360) | | CVE-2019-12384 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-12384](https://nvd.nist.gov/vuln/detail/CVE-2019-12384) | | CVE-2019-14379 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-14379](https://nvd.nist.gov/vuln/detail/CVE-2019-14379) | | CVE-2019-14439 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-14439](https://nvd.nist.gov/vuln/detail/CVE-2019-14439) | | CVE-2019-16335 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-16335](https://nvd.nist.gov/vuln/detail/CVE-2019-16335) | | CVE-2019-16943 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-16943](https://nvd.nist.gov/vuln/detail/CVE-2019-16943) | | CVE-2020-10672 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2020-10672](https://nvd.nist.gov/vuln/detail/CVE-2020-10672) | | CVE-2020-10673 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2020-10673](https://nvd.nist.gov/vuln/detail/CVE-2020-10673) | | CVE-2020-11112 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2020-11112](https://nvd.nist.gov/vuln/detail/CVE-2020-11112) | | CVE-2020-13949 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2020-13949](https://nvd.nist.gov/vuln/detail/CVE-2020-13949) | | CVE-2020-14195 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2020-14195](https://nvd.nist.gov/vuln/detail/CVE-2020-14195) | | CVE-2020-17521 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2020-17521](https://nvd.nist.gov/vuln/detail/CVE-2020-17521) | | CVE-2020-24616 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2020-24616](https://nvd.nist.gov/vuln/detail/CVE-2020-24616) | | CVE-2020-24750 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2020-24750](https://nvd.nist.gov/vuln/detail/CVE-2020-24750) | | CVE-2020-25649 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2020-25649](https://nvd.nist.gov/vuln/detail/CVE-2020-25649) | | CVE-2020-36183 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2020-36183](https://nvd.nist.gov/vuln/detail/CVE-2020-36183) | | CVE-2020-36186 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2020-36186](https://nvd.nist.gov/vuln/detail/CVE-2020-36186) | | CVE-2020-36188 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2020-36188](https://nvd.nist.gov/vuln/detail/CVE-2020-36188) | | CVE-2020-36189 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2020-36189](https://nvd.nist.gov/vuln/detail/CVE-2020-36189) | | CVE-2020-8908 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2020-8908](https://nvd.nist.gov/vuln/detail/CVE-2020-8908) | | CVE-2021-37137 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2021-37137](https://nvd.nist.gov/vuln/detail/CVE-2021-37137) | | CVE-2021-38153 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2021-38153](https://nvd.nist.gov/vuln/detail/CVE-2021-38153) | | CVE-2022-25857 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2022-25857](https://nvd.nist.gov/vuln/detail/CVE-2022-25857) | | CVE-2022-3509 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2022-3509](https://nvd.nist.gov/vuln/detail/CVE-2022-3509) | | CVE-2022-37866 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2022-37866](https://nvd.nist.gov/vuln/detail/CVE-2022-37866) | | CVE-2022-39135 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2022-39135](https://nvd.nist.gov/vuln/detail/CVE-2022-39135) | | CVE-2022-41966 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2022-41966](https://nvd.nist.gov/vuln/detail/CVE-2022-41966) | | CVE-2022-45688 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2022-45688](https://nvd.nist.gov/vuln/detail/CVE-2022-45688) | | CVE-2023-34455 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2023-34455](https://nvd.nist.gov/vuln/detail/CVE-2023-34455) | | CVE-2023-3635 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2023-3635](https://nvd.nist.gov/vuln/detail/CVE-2023-3635) | | CVE-2023-38647 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2023-38647](https://nvd.nist.gov/vuln/detail/CVE-2023-38647) | | CVE-2023-40167 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2023-40167](https://nvd.nist.gov/vuln/detail/CVE-2023-40167) | | CVE-2023-43642 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2023-43642](https://nvd.nist.gov/vuln/detail/CVE-2023-43642) | | CVE-2023-44981 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2023-44981](https://nvd.nist.gov/vuln/detail/CVE-2023-44981) | | CVE-2023-50387 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2023-50387](https://nvd.nist.gov/vuln/detail/CVE-2023-50387) | | CVE-2024-23454 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2024-23454](https://nvd.nist.gov/vuln/detail/CVE-2024-23454) | | CVE-2024-29025 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2024-29025](https://nvd.nist.gov/vuln/detail/CVE-2024-29025) | | CVE-2024-36124 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2024-36124](https://nvd.nist.gov/vuln/detail/CVE-2024-36124) | | CVE-2024-8184 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2024-8184](https://nvd.nist.gov/vuln/detail/CVE-2024-8184) | | CVE-2012-0881 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2012-0881](https://nvd.nist.gov/vuln/detail/CVE-2012-0881) | | CVE-2012-5783 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2012-5783](https://nvd.nist.gov/vuln/detail/CVE-2012-5783) | | CVE-2015-5237 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2015-5237](https://nvd.nist.gov/vuln/detail/CVE-2015-5237) | | CVE-2016-5018 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2016-5018](https://nvd.nist.gov/vuln/detail/CVE-2016-5018) | | CVE-2016-6796 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2016-6796](https://nvd.nist.gov/vuln/detail/CVE-2016-6796) | | CVE-2017-15095 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2017-15095](https://nvd.nist.gov/vuln/detail/CVE-2017-15095) | | CVE-2018-1000180 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2018-1000180](https://nvd.nist.gov/vuln/detail/CVE-2018-1000180) | | CVE-2018-10054 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2018-10054](https://nvd.nist.gov/vuln/detail/CVE-2018-10054) | | CVE-2018-10237 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2018-10237](https://nvd.nist.gov/vuln/detail/CVE-2018-10237) | | CVE-2018-11798 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2018-11798](https://nvd.nist.gov/vuln/detail/CVE-2018-11798) | | CVE-2018-12023 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2018-12023](https://nvd.nist.gov/vuln/detail/CVE-2018-12023) | | CVE-2018-14721 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2018-14721](https://nvd.nist.gov/vuln/detail/CVE-2018-14721) | | CVE-2018-19361 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2018-19361](https://nvd.nist.gov/vuln/detail/CVE-2018-19361) | | CVE-2018-5968 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2018-5968](https://nvd.nist.gov/vuln/detail/CVE-2018-5968) | | CVE-2018-7489 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2018-7489](https://nvd.nist.gov/vuln/detail/CVE-2018-7489) | | CVE-2019-0205 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2019-0205](https://nvd.nist.gov/vuln/detail/CVE-2019-0205) | | CVE-2019-12086 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2019-12086](https://nvd.nist.gov/vuln/detail/CVE-2019-12086) | | CVE-2019-12814 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2019-12814](https://nvd.nist.gov/vuln/detail/CVE-2019-12814) | | CVE-2019-14540 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2019-14540](https://nvd.nist.gov/vuln/detail/CVE-2019-14540) | | CVE-2019-14892 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2019-14892](https://nvd.nist.gov/vuln/detail/CVE-2019-14892) | | CVE-2019-14893 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2019-14893](https://nvd.nist.gov/vuln/detail/CVE-2019-14893) | | CVE-2019-16869 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2019-16869](https://nvd.nist.gov/vuln/detail/CVE-2019-16869) | | CVE-2019-16942 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2019-16942](https://nvd.nist.gov/vuln/detail/CVE-2019-16942) | | CVE-2019-17267 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2019-17267](https://nvd.nist.gov/vuln/detail/CVE-2019-17267) | | CVE-2019-17531 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2019-17531](https://nvd.nist.gov/vuln/detail/CVE-2019-17531) | | CVE-2019-20330 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2019-20330](https://nvd.nist.gov/vuln/detail/CVE-2019-20330) | | CVE-2019-20444 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2019-20444](https://nvd.nist.gov/vuln/detail/CVE-2019-20444) | | CVE-2019-20445 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2019-20445](https://nvd.nist.gov/vuln/detail/CVE-2019-20445) | | CVE-2020-10969 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-10969](https://nvd.nist.gov/vuln/detail/CVE-2020-10969) | | CVE-2020-11111 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-11111](https://nvd.nist.gov/vuln/detail/CVE-2020-11111) | | CVE-2020-11619 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-11619](https://nvd.nist.gov/vuln/detail/CVE-2020-11619) | | CVE-2020-11620 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-11620](https://nvd.nist.gov/vuln/detail/CVE-2020-11620) | | CVE-2020-13955 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-13955](https://nvd.nist.gov/vuln/detail/CVE-2020-13955) | | CVE-2020-13956 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-13956](https://nvd.nist.gov/vuln/detail/CVE-2020-13956) | | CVE-2020-14060 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-14060](https://nvd.nist.gov/vuln/detail/CVE-2020-14060) | | CVE-2020-14061 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-14061](https://nvd.nist.gov/vuln/detail/CVE-2020-14061) | | CVE-2020-15250 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-15250](https://nvd.nist.gov/vuln/detail/CVE-2020-15250) | | CVE-2020-26939 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-26939](https://nvd.nist.gov/vuln/detail/CVE-2020-26939) | | CVE-2020-35491 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-35491](https://nvd.nist.gov/vuln/detail/CVE-2020-35491) | | CVE-2020-35728 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-35728](https://nvd.nist.gov/vuln/detail/CVE-2020-35728) | | CVE-2020-36179 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36179](https://nvd.nist.gov/vuln/detail/CVE-2020-36179) | | CVE-2020-36181 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36181](https://nvd.nist.gov/vuln/detail/CVE-2020-36181) | | CVE-2020-36182 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36182](https://nvd.nist.gov/vuln/detail/CVE-2020-36182) | | CVE-2020-36184 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36184](https://nvd.nist.gov/vuln/detail/CVE-2020-36184) | | CVE-2020-36185 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36185](https://nvd.nist.gov/vuln/detail/CVE-2020-36185) | | CVE-2020-36187 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36187](https://nvd.nist.gov/vuln/detail/CVE-2020-36187) | | CVE-2020-36518 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36518](https://nvd.nist.gov/vuln/detail/CVE-2020-36518) | | CVE-2020-8840 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-8840](https://nvd.nist.gov/vuln/detail/CVE-2020-8840) | | CVE-2020-9546 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-9546](https://nvd.nist.gov/vuln/detail/CVE-2020-9546) | | CVE-2020-9547 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-9547](https://nvd.nist.gov/vuln/detail/CVE-2020-9547) | | CVE-2020-9548 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-9548](https://nvd.nist.gov/vuln/detail/CVE-2020-9548) | | CVE-2021-20190 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-20190](https://nvd.nist.gov/vuln/detail/CVE-2021-20190) | | CVE-2021-21290 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-21290](https://nvd.nist.gov/vuln/detail/CVE-2021-21290) | | CVE-2021-22569 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-22569](https://nvd.nist.gov/vuln/detail/CVE-2021-22569) | | CVE-2021-28170 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-28170](https://nvd.nist.gov/vuln/detail/CVE-2021-28170) | | CVE-2021-29425 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-29425](https://nvd.nist.gov/vuln/detail/CVE-2021-29425) | | CVE-2021-31684 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-31684](https://nvd.nist.gov/vuln/detail/CVE-2021-31684) | | CVE-2021-34538 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-34538](https://nvd.nist.gov/vuln/detail/CVE-2021-34538) | | CVE-2021-36374 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-36374](https://nvd.nist.gov/vuln/detail/CVE-2021-36374) | | CVE-2021-37533 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-37533](https://nvd.nist.gov/vuln/detail/CVE-2021-37533) | | CVE-2021-43859 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-43859](https://nvd.nist.gov/vuln/detail/CVE-2021-43859) | | CVE-2022-1415 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-1415](https://nvd.nist.gov/vuln/detail/CVE-2022-1415) | | CVE-2022-1471 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-1471](https://nvd.nist.gov/vuln/detail/CVE-2022-1471) | | CVE-2022-3171 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-3171](https://nvd.nist.gov/vuln/detail/CVE-2022-3171) | | CVE-2022-3510 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-3510](https://nvd.nist.gov/vuln/detail/CVE-2022-3510) | | CVE-2022-36364 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-36364](https://nvd.nist.gov/vuln/detail/CVE-2022-36364) | | CVE-2022-37865 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-37865](https://nvd.nist.gov/vuln/detail/CVE-2022-37865) | | CVE-2022-38749 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-38749](https://nvd.nist.gov/vuln/detail/CVE-2022-38749) | | CVE-2022-38751 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-38751](https://nvd.nist.gov/vuln/detail/CVE-2022-38751) | | CVE-2022-40150 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-40150](https://nvd.nist.gov/vuln/detail/CVE-2022-40150) | | CVE-2022-40151 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-40151](https://nvd.nist.gov/vuln/detail/CVE-2022-40151) | | CVE-2022-40152 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-40152](https://nvd.nist.gov/vuln/detail/CVE-2022-40152) | | CVE-2022-41854 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-41854](https://nvd.nist.gov/vuln/detail/CVE-2022-41854) | | CVE-2022-42003 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-42003](https://nvd.nist.gov/vuln/detail/CVE-2022-42003) | | CVE-2022-42889 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-42889](https://nvd.nist.gov/vuln/detail/CVE-2022-42889) | | CVE-2022-45685 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-45685](https://nvd.nist.gov/vuln/detail/CVE-2022-45685) | | CVE-2022-45693 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-45693](https://nvd.nist.gov/vuln/detail/CVE-2022-45693) | | CVE-2022-46337 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-46337](https://nvd.nist.gov/vuln/detail/CVE-2022-46337) | | CVE-2023-0833 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-0833](https://nvd.nist.gov/vuln/detail/CVE-2023-0833) | | CVE-2023-1370 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-1370](https://nvd.nist.gov/vuln/detail/CVE-2023-1370) | | CVE-2023-1436 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-1436](https://nvd.nist.gov/vuln/detail/CVE-2023-1436) | | CVE-2023-20863 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-20863](https://nvd.nist.gov/vuln/detail/CVE-2023-20863) | | CVE-2023-25194 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-25194](https://nvd.nist.gov/vuln/detail/CVE-2023-25194) | | CVE-2023-33201 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-33201](https://nvd.nist.gov/vuln/detail/CVE-2023-33201) | | CVE-2023-33202 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-33202](https://nvd.nist.gov/vuln/detail/CVE-2023-33202) | | CVE-2023-34453 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-34453](https://nvd.nist.gov/vuln/detail/CVE-2023-34453) | | CVE-2023-34454 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-34454](https://nvd.nist.gov/vuln/detail/CVE-2023-34454) | | CVE-2023-39410 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-39410](https://nvd.nist.gov/vuln/detail/CVE-2023-39410) | | CVE-2023-42503 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-42503](https://nvd.nist.gov/vuln/detail/CVE-2023-42503) | | CVE-2023-5072 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-5072](https://nvd.nist.gov/vuln/detail/CVE-2023-5072) | | CVE-2023-50868 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-50868](https://nvd.nist.gov/vuln/detail/CVE-2023-50868) | | CVE-2023-52428 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-52428](https://nvd.nist.gov/vuln/detail/CVE-2023-52428) | | CVE-2023-6378 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-6378](https://nvd.nist.gov/vuln/detail/CVE-2023-6378) | | CVE-2023-6481 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-6481](https://nvd.nist.gov/vuln/detail/CVE-2023-6481) | | CVE-2024-21634 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-21634](https://nvd.nist.gov/vuln/detail/CVE-2024-21634) | | CVE-2024-23944 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-23944](https://nvd.nist.gov/vuln/detail/CVE-2024-23944) | | CVE-2024-29131 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-29131](https://nvd.nist.gov/vuln/detail/CVE-2024-29131) | | CVE-2024-30171 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-30171](https://nvd.nist.gov/vuln/detail/CVE-2024-30171) | | CVE-2024-30172 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-30172](https://nvd.nist.gov/vuln/detail/CVE-2024-30172) | | CVE-2024-34447 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-34447](https://nvd.nist.gov/vuln/detail/CVE-2024-34447) | | CVE-2024-35255 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-35255](https://nvd.nist.gov/vuln/detail/CVE-2024-35255) | | CVE-2024-36114 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-36114](https://nvd.nist.gov/vuln/detail/CVE-2024-36114) | | CVE-2024-38808 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-38808](https://nvd.nist.gov/vuln/detail/CVE-2024-38808) | | CVE-2024-39676 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-39676](https://nvd.nist.gov/vuln/detail/CVE-2024-39676) | | CVE-2024-51504 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-51504](https://nvd.nist.gov/vuln/detail/CVE-2024-51504) | | CVE-2024-7254 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-7254](https://nvd.nist.gov/vuln/detail/CVE-2024-7254) | | CVE-2017-17485 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2017-17485](https://nvd.nist.gov/vuln/detail/CVE-2017-17485) | | CVE-2018-14719 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2018-14719](https://nvd.nist.gov/vuln/detail/CVE-2018-14719) | | CVE-2018-14720 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2018-14720](https://nvd.nist.gov/vuln/detail/CVE-2018-14720) | | CVE-2018-19362 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2018-19362](https://nvd.nist.gov/vuln/detail/CVE-2018-19362) | | CVE-2020-10650 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2020-10650](https://nvd.nist.gov/vuln/detail/CVE-2020-10650) | | CVE-2020-10968 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2020-10968](https://nvd.nist.gov/vuln/detail/CVE-2020-10968) | | CVE-2020-13936 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2020-13936](https://nvd.nist.gov/vuln/detail/CVE-2020-13936) | | CVE-2020-14062 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2020-14062](https://nvd.nist.gov/vuln/detail/CVE-2020-14062) | | CVE-2020-15522 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2020-15522](https://nvd.nist.gov/vuln/detail/CVE-2020-15522) | | CVE-2020-1945 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2020-1945](https://nvd.nist.gov/vuln/detail/CVE-2020-1945) | | CVE-2020-28052 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2020-28052](https://nvd.nist.gov/vuln/detail/CVE-2020-28052) | | CVE-2020-35490 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2020-35490](https://nvd.nist.gov/vuln/detail/CVE-2020-35490) | | CVE-2021-36373 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2021-36373](https://nvd.nist.gov/vuln/detail/CVE-2021-36373) | | CVE-2022-25647 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2022-25647](https://nvd.nist.gov/vuln/detail/CVE-2022-25647) | | CVE-2022-38750 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2022-38750](https://nvd.nist.gov/vuln/detail/CVE-2022-38750) | | CVE-2022-38752 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2022-38752](https://nvd.nist.gov/vuln/detail/CVE-2022-38752) | | CVE-2022-42004 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2022-42004](https://nvd.nist.gov/vuln/detail/CVE-2022-42004) | | CVE-2022-46751 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2022-46751](https://nvd.nist.gov/vuln/detail/CVE-2022-46751) | | CVE-2023-20883 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2023-20883](https://nvd.nist.gov/vuln/detail/CVE-2023-20883) | | CVE-2023-26048 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2023-26048](https://nvd.nist.gov/vuln/detail/CVE-2023-26048) | | CVE-2023-2976 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2023-2976](https://nvd.nist.gov/vuln/detail/CVE-2023-2976) | | CVE-2023-36478 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2023-36478](https://nvd.nist.gov/vuln/detail/CVE-2023-36478) | | CVE-2023-51074 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2023-51074](https://nvd.nist.gov/vuln/detail/CVE-2023-51074) | | CVE-2024-25638 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-25638](https://nvd.nist.gov/vuln/detail/CVE-2024-25638) | | CVE-2024-25710 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-25710](https://nvd.nist.gov/vuln/detail/CVE-2024-25710) | | CVE-2024-26308 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-26308](https://nvd.nist.gov/vuln/detail/CVE-2024-26308) | | CVE-2024-29133 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-29133](https://nvd.nist.gov/vuln/detail/CVE-2024-29133) | | CVE-2024-47554 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-47554](https://nvd.nist.gov/vuln/detail/CVE-2024-47554) | | CVE-2024-47561 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-47561](https://nvd.nist.gov/vuln/detail/CVE-2024-47561) | | CVE-2024-6763 | MEDIUM | [https://nvd.nist.gov/vuln/detail/CVE-2024-6763](https://nvd.nist.gov/vuln/detail/CVE-2024-6763) | | CVE-2012-6153 | LOW | [https://nvd.nist.gov/vuln/detail/CVE-2012-6153](https://nvd.nist.gov/vuln/detail/CVE-2012-6153) | | CVE-2020-11113 | LOW | [https://nvd.nist.gov/vuln/detail/CVE-2020-11113](https://nvd.nist.gov/vuln/detail/CVE-2020-11113) | | CVE-2021-22570 | LOW | [https://nvd.nist.gov/vuln/detail/CVE-2021-22570](https://nvd.nist.gov/vuln/detail/CVE-2021-22570) | | CVE-2021-27568 | LOW | [https://nvd.nist.gov/vuln/detail/CVE-2021-27568](https://nvd.nist.gov/vuln/detail/CVE-2021-27568) | | CVE-2024-29857 | LOW | [https://nvd.nist.gov/vuln/detail/CVE-2024-29857](https://nvd.nist.gov/vuln/detail/CVE-2024-29857) | | CVE-2018-1313 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2018-1313](https://nvd.nist.gov/vuln/detail/CVE-2018-1313) | | CVE-2020-36180 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2020-36180](https://nvd.nist.gov/vuln/detail/CVE-2020-36180) | | CVE-2021-37136 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2021-37136](https://nvd.nist.gov/vuln/detail/CVE-2021-37136) | | CVE-2022-24329 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2022-24329](https://nvd.nist.gov/vuln/detail/CVE-2022-24329) | | CVE-2022-40149 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2022-40149](https://nvd.nist.gov/vuln/detail/CVE-2022-40149) | | CVE-2023-26049 | UNTRIAGED | [https://nvd.nist.gov/vuln/detail/CVE-2023-26049](https://nvd.nist.gov/vuln/detail/CVE-2023-26049) | ## 7.4.0 release notes The following release notes include information for Amazon EMR release 7.4.0. **New features** + **Application upgrades** – Amazon EMR 7.4.0 application upgrades include AWS SDK for Java 2.28.8, Delta 3.2.0-amzn-1, Flink 1.19.1, Hadoop 3.4.0, HBase 2.5.5, Hudi 0.15.0-amzn-1, Iceberg 1.6.1-amzn-0, Phoenix 5.2.0, Presto 0.287, Spark 3.5.2, Trino 446, and Zookeeper 3.9.2. + This release adds 7 open-source endpoints that support in-transit encryption over the network. This improvement includes open-source frameworks such as Hue, Livy, JupyterEnterpriseGateway, Zookeeper, and Ranger KMS. If you enabled in-transit encryption and want to connect to these endpoints, you might have to modify your code to support TLS communication. You can disable TLS for any of these endpoints as needed. For a full list of the supported endpoints, see [Understanding in-transit encryption](https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-encryption-support-matrix.html). **Known issues** + The `yarn.resourcemanager.decommissioning-nodes-watcher.wait-for-shuffle-data` property for the Yarn ResourceManager, which previously supported Hive shuffle handlers since emr-6.11.0, now also applies graceful decommissioning to the files served by the Spark external shuffle service, when running Spark on Yarn in emr-7.4.0. Note that The default behavior of the Spark external shuffle service, when running Spark on Yarn, is for the Yarn NodeManager to remove application shuffle files when the application is terminated. This might have an impact on the speed of node decommissioning and compute utilization. For long running applications, consider setting `spark.shuffle.service.removeShuffle` to `true` to remove shuffle files no longer in use, which enables faster decommissioning of nodes with no active shuffle data. + Amazon S3 Select availability – Amazon S3 Select does not support cross-region queries for Presto and Trino. For more information about specific features available for Presto and Trino releases, see [Presto and Trino](https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-presto.html). + Amazon EMR release 7.4 and later don't support Apache Ranger. + Apache Sqoop is not compatible with Apache Hadoop 3.4 because of a known issue SQOOP-3485. Amazon EMR Release 7.4 includes but does not support Apache Sqoop. Apache Sqoop will be removed from Amazon EMR Release 7.5 and higher releases. + When managed scaling is enabled, the YARN ResourceManager (RM) experiences a critical deadlock issue, causing it to become unresponsive, when transitioning from the **DECOMMISSIONING** to **DECOMMISSIONED** state with simultaneous operations. This happens particularly in large-cluster environments. When it occurs, both node-state transitions and application-state transitions fail to process, effectively paralyzing the core functionality of the ResourceManager. As a workaround, restart the RM JVM using the following command on the primary node: ``` sudo systemctl restart hadoop-yarn-resourcemanager ``` + When an EMR cluster is started with YARN schedulers other than the `CapacityScheduler` or `FairScheduler` (for example, the FIFO Scheduler), the `YARNMemoryAvailablePercentage` metric is not pushed to CloudWatch. This issue impacts down-scaling in managed-scaling enabled clusters. It might also affect custom managed scaling if it depends on the `YARNMemoryAvailablePercentage` metric. As a workaround, use the `CapacityScheduler` or `FairScheduler`. + **HBase table creation in Hive with Glue catalog** – From EMR-7.3.0 to EMR-7.10.0, there is a bug due to Hive Iceberg integration, which causes HBase table creation in Hive to fail when Glue Data Catalog is used as the metastore. Please reach out to the AWS support team if you encounter this issue. + **Tez UI** – From EMR-7.3 to EMR-7.10, there is a bug where accessing Tez UI through YARN Resource Manager Web UI fails when in-transit encryption is enabled. This occurs because YARN proxy attempts to use HTTP protocol while Tez UI requires HTTPS. Please modify the `tez.tez-ui.history-url.base` configuration in `tez-site` from `http` to `https` to resolve this issue. **Changes, enhancements, and resolved issues** + Starting with Spark 3.3.1 (supported in EMR versions 6.10 and above), all executors in a decommissioning host are set to a new `ExecutorState`, called *DECOMMISSIONING* state. The executors being decommissioned cannot be used by Yarn to allocate tasks and thus it will request for new executors, if needed, for the tasks being executed. Thus, if you disable Spark DRA while using EMR Managed Scaling, EMR Auto Scaling, or any custom scaling mechanism on EMR-EC2 clusters, then Yarn may request maximum permissible executors for each job. In order to avoid this issue, leave the `spark.dynamicAllocation.enabled` property set to `TRUE` (which is the default) when you are using the above combination of features. In addition, you can also set minimum and maximum executor constraints by setting values for `spark.dynamicAllocation.maxExecutors` and `spark.dynamicAllocation.minExecutors` properties for your Spark jobs, to restrict the number of executors allocated during the job’s execution. + Amazon EMR uses different builds of the TensorFlow library depending on the instance types that you choose for your cluster. Amazon EMR also supports TensorFlow for clusters with aarch64 instance types (Graviton instances) from EMR-7.5.0 and above. + **Optimizing S3 Request Scaling for Iceberg Workloads** – For customers that use Iceberg bundled with EMR 34 v7.4 releases, S3 has updated the default entropy scheme to a 20-character base-2 (e.g. 35 01010110100110110010) hash. This new base-2 entropy scheme reduces the character range and allows S3 to automatically scale request capacity more quickly to match the demands of the target Iceberg workload. It also reduces the amount of time that a workload observes 503 throttle responses. + [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-740-release.html) ## 7.4.0 default Java versions Amazon EMR release 7.4 and higher ships with Amazon Corretto 17 (built on OpenJDK) by default for applications that support Corretto 17 (JDK 17), with the exception of Apache Livy. The following table shows the default Java versions for applications in Amazon EMR 7.4.0. If you want to change the default JVM on your cluster, follow the instructions in [Configure applications to use a specific Java Virtual Machine](configuring-java8.md) for each application that runs on the cluster. You can only use one Java runtime version for a cluster. Amazon EMR doesn't support running different nodes or applications on different runtime versions on the same cluster. | Application | Java / Amazon Corretto version (default is bold) | | --- | --- | | Delta | 17, 11, 8 | | Flink | 17, 11, 8 | | HBase | 17, 11, 8 | | HCatalog | 17, 11, 8 | | Hadoop | 17, 11, 8 | | Hive | 17, 11, 8 | | Hudi | 17, 11, 8 | | Iceberg | 17, 11, 8 | | Livy | 17, 11, 8 | | Oozie | 17, 11, 8 | | Phoenix | 17, 11, 8 | | PrestoDB | 8 | | Spark | 17, 11, 8 | | Spark RAPIDS | 17, 11, 8 | | Sqoop | 8 | | Tez | 17, 11, 8 | | Trino | 21, 17 | | Zeppelin | 17, 11, 8 | | Pig | 8 | | Zookeeper | 8 | ## 7.4.0 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | adot-java-agent | 1.31.0 | A Java Agent that collects metrics from application daemons. | | delta | 3.2.0-amzn-1 | Delta lake is an open table format for huge analytic datasets | | emr-amazon-cloudwatch-agent | 1.300032.2-amzn-0 | An application that collects internal system-level metrics and custom application metrics from Amazon EC2 instances. | | emr-ddb | 5.6.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 3.13.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.17.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-notebook-env | 1.17.0 | Conda env for emr notebook which includes jupyter enterprise gateway | | emr-s3-dist-cp | 2.34.0 | Distributed copy application optimized for Amazon S3. | | emr-s3-select | 2.13.0 | EMR S3Select Connector | | emr-wal-cli | 1.3.1 | Cli used for emrwal list/deletion. | | emrfs | 2.65.0 | Amazon S3 connector for Hadoop ecosystem applications. | | flink-client | 1.19.1-amzn-0 | Apache Flink command line client scripts and applications. | | flink-jobmanager-config | 1.19.1-amzn-0 | Managing resources on EMR nodes for Apache Flink JobManager. | | hadoop-client | 3.4.0-amzn-0 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 3.4.0-amzn-0 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 3.4.0-amzn-0 | HDFS command-line client and library | | hadoop-hdfs-namenode | 3.4.0-amzn-0 | HDFS service for tracking file names and block locations. | | hadoop-hdfs-journalnode | 3.4.0-amzn-0 | HDFS service for managing the Hadoop filesystem journal on HA clusters. | | hadoop-httpfs-server | 3.4.0-amzn-0 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 3.4.0-amzn-0 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 3.4.0-amzn-0 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 3.4.0-amzn-0 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 3.4.0-amzn-0 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 3.4.0-amzn-0 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 2.5.5-amzn-0 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 2.5.5-amzn-0 | Service for serving one or more HBase regions. | | hbase-client | 2.5.5-amzn-0 | HBase command-line client. | | hbase-rest-server | 2.5.5-amzn-0 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 2.5.5-amzn-0 | Service providing a Thrift endpoint to HBase. | | hbase-operator-tools | 2.5.5-amzn-0 | Repair tool for Apache HBase clusters. | | hcatalog-client | 3.1.3-amzn-13 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 3.1.3-amzn-13 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 3.1.3-amzn-13 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 3.1.3-amzn-13 | Hive command line client. | | hive-hbase | 3.1.3-amzn-13 | Hive-hbase client. | | hive-metastore-server | 3.1.3-amzn-13 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server2 | 3.1.3-amzn-13 | Service for accepting Hive queries as web requests. | | hudi | 0.15.0-amzn-1 | Incremental processing framework to power data pipeline at low latency and high efficiency. | | hudi-presto | 0.15.0-amzn-1 | Bundle library for running Presto with Hudi. | | hudi-trino | 0.15.0-amzn-1 | Bundle library for running Trino with Hudi. | | hudi-spark | 0.15.0-amzn-1 | Bundle library for running Spark with Hudi. | | hue-server | 4.11.0 | Web application for analyzing data using Hadoop ecosystem applications | | iceberg | 1.6.1-amzn-0 | Apache Iceberg is an open table format for huge analytic datasets | | jupyterhub | 1.5.0 | Multi-user server for Jupyter notebooks | | livy-server | 0.8.0-incubating | REST interface for interacting with Apache Spark | | nginx | 1.12.1 | nginx [engine x] is an HTTP and reverse proxy server | | mariadb-server | 5.5.68\$1 | MariaDB database server. | | nvidia-cuda | 12.3.0 | Nvidia drivers and Cuda toolkit | | oozie-client | 5.2.1 | Oozie command-line client. | | oozie-server | 5.2.1 | Service for accepting Oozie workflow requests. | | opencv | 4.7.0 | Open Source Computer Vision Library. | | phoenix-library | 5.2.0 | The phoenix libraries for server and client | | phoenix-connectors | 6.0.0 | Apache Phoenix-Connectors for Spark-3 | | phoenix-query-server | 6.0.0 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.287-amzn-0 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.287-amzn-0 | Service for executing pieces of a query. | | presto-client | 0.287-amzn-0 | Presto command-line client which is installed on an HA cluster's stand-by masters where Presto server is not started. | | trino-coordinator | 446-amzn-0 | Service for accepting queries and managing query execution among trino-workers. | | trino-worker | 446-amzn-0 | Service for executing pieces of a query. | | trino-client | 446-amzn-0 | Trino command-line client which is installed on an HA cluster's stand-by masters where Trino server is not started. | | pig-client | 0.17.0 | Pig command-line client. | | r | 4.3.2 | The R Project for Statistical Computing | | ranger-kms-server | 2.0.0 | Apache Ranger Key Management System | | spark-client | 3.5.2-amzn-0 | Spark command-line clients. | | spark-history-server | 3.5.2-amzn-0 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 3.5.2-amzn-0 | In-memory execution engine for YARN. | | spark-yarn-slave | 3.5.2-amzn-0 | Apache Spark libraries needed by YARN slaves. | | spark-rapids | 24.08.1-amzn-0 | Nvidia Spark RAPIDS plugin that accelerates Apache Spark with GPUs. | | sqoop-client | 1.4.7 | Apache Sqoop command-line client. | | tensorflow | 2.16.1 | TensorFlow open source software library for high performance numerical computation. | | tez-on-yarn | 0.10.2-amzn-11 | The tez YARN application and libraries. | | tez-on-worker | 0.10.2-amzn-11 | The tez YARN application and libraries for worker nodes. | | zeppelin-server | 0.11.1 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.9.2-amzn-0 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.9.2-amzn-0 | ZooKeeper command line client. | ## 7.4.0 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). Reconfiguration actions occur when you specify a configuration for instance groups in a running cluster. Amazon EMR only initiates reconfiguration actions for the classifications that you modify. For more information, see [Reconfigure an instance group in a running cluster](emr-configure-apps-running-cluster.md). **emr-7.4.0 classifications** | Classifications | Description | Reconfiguration Actions | | --- | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | Restarts the ResourceManager service. | | container-executor | Change values in Hadoop YARN's container-executor.cfg file. | Not available. | | container-log4j | Change values in Hadoop YARN's container-log4j.properties file. | Not available. | | core-site | Change values in Hadoop's core-site.xml file. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts Hadoop KMS, Ranger KMS, HiveServer2, Hive MetaStore, Hadoop Httpfs, and MapReduce-HistoryServer. | | docker-conf | Change docker related settings. | Not available. | | emrfs-site | Change EMRFS settings. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts HBaseRegionserver, HBaseMaster, HBaseThrift, HBaseRest, HiveServer2, Hive MetaStore, Hadoop Httpfs, and MapReduce-HistoryServer. | | flink-conf | Change flink-conf.yaml settings. | Restarts Flink history server. | | flink-log4j | Change Flink log4j.properties settings. | Restarts Flink history server. | | flink-log4j-session | Change Flink log4j-session.properties settings for Kubernetes/Yarn session. | Restarts Flink history server. | | flink-log4j-cli | Change Flink log4j-cli.properties settings. | Restarts Flink history server. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts PhoenixQueryserver, HiveServer2, Hive MetaStore, and MapReduce-HistoryServer. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | Restarts the Hadoop HDFS services SecondaryNamenode, Datanode, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts Hadoop KMS, Hadoop Httpfs, and MapReduce-HistoryServer. | | hadoop-ssl-server | Change hadoop ssl server configuration | Not available. | | hadoop-ssl-client | Change hadoop ssl client configuration | Not available. | | hbase | Amazon EMR-curated settings for Apache HBase. | Custom EMR specific property. Sets emrfs-site and hbase-site configs. See those for their associated restarts. | | hbase-env | Change values in HBase's environment. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbase.properties file. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | Not available. | | hbase-site | Change values in HBase's hbase-site.xml file. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. Additionally restarts Phoenix QueryServer. | | hdfs-encryption-zones | Configure HDFS encryption zones. | This classification should not be reconfigured. | | hdfs-env | Change values in the HDFS environment. | Restarts Hadoop HDFS services Namenode, Datanode, and ZKFC. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Additionally restarts Hadoop Httpfs. | | hcatalog-env | Change values in HCatalog's environment. | Restarts Hive HCatalog Server. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | Restarts Hive HCatalog Server. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | Restarts Hive HCatalog Server. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | Restarts Hive WebHCat server. | | hcatalog-webhcat-log4j2 | Change values in HCatalog WebHCat's log4j2.properties. | Restarts Hive WebHCat server. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | Restarts Hive WebHCat server. | | hive | Amazon EMR-curated settings for Apache Hive. | Sets configurations to launch Hive LLAP service. | | hive-beeline-log4j2 | Change values in Hive's beeline-log4j2.properties file. | Not available. | | hive-parquet-logging | Change values in Hive's parquet-logging.properties file. | Not available. | | hive-env | Change values in the Hive environment. | Restarts HiveServer2, HiveMetastore, and Hive HCatalog-Server. Runs Hive schemaTool CLI commands to verify hive-metastore. | | hive-exec-log4j2 | Change values in Hive's hive-exec-log4j2.properties file. | Not available. | | hive-llap-daemon-log4j2 | Change values in Hive's llap-daemon-log4j2.properties file. | Not available. | | hive-log4j2 | Change values in Hive's hive-log4j2.properties file. | Not available. | | hive-site | Change values in Hive's hive-site.xml file | Restarts HiveServer2, HiveMetastore, and Hive HCatalog-Server. Runs Hive schemaTool CLI commands to verify hive-metastore. Also restarts Oozie and Zeppelin. | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | Not available. | | hue-ini | Change values in Hue's ini file | Restarts Hue. Also activates Hue config override CLI commands to pick up new configurations. | | httpfs-env | Change values in the HTTPFS environment. | Restarts Hadoop Httpfs service. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | Restarts Hadoop Httpfs service. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | Not available. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | Restarts Hadoop-KMS service. | | hadoop-kms-java-home | Change Hadoop's KMS java home | Not available. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | Not available. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | Restarts Hadoop-KMS and Ranger-KMS service. | | hudi-env | Change values in the Hudi environment. | Not available. | | hudi-defaults | Change values in Hudi's hudi-defaults.conf file. | Not available. | | iceberg-defaults | Change values in Iceberg's iceberg-defaults.conf file. | Not available. | | delta-defaults | Change values in Delta's delta-defaults.conf file. | Not available. | | jupyter-notebook-conf | Change values in Jupyter Notebook's jupyter\$1notebook\$1config.py file. | Not available. | | jupyter-hub-conf | Change values in JupyterHubs's jupyterhub\$1config.py file. | Not available. | | jupyter-s3-conf | Configure Jupyter Notebook S3 persistence. | Not available. | | jupyter-sparkmagic-conf | Change values in Sparkmagic's config.json file. | Not available. | | livy-conf | Change values in Livy's livy.conf file. | Restarts Livy Server. | | livy-env | Change values in the Livy environment. | Restarts Livy Server. | | livy-log4j2 | Change Livy log4j2.properties settings. | Restarts Livy Server. | | mapred-env | Change values in the MapReduce application's environment. | Restarts Hadoop MapReduce-HistoryServer. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | Restarts Hadoop MapReduce-HistoryServer. | | oozie-env | Change values in Oozie's environment. | Restarts Oozie. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | Restarts Oozie. | | oozie-site | Change values in Oozie's oozie-site.xml file. | Restarts Oozie. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | Not available. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | Not available. | | phoenix-log4j2 | Change values in Phoenix's log4j2.properties file. | Restarts Phoenix-QueryServer. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | Not available. | | pig-env | Change values in the Pig environment. | Not available. | | pig-properties | Change values in Pig's pig.properties file. | Restarts Oozie. | | pig-log4j | Change values in Pig's log4j.properties file. | Not available. | | presto-log | Change values in Presto's log.properties file. | Restarts Presto-Server (for PrestoDB) | | presto-config | Change values in Presto's config.properties file. | Restarts Presto-Server (for PrestoDB) | | presto-password-authenticator | Change values in Presto's password-authenticator.properties file. | Not available. | | presto-env | Change values in Presto's presto-env.sh file. | Restarts Presto-Server (for PrestoDB) | | presto-node | Change values in Presto's node.properties file. | Not available. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | Not available. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | Not available. | | presto-connector-hive | Change values in Presto's hive.properties file. | Restarts Presto-Server (for PrestoDB) | | presto-connector-jmx | Change values in Presto's jmx.properties file. | Not available. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | Not available. | | presto-connector-lakeformation | Change values in Presto's lakeformation.properties file. | Restarts Presto-Server (for PrestoDB) | | presto-connector-localfile | Change values in Presto's localfile.properties file. | Not available. | | presto-connector-memory | Change values in Presto's memory.properties file. | Not available. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | Not available. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | Not available. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | Not available. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | Not available. | | presto-connector-redis | Change values in Presto's redis.properties file. | Not available. | | presto-connector-redshift | Change values in Presto's redshift.properties file. | Not available. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | Not available. | | presto-connector-tpcds | Change values in Presto's tpcds.properties file. | Not available. | | trino-log | Change values in Trino's log.properties file. | Restarts Trino-Server (for Trino) | | trino-config | Change values in Trino's config.properties file. | Restarts Trino-Server (for Trino) | | trino-password-authenticator | Change values in Trino's password-authenticator.properties file. | Restarts Trino-Server (for Trino) | | trino-env | Change values in Trino's trino-env.sh file. | Restarts Trino-Server (for Trino) | | trino-node | Change values in Trino's node.properties file. | Not available. | | trino-connector-blackhole | Change values in Trino's blackhole.properties file. | Not available. | | trino-connector-cassandra | Change values in Trino's cassandra.properties file. | Not available. | | trino-connector-delta | Change values in Trino's delta.properties file. | Restarts Trino-Server (for Trino) | | trino-connector-hive | Change values in Trino's hive.properties file. | Restarts Trino-Server (for Trino) | | trino-exchange-manager | Change values in Trino's exchange-manager.properties file. | Restarts Trino-Server (for Trino) | | trino-connector-iceberg | Change values in Trino's iceberg.properties file. | Restarts Trino-Server (for Trino) | | trino-connector-hudi | Change values in Trino's hudi.properties file. | Restarts Trino-Server (for Trino) | | trino-connector-jmx | Change values in Trino's jmx.properties file. | Not available. | | trino-connector-kafka | Change values in Trino's kafka.properties file. | Not available. | | trino-connector-localfile | Change values in Trino's localfile.properties file. | Not available. | | trino-connector-memory | Change values in Trino's memory.properties file. | Not available. | | trino-connector-mongodb | Change values in Trino's mongodb.properties file. | Not available. | | trino-connector-mysql | Change values in Trino's mysql.properties file. | Not available. | | trino-connector-postgresql | Change values in Trino's postgresql.properties file. | Not available. | | trino-connector-raptor | Change values in Trino's raptor.properties file. | Not available. | | trino-connector-redis | Change values in Trino's redis.properties file. | Not available. | | trino-connector-redshift | Change values in Trino's redshift.properties file. | Not available. | | trino-connector-tpch | Change values in Trino's tpch.properties file. | Not available. | | trino-connector-tpcds | Change values in Trino's tpcds.properties file. | Not available. | | ranger-kms-dbks-site | Change values in dbks-site.xml file of Ranger KMS. | Restarts Ranger KMS Server. | | ranger-kms-site | Change values in ranger-kms-site.xml file of Ranger KMS. | Restarts Ranger KMS Server. | | ranger-kms-env | Change values in the Ranger KMS environment. | Restarts Ranger KMS Server. | | ranger-kms-logback | Change values in kms-logback.xml file of Ranger KMS. | Not available. | | ranger-kms-db-ca | Change values for CA file on S3 for MySQL SSL connection with Ranger KMS. | Not available. | | spark | Amazon EMR-curated settings for Apache Spark. | This property modifies spark-defaults. See actions there. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | Restarts Spark history server and Spark thrift server. | | spark-env | Change values in the Spark environment. | Restarts Spark history server and Spark thrift server. | | spark-hive-site | Change values in Spark's hive-site.xml file | Not available. | | spark-log4j2 | Change values in Spark's log4j2.properties file. | Restarts Spark history server and Spark thrift server. | | spark-metrics | Change values in Spark's metrics.properties file. | Restarts Spark history server and Spark thrift server. | | sqoop-env | Change values in Sqoop's environment. | Not available. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | Not available. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | Not available. | | tez-site | Change values in Tez's tez-site.xml file. | Restart Oozie and HiveServer2. | | yarn-env | Change values in the YARN environment. | Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts MapReduce-HistoryServer. | | yarn-site | Change values in YARN's yarn-site.xml file. | Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts Livy Server and MapReduce-HistoryServer. | | zeppelin-env | Change values in the Zeppelin environment. | Restarts Zeppelin. | | zeppelin-site | Change configuration settings in zeppelin-site.xml. | Restarts Zeppelin. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | Restarts Zookeeper server. | | zookeeper-logback | Change values in ZooKeeper's logback.xml file. | Restarts Zookeeper server. | | emr-metrics | Change emr metric settings for this node. | Restarts the CloudWatchAgent service. | ## 7.4.0 change log **Change log for 7.4.0 release and release notes** | Date | Event | Description | | --- | --- | --- | | 2024-11-18 | Docs publication | Amazon EMR 7.4.0 release notes first published | | 2024-11-13 | Deployment complete | Amazon EMR 7.4.0 fully deployed to all [supported Regions](https://aws.amazon.com/about-aws/global-infrastructure/regional-product-services/) | | 2024-10-31 | Start of deployment | Amazon EMR 7.4.0 first deployed to initial commercial Regions | # Amazon EMR release 7.3.0 ## 7.3.0 supported lifecycle The following table describes the supported lifecycle dates for Amazon EMR release 7.3.0. | Support phase | Date | | --- | --- | | Initial release date | October 16, 2024 | | Standard support until | October 15, 2026 | | End of support | October 16, 2026 | | End of life | October 16, 2027 | ## 7.3.0 application versions This release includes the following applications: [https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/Install-CloudWatch-Agent.html](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/Install-CloudWatch-Agent.html), [https://delta.io/](https://delta.io/), [https://flink.apache.org/](https://flink.apache.org/), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [https://hudi.apache.org](https://hudi.apache.org), [http://gethue.com/](http://gethue.com/), [https://iceberg.apache.org/](https://iceberg.apache.org/), [https://jupyter-enterprise-gateway.readthedocs.io/en/latest/](https://jupyter-enterprise-gateway.readthedocs.io/en/latest/), [https://jupyterhub.readthedocs.io/en/latest/#](https://jupyterhub.readthedocs.io/en/latest/#), [https://livy.incubator.apache.org/](https://livy.incubator.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://www.tensorflow.org/](https://www.tensorflow.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://trino.io/](https://trino.io/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-7.3.0 | emr-7.2.0 | emr-7.1.0 | emr-7.0.0 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 2.25.70, 1.12.747 | 2.23.18, 1.12.705 | 2.23.18, 1.12.656 | 2.20.160-amzn-0, 1.12.569 | | Python | 3.9, 3.11 | 3.9, 3.11 | 3.9, 3.11 | 3.9 | | Scala | 2.12.18 | 2.12.17 | 2.12.17 | 2.12.17 | | AmazonCloudWatchAgent | 1.300032.2-amzn-0 | 1.300032.2-amzn-0 | 1.300032.2-amzn-0 | 1.300031.1-amzn-0 | | Delta | 3.2.0-amzn-0 | 3.1.0 | 3.0.0 | 3.0.0 | | Flink | 1.18.1-amzn-2 | 1.18.1-amzn-1 | 1.18.1-amzn-0 | 1.18.0-amzn-0 | | Ganglia | - | - | - | - | | HBase | 2.4.17-amzn-7 | 2.4.17-amzn-6 | 2.4.17-amzn-5 | 2.4.17-amzn-4 | | HCatalog | 3.1.3-amzn-12 | 3.1.3-amzn-11 | 3.1.3-amzn-10 | 3.1.3-amzn-9 | | Hadoop | 3.3.6-amzn-5 | 3.3.6-amzn-4 | 3.3.6-amzn-3 | 3.3.6-amzn-2 | | Hive | 3.1.3-amzn-12 | 3.1.3-amzn-11 | 3.1.3-amzn-10 | 3.1.3-amzn-9 | | Hudi | 0.15.0-amzn-0 | 0.14.1-amzn-1 | 0.14.1-amzn-0 | 0.14.0-amzn-1 | | Hue | 4.11.0 | 4.11.0 | 4.11.0 | 4.11.0 | | Iceberg | 1.5.2-amzn-0 | 1.5.0-amzn-0 | 1.4.3-amzn-0 | 1.4.2-amzn-0 | | JupyterEnterpriseGateway | 2.6.0 | 2.6.0 | 2.6.0 | 2.6.0 | | JupyterHub | 1.5.0 | 1.5.0 | 1.5.0 | 1.5.0 | | Livy | 0.8.0-incubating | 0.8.0-incubating | 0.8.0-incubating | 0.7.1-incubating | | MXNet | - | 1.9.1 | 1.9.1 | 1.9.1 | | Mahout | - | - | - | - | | Oozie | 5.2.1 | 5.2.1 | 5.2.1 | 5.2.1 | | Phoenix | 5.1.3 | 5.1.3 | 5.1.3 | 5.1.3 | | Pig | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | | Presto | 0.285-amzn-1 | 0.285-amzn-0 | 0.284-amzn-0 | 0.283-amzn-1 | | Spark | 3.5.1-amzn-1 | 3.5.1-amzn-0 | 3.5.0-amzn-1 | 3.5.0-amzn-0 | | Sqoop | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | | TensorFlow | 2.16.1 | 2.11.0 | 2.11.0 | 2.11.0 | | Tez | 0.10.2-amzn-10 | 0.10.2-amzn-9 | 0.10.2-amzn-8 | 0.10.2-amzn-7 | | Trino (PrestoSQL) | 442-amzn-0 | 436-amzn-0 | 435-amzn-0 | 426-amzn-1 | | Zeppelin | 0.11.1 | 0.10.1 | 0.10.1 | 0.10.1 | | ZooKeeper | 3.9.1-amzn-1 | 3.9.1-amzn-1 | 3.9.1-amzn-0 | 3.5.10 | # 7.3.0 release components details See the following table for more information about the *Extras* packages in Amazon EMR 7.3.0. | Application | Application Type | Package | Package Type | Package Version | | --- | --- | --- | --- | --- | | AmazonCloudWatchAgent | Core | hadoop-hdfs-journalnode | Extras | 3.3.6.amzn.5-1.amzn2023 | | AmazonCloudWatchAgent | Core | ranger-kms | Extras | 2.0.0-1.amzn2023 | | AmazonCloudWatchAgent | Core | s3-dist-cp | Extras | 2.33.0-1.amzn2023 | | AmazonCloudWatchAgent | Core | zookeeper | Extras | 3.9.1.amzn.1-1.amzn2023 | | AmazonCloudWatchAgent | Core | zookeeper-server | Extras | 3.9.1.amzn.1-1.amzn2023 | | Flink | Core | hadoop-client | Extras | 3.3.6.amzn.5-1.amzn2023 | | Flink | Core | hadoop-hdfs | Extras | 3.3.6.amzn.5-1.amzn2023 | | Flink | Core | hadoop-hdfs-datanode | Extras | 3.3.6.amzn.5-1.amzn2023 | | Flink | Core | hadoop-hdfs-journalnode | Extras | 3.3.6.amzn.5-1.amzn2023 | | Flink | Core | hadoop-hdfs-namenode | Extras | 3.3.6.amzn.5-1.amzn2023 | | Flink | Core | hadoop-httpfs | Extras | 3.3.6.amzn.5-1.amzn2023 | | Flink | Core | hadoop-kms | Extras | 3.3.6.amzn.5-1.amzn2023 | | Flink | Core | hadoop-mapreduce | Extras | 3.3.6.amzn.5-1.amzn2023 | | Flink | Core | hadoop-mapreduce-historyserver | Extras | 3.3.6.amzn.5-1.amzn2023 | | Flink | Core | hadoop-yarn | Extras | 3.3.6.amzn.5-1.amzn2023 | | Flink | Core | hadoop-yarn-nodemanager | Extras | 3.3.6.amzn.5-1.amzn2023 | | Flink | Core | hadoop-yarn-proxyserver | Extras | 3.3.6.amzn.5-1.amzn2023 | | Flink | Core | hadoop-yarn-resourcemanager | Extras | 3.3.6.amzn.5-1.amzn2023 | | Flink | Core | hadoop-yarn-timelineserver | Extras | 3.3.6.amzn.5-1.amzn2023 | | Flink | Core | hudi | Extras | 0.15.0.amzn.0-1.amzn2023 | | Flink | Core | ranger-kms | Extras | 2.0.0-1.amzn2023 | | Flink | Core | hadoop | Extras | 3.3.6.amzn.5-1.amzn2023 | | HBase | Core | s3-dist-cp | Extras | 2.33.0-1.amzn2023 | | HCatalog | Core | hadoop-yarn-proxyserver | Extras | 3.3.6.amzn.5-1.amzn2023 | | HCatalog | Core | mariadb-connector-java | Extras | | | Hive | Core | hadoop-httpfs | Extras | 3.3.6.amzn.5-1.amzn2023 | | Hive | Core | hadoop-yarn-proxyserver | Extras | 3.3.6.amzn.5-1.amzn2023 | | Hive | Core | hive-hbase | Extras | 3.1.3.amzn.12-1.amzn2023 | | Hive | Core | mariadb-connector-java | Extras | 2.7.2-1 | | Hive | Core | s3-dist-cp | Extras | 2.33.0-1.amzn2023 | | Phoenix | Core | s3-dist-cp | Extras | 2.33.0-1.amzn2023 | | Spark | Core | spark-datanucleus | Extras | 3.5.1.amzn.0-1.amzn2023 | | Tez | Core | hadoop-mapreduce | Extras | 3.3.6.amzn.5-1.amzn2023 | | Tez | Core | hadoop-mapreduce-historyserver | Extras | 3.3.6.amzn.5-1.amzn2023 | | Tez | Core | hadoop-yarn-proxyserver | Extras | 3.3.6.amzn.5-1.amzn2023 | | Trino | Core | bigtop-utils | Extras | 1.2.0-1.amzn2023 | | Trino | Core | hadoop-client | Extras | 3.3.6.amzn.5-1.amzn2023 | | Trino | Core | hadoop-hdfs | Extras | 3.3.6.amzn.5-1.amzn2023 | | Trino | Core | hadoop-hdfs-datanode | Extras | 3.3.6.amzn.5-1.amzn2023 | | Trino | Core | hadoop-hdfs-journalnode | Extras | 3.3.6.amzn.5-1.amzn2023 | | Trino | Core | hadoop-hdfs-namenode | Extras | 3.3.6.amzn.5-1.amzn2023 | | Trino | Core | hadoop-kms | Extras | 3.3.6.amzn.5-1.amzn2023 | | Trino | Core | hadoop-mapreduce | Extras | 3.3.6.amzn.5-1.amzn2023 | | Trino | Core | hadoop-mapreduce-historyserver | Extras | 3.3.6.amzn.5-1.amzn2023 | | Trino | Core | hadoop-yarn | Extras | 3.3.6.amzn.5-1.amzn2023 | | Trino | Core | hadoop-yarn-nodemanager | Extras | 3.3.6.amzn.5-1.amzn2023 | | Trino | Core | hadoop-yarn-proxyserver | Extras | 3.3.6.amzn.5-1.amzn2023 | | Trino | Core | hadoop-yarn-resourcemanager | Extras | 3.3.6.amzn.5-1.amzn2023 | | Trino | Core | hadoop-yarn-timelineserver | Extras | 3.3.6.amzn.5-1.amzn2023 | | Trino | Core | hive | Extras | 3.1.3.amzn.12-1.amzn2023 | | Trino | Core | hive-hcatalog-server | Extras | 3.1.3.amzn.12-1.amzn2023 | | Trino | Core | mariadb-connector-java | Extras | 2.7.2-1 | | Trino | Core | ranger-kms | Extras | 2.0.0-1.amzn2023 | | Trino | Core | zookeeper | Extras | 3.9.1.amzn.1-1.amzn2023 | | Trino | Core | zookeeper-server | Extras | 3.9.1.amzn.1-1.amzn2023 | | Trino | Core | bigtop-groovy | Extras | 2.5.4-1.amzn2023 | | Trino | Core | bigtop-jsvc | Extras | 1.2.4-1.amzn2023 | | Trino | Core | hadoop | Extras | 3.3.6.amzn.5-1.amzn2023 | | Trino | Core | hadoop-lzo | Extras | 0.4.19-1.amzn2023 | | Trino | Core | hive-hcatalog | Extras | 3.1.3.amzn.12-1.amzn2023 | | Trino | Core | hive-jdbc | Extras | 3.1.3.amzn.12-1.amzn2023 | | Zookeeper | Core | emrfs | Extras | 2.64.0-1.amzn2023 | | Zookeeper | Core | hadoop-client | Extras | 3.3.6.amzn.5-1.amzn2023 | | Zookeeper | Core | hadoop-hdfs | Extras | 3.3.6.amzn.5-1.amzn2023 | | Zookeeper | Core | hadoop-hdfs-datanode | Extras | 3.3.6.amzn.5-1.amzn2023 | | Zookeeper | Core | hadoop-hdfs-journalnode | Extras | 3.3.6.amzn.5-1.amzn2023 | | Zookeeper | Core | hadoop-hdfs-namenode | Extras | 3.3.6.amzn.5-1.amzn2023 | | Zookeeper | Core | hadoop-httpfs | Extras | 3.3.6.amzn.5-1.amzn2023 | | Zookeeper | Core | hadoop-kms | Extras | 3.3.6.amzn.5-1.amzn2023 | | Zookeeper | Core | hadoop-mapreduce | Extras | 3.3.6.amzn.5-1.amzn2023 | | Zookeeper | Core | hadoop-mapreduce-historyserver | Extras | 3.3.6.amzn.5-1.amzn2023 | | Zookeeper | Core | hadoop-yarn | Extras | 3.3.6.amzn.5-1.amzn2023 | | Zookeeper | Core | hadoop-yarn-nodemanager | Extras | 3.3.6.amzn.5-1.amzn2023 | | Zookeeper | Core | hadoop-yarn-proxyserver | Extras | 3.3.6.amzn.5-1.amzn2023 | | Zookeeper | Core | hadoop-yarn-resourcemanager | Extras | 3.3.6.amzn.5-1.amzn2023 | | Zookeeper | Core | hadoop-yarn-timelineserver | Extras | 3.3.6.amzn.5-1.amzn2023 | | Zookeeper | Core | ranger-kms | Extras | 2.0.0-1.amzn2023 | # 7.3.0 common vulnerabilities and exposures The following table lists all CVEs that don't impact EMR clusters that run on recommended configurations of Amazon EMR 7.3.0. Amazon EMR is dependent on upstream open-source for availability of fixes and will provide the latest stable version as part of the Core Engine components within 90 days of Amazon EMR verifying the fixes. This table was updated on October 3, 2024. | CVE ID | Severity | CVE details URL | | --- | --- | --- | | CVE-2015-1832 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2015-1832](https://nvd.nist.gov/vuln/detail/CVE-2015-1832) | | CVE-2016-5018 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2016-5018](https://nvd.nist.gov/vuln/detail/CVE-2016-5018) | | CVE-2018-11307 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-11307](https://nvd.nist.gov/vuln/detail/CVE-2018-11307) | | CVE-2018-14718 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-14718](https://nvd.nist.gov/vuln/detail/CVE-2018-14718) | | CVE-2018-14719 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-14719](https://nvd.nist.gov/vuln/detail/CVE-2018-14719) | | CVE-2018-14720 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-14720](https://nvd.nist.gov/vuln/detail/CVE-2018-14720) | | CVE-2018-14721 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-14721](https://nvd.nist.gov/vuln/detail/CVE-2018-14721) | | CVE-2018-19360 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-19360](https://nvd.nist.gov/vuln/detail/CVE-2018-19360) | | CVE-2018-19361 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-19361](https://nvd.nist.gov/vuln/detail/CVE-2018-19361) | | CVE-2018-19362 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-19362](https://nvd.nist.gov/vuln/detail/CVE-2018-19362) | | CVE-2018-7489 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-7489](https://nvd.nist.gov/vuln/detail/CVE-2018-7489) | | CVE-2019-14379 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-14379](https://nvd.nist.gov/vuln/detail/CVE-2019-14379) | | CVE-2019-14540 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-14540](https://nvd.nist.gov/vuln/detail/CVE-2019-14540) | | CVE-2019-14892 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-14892](https://nvd.nist.gov/vuln/detail/CVE-2019-14892) | | CVE-2019-14893 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-14893](https://nvd.nist.gov/vuln/detail/CVE-2019-14893) | | CVE-2019-16335 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-16335](https://nvd.nist.gov/vuln/detail/CVE-2019-16335) | | CVE-2019-16942 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-16942](https://nvd.nist.gov/vuln/detail/CVE-2019-16942) | | CVE-2019-16943 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-16943](https://nvd.nist.gov/vuln/detail/CVE-2019-16943) | | CVE-2019-17267 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-17267](https://nvd.nist.gov/vuln/detail/CVE-2019-17267) | | CVE-2019-17531 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-17531](https://nvd.nist.gov/vuln/detail/CVE-2019-17531) | | CVE-2019-20330 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-20330](https://nvd.nist.gov/vuln/detail/CVE-2019-20330) | | CVE-2019-20444 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-20444](https://nvd.nist.gov/vuln/detail/CVE-2019-20444) | | CVE-2019-20445 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-20445](https://nvd.nist.gov/vuln/detail/CVE-2019-20445) | | CVE-2020-8840 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2020-8840](https://nvd.nist.gov/vuln/detail/CVE-2020-8840) | | CVE-2020-9546 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2020-9546](https://nvd.nist.gov/vuln/detail/CVE-2020-9546) | | CVE-2020-9547 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2020-9547](https://nvd.nist.gov/vuln/detail/CVE-2020-9547) | | CVE-2020-9548 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2020-9548](https://nvd.nist.gov/vuln/detail/CVE-2020-9548) | | CVE-2022-1471 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2022-1471](https://nvd.nist.gov/vuln/detail/CVE-2022-1471) | | CVE-2022-1471 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2022-1471](https://nvd.nist.gov/vuln/detail/CVE-2022-1471) | | CVE-2022-1471 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2022-1471](https://nvd.nist.gov/vuln/detail/CVE-2022-1471) | | CVE-2022-37865 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2022-37865](https://nvd.nist.gov/vuln/detail/CVE-2022-37865) | | CVE-2022-39135 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2022-39135](https://nvd.nist.gov/vuln/detail/CVE-2022-39135) | | CVE-2022-46337 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2022-46337](https://nvd.nist.gov/vuln/detail/CVE-2022-46337) | | CVE-2022-46337 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2022-46337](https://nvd.nist.gov/vuln/detail/CVE-2022-46337) | | CVE-2023-44981 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2023-44981](https://nvd.nist.gov/vuln/detail/CVE-2023-44981) | | CVE-2023-44981 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2023-44981](https://nvd.nist.gov/vuln/detail/CVE-2023-44981) | | CVE-2023-44981 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2023-44981](https://nvd.nist.gov/vuln/detail/CVE-2023-44981) | | CVE-2012-0881 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2012-0881](https://nvd.nist.gov/vuln/detail/CVE-2012-0881) | | CVE-2013-4002 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2013-4002](https://nvd.nist.gov/vuln/detail/CVE-2013-4002) | | CVE-2015-2156 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2015-2156](https://nvd.nist.gov/vuln/detail/CVE-2015-2156) | | CVE-2015-5237 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2015-5237](https://nvd.nist.gov/vuln/detail/CVE-2015-5237) | | CVE-2016-6796 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2016-6796](https://nvd.nist.gov/vuln/detail/CVE-2016-6796) | | CVE-2017-18640 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2017-18640](https://nvd.nist.gov/vuln/detail/CVE-2017-18640) | | CVE-2017-18640 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2017-18640](https://nvd.nist.gov/vuln/detail/CVE-2017-18640) | | CVE-2018-1000180 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2018-1000180](https://nvd.nist.gov/vuln/detail/CVE-2018-1000180) | | CVE-2018-12022 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2018-12022](https://nvd.nist.gov/vuln/detail/CVE-2018-12022) | | CVE-2018-12023 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2018-12023](https://nvd.nist.gov/vuln/detail/CVE-2018-12023) | | CVE-2019-0205 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2019-0205](https://nvd.nist.gov/vuln/detail/CVE-2019-0205) | | CVE-2019-0205 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2019-0205](https://nvd.nist.gov/vuln/detail/CVE-2019-0205) | | CVE-2019-12086 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2019-12086](https://nvd.nist.gov/vuln/detail/CVE-2019-12086) | | CVE-2019-14439 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2019-14439](https://nvd.nist.gov/vuln/detail/CVE-2019-14439) | | CVE-2019-16869 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2019-16869](https://nvd.nist.gov/vuln/detail/CVE-2019-16869) | | CVE-2020-10650 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-10650](https://nvd.nist.gov/vuln/detail/CVE-2020-10650) | | CVE-2020-10672 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-10672](https://nvd.nist.gov/vuln/detail/CVE-2020-10672) | | CVE-2020-10673 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-10673](https://nvd.nist.gov/vuln/detail/CVE-2020-10673) | | CVE-2020-10968 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-10968](https://nvd.nist.gov/vuln/detail/CVE-2020-10968) | | CVE-2020-10969 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-10969](https://nvd.nist.gov/vuln/detail/CVE-2020-10969) | | CVE-2020-11111 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-11111](https://nvd.nist.gov/vuln/detail/CVE-2020-11111) | | CVE-2020-11112 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-11112](https://nvd.nist.gov/vuln/detail/CVE-2020-11112) | | CVE-2020-11113 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-11113](https://nvd.nist.gov/vuln/detail/CVE-2020-11113) | | CVE-2020-11619 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-11619](https://nvd.nist.gov/vuln/detail/CVE-2020-11619) | | CVE-2020-11620 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-11620](https://nvd.nist.gov/vuln/detail/CVE-2020-11620) | | CVE-2020-13936 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-13936](https://nvd.nist.gov/vuln/detail/CVE-2020-13936) | | CVE-2020-13949 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-13949](https://nvd.nist.gov/vuln/detail/CVE-2020-13949) | | CVE-2020-13949 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-13949](https://nvd.nist.gov/vuln/detail/CVE-2020-13949) | | CVE-2020-13949 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-13949](https://nvd.nist.gov/vuln/detail/CVE-2020-13949) | | CVE-2020-14060 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-14060](https://nvd.nist.gov/vuln/detail/CVE-2020-14060) | | CVE-2020-14061 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-14061](https://nvd.nist.gov/vuln/detail/CVE-2020-14061) | | CVE-2020-14062 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-14062](https://nvd.nist.gov/vuln/detail/CVE-2020-14062) | | CVE-2020-14195 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-14195](https://nvd.nist.gov/vuln/detail/CVE-2020-14195) | | CVE-2020-24616 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-24616](https://nvd.nist.gov/vuln/detail/CVE-2020-24616) | | CVE-2020-24750 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-24750](https://nvd.nist.gov/vuln/detail/CVE-2020-24750) | | CVE-2020-25649 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-25649](https://nvd.nist.gov/vuln/detail/CVE-2020-25649) | | CVE-2020-28052 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-28052](https://nvd.nist.gov/vuln/detail/CVE-2020-28052) | | CVE-2020-35490 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-35490](https://nvd.nist.gov/vuln/detail/CVE-2020-35490) | | CVE-2020-35491 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-35491](https://nvd.nist.gov/vuln/detail/CVE-2020-35491) | | CVE-2020-35728 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-35728](https://nvd.nist.gov/vuln/detail/CVE-2020-35728) | | CVE-2020-36179 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36179](https://nvd.nist.gov/vuln/detail/CVE-2020-36179) | | CVE-2020-36180 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36180](https://nvd.nist.gov/vuln/detail/CVE-2020-36180) | | CVE-2020-36181 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36181](https://nvd.nist.gov/vuln/detail/CVE-2020-36181) | | CVE-2020-36182 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36182](https://nvd.nist.gov/vuln/detail/CVE-2020-36182) | | CVE-2020-36183 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36183](https://nvd.nist.gov/vuln/detail/CVE-2020-36183) | | CVE-2020-36184 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36184](https://nvd.nist.gov/vuln/detail/CVE-2020-36184) | | CVE-2020-36185 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36185](https://nvd.nist.gov/vuln/detail/CVE-2020-36185) | | CVE-2020-36186 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36186](https://nvd.nist.gov/vuln/detail/CVE-2020-36186) | | CVE-2020-36187 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36187](https://nvd.nist.gov/vuln/detail/CVE-2020-36187) | | CVE-2020-36188 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36188](https://nvd.nist.gov/vuln/detail/CVE-2020-36188) | | CVE-2020-36189 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36189](https://nvd.nist.gov/vuln/detail/CVE-2020-36189) | | CVE-2020-36518 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36518](https://nvd.nist.gov/vuln/detail/CVE-2020-36518) | | CVE-2021-20190 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-20190](https://nvd.nist.gov/vuln/detail/CVE-2021-20190) | | CVE-2021-31684 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-31684](https://nvd.nist.gov/vuln/detail/CVE-2021-31684) | | CVE-2021-31684 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-31684](https://nvd.nist.gov/vuln/detail/CVE-2021-31684) | | CVE-2021-34538 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-34538](https://nvd.nist.gov/vuln/detail/CVE-2021-34538) | | CVE-2021-34538 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-34538](https://nvd.nist.gov/vuln/detail/CVE-2021-34538) | | CVE-2021-34538 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-34538](https://nvd.nist.gov/vuln/detail/CVE-2021-34538) | | CVE-2021-35515 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-35515](https://nvd.nist.gov/vuln/detail/CVE-2021-35515) | | CVE-2021-35516 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-35516](https://nvd.nist.gov/vuln/detail/CVE-2021-35516) | | CVE-2021-35517 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-35517](https://nvd.nist.gov/vuln/detail/CVE-2021-35517) | | CVE-2021-36090 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-36090](https://nvd.nist.gov/vuln/detail/CVE-2021-36090) | | CVE-2021-37136 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-37136](https://nvd.nist.gov/vuln/detail/CVE-2021-37136) | | CVE-2021-37137 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-37137](https://nvd.nist.gov/vuln/detail/CVE-2021-37137) | | CVE-2022-25647 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-25647](https://nvd.nist.gov/vuln/detail/CVE-2022-25647) | | CVE-2022-25857 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-25857](https://nvd.nist.gov/vuln/detail/CVE-2022-25857) | | CVE-2022-25857 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-25857](https://nvd.nist.gov/vuln/detail/CVE-2022-25857) | | CVE-2022-25857 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-25857](https://nvd.nist.gov/vuln/detail/CVE-2022-25857) | | CVE-2022-3171 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-3171](https://nvd.nist.gov/vuln/detail/CVE-2022-3171) | | CVE-2022-3171 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-3171](https://nvd.nist.gov/vuln/detail/CVE-2022-3171) | | CVE-2022-3171 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-3171](https://nvd.nist.gov/vuln/detail/CVE-2022-3171) | | CVE-2022-3509 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-3509](https://nvd.nist.gov/vuln/detail/CVE-2022-3509) | | CVE-2022-3509 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-3509](https://nvd.nist.gov/vuln/detail/CVE-2022-3509) | | CVE-2022-3509 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-3509](https://nvd.nist.gov/vuln/detail/CVE-2022-3509) | | CVE-2022-3510 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-3510](https://nvd.nist.gov/vuln/detail/CVE-2022-3510) | | CVE-2022-3510 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-3510](https://nvd.nist.gov/vuln/detail/CVE-2022-3510) | | CVE-2022-3510 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-3510](https://nvd.nist.gov/vuln/detail/CVE-2022-3510) | | CVE-2022-36364 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-36364](https://nvd.nist.gov/vuln/detail/CVE-2022-36364) | | CVE-2022-37866 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-37866](https://nvd.nist.gov/vuln/detail/CVE-2022-37866) | | CVE-2022-40149 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-40149](https://nvd.nist.gov/vuln/detail/CVE-2022-40149) | | CVE-2022-40149 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-40149](https://nvd.nist.gov/vuln/detail/CVE-2022-40149) | | CVE-2022-40149 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-40149](https://nvd.nist.gov/vuln/detail/CVE-2022-40149) | | CVE-2022-40150 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-40150](https://nvd.nist.gov/vuln/detail/CVE-2022-40150) | | CVE-2022-40150 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-40150](https://nvd.nist.gov/vuln/detail/CVE-2022-40150) | | CVE-2022-40150 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-40150](https://nvd.nist.gov/vuln/detail/CVE-2022-40150) | | CVE-2022-40152 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-40152](https://nvd.nist.gov/vuln/detail/CVE-2022-40152) | | CVE-2022-42003 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-42003](https://nvd.nist.gov/vuln/detail/CVE-2022-42003) | | CVE-2022-42003 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-42003](https://nvd.nist.gov/vuln/detail/CVE-2022-42003) | | CVE-2022-42003 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-42003](https://nvd.nist.gov/vuln/detail/CVE-2022-42003) | | CVE-2022-42003 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-42003](https://nvd.nist.gov/vuln/detail/CVE-2022-42003) | | CVE-2022-42004 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-42004](https://nvd.nist.gov/vuln/detail/CVE-2022-42004) | | CVE-2022-42004 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-42004](https://nvd.nist.gov/vuln/detail/CVE-2022-42004) | | CVE-2022-42004 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-42004](https://nvd.nist.gov/vuln/detail/CVE-2022-42004) | | CVE-2022-45685 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-45685](https://nvd.nist.gov/vuln/detail/CVE-2022-45685) | | CVE-2022-45685 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-45685](https://nvd.nist.gov/vuln/detail/CVE-2022-45685) | | CVE-2022-45685 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-45685](https://nvd.nist.gov/vuln/detail/CVE-2022-45685) | | CVE-2022-45688 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-45688](https://nvd.nist.gov/vuln/detail/CVE-2022-45688) | | CVE-2022-45693 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-45693](https://nvd.nist.gov/vuln/detail/CVE-2022-45693) | | CVE-2022-45693 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-45693](https://nvd.nist.gov/vuln/detail/CVE-2022-45693) | | CVE-2022-45693 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-45693](https://nvd.nist.gov/vuln/detail/CVE-2022-45693) | | CVE-2022-46751 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-46751](https://nvd.nist.gov/vuln/detail/CVE-2022-46751) | | CVE-2023-1370 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-1370](https://nvd.nist.gov/vuln/detail/CVE-2023-1370) | | CVE-2023-1370 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-1370](https://nvd.nist.gov/vuln/detail/CVE-2023-1370) | | CVE-2023-1436 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-1436](https://nvd.nist.gov/vuln/detail/CVE-2023-1436) | | CVE-2023-1436 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-1436](https://nvd.nist.gov/vuln/detail/CVE-2023-1436) | | CVE-2023-1436 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-1436](https://nvd.nist.gov/vuln/detail/CVE-2023-1436) | | CVE-2023-2976 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-2976](https://nvd.nist.gov/vuln/detail/CVE-2023-2976) | | CVE-2023-2976 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-2976](https://nvd.nist.gov/vuln/detail/CVE-2023-2976) | | CVE-2023-2976 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-2976](https://nvd.nist.gov/vuln/detail/CVE-2023-2976) | | CVE-2023-2976 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-2976](https://nvd.nist.gov/vuln/detail/CVE-2023-2976) | | CVE-2023-2976 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-2976](https://nvd.nist.gov/vuln/detail/CVE-2023-2976) | | CVE-2023-2976 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-2976](https://nvd.nist.gov/vuln/detail/CVE-2023-2976) | | CVE-2023-2976 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-2976](https://nvd.nist.gov/vuln/detail/CVE-2023-2976) | | CVE-2023-2976 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-2976](https://nvd.nist.gov/vuln/detail/CVE-2023-2976) | | CVE-2023-2976 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-2976](https://nvd.nist.gov/vuln/detail/CVE-2023-2976) | | CVE-2023-2976 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-2976](https://nvd.nist.gov/vuln/detail/CVE-2023-2976) | | CVE-2023-34453 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-34453](https://nvd.nist.gov/vuln/detail/CVE-2023-34453) | | CVE-2023-34454 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-34454](https://nvd.nist.gov/vuln/detail/CVE-2023-34454) | | CVE-2023-34455 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-34455](https://nvd.nist.gov/vuln/detail/CVE-2023-34455) | | CVE-2023-3635 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-3635](https://nvd.nist.gov/vuln/detail/CVE-2023-3635) | | CVE-2023-36478 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-36478](https://nvd.nist.gov/vuln/detail/CVE-2023-36478) | | CVE-2023-36478 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-36478](https://nvd.nist.gov/vuln/detail/CVE-2023-36478) | | CVE-2023-39410 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-39410](https://nvd.nist.gov/vuln/detail/CVE-2023-39410) | | CVE-2023-39410 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-39410](https://nvd.nist.gov/vuln/detail/CVE-2023-39410) | | CVE-2023-43642 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-43642](https://nvd.nist.gov/vuln/detail/CVE-2023-43642) | | CVE-2023-43642 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-43642](https://nvd.nist.gov/vuln/detail/CVE-2023-43642) | | CVE-2023-50387 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-50387](https://nvd.nist.gov/vuln/detail/CVE-2023-50387) | | CVE-2023-5072 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-5072](https://nvd.nist.gov/vuln/detail/CVE-2023-5072) | | CVE-2023-6378 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-6378](https://nvd.nist.gov/vuln/detail/CVE-2023-6378) | | CVE-2023-6378 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-6378](https://nvd.nist.gov/vuln/detail/CVE-2023-6378) | | CVE-2023-6481 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-6481](https://nvd.nist.gov/vuln/detail/CVE-2023-6481) | | CVE-2023-6481 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-6481](https://nvd.nist.gov/vuln/detail/CVE-2023-6481) | | CVE-2024-21634 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-21634](https://nvd.nist.gov/vuln/detail/CVE-2024-21634) | ## 7.3.0 release notes The following release notes include information for Amazon EMR release 7.3.0. Changes are relative to 7.3.0. **New features** + **Application upgrades** – Amazon EMR 7.3.0 application upgrades include Delta 3.2.0-amzn-0, Hudi 0.15.0-amzn-0, TensorFlow 2.16.1, Trino 442, and Zeppelin 0.11.1. + This release adds 22 open-source endpoints that support in-transit encryption over the network. This improvement includes open-source frameworks such as Hadoop, Hive, Tez, HBase, Phoenix, and Flink. If you enabled in-transit encryption and want to connect to these endpoints, you might have to modify your code to support TLS communication. You can disable TLS for any of these endpoints as needed. For a full list of the supported endpoints, see [Understanding in-transit encryption](https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-encryption-support-matrix.html). + With Amazon EMR 7.3.0, Apache Hive with Iceberg now supports the AWS Glue Data Catalog as a metastore, which gives you benefits such as a centralized repository to store metadata, automatically crawl data sources and extract metadata, and integration with other services for seamless workflows. **Known issues** + Python 3.11 isn't supported with EMR Studio. + If you use in-transit encryption and Kerberos authentication, you won't be able to use Spark applications that depend on the Hive metastore. This issue is fixed in [HIVE-16340](https://issues.apache.org/jira/browse/HIVE-16340) and will be fully resolved in [SPARK-44114](https://issues.apache.org/jira/browse/SPARK-44114) when open-source Spark can upgrade to Hive 3. In the meantime, you can temporarily work around this issue by setting `hive.metastore.use.SSL` to `false`. For more information, see [Configure applications](https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-configure-apps.html). + If you don't use Kerberos authentication, you must configure Spark to use SSL. To do so, set the following configurations when you provision your cluster. ``` [{ "Classification": "spark-defaults", "Properties": { "spark.hadoop.hive.metastore.truststore.path": "/usr/share/aws/emr/security/conf/keystore-thrift.jks", "spark.hadoop.hive.metastore.use.SSL": "true" } }] ``` + If you use custom domains, some links in the Spark History Server UI in the AWS console might not work. Upload the following script to an S3 bucket, and then [ add a step](https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-add-steps-console.html) to use the shell script. ``` #!/bin/bash if grep -q ":8042" /etc/nginx/nginx.conf; then sudo sed -i '=s#\^(\\/nm\\/(?.*:8042)\\/(?.*)\$)#^(\\/nm\\/(?(\\.|-|[a-zA-Z0-9])*:8042)\\/(?(node|conf|logs|stacks|jmx).*)\$)#g' /etc/nginx/nginx.conf else sudo sed -i 's#\^(\\/nm\\/(?.*:8044)\\/(?.*)\$)#^(\\/nm\\/(?(\\.|-|[a-zA-Z0-9])*:8044)\\/(?(node|conf|logs|stacks|jmx).*)\$)#g' /etc/nginx/nginx.conf fi sudo systemctl restart nginx ``` + **HBase table creation in Hive with Glue catalog** – From EMR-7.3.0 to EMR-7.10.0, there is a bug due to Hive Iceberg integration, which causes HBase table creation in Hive to fail when Glue Data Catalog is used as the metastore. Please reach out to the AWS support team if you encounter this issue. + **Tez UI** – From EMR-7.3 to EMR-7.10, there is a bug where accessing Tez UI through YARN Resource Manager Web UI fails when in-transit encryption is enabled. This occurs because YARN proxy attempts to use HTTP protocol while Tez UI requires HTTPS. Please modify the `tez.tez-ui.history-url.base` configuration in `tez-site` from `http` to `https` to resolve this issue. **Changes, enhancements, and resolved issues** + Starting with Spark 3.3.1 (supported in EMR versions 6.10 and above), all executors in a decommissioning host are set to a new `ExecutorState`, called *DECOMMISSIONING* state. The executors being decommissioned cannot be used by Yarn to allocate tasks and thus it will request for new executors, if needed, for the tasks being executed. Thus, if you disable Spark DRA while using EMR Managed Scaling, EMR Auto Scaling, or any custom scaling mechanism on EMR-EC2 clusters, then Yarn may request maximum permissible executors for each job. In order to avoid this issue, leave the `spark.dynamicAllocation.enabled` property set to `TRUE` (which is the default) when you are using the above combination of features. In addition, you can also set minimum and maximum executor constraints by setting values for `spark.dynamicAllocation.maxExecutors` and `spark.dynamicAllocation.minExecutors` properties for your Spark jobs, to restrict the number of executors allocated during the job’s execution. + To support TLS hostname verification for YARN, this release changes the `yarn.resourcemanager.hostname` configuration from the primary instance IP address to the primary instance private DNS name. + [MXNet project](https://github.com/apache/mxnet) has reached end of life and is now archived. [Amazon EMR release 7.2.0](emr-720-release.md) is the last release that includes MXNet. + When you launch a cluster with *the latest patch release* of Amazon EMR 5.36 or higher, 6.6 or higher, or 7.0 or higher, Amazon EMR uses the latest Amazon Linux 2023 or Amazon Linux 2 release for the default Amazon EMR AMI. For more information, see [Using the default Amazon Linux AMI for Amazon EMR](https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-default-ami.html). [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-730-release.html) ## 7.3.0 default Java versions Amazon EMR release 7.3 and higher ships with Amazon Corretto 17 (built on OpenJDK) by default for applications that support Corretto 17 (JDK 17), with the exception of Apache Livy. The following table shows the default Java versions for applications in Amazon EMR 7.3.0. If you want to change the default JVM on your cluster, follow the instructions in [Configure applications to use a specific Java Virtual Machine](configuring-java8.md) for each application that runs on the cluster. You can only use one Java runtime version for a cluster. Amazon EMR doesn't support running different nodes or applications on different runtime versions on the same cluster. | Application | Java / Amazon Corretto version (default is bold) | | --- | --- | | Delta | 17, 11, 8 | | Flink | 17, 11, 8 | | HBase | 17, 11, 8 | | HCatalog | 17, 11, 8 | | Hadoop | 17, 11, 8 | | Hive | 17, 11, 8 | | Hudi | 17, 11, 8 | | Iceberg | 17, 11, 8 | | Livy | 17, 11, 8 | | Oozie | 17, 11, 8 | | Phoenix | 17, 11, 8 | | PrestoDB | 8 | | Spark | 17, 11, 8 | | Spark RAPIDS | 17, 11, 8 | | Sqoop | 8 | | Tez | 17, 11, 8 | | Trino | 21, 17 | | Zeppelin | 8 | | Pig | 8 | | Zookeeper | 17, 11, 8 | ## 7.3.0 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | adot-java-agent | 1.31.0 | A Java Agent that collects metrics from application daemons. | | delta | 3.2.0-amzn-0 | Delta lake is an open table format for huge analytic datasets | | emr-amazon-cloudwatch-agent | 1.300032.2-amzn-0 | An application that collects internal system-level metrics and custom application metrics from Amazon EC2 instances. | | emr-ddb | 5.5.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 3.12.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.16.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-notebook-env | 1.16.0 | Conda env for emr notebook which includes jupyter enterprise gateway | | emr-s3-dist-cp | 2.33.0 | Distributed copy application optimized for Amazon S3. | | emr-s3-select | 2.12.0 | EMR S3Select Connector | | emr-wal-cli | 1.3.0 | Cli used for emrwal list/deletion. | | emrfs | 2.64.0 | Amazon S3 connector for Hadoop ecosystem applications. | | flink-client | 1.18.1-amzn-2 | Apache Flink command line client scripts and applications. | | flink-jobmanager-config | 1.18.1-amzn-2 | Managing resources on EMR nodes for Apache Flink JobManager. | | hadoop-client | 3.3.6-amzn-5 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 3.3.6-amzn-5 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 3.3.6-amzn-5 | HDFS command-line client and library | | hadoop-hdfs-namenode | 3.3.6-amzn-5 | HDFS service for tracking file names and block locations. | | hadoop-hdfs-journalnode | 3.3.6-amzn-5 | HDFS service for managing the Hadoop filesystem journal on HA clusters. | | hadoop-httpfs-server | 3.3.6-amzn-5 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 3.3.6-amzn-5 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 3.3.6-amzn-5 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 3.3.6-amzn-5 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 3.3.6-amzn-5 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 3.3.6-amzn-5 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 2.4.17-amzn-7 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 2.4.17-amzn-7 | Service for serving one or more HBase regions. | | hbase-client | 2.4.17-amzn-7 | HBase command-line client. | | hbase-rest-server | 2.4.17-amzn-7 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 2.4.17-amzn-7 | Service providing a Thrift endpoint to HBase. | | hbase-operator-tools | 2.4.17-amzn-7 | Repair tool for Apache HBase clusters. | | hcatalog-client | 3.1.3-amzn-12 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 3.1.3-amzn-12 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 3.1.3-amzn-12 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 3.1.3-amzn-12 | Hive command line client. | | hive-hbase | 3.1.3-amzn-12 | Hive-hbase client. | | hive-metastore-server | 3.1.3-amzn-12 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server2 | 3.1.3-amzn-12 | Service for accepting Hive queries as web requests. | | hudi | 0.15.0-amzn-0 | Incremental processing framework to power data pipeline at low latency and high efficiency. | | hudi-presto | 0.15.0-amzn-0 | Bundle library for running Presto with Hudi. | | hudi-trino | 0.15.0-amzn-0 | Bundle library for running Trino with Hudi. | | hudi-spark | 0.15.0-amzn-0 | Bundle library for running Spark with Hudi. | | hue-server | 4.11.0 | Web application for analyzing data using Hadoop ecosystem applications | | iceberg | 1.5.2-amzn-0 | Apache Iceberg is an open table format for huge analytic datasets | | jupyterhub | 1.5.0 | Multi-user server for Jupyter notebooks | | livy-server | 0.8.0-incubating | REST interface for interacting with Apache Spark | | nginx | 1.12.1 | nginx [engine x] is an HTTP and reverse proxy server | | mariadb-server | 5.5.68\$1 | MariaDB database server. | | nvidia-cuda | 12.3.0 | Nvidia drivers and Cuda toolkit | | oozie-client | 5.2.1 | Oozie command-line client. | | oozie-server | 5.2.1 | Service for accepting Oozie workflow requests. | | opencv | 4.7.0 | Open Source Computer Vision Library. | | phoenix-library | 5.1.3 | The phoenix libraries for server and client | | phoenix-connectors | 6.0.0 | Apache Phoenix-Connectors for Spark-3 | | phoenix-query-server | 6.0.0 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.285-amzn-1 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.285-amzn-1 | Service for executing pieces of a query. | | presto-client | 0.285-amzn-1 | Presto command-line client which is installed on an HA cluster's stand-by masters where Presto server is not started. | | trino-coordinator | 442-amzn-0 | Service for accepting queries and managing query execution among trino-workers. | | trino-worker | 442-amzn-0 | Service for executing pieces of a query. | | trino-client | 442-amzn-0 | Trino command-line client which is installed on an HA cluster's stand-by masters where Trino server is not started. | | pig-client | 0.17.0 | Pig command-line client. | | r | 4.3.2 | The R Project for Statistical Computing | | ranger-kms-server | 2.0.0 | Apache Ranger Key Management System | | spark-client | 3.5.1-amzn-1 | Spark command-line clients. | | spark-history-server | 3.5.1-amzn-1 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 3.5.1-amzn-1 | In-memory execution engine for YARN. | | spark-yarn-slave | 3.5.1-amzn-1 | Apache Spark libraries needed by YARN slaves. | | spark-rapids | 24.06.1-amzn-0 | Nvidia Spark RAPIDS plugin that accelerates Apache Spark with GPUs. | | sqoop-client | 1.4.7 | Apache Sqoop command-line client. | | tensorflow | 2.16.1 | TensorFlow open source software library for high performance numerical computation. | | tez-on-yarn | 0.10.2-amzn-10 | The tez YARN application and libraries. | | tez-on-worker | 0.10.2-amzn-10 | The tez YARN application and libraries for worker nodes. | | zeppelin-server | 0.11.1 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.9.1-amzn-1 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.9.1-amzn-1 | ZooKeeper command line client. | ## 7.3.0 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). Reconfiguration actions occur when you specify a configuration for instance groups in a running cluster. Amazon EMR only initiates reconfiguration actions for the classifications that you modify. For more information, see [Reconfigure an instance group in a running cluster](emr-configure-apps-running-cluster.md). **emr-7.3.0 classifications** | Classifications | Description | Reconfiguration Actions | | --- | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | Restarts the ResourceManager service. | | container-executor | Change values in Hadoop YARN's container-executor.cfg file. | Not available. | | container-log4j | Change values in Hadoop YARN's container-log4j.properties file. | Not available. | | core-site | Change values in Hadoop's core-site.xml file. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts Hadoop KMS, Ranger KMS, HiveServer2, Hive MetaStore, Hadoop Httpfs, and MapReduce-HistoryServer. | | docker-conf | Change docker related settings. | Not available. | | emrfs-site | Change EMRFS settings. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts HBaseRegionserver, HBaseMaster, HBaseThrift, HBaseRest, HiveServer2, Hive MetaStore, Hadoop Httpfs, and MapReduce-HistoryServer. | | flink-conf | Change flink-conf.yaml settings. | Restarts Flink history server. | | flink-log4j | Change Flink log4j.properties settings. | Restarts Flink history server. | | flink-log4j-session | Change Flink log4j-session.properties settings for Kubernetes/Yarn session. | Restarts Flink history server. | | flink-log4j-cli | Change Flink log4j-cli.properties settings. | Restarts Flink history server. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts PhoenixQueryserver, HiveServer2, Hive MetaStore, and MapReduce-HistoryServer. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | Restarts the Hadoop HDFS services SecondaryNamenode, Datanode, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts Hadoop KMS, Hadoop Httpfs, and MapReduce-HistoryServer. | | hadoop-ssl-server | Change hadoop ssl server configuration | Not available. | | hadoop-ssl-client | Change hadoop ssl client configuration | Not available. | | hbase | Amazon EMR-curated settings for Apache HBase. | Custom EMR specific property. Sets emrfs-site and hbase-site configs. See those for their associated restarts. | | hbase-env | Change values in HBase's environment. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbase.properties file. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | Not available. | | hbase-site | Change values in HBase's hbase-site.xml file. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. Additionally restarts Phoenix QueryServer. | | hdfs-encryption-zones | Configure HDFS encryption zones. | This classification should not be reconfigured. | | hdfs-env | Change values in the HDFS environment. | Restarts Hadoop HDFS services Namenode, Datanode, and ZKFC. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Additionally restarts Hadoop Httpfs. | | hcatalog-env | Change values in HCatalog's environment. | Restarts Hive HCatalog Server. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | Restarts Hive HCatalog Server. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | Restarts Hive HCatalog Server. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | Restarts Hive WebHCat server. | | hcatalog-webhcat-log4j2 | Change values in HCatalog WebHCat's log4j2.properties. | Restarts Hive WebHCat server. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | Restarts Hive WebHCat server. | | hive | Amazon EMR-curated settings for Apache Hive. | Sets configurations to launch Hive LLAP service. | | hive-beeline-log4j2 | Change values in Hive's beeline-log4j2.properties file. | Not available. | | hive-parquet-logging | Change values in Hive's parquet-logging.properties file. | Not available. | | hive-env | Change values in the Hive environment. | Restarts HiveServer2, HiveMetastore, and Hive HCatalog-Server. Runs Hive schemaTool CLI commands to verify hive-metastore. | | hive-exec-log4j2 | Change values in Hive's hive-exec-log4j2.properties file. | Not available. | | hive-llap-daemon-log4j2 | Change values in Hive's llap-daemon-log4j2.properties file. | Not available. | | hive-log4j2 | Change values in Hive's hive-log4j2.properties file. | Not available. | | hive-site | Change values in Hive's hive-site.xml file | Restarts HiveServer2, HiveMetastore, and Hive HCatalog-Server. Runs Hive schemaTool CLI commands to verify hive-metastore. Also restarts Oozie and Zeppelin. | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | Not available. | | hue-ini | Change values in Hue's ini file | Restarts Hue. Also activates Hue config override CLI commands to pick up new configurations. | | httpfs-env | Change values in the HTTPFS environment. | Restarts Hadoop Httpfs service. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | Restarts Hadoop Httpfs service. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | Not available. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | Restarts Hadoop-KMS service. | | hadoop-kms-java-home | Change Hadoop's KMS java home | Not available. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | Not available. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | Restarts Hadoop-KMS and Ranger-KMS service. | | hudi-env | Change values in the Hudi environment. | Not available. | | hudi-defaults | Change values in Hudi's hudi-defaults.conf file. | Not available. | | iceberg-defaults | Change values in Iceberg's iceberg-defaults.conf file. | Not available. | | delta-defaults | Change values in Delta's delta-defaults.conf file. | Not available. | | jupyter-notebook-conf | Change values in Jupyter Notebook's jupyter\$1notebook\$1config.py file. | Not available. | | jupyter-hub-conf | Change values in JupyterHubs's jupyterhub\$1config.py file. | Not available. | | jupyter-s3-conf | Configure Jupyter Notebook S3 persistence. | Not available. | | jupyter-sparkmagic-conf | Change values in Sparkmagic's config.json file. | Not available. | | livy-conf | Change values in Livy's livy.conf file. | Restarts Livy Server. | | livy-env | Change values in the Livy environment. | Restarts Livy Server. | | livy-log4j2 | Change Livy log4j2.properties settings. | Restarts Livy Server. | | mapred-env | Change values in the MapReduce application's environment. | Restarts Hadoop MapReduce-HistoryServer. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | Restarts Hadoop MapReduce-HistoryServer. | | oozie-env | Change values in Oozie's environment. | Restarts Oozie. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | Restarts Oozie. | | oozie-site | Change values in Oozie's oozie-site.xml file. | Restarts Oozie. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | Not available. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | Not available. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | Restarts Phoenix-QueryServer. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | Not available. | | pig-env | Change values in the Pig environment. | Not available. | | pig-properties | Change values in Pig's pig.properties file. | Restarts Oozie. | | pig-log4j | Change values in Pig's log4j.properties file. | Not available. | | presto-log | Change values in Presto's log.properties file. | Restarts Presto-Server (for PrestoDB) | | presto-config | Change values in Presto's config.properties file. | Restarts Presto-Server (for PrestoDB) | | presto-password-authenticator | Change values in Presto's password-authenticator.properties file. | Not available. | | presto-env | Change values in Presto's presto-env.sh file. | Restarts Presto-Server (for PrestoDB) | | presto-node | Change values in Presto's node.properties file. | Not available. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | Not available. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | Not available. | | presto-connector-hive | Change values in Presto's hive.properties file. | Restarts Presto-Server (for PrestoDB) | | presto-connector-jmx | Change values in Presto's jmx.properties file. | Not available. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | Not available. | | presto-connector-lakeformation | Change values in Presto's lakeformation.properties file. | Restarts Presto-Server (for PrestoDB) | | presto-connector-localfile | Change values in Presto's localfile.properties file. | Not available. | | presto-connector-memory | Change values in Presto's memory.properties file. | Not available. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | Not available. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | Not available. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | Not available. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | Not available. | | presto-connector-redis | Change values in Presto's redis.properties file. | Not available. | | presto-connector-redshift | Change values in Presto's redshift.properties file. | Not available. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | Not available. | | presto-connector-tpcds | Change values in Presto's tpcds.properties file. | Not available. | | trino-log | Change values in Trino's log.properties file. | Restarts Trino-Server (for Trino) | | trino-config | Change values in Trino's config.properties file. | Restarts Trino-Server (for Trino) | | trino-password-authenticator | Change values in Trino's password-authenticator.properties file. | Restarts Trino-Server (for Trino) | | trino-env | Change values in Trino's trino-env.sh file. | Restarts Trino-Server (for Trino) | | trino-node | Change values in Trino's node.properties file. | Not available. | | trino-connector-blackhole | Change values in Trino's blackhole.properties file. | Not available. | | trino-connector-cassandra | Change values in Trino's cassandra.properties file. | Not available. | | trino-connector-delta | Change values in Trino's delta.properties file. | Restarts Trino-Server (for Trino) | | trino-connector-hive | Change values in Trino's hive.properties file. | Restarts Trino-Server (for Trino) | | trino-exchange-manager | Change values in Trino's exchange-manager.properties file. | Restarts Trino-Server (for Trino) | | trino-connector-iceberg | Change values in Trino's iceberg.properties file. | Restarts Trino-Server (for Trino) | | trino-connector-hudi | Change values in Trino's hudi.properties file. | Restarts Trino-Server (for Trino) | | trino-connector-jmx | Change values in Trino's jmx.properties file. | Not available. | | trino-connector-kafka | Change values in Trino's kafka.properties file. | Not available. | | trino-connector-localfile | Change values in Trino's localfile.properties file. | Not available. | | trino-connector-memory | Change values in Trino's memory.properties file. | Not available. | | trino-connector-mongodb | Change values in Trino's mongodb.properties file. | Not available. | | trino-connector-mysql | Change values in Trino's mysql.properties file. | Not available. | | trino-connector-postgresql | Change values in Trino's postgresql.properties file. | Not available. | | trino-connector-raptor | Change values in Trino's raptor.properties file. | Not available. | | trino-connector-redis | Change values in Trino's redis.properties file. | Not available. | | trino-connector-redshift | Change values in Trino's redshift.properties file. | Not available. | | trino-connector-tpch | Change values in Trino's tpch.properties file. | Not available. | | trino-connector-tpcds | Change values in Trino's tpcds.properties file. | Not available. | | ranger-kms-dbks-site | Change values in dbks-site.xml file of Ranger KMS. | Restarts Ranger KMS Server. | | ranger-kms-site | Change values in ranger-kms-site.xml file of Ranger KMS. | Restarts Ranger KMS Server. | | ranger-kms-env | Change values in the Ranger KMS environment. | Restarts Ranger KMS Server. | | ranger-kms-logback | Change values in kms-logback.xml file of Ranger KMS. | Not available. | | ranger-kms-db-ca | Change values for CA file on S3 for MySQL SSL connection with Ranger KMS. | Not available. | | spark | Amazon EMR-curated settings for Apache Spark. | This property modifies spark-defaults. See actions there. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | Restarts Spark history server and Spark thrift server. | | spark-env | Change values in the Spark environment. | Restarts Spark history server and Spark thrift server. | | spark-hive-site | Change values in Spark's hive-site.xml file | Not available. | | spark-log4j2 | Change values in Spark's log4j2.properties file. | Restarts Spark history server and Spark thrift server. | | spark-metrics | Change values in Spark's metrics.properties file. | Restarts Spark history server and Spark thrift server. | | sqoop-env | Change values in Sqoop's environment. | Not available. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | Not available. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | Not available. | | tez-site | Change values in Tez's tez-site.xml file. | Restart Oozie and HiveServer2. | | yarn-env | Change values in the YARN environment. | Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts MapReduce-HistoryServer. | | yarn-site | Change values in YARN's yarn-site.xml file. | Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts Livy Server and MapReduce-HistoryServer. | | zeppelin-env | Change values in the Zeppelin environment. | Restarts Zeppelin. | | zeppelin-site | Change configuration settings in zeppelin-site.xml. | Restarts Zeppelin. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | Restarts Zookeeper server. | | zookeeper-logback | Change values in ZooKeeper's logback.xml file. | Restarts Zookeeper server. | | emr-metrics | Change emr metric settings for this node. | Restarts the CloudWatchAgent service. | ## 7.3.0 change log **Change log for 7.3.0 release and release notes** | Date | Event | Description | | --- | --- | --- | | 2024-10-17 | Docs publication | Amazon EMR 7.3.0 release notes first published | | 2024-10-16 | Deployment complete | Amazon EMR 7.3.0 fully deployed to all [supported Regions](https://aws.amazon.com/about-aws/global-infrastructure/regional-product-services/) | | 2024-10-14 | Initial release | Amazon EMR 7.3.0 first deployed to initial commercial Regions | # Amazon EMR release 7.2.0 ## 7.2.0 supported lifecycle The following table describes the supported lifecycle dates for Amazon EMR release 7.2.0. | Support phase | Date | | --- | --- | | Initial release date | July 25, 2024 | | Standard support until | July 24, 2026 | | End of support | July 25, 2026 | | End of life | July 25, 2027 | ## 7.2.0 application versions This release includes the following applications: [https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/Install-CloudWatch-Agent.html](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/Install-CloudWatch-Agent.html), [https://delta.io/](https://delta.io/), [https://flink.apache.org/](https://flink.apache.org/), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [https://hudi.apache.org](https://hudi.apache.org), [http://gethue.com/](http://gethue.com/), [https://iceberg.apache.org/](https://iceberg.apache.org/), [https://jupyter-enterprise-gateway.readthedocs.io/en/latest/](https://jupyter-enterprise-gateway.readthedocs.io/en/latest/), [https://jupyterhub.readthedocs.io/en/latest/#](https://jupyterhub.readthedocs.io/en/latest/#), [https://livy.incubator.apache.org/](https://livy.incubator.apache.org/), [https://mxnet.incubator.apache.org/](https://mxnet.incubator.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://www.tensorflow.org/](https://www.tensorflow.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://trino.io/](https://trino.io/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-7.3.0 | emr-7.2.0 | emr-7.1.0 | emr-7.0.0 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 2.25.70, 1.12.747 | 2.23.18, 1.12.705 | 2.23.18, 1.12.656 | 2.20.160-amzn-0, 1.12.569 | | Python | 3.9, 3.11 | 3.9, 3.11 | 3.9, 3.11 | 3.9 | | Scala | 2.12.18 | 2.12.17 | 2.12.17 | 2.12.17 | | AmazonCloudWatchAgent | 1.300032.2-amzn-0 | 1.300032.2-amzn-0 | 1.300032.2-amzn-0 | 1.300031.1-amzn-0 | | Delta | 3.2.0-amzn-0 | 3.1.0 | 3.0.0 | 3.0.0 | | Flink | 1.18.1-amzn-2 | 1.18.1-amzn-1 | 1.18.1-amzn-0 | 1.18.0-amzn-0 | | Ganglia | - | - | - | - | | HBase | 2.4.17-amzn-7 | 2.4.17-amzn-6 | 2.4.17-amzn-5 | 2.4.17-amzn-4 | | HCatalog | 3.1.3-amzn-12 | 3.1.3-amzn-11 | 3.1.3-amzn-10 | 3.1.3-amzn-9 | | Hadoop | 3.3.6-amzn-5 | 3.3.6-amzn-4 | 3.3.6-amzn-3 | 3.3.6-amzn-2 | | Hive | 3.1.3-amzn-12 | 3.1.3-amzn-11 | 3.1.3-amzn-10 | 3.1.3-amzn-9 | | Hudi | 0.15.0-amzn-0 | 0.14.1-amzn-1 | 0.14.1-amzn-0 | 0.14.0-amzn-1 | | Hue | 4.11.0 | 4.11.0 | 4.11.0 | 4.11.0 | | Iceberg | 1.5.2-amzn-0 | 1.5.0-amzn-0 | 1.4.3-amzn-0 | 1.4.2-amzn-0 | | JupyterEnterpriseGateway | 2.6.0 | 2.6.0 | 2.6.0 | 2.6.0 | | JupyterHub | 1.5.0 | 1.5.0 | 1.5.0 | 1.5.0 | | Livy | 0.8.0-incubating | 0.8.0-incubating | 0.8.0-incubating | 0.7.1-incubating | | MXNet | - | 1.9.1 | 1.9.1 | 1.9.1 | | Mahout | - | - | - | - | | Oozie | 5.2.1 | 5.2.1 | 5.2.1 | 5.2.1 | | Phoenix | 5.1.3 | 5.1.3 | 5.1.3 | 5.1.3 | | Pig | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | | Presto | 0.285-amzn-1 | 0.285-amzn-0 | 0.284-amzn-0 | 0.283-amzn-1 | | Spark | 3.5.1-amzn-1 | 3.5.1-amzn-0 | 3.5.0-amzn-1 | 3.5.0-amzn-0 | | Sqoop | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | | TensorFlow | 2.16.1 | 2.11.0 | 2.11.0 | 2.11.0 | | Tez | 0.10.2-amzn-10 | 0.10.2-amzn-9 | 0.10.2-amzn-8 | 0.10.2-amzn-7 | | Trino (PrestoSQL) | 442-amzn-0 | 436-amzn-0 | 435-amzn-0 | 426-amzn-1 | | Zeppelin | 0.11.1 | 0.10.1 | 0.10.1 | 0.10.1 | | ZooKeeper | 3.9.1-amzn-1 | 3.9.1-amzn-1 | 3.9.1-amzn-0 | 3.5.10 | # 7.2.0 release components details See the following table for more information about the *Extras* packages in Amazon EMR 7.2.0. | Application | Application Type | Package | Package Type | Package Version | | --- | --- | --- | --- | --- | | AmazonCloudWatchAgent | Core | hadoop-hdfs-journalnode | Extras | 3.3.6.amzn.4-1.amzn2023 | | AmazonCloudWatchAgent | Core | ranger-kms | Extras | 2.0.0-1.amzn2023 | | AmazonCloudWatchAgent | Core | s3-dist-cp | Extras | 2.32.0-1.amzn2023 | | AmazonCloudWatchAgent | Core | zookeeper | Extras | 3.9.1.amzn.1-1.amzn2023 | | AmazonCloudWatchAgent | Core | zookeeper-server | Extras | 3.9.1.amzn.1-1.amzn2023 | | Flink | Core | hadoop-client | Extras | 3.3.6.amzn.4-1.amzn2023 | | Flink | Core | hadoop-hdfs | Extras | 3.3.6.amzn.4-1.amzn2023 | | Flink | Core | hadoop-hdfs-datanode | Extras | 3.3.6.amzn.4-1.amzn2023 | | Flink | Core | hadoop-hdfs-journalnode | Extras | 3.3.6.amzn.4-1.amzn2023 | | Flink | Core | hadoop-hdfs-namenode | Extras | 3.3.6.amzn.4-1.amzn2023 | | Flink | Core | hadoop-httpfs | Extras | 3.3.6.amzn.4-1.amzn2023 | | Flink | Core | hadoop-kms | Extras | 3.3.6.amzn.4-1.amzn2023 | | Flink | Core | hadoop-mapreduce | Extras | 3.3.6.amzn.4-1.amzn2023 | | Flink | Core | hadoop-mapreduce-historyserver | Extras | 3.3.6.amzn.4-1.amzn2023 | | Flink | Core | hadoop-yarn | Extras | 3.3.6.amzn.4-1.amzn2023 | | Flink | Core | hadoop-yarn-nodemanager | Extras | 3.3.6.amzn.4-1.amzn2023 | | Flink | Core | hadoop-yarn-proxyserver | Extras | 3.3.6.amzn.4-1.amzn2023 | | Flink | Core | hadoop-yarn-resourcemanager | Extras | 3.3.6.amzn.4-1.amzn2023 | | Flink | Core | hadoop-yarn-timelineserver | Extras | 3.3.6.amzn.4-1.amzn2023 | | Flink | Core | hudi | Extras | 0.14.1.amzn.1-1.amzn2023 | | Flink | Core | ranger-kms | Extras | 2.0.0-1.amzn2023 | | Flink | Core | hadoop | Extras | 3.3.6.amzn.4-1.amzn2023 | | HBase | Core | s3-dist-cp | Extras | 2.32.0-1.amzn2023 | | HCatalog | Core | hadoop-yarn-proxyserver | Extras | 3.3.6.amzn.4-1.amzn2023 | | HCatalog | Core | mariadb-connector-java | Extras | 2.7.2-1 | | Hive | Core | hadoop-httpfs | Extras | 3.3.6.amzn.4-1.amzn2023 | | Hive | Core | hadoop-yarn-proxyserver | Extras | 3.3.6.amzn.4-1.amzn2023 | | Hive | Core | hive-hbase | Extras | 3.1.3.amzn.11-1.amzn2023 | | Hive | Core | mariadb-connector-java | Extras | 2.7.2-1 | | Hive | Core | s3-dist-cp | Extras | 2.32.0-1.amzn2023 | | Phoenix | Core | s3-dist-cp | Extras | 2.32.0-1.amzn2023 | | Spark | Core | spark-datanucleus | Extras | 3.5.1.amzn.0-1.amzn2023 | | Tez | Core | hadoop-mapreduce | Extras | 3.3.6.amzn.4-1.amzn2023 | | Tez | Core | hadoop-mapreduce-historyserver | Extras | 3.3.6.amzn.4-1.amzn2023 | | Tez | Core | hadoop-yarn-proxyserver | Extras | 3.3.6.amzn.4-1.amzn2023 | | Trino | Core | bigtop-utils | Extras | 1.2.0-1.amzn2023 | | Trino | Core | hadoop-client | Extras | 3.3.6.amzn.4-1.amzn2023 | | Trino | Core | hadoop-hdfs | Extras | 3.3.6.amzn.4-1.amzn2023 | | Trino | Core | hadoop-hdfs-datanode | Extras | 3.3.6.amzn.4-1.amzn2023 | | Trino | Core | hadoop-hdfs-journalnode | Extras | 3.3.6.amzn.4-1.amzn2023 | | Trino | Core | hadoop-hdfs-namenode | Extras | 3.3.6.amzn.4-1.amzn2023 | | Trino | Core | hadoop-kms | Extras | 3.3.6.amzn.4-1.amzn2023 | | Trino | Core | hadoop-mapreduce | Extras | 3.3.6.amzn.4-1.amzn2023 | | Trino | Core | hadoop-mapreduce-historyserver | Extras | 3.3.6.amzn.4-1.amzn2023 | | Trino | Core | hadoop-yarn | Extras | 3.3.6.amzn.4-1.amzn2023 | | Trino | Core | hadoop-yarn-nodemanager | Extras | 3.3.6.amzn.4-1.amzn2023 | | Trino | Core | hadoop-yarn-proxyserver | Extras | 3.3.6.amzn.4-1.amzn2023 | | Trino | Core | hadoop-yarn-resourcemanager | Extras | 3.3.6.amzn.4-1.amzn2023 | | Trino | Core | hadoop-yarn-timelineserver | Extras | 3.3.6.amzn.4-1.amzn2023 | | Trino | Core | hive | Extras | 3.1.3.amzn.11-1.amzn2023 | | Trino | Core | hive-hcatalog-server | Extras | 3.1.3.amzn.11-1.amzn2023 | | Trino | Core | mariadb-connector-java | Extras | 2.7.2-1 | | Trino | Core | ranger-kms | Extras | 2.0.0-1.amzn2023 | | Trino | Core | zookeeper | Extras | 3.9.1.amzn.1-1.amzn2023 | | Trino | Core | zookeeper-server | Extras | 3.9.1.amzn.1-1.amzn2023 | | Trino | Core | bigtop-groovy | Extras | 2.5.4-1.amzn2023 | | Trino | Core | bigtop-jsvc | Extras | 1.2.4-1.amzn2023 | | Trino | Core | hadoop | Extras | 3.3.6.amzn.4-1.amzn2023 | | Trino | Core | hadoop-lzo | Extras | 0.4.19-1.amzn2023 | | Trino | Core | hive-hcatalog | Extras | 3.1.3.amzn.11-1.amzn2023 | | Trino | Core | hive-jdbc | Extras | 3.1.3.amzn.11-1.amzn2023 | | Zookeeper | Core | emrfs | Extras | 2.63.0-1.amzn2023 | | Zookeeper | Core | hadoop-client | Extras | 3.3.6.amzn.4-1.amzn2023 | | Zookeeper | Core | hadoop-hdfs | Extras | 3.3.6.amzn.4-1.amzn2023 | | Zookeeper | Core | hadoop-hdfs-datanode | Extras | 3.3.6.amzn.4-1.amzn2023 | | Zookeeper | Core | hadoop-hdfs-journalnode | Extras | 3.3.6.amzn.4-1.amzn2023 | | Zookeeper | Core | hadoop-hdfs-namenode | Extras | 3.3.6.amzn.4-1.amzn2023 | | Zookeeper | Core | hadoop-httpfs | Extras | 3.3.6.amzn.4-1.amzn2023 | | Zookeeper | Core | hadoop-kms | Extras | 3.3.6.amzn.4-1.amzn2023 | | Zookeeper | Core | hadoop-mapreduce | Extras | 3.3.6.amzn.4-1.amzn2023 | | Zookeeper | Core | hadoop-mapreduce-historyserver | Extras | 3.3.6.amzn.4-1.amzn2023 | | Zookeeper | Core | hadoop-yarn | Extras | 3.3.6.amzn.4-1.amzn2023 | | Zookeeper | Core | hadoop-yarn-nodemanager | Extras | 3.3.6.amzn.4-1.amzn2023 | | Zookeeper | Core | hadoop-yarn-proxyserver | Extras | 3.3.6.amzn.4-1.amzn2023 | | Zookeeper | Core | hadoop-yarn-resourcemanager | Extras | 3.3.6.amzn.4-1.amzn2023 | | Zookeeper | Core | hadoop-yarn-timelineserver | Extras | 3.3.6.amzn.4-1.amzn2023 | | Zookeeper | Core | ranger-kms | Extras | 2.0.0-1.amzn2023 | # 7.2.0 common vulnerabilities and exposures The following table lists all CVEs that don't impact EMR clusters that run on recommended configurations of Amazon EMR 7.2.0. Amazon EMR is dependent on upstream open-source for availability of fixes and will provide the latest stable version as part of the Core Engine components within 90 days of Amazon EMR verifying the fixes. This table was updated on September 15, 2024. | CVE ID | Severity | CVE details URL | | --- | --- | --- | | CVE-2015-1832 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2015-1832](https://nvd.nist.gov/vuln/detail/CVE-2015-1832) | | CVE-2015-4852 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2015-4852](https://nvd.nist.gov/vuln/detail/CVE-2015-4852) | | CVE-2015-7501 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2015-7501](https://nvd.nist.gov/vuln/detail/CVE-2015-7501) | | CVE-2016-5018 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2016-5018](https://nvd.nist.gov/vuln/detail/CVE-2016-5018) | | CVE-2017-1000487 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2017-1000487](https://nvd.nist.gov/vuln/detail/CVE-2017-1000487) | | CVE-2017-12629 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2017-12629](https://nvd.nist.gov/vuln/detail/CVE-2017-12629) | | CVE-2017-15095 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2017-15095](https://nvd.nist.gov/vuln/detail/CVE-2017-15095) | | CVE-2017-15095 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2017-15095](https://nvd.nist.gov/vuln/detail/CVE-2017-15095) | | CVE-2017-17485 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2017-17485](https://nvd.nist.gov/vuln/detail/CVE-2017-17485) | | CVE-2017-17485 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2017-17485](https://nvd.nist.gov/vuln/detail/CVE-2017-17485) | | CVE-2017-7525 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2017-7525](https://nvd.nist.gov/vuln/detail/CVE-2017-7525) | | CVE-2017-7657 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2017-7657](https://nvd.nist.gov/vuln/detail/CVE-2017-7657) | | CVE-2017-7658 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2017-7658](https://nvd.nist.gov/vuln/detail/CVE-2017-7658) | | CVE-2018-11307 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-11307](https://nvd.nist.gov/vuln/detail/CVE-2018-11307) | | CVE-2018-11307 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-11307](https://nvd.nist.gov/vuln/detail/CVE-2018-11307) | | CVE-2018-11307 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-11307](https://nvd.nist.gov/vuln/detail/CVE-2018-11307) | | CVE-2018-14718 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-14718](https://nvd.nist.gov/vuln/detail/CVE-2018-14718) | | CVE-2018-14718 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-14718](https://nvd.nist.gov/vuln/detail/CVE-2018-14718) | | CVE-2018-14718 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-14718](https://nvd.nist.gov/vuln/detail/CVE-2018-14718) | | CVE-2018-14719 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-14719](https://nvd.nist.gov/vuln/detail/CVE-2018-14719) | | CVE-2018-14719 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-14719](https://nvd.nist.gov/vuln/detail/CVE-2018-14719) | | CVE-2018-14719 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-14719](https://nvd.nist.gov/vuln/detail/CVE-2018-14719) | | CVE-2018-14720 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-14720](https://nvd.nist.gov/vuln/detail/CVE-2018-14720) | | CVE-2018-14720 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-14720](https://nvd.nist.gov/vuln/detail/CVE-2018-14720) | | CVE-2018-14720 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-14720](https://nvd.nist.gov/vuln/detail/CVE-2018-14720) | | CVE-2018-14721 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-14721](https://nvd.nist.gov/vuln/detail/CVE-2018-14721) | | CVE-2018-14721 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-14721](https://nvd.nist.gov/vuln/detail/CVE-2018-14721) | | CVE-2018-14721 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-14721](https://nvd.nist.gov/vuln/detail/CVE-2018-14721) | | CVE-2018-17190 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-17190](https://nvd.nist.gov/vuln/detail/CVE-2018-17190) | | CVE-2018-19360 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-19360](https://nvd.nist.gov/vuln/detail/CVE-2018-19360) | | CVE-2018-19360 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-19360](https://nvd.nist.gov/vuln/detail/CVE-2018-19360) | | CVE-2018-19360 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-19360](https://nvd.nist.gov/vuln/detail/CVE-2018-19360) | | CVE-2018-19361 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-19361](https://nvd.nist.gov/vuln/detail/CVE-2018-19361) | | CVE-2018-19361 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-19361](https://nvd.nist.gov/vuln/detail/CVE-2018-19361) | | CVE-2018-19361 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-19361](https://nvd.nist.gov/vuln/detail/CVE-2018-19361) | | CVE-2018-19362 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-19362](https://nvd.nist.gov/vuln/detail/CVE-2018-19362) | | CVE-2018-19362 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-19362](https://nvd.nist.gov/vuln/detail/CVE-2018-19362) | | CVE-2018-19362 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-19362](https://nvd.nist.gov/vuln/detail/CVE-2018-19362) | | CVE-2018-7489 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-7489](https://nvd.nist.gov/vuln/detail/CVE-2018-7489) | | CVE-2018-7489 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-7489](https://nvd.nist.gov/vuln/detail/CVE-2018-7489) | | CVE-2018-7489 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2018-7489](https://nvd.nist.gov/vuln/detail/CVE-2018-7489) | | CVE-2019-10202 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-10202](https://nvd.nist.gov/vuln/detail/CVE-2019-10202) | | CVE-2019-14379 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-14379](https://nvd.nist.gov/vuln/detail/CVE-2019-14379) | | CVE-2019-14379 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-14379](https://nvd.nist.gov/vuln/detail/CVE-2019-14379) | | CVE-2019-14379 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-14379](https://nvd.nist.gov/vuln/detail/CVE-2019-14379) | | CVE-2019-14379 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-14379](https://nvd.nist.gov/vuln/detail/CVE-2019-14379) | | CVE-2019-14540 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-14540](https://nvd.nist.gov/vuln/detail/CVE-2019-14540) | | CVE-2019-14540 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-14540](https://nvd.nist.gov/vuln/detail/CVE-2019-14540) | | CVE-2019-14540 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-14540](https://nvd.nist.gov/vuln/detail/CVE-2019-14540) | | CVE-2019-14540 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-14540](https://nvd.nist.gov/vuln/detail/CVE-2019-14540) | | CVE-2019-14892 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-14892](https://nvd.nist.gov/vuln/detail/CVE-2019-14892) | | CVE-2019-14892 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-14892](https://nvd.nist.gov/vuln/detail/CVE-2019-14892) | | CVE-2019-14892 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-14892](https://nvd.nist.gov/vuln/detail/CVE-2019-14892) | | CVE-2019-14892 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-14892](https://nvd.nist.gov/vuln/detail/CVE-2019-14892) | | CVE-2019-14893 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-14893](https://nvd.nist.gov/vuln/detail/CVE-2019-14893) | | CVE-2019-14893 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-14893](https://nvd.nist.gov/vuln/detail/CVE-2019-14893) | | CVE-2019-14893 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-14893](https://nvd.nist.gov/vuln/detail/CVE-2019-14893) | | CVE-2019-16335 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-16335](https://nvd.nist.gov/vuln/detail/CVE-2019-16335) | | CVE-2019-16335 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-16335](https://nvd.nist.gov/vuln/detail/CVE-2019-16335) | | CVE-2019-16335 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-16335](https://nvd.nist.gov/vuln/detail/CVE-2019-16335) | | CVE-2019-16335 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-16335](https://nvd.nist.gov/vuln/detail/CVE-2019-16335) | | CVE-2019-16942 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-16942](https://nvd.nist.gov/vuln/detail/CVE-2019-16942) | | CVE-2019-16942 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-16942](https://nvd.nist.gov/vuln/detail/CVE-2019-16942) | | CVE-2019-16942 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-16942](https://nvd.nist.gov/vuln/detail/CVE-2019-16942) | | CVE-2019-16942 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-16942](https://nvd.nist.gov/vuln/detail/CVE-2019-16942) | | CVE-2019-16943 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-16943](https://nvd.nist.gov/vuln/detail/CVE-2019-16943) | | CVE-2019-16943 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-16943](https://nvd.nist.gov/vuln/detail/CVE-2019-16943) | | CVE-2019-16943 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-16943](https://nvd.nist.gov/vuln/detail/CVE-2019-16943) | | CVE-2019-16943 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-16943](https://nvd.nist.gov/vuln/detail/CVE-2019-16943) | | CVE-2019-17195 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-17195](https://nvd.nist.gov/vuln/detail/CVE-2019-17195) | | CVE-2019-17195 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-17195](https://nvd.nist.gov/vuln/detail/CVE-2019-17195) | | CVE-2019-17267 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-17267](https://nvd.nist.gov/vuln/detail/CVE-2019-17267) | | CVE-2019-17267 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-17267](https://nvd.nist.gov/vuln/detail/CVE-2019-17267) | | CVE-2019-17267 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-17267](https://nvd.nist.gov/vuln/detail/CVE-2019-17267) | | CVE-2019-17267 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-17267](https://nvd.nist.gov/vuln/detail/CVE-2019-17267) | | CVE-2019-17531 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-17531](https://nvd.nist.gov/vuln/detail/CVE-2019-17531) | | CVE-2019-17531 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-17531](https://nvd.nist.gov/vuln/detail/CVE-2019-17531) | | CVE-2019-17531 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-17531](https://nvd.nist.gov/vuln/detail/CVE-2019-17531) | | CVE-2019-17531 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-17531](https://nvd.nist.gov/vuln/detail/CVE-2019-17531) | | CVE-2019-20330 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-20330](https://nvd.nist.gov/vuln/detail/CVE-2019-20330) | | CVE-2019-20330 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-20330](https://nvd.nist.gov/vuln/detail/CVE-2019-20330) | | CVE-2019-20330 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-20330](https://nvd.nist.gov/vuln/detail/CVE-2019-20330) | | CVE-2019-20330 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-20330](https://nvd.nist.gov/vuln/detail/CVE-2019-20330) | | CVE-2019-20444 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-20444](https://nvd.nist.gov/vuln/detail/CVE-2019-20444) | | CVE-2019-20444 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-20444](https://nvd.nist.gov/vuln/detail/CVE-2019-20444) | | CVE-2019-20445 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-20445](https://nvd.nist.gov/vuln/detail/CVE-2019-20445) | | CVE-2019-20445 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2019-20445](https://nvd.nist.gov/vuln/detail/CVE-2019-20445) | | CVE-2020-7692 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2020-7692](https://nvd.nist.gov/vuln/detail/CVE-2020-7692) | | CVE-2020-7692 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2020-7692](https://nvd.nist.gov/vuln/detail/CVE-2020-7692) | | CVE-2020-8840 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2020-8840](https://nvd.nist.gov/vuln/detail/CVE-2020-8840) | | CVE-2020-8840 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2020-8840](https://nvd.nist.gov/vuln/detail/CVE-2020-8840) | | CVE-2020-8840 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2020-8840](https://nvd.nist.gov/vuln/detail/CVE-2020-8840) | | CVE-2020-8840 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2020-8840](https://nvd.nist.gov/vuln/detail/CVE-2020-8840) | | CVE-2020-9480 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2020-9480](https://nvd.nist.gov/vuln/detail/CVE-2020-9480) | | CVE-2020-9546 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2020-9546](https://nvd.nist.gov/vuln/detail/CVE-2020-9546) | | CVE-2020-9546 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2020-9546](https://nvd.nist.gov/vuln/detail/CVE-2020-9546) | | CVE-2020-9546 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2020-9546](https://nvd.nist.gov/vuln/detail/CVE-2020-9546) | | CVE-2020-9546 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2020-9546](https://nvd.nist.gov/vuln/detail/CVE-2020-9546) | | CVE-2020-9547 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2020-9547](https://nvd.nist.gov/vuln/detail/CVE-2020-9547) | | CVE-2020-9547 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2020-9547](https://nvd.nist.gov/vuln/detail/CVE-2020-9547) | | CVE-2020-9547 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2020-9547](https://nvd.nist.gov/vuln/detail/CVE-2020-9547) | | CVE-2020-9547 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2020-9547](https://nvd.nist.gov/vuln/detail/CVE-2020-9547) | | CVE-2020-9548 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2020-9548](https://nvd.nist.gov/vuln/detail/CVE-2020-9548) | | CVE-2020-9548 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2020-9548](https://nvd.nist.gov/vuln/detail/CVE-2020-9548) | | CVE-2020-9548 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2020-9548](https://nvd.nist.gov/vuln/detail/CVE-2020-9548) | | CVE-2020-9548 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2020-9548](https://nvd.nist.gov/vuln/detail/CVE-2020-9548) | | CVE-2021-26291 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2021-26291](https://nvd.nist.gov/vuln/detail/CVE-2021-26291) | | CVE-2021-26291 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2021-26291](https://nvd.nist.gov/vuln/detail/CVE-2021-26291) | | CVE-2021-32797 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2021-32797](https://nvd.nist.gov/vuln/detail/CVE-2021-32797) | | CVE-2021-32798 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2021-32798](https://nvd.nist.gov/vuln/detail/CVE-2021-32798) | | CVE-2021-35958 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2021-35958](https://nvd.nist.gov/vuln/detail/CVE-2021-35958) | | CVE-2021-37404 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2021-37404](https://nvd.nist.gov/vuln/detail/CVE-2021-37404) | | CVE-2022-1471 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2022-1471](https://nvd.nist.gov/vuln/detail/CVE-2022-1471) | | CVE-2022-1471 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2022-1471](https://nvd.nist.gov/vuln/detail/CVE-2022-1471) | | CVE-2022-1471 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2022-1471](https://nvd.nist.gov/vuln/detail/CVE-2022-1471) | | CVE-2022-1471 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2022-1471](https://nvd.nist.gov/vuln/detail/CVE-2022-1471) | | CVE-2022-25168 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2022-25168](https://nvd.nist.gov/vuln/detail/CVE-2022-25168) | | CVE-2022-26612 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2022-26612](https://nvd.nist.gov/vuln/detail/CVE-2022-26612) | | CVE-2022-32532 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2022-32532](https://nvd.nist.gov/vuln/detail/CVE-2022-32532) | | CVE-2022-33980 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2022-33980](https://nvd.nist.gov/vuln/detail/CVE-2022-33980) | | CVE-2022-37865 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2022-37865](https://nvd.nist.gov/vuln/detail/CVE-2022-37865) | | CVE-2022-39135 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2022-39135](https://nvd.nist.gov/vuln/detail/CVE-2022-39135) | | CVE-2022-40664 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2022-40664](https://nvd.nist.gov/vuln/detail/CVE-2022-40664) | | CVE-2022-42889 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2022-42889](https://nvd.nist.gov/vuln/detail/CVE-2022-42889) | | CVE-2022-46337 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2022-46337](https://nvd.nist.gov/vuln/detail/CVE-2022-46337) | | CVE-2022-46337 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2022-46337](https://nvd.nist.gov/vuln/detail/CVE-2022-46337) | | CVE-2022-46337 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2022-46337](https://nvd.nist.gov/vuln/detail/CVE-2022-46337) | | CVE-2023-38647 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2023-38647](https://nvd.nist.gov/vuln/detail/CVE-2023-38647) | | CVE-2023-38647 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2023-38647](https://nvd.nist.gov/vuln/detail/CVE-2023-38647) | | CVE-2023-44981 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2023-44981](https://nvd.nist.gov/vuln/detail/CVE-2023-44981) | | CVE-2023-44981 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2023-44981](https://nvd.nist.gov/vuln/detail/CVE-2023-44981) | | CVE-2023-44981 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2023-44981](https://nvd.nist.gov/vuln/detail/CVE-2023-44981) | | CVE-2023-44981 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2023-44981](https://nvd.nist.gov/vuln/detail/CVE-2023-44981) | | CVE-2023-44981 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2023-44981](https://nvd.nist.gov/vuln/detail/CVE-2023-44981) | | CVE-2023-44981 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2023-44981](https://nvd.nist.gov/vuln/detail/CVE-2023-44981) | | CVE-2024-1597 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2024-1597](https://nvd.nist.gov/vuln/detail/CVE-2024-1597) | | CVE-2024-1597 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2024-1597](https://nvd.nist.gov/vuln/detail/CVE-2024-1597) | | CVE-2024-1597 | CRITICAL | [https://nvd.nist.gov/vuln/detail/CVE-2024-1597](https://nvd.nist.gov/vuln/detail/CVE-2024-1597) | | CVE-2012-0881 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2012-0881](https://nvd.nist.gov/vuln/detail/CVE-2012-0881) | | CVE-2012-0881 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2012-0881](https://nvd.nist.gov/vuln/detail/CVE-2012-0881) | | CVE-2013-4002 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2013-4002](https://nvd.nist.gov/vuln/detail/CVE-2013-4002) | | CVE-2013-4002 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2013-4002](https://nvd.nist.gov/vuln/detail/CVE-2013-4002) | | CVE-2014-0114 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2014-0114](https://nvd.nist.gov/vuln/detail/CVE-2014-0114) | | CVE-2014-0114 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2014-0114](https://nvd.nist.gov/vuln/detail/CVE-2014-0114) | | CVE-2015-2080 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2015-2080](https://nvd.nist.gov/vuln/detail/CVE-2015-2080) | | CVE-2015-2156 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2015-2156](https://nvd.nist.gov/vuln/detail/CVE-2015-2156) | | CVE-2015-2156 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2015-2156](https://nvd.nist.gov/vuln/detail/CVE-2015-2156) | | CVE-2015-5237 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2015-5237](https://nvd.nist.gov/vuln/detail/CVE-2015-5237) | | CVE-2015-5237 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2015-5237](https://nvd.nist.gov/vuln/detail/CVE-2015-5237) | | CVE-2015-6420 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2015-6420](https://nvd.nist.gov/vuln/detail/CVE-2015-6420) | | CVE-2016-1000338 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2016-1000338](https://nvd.nist.gov/vuln/detail/CVE-2016-1000338) | | CVE-2016-1000340 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2016-1000340](https://nvd.nist.gov/vuln/detail/CVE-2016-1000340) | | CVE-2016-1000342 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2016-1000342](https://nvd.nist.gov/vuln/detail/CVE-2016-1000342) | | CVE-2016-1000343 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2016-1000343](https://nvd.nist.gov/vuln/detail/CVE-2016-1000343) | | CVE-2016-1000344 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2016-1000344](https://nvd.nist.gov/vuln/detail/CVE-2016-1000344) | | CVE-2016-1000352 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2016-1000352](https://nvd.nist.gov/vuln/detail/CVE-2016-1000352) | | CVE-2016-6796 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2016-6796](https://nvd.nist.gov/vuln/detail/CVE-2016-6796) | | CVE-2017-1000034 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2017-1000034](https://nvd.nist.gov/vuln/detail/CVE-2017-1000034) | | CVE-2017-12612 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2017-12612](https://nvd.nist.gov/vuln/detail/CVE-2017-12612) | | CVE-2017-15288 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2017-15288](https://nvd.nist.gov/vuln/detail/CVE-2017-15288) | | CVE-2017-18640 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2017-18640](https://nvd.nist.gov/vuln/detail/CVE-2017-18640) | | CVE-2017-18640 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2017-18640](https://nvd.nist.gov/vuln/detail/CVE-2017-18640) | | CVE-2017-5637 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2017-5637](https://nvd.nist.gov/vuln/detail/CVE-2017-5637) | | CVE-2017-7656 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2017-7656](https://nvd.nist.gov/vuln/detail/CVE-2017-7656) | | CVE-2017-7687 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2017-7687](https://nvd.nist.gov/vuln/detail/CVE-2017-7687) | | CVE-2017-9735 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2017-9735](https://nvd.nist.gov/vuln/detail/CVE-2017-9735) | | CVE-2017-9790 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2017-9790](https://nvd.nist.gov/vuln/detail/CVE-2017-9790) | | CVE-2018-10054 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2018-10054](https://nvd.nist.gov/vuln/detail/CVE-2018-10054) | | CVE-2018-10936 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2018-10936](https://nvd.nist.gov/vuln/detail/CVE-2018-10936) | | CVE-2018-11793 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2018-11793](https://nvd.nist.gov/vuln/detail/CVE-2018-11793) | | CVE-2018-11804 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2018-11804](https://nvd.nist.gov/vuln/detail/CVE-2018-11804) | | CVE-2018-12022 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2018-12022](https://nvd.nist.gov/vuln/detail/CVE-2018-12022) | | CVE-2018-12022 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2018-12022](https://nvd.nist.gov/vuln/detail/CVE-2018-12022) | | CVE-2018-12022 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2018-12022](https://nvd.nist.gov/vuln/detail/CVE-2018-12022) | | CVE-2018-12023 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2018-12023](https://nvd.nist.gov/vuln/detail/CVE-2018-12023) | | CVE-2018-12023 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2018-12023](https://nvd.nist.gov/vuln/detail/CVE-2018-12023) | | CVE-2018-12023 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2018-12023](https://nvd.nist.gov/vuln/detail/CVE-2018-12023) | | CVE-2018-3258 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2018-3258](https://nvd.nist.gov/vuln/detail/CVE-2018-3258) | | CVE-2018-3827 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2018-3827](https://nvd.nist.gov/vuln/detail/CVE-2018-3827) | | CVE-2018-3831 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2018-3831](https://nvd.nist.gov/vuln/detail/CVE-2018-3831) | | CVE-2018-5968 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2018-5968](https://nvd.nist.gov/vuln/detail/CVE-2018-5968) | | CVE-2018-5968 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2018-5968](https://nvd.nist.gov/vuln/detail/CVE-2018-5968) | | CVE-2018-8012 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2018-8012](https://nvd.nist.gov/vuln/detail/CVE-2018-8012) | | CVE-2018-8768 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2018-8768](https://nvd.nist.gov/vuln/detail/CVE-2018-8768) | | CVE-2019-0204 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2019-0204](https://nvd.nist.gov/vuln/detail/CVE-2019-0204) | | CVE-2019-0205 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2019-0205](https://nvd.nist.gov/vuln/detail/CVE-2019-0205) | | CVE-2019-0205 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2019-0205](https://nvd.nist.gov/vuln/detail/CVE-2019-0205) | | CVE-2019-0205 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2019-0205](https://nvd.nist.gov/vuln/detail/CVE-2019-0205) | | CVE-2019-0205 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2019-0205](https://nvd.nist.gov/vuln/detail/CVE-2019-0205) | | CVE-2019-10086 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2019-10086](https://nvd.nist.gov/vuln/detail/CVE-2019-10086) | | CVE-2019-10086 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2019-10086](https://nvd.nist.gov/vuln/detail/CVE-2019-10086) | | CVE-2019-10086 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2019-10086](https://nvd.nist.gov/vuln/detail/CVE-2019-10086) | | CVE-2019-10086 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2019-10086](https://nvd.nist.gov/vuln/detail/CVE-2019-10086) | | CVE-2019-10099 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2019-10099](https://nvd.nist.gov/vuln/detail/CVE-2019-10099) | | CVE-2019-10172 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2019-10172](https://nvd.nist.gov/vuln/detail/CVE-2019-10172) | | CVE-2019-12086 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2019-12086](https://nvd.nist.gov/vuln/detail/CVE-2019-12086) | | CVE-2019-12086 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2019-12086](https://nvd.nist.gov/vuln/detail/CVE-2019-12086) | | CVE-2019-12086 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2019-12086](https://nvd.nist.gov/vuln/detail/CVE-2019-12086) | | CVE-2019-12086 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2019-12086](https://nvd.nist.gov/vuln/detail/CVE-2019-12086) | | CVE-2019-14439 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2019-14439](https://nvd.nist.gov/vuln/detail/CVE-2019-14439) | | CVE-2019-14439 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2019-14439](https://nvd.nist.gov/vuln/detail/CVE-2019-14439) | | CVE-2019-14439 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2019-14439](https://nvd.nist.gov/vuln/detail/CVE-2019-14439) | | CVE-2019-14439 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2019-14439](https://nvd.nist.gov/vuln/detail/CVE-2019-14439) | | CVE-2019-16869 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2019-16869](https://nvd.nist.gov/vuln/detail/CVE-2019-16869) | | CVE-2019-16869 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2019-16869](https://nvd.nist.gov/vuln/detail/CVE-2019-16869) | | CVE-2019-17566 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2019-17566](https://nvd.nist.gov/vuln/detail/CVE-2019-17566) | | CVE-2019-17566 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2019-17566](https://nvd.nist.gov/vuln/detail/CVE-2019-17566) | | CVE-2019-7611 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2019-7611](https://nvd.nist.gov/vuln/detail/CVE-2019-7611) | | CVE-2020-10650 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-10650](https://nvd.nist.gov/vuln/detail/CVE-2020-10650) | | CVE-2020-10650 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-10650](https://nvd.nist.gov/vuln/detail/CVE-2020-10650) | | CVE-2020-10650 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-10650](https://nvd.nist.gov/vuln/detail/CVE-2020-10650) | | CVE-2020-10650 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-10650](https://nvd.nist.gov/vuln/detail/CVE-2020-10650) | | CVE-2020-10672 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-10672](https://nvd.nist.gov/vuln/detail/CVE-2020-10672) | | CVE-2020-10672 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-10672](https://nvd.nist.gov/vuln/detail/CVE-2020-10672) | | CVE-2020-10672 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-10672](https://nvd.nist.gov/vuln/detail/CVE-2020-10672) | | CVE-2020-10672 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-10672](https://nvd.nist.gov/vuln/detail/CVE-2020-10672) | | CVE-2020-10673 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-10673](https://nvd.nist.gov/vuln/detail/CVE-2020-10673) | | CVE-2020-10673 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-10673](https://nvd.nist.gov/vuln/detail/CVE-2020-10673) | | CVE-2020-10673 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-10673](https://nvd.nist.gov/vuln/detail/CVE-2020-10673) | | CVE-2020-10673 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-10673](https://nvd.nist.gov/vuln/detail/CVE-2020-10673) | | CVE-2020-10968 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-10968](https://nvd.nist.gov/vuln/detail/CVE-2020-10968) | | CVE-2020-10968 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-10968](https://nvd.nist.gov/vuln/detail/CVE-2020-10968) | | CVE-2020-10968 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-10968](https://nvd.nist.gov/vuln/detail/CVE-2020-10968) | | CVE-2020-10968 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-10968](https://nvd.nist.gov/vuln/detail/CVE-2020-10968) | | CVE-2020-10969 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-10969](https://nvd.nist.gov/vuln/detail/CVE-2020-10969) | | CVE-2020-10969 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-10969](https://nvd.nist.gov/vuln/detail/CVE-2020-10969) | | CVE-2020-10969 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-10969](https://nvd.nist.gov/vuln/detail/CVE-2020-10969) | | CVE-2020-10969 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-10969](https://nvd.nist.gov/vuln/detail/CVE-2020-10969) | | CVE-2020-11111 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-11111](https://nvd.nist.gov/vuln/detail/CVE-2020-11111) | | CVE-2020-11111 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-11111](https://nvd.nist.gov/vuln/detail/CVE-2020-11111) | | CVE-2020-11111 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-11111](https://nvd.nist.gov/vuln/detail/CVE-2020-11111) | | CVE-2020-11111 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-11111](https://nvd.nist.gov/vuln/detail/CVE-2020-11111) | | CVE-2020-11112 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-11112](https://nvd.nist.gov/vuln/detail/CVE-2020-11112) | | CVE-2020-11112 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-11112](https://nvd.nist.gov/vuln/detail/CVE-2020-11112) | | CVE-2020-11112 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-11112](https://nvd.nist.gov/vuln/detail/CVE-2020-11112) | | CVE-2020-11113 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-11113](https://nvd.nist.gov/vuln/detail/CVE-2020-11113) | | CVE-2020-11113 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-11113](https://nvd.nist.gov/vuln/detail/CVE-2020-11113) | | CVE-2020-11113 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-11113](https://nvd.nist.gov/vuln/detail/CVE-2020-11113) | | CVE-2020-11619 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-11619](https://nvd.nist.gov/vuln/detail/CVE-2020-11619) | | CVE-2020-11619 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-11619](https://nvd.nist.gov/vuln/detail/CVE-2020-11619) | | CVE-2020-11619 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-11619](https://nvd.nist.gov/vuln/detail/CVE-2020-11619) | | CVE-2020-11620 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-11620](https://nvd.nist.gov/vuln/detail/CVE-2020-11620) | | CVE-2020-11620 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-11620](https://nvd.nist.gov/vuln/detail/CVE-2020-11620) | | CVE-2020-11620 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-11620](https://nvd.nist.gov/vuln/detail/CVE-2020-11620) | | CVE-2020-11620 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-11620](https://nvd.nist.gov/vuln/detail/CVE-2020-11620) | | CVE-2020-11988 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-11988](https://nvd.nist.gov/vuln/detail/CVE-2020-11988) | | CVE-2020-13692 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-13692](https://nvd.nist.gov/vuln/detail/CVE-2020-13692) | | CVE-2020-13936 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-13936](https://nvd.nist.gov/vuln/detail/CVE-2020-13936) | | CVE-2020-13949 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-13949](https://nvd.nist.gov/vuln/detail/CVE-2020-13949) | | CVE-2020-13949 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-13949](https://nvd.nist.gov/vuln/detail/CVE-2020-13949) | | CVE-2020-13949 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-13949](https://nvd.nist.gov/vuln/detail/CVE-2020-13949) | | CVE-2020-13949 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-13949](https://nvd.nist.gov/vuln/detail/CVE-2020-13949) | | CVE-2020-13949 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-13949](https://nvd.nist.gov/vuln/detail/CVE-2020-13949) | | CVE-2020-14060 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-14060](https://nvd.nist.gov/vuln/detail/CVE-2020-14060) | | CVE-2020-14060 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-14060](https://nvd.nist.gov/vuln/detail/CVE-2020-14060) | | CVE-2020-14060 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-14060](https://nvd.nist.gov/vuln/detail/CVE-2020-14060) | | CVE-2020-14060 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-14060](https://nvd.nist.gov/vuln/detail/CVE-2020-14060) | | CVE-2020-14061 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-14061](https://nvd.nist.gov/vuln/detail/CVE-2020-14061) | | CVE-2020-14061 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-14061](https://nvd.nist.gov/vuln/detail/CVE-2020-14061) | | CVE-2020-14061 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-14061](https://nvd.nist.gov/vuln/detail/CVE-2020-14061) | | CVE-2020-14061 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-14061](https://nvd.nist.gov/vuln/detail/CVE-2020-14061) | | CVE-2020-14062 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-14062](https://nvd.nist.gov/vuln/detail/CVE-2020-14062) | | CVE-2020-14062 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-14062](https://nvd.nist.gov/vuln/detail/CVE-2020-14062) | | CVE-2020-14062 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-14062](https://nvd.nist.gov/vuln/detail/CVE-2020-14062) | | CVE-2020-14062 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-14062](https://nvd.nist.gov/vuln/detail/CVE-2020-14062) | | CVE-2020-14195 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-14195](https://nvd.nist.gov/vuln/detail/CVE-2020-14195) | | CVE-2020-14195 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-14195](https://nvd.nist.gov/vuln/detail/CVE-2020-14195) | | CVE-2020-14195 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-14195](https://nvd.nist.gov/vuln/detail/CVE-2020-14195) | | CVE-2020-14195 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-14195](https://nvd.nist.gov/vuln/detail/CVE-2020-14195) | | CVE-2020-17533 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-17533](https://nvd.nist.gov/vuln/detail/CVE-2020-17533) | | CVE-2020-24616 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-24616](https://nvd.nist.gov/vuln/detail/CVE-2020-24616) | | CVE-2020-24616 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-24616](https://nvd.nist.gov/vuln/detail/CVE-2020-24616) | | CVE-2020-24616 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-24616](https://nvd.nist.gov/vuln/detail/CVE-2020-24616) | | CVE-2020-24616 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-24616](https://nvd.nist.gov/vuln/detail/CVE-2020-24616) | | CVE-2020-24750 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-24750](https://nvd.nist.gov/vuln/detail/CVE-2020-24750) | | CVE-2020-24750 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-24750](https://nvd.nist.gov/vuln/detail/CVE-2020-24750) | | CVE-2020-24750 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-24750](https://nvd.nist.gov/vuln/detail/CVE-2020-24750) | | CVE-2020-24750 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-24750](https://nvd.nist.gov/vuln/detail/CVE-2020-24750) | | CVE-2020-25649 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-25649](https://nvd.nist.gov/vuln/detail/CVE-2020-25649) | | CVE-2020-25649 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-25649](https://nvd.nist.gov/vuln/detail/CVE-2020-25649) | | CVE-2020-25649 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-25649](https://nvd.nist.gov/vuln/detail/CVE-2020-25649) | | CVE-2020-25649 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-25649](https://nvd.nist.gov/vuln/detail/CVE-2020-25649) | | CVE-2020-28491 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-28491](https://nvd.nist.gov/vuln/detail/CVE-2020-28491) | | CVE-2020-35211 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-35211](https://nvd.nist.gov/vuln/detail/CVE-2020-35211) | | CVE-2020-35490 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-35490](https://nvd.nist.gov/vuln/detail/CVE-2020-35490) | | CVE-2020-35490 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-35490](https://nvd.nist.gov/vuln/detail/CVE-2020-35490) | | CVE-2020-35490 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-35490](https://nvd.nist.gov/vuln/detail/CVE-2020-35490) | | CVE-2020-35490 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-35490](https://nvd.nist.gov/vuln/detail/CVE-2020-35490) | | CVE-2020-35490 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-35490](https://nvd.nist.gov/vuln/detail/CVE-2020-35490) | | CVE-2020-35491 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-35491](https://nvd.nist.gov/vuln/detail/CVE-2020-35491) | | CVE-2020-35491 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-35491](https://nvd.nist.gov/vuln/detail/CVE-2020-35491) | | CVE-2020-35491 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-35491](https://nvd.nist.gov/vuln/detail/CVE-2020-35491) | | CVE-2020-35491 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-35491](https://nvd.nist.gov/vuln/detail/CVE-2020-35491) | | CVE-2020-35491 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-35491](https://nvd.nist.gov/vuln/detail/CVE-2020-35491) | | CVE-2020-35728 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-35728](https://nvd.nist.gov/vuln/detail/CVE-2020-35728) | | CVE-2020-35728 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-35728](https://nvd.nist.gov/vuln/detail/CVE-2020-35728) | | CVE-2020-35728 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-35728](https://nvd.nist.gov/vuln/detail/CVE-2020-35728) | | CVE-2020-35728 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-35728](https://nvd.nist.gov/vuln/detail/CVE-2020-35728) | | CVE-2020-35728 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-35728](https://nvd.nist.gov/vuln/detail/CVE-2020-35728) | | CVE-2020-36179 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36179](https://nvd.nist.gov/vuln/detail/CVE-2020-36179) | | CVE-2020-36179 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36179](https://nvd.nist.gov/vuln/detail/CVE-2020-36179) | | CVE-2020-36179 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36179](https://nvd.nist.gov/vuln/detail/CVE-2020-36179) | | CVE-2020-36179 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36179](https://nvd.nist.gov/vuln/detail/CVE-2020-36179) | | CVE-2020-36179 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36179](https://nvd.nist.gov/vuln/detail/CVE-2020-36179) | | CVE-2020-36180 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36180](https://nvd.nist.gov/vuln/detail/CVE-2020-36180) | | CVE-2020-36180 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36180](https://nvd.nist.gov/vuln/detail/CVE-2020-36180) | | CVE-2020-36180 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36180](https://nvd.nist.gov/vuln/detail/CVE-2020-36180) | | CVE-2020-36180 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36180](https://nvd.nist.gov/vuln/detail/CVE-2020-36180) | | CVE-2020-36180 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36180](https://nvd.nist.gov/vuln/detail/CVE-2020-36180) | | CVE-2020-36181 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36181](https://nvd.nist.gov/vuln/detail/CVE-2020-36181) | | CVE-2020-36181 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36181](https://nvd.nist.gov/vuln/detail/CVE-2020-36181) | | CVE-2020-36181 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36181](https://nvd.nist.gov/vuln/detail/CVE-2020-36181) | | CVE-2020-36181 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36181](https://nvd.nist.gov/vuln/detail/CVE-2020-36181) | | CVE-2020-36181 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36181](https://nvd.nist.gov/vuln/detail/CVE-2020-36181) | | CVE-2020-36182 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36182](https://nvd.nist.gov/vuln/detail/CVE-2020-36182) | | CVE-2020-36182 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36182](https://nvd.nist.gov/vuln/detail/CVE-2020-36182) | | CVE-2020-36182 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36182](https://nvd.nist.gov/vuln/detail/CVE-2020-36182) | | CVE-2020-36182 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36182](https://nvd.nist.gov/vuln/detail/CVE-2020-36182) | | CVE-2020-36182 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36182](https://nvd.nist.gov/vuln/detail/CVE-2020-36182) | | CVE-2020-36183 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36183](https://nvd.nist.gov/vuln/detail/CVE-2020-36183) | | CVE-2020-36183 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36183](https://nvd.nist.gov/vuln/detail/CVE-2020-36183) | | CVE-2020-36183 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36183](https://nvd.nist.gov/vuln/detail/CVE-2020-36183) | | CVE-2020-36183 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36183](https://nvd.nist.gov/vuln/detail/CVE-2020-36183) | | CVE-2020-36183 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36183](https://nvd.nist.gov/vuln/detail/CVE-2020-36183) | | CVE-2020-36184 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36184](https://nvd.nist.gov/vuln/detail/CVE-2020-36184) | | CVE-2020-36184 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36184](https://nvd.nist.gov/vuln/detail/CVE-2020-36184) | | CVE-2020-36184 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36184](https://nvd.nist.gov/vuln/detail/CVE-2020-36184) | | CVE-2020-36184 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36184](https://nvd.nist.gov/vuln/detail/CVE-2020-36184) | | CVE-2020-36184 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36184](https://nvd.nist.gov/vuln/detail/CVE-2020-36184) | | CVE-2020-36185 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36185](https://nvd.nist.gov/vuln/detail/CVE-2020-36185) | | CVE-2020-36185 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36185](https://nvd.nist.gov/vuln/detail/CVE-2020-36185) | | CVE-2020-36185 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36185](https://nvd.nist.gov/vuln/detail/CVE-2020-36185) | | CVE-2020-36185 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36185](https://nvd.nist.gov/vuln/detail/CVE-2020-36185) | | CVE-2020-36185 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36185](https://nvd.nist.gov/vuln/detail/CVE-2020-36185) | | CVE-2020-36186 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36186](https://nvd.nist.gov/vuln/detail/CVE-2020-36186) | | CVE-2020-36186 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36186](https://nvd.nist.gov/vuln/detail/CVE-2020-36186) | | CVE-2020-36186 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36186](https://nvd.nist.gov/vuln/detail/CVE-2020-36186) | | CVE-2020-36186 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36186](https://nvd.nist.gov/vuln/detail/CVE-2020-36186) | | CVE-2020-36186 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36186](https://nvd.nist.gov/vuln/detail/CVE-2020-36186) | | CVE-2020-36187 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36187](https://nvd.nist.gov/vuln/detail/CVE-2020-36187) | | CVE-2020-36187 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36187](https://nvd.nist.gov/vuln/detail/CVE-2020-36187) | | CVE-2020-36187 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36187](https://nvd.nist.gov/vuln/detail/CVE-2020-36187) | | CVE-2020-36187 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36187](https://nvd.nist.gov/vuln/detail/CVE-2020-36187) | | CVE-2020-36187 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36187](https://nvd.nist.gov/vuln/detail/CVE-2020-36187) | | CVE-2020-36188 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36188](https://nvd.nist.gov/vuln/detail/CVE-2020-36188) | | CVE-2020-36188 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36188](https://nvd.nist.gov/vuln/detail/CVE-2020-36188) | | CVE-2020-36188 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36188](https://nvd.nist.gov/vuln/detail/CVE-2020-36188) | | CVE-2020-36188 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36188](https://nvd.nist.gov/vuln/detail/CVE-2020-36188) | | CVE-2020-36188 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36188](https://nvd.nist.gov/vuln/detail/CVE-2020-36188) | | CVE-2020-36189 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36189](https://nvd.nist.gov/vuln/detail/CVE-2020-36189) | | CVE-2020-36189 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36189](https://nvd.nist.gov/vuln/detail/CVE-2020-36189) | | CVE-2020-36189 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36189](https://nvd.nist.gov/vuln/detail/CVE-2020-36189) | | CVE-2020-36189 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36189](https://nvd.nist.gov/vuln/detail/CVE-2020-36189) | | CVE-2020-36189 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36189](https://nvd.nist.gov/vuln/detail/CVE-2020-36189) | | CVE-2020-36518 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36518](https://nvd.nist.gov/vuln/detail/CVE-2020-36518) | | CVE-2020-36518 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36518](https://nvd.nist.gov/vuln/detail/CVE-2020-36518) | | CVE-2020-36518 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36518](https://nvd.nist.gov/vuln/detail/CVE-2020-36518) | | CVE-2020-36518 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36518](https://nvd.nist.gov/vuln/detail/CVE-2020-36518) | | CVE-2020-36518 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36518](https://nvd.nist.gov/vuln/detail/CVE-2020-36518) | | CVE-2020-36518 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-36518](https://nvd.nist.gov/vuln/detail/CVE-2020-36518) | | CVE-2020-9492 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2020-9492](https://nvd.nist.gov/vuln/detail/CVE-2020-9492) | | CVE-2021-20190 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-20190](https://nvd.nist.gov/vuln/detail/CVE-2021-20190) | | CVE-2021-20190 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-20190](https://nvd.nist.gov/vuln/detail/CVE-2021-20190) | | CVE-2021-20190 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-20190](https://nvd.nist.gov/vuln/detail/CVE-2021-20190) | | CVE-2021-20190 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-20190](https://nvd.nist.gov/vuln/detail/CVE-2021-20190) | | CVE-2021-20190 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-20190](https://nvd.nist.gov/vuln/detail/CVE-2021-20190) | | CVE-2021-22573 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-22573](https://nvd.nist.gov/vuln/detail/CVE-2021-22573) | | CVE-2021-22573 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-22573](https://nvd.nist.gov/vuln/detail/CVE-2021-22573) | | CVE-2021-22573 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-22573](https://nvd.nist.gov/vuln/detail/CVE-2021-22573) | | CVE-2021-25646 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-25646](https://nvd.nist.gov/vuln/detail/CVE-2021-25646) | | CVE-2021-26919 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-26919](https://nvd.nist.gov/vuln/detail/CVE-2021-26919) | | CVE-2021-28165 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-28165](https://nvd.nist.gov/vuln/detail/CVE-2021-28165) | | CVE-2021-31684 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-31684](https://nvd.nist.gov/vuln/detail/CVE-2021-31684) | | CVE-2021-31684 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-31684](https://nvd.nist.gov/vuln/detail/CVE-2021-31684) | | CVE-2021-31684 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-31684](https://nvd.nist.gov/vuln/detail/CVE-2021-31684) | | CVE-2021-33036 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-33036](https://nvd.nist.gov/vuln/detail/CVE-2021-33036) | | CVE-2021-33813 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-33813](https://nvd.nist.gov/vuln/detail/CVE-2021-33813) | | CVE-2021-33813 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-33813](https://nvd.nist.gov/vuln/detail/CVE-2021-33813) | | CVE-2021-34538 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-34538](https://nvd.nist.gov/vuln/detail/CVE-2021-34538) | | CVE-2021-34538 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-34538](https://nvd.nist.gov/vuln/detail/CVE-2021-34538) | | CVE-2021-34538 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-34538](https://nvd.nist.gov/vuln/detail/CVE-2021-34538) | | CVE-2021-35515 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-35515](https://nvd.nist.gov/vuln/detail/CVE-2021-35515) | | CVE-2021-35515 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-35515](https://nvd.nist.gov/vuln/detail/CVE-2021-35515) | | CVE-2021-35515 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-35515](https://nvd.nist.gov/vuln/detail/CVE-2021-35515) | | CVE-2021-35515 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-35515](https://nvd.nist.gov/vuln/detail/CVE-2021-35515) | | CVE-2021-35515 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-35515](https://nvd.nist.gov/vuln/detail/CVE-2021-35515) | | CVE-2021-35515 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-35515](https://nvd.nist.gov/vuln/detail/CVE-2021-35515) | | CVE-2021-35515 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-35515](https://nvd.nist.gov/vuln/detail/CVE-2021-35515) | | CVE-2021-35516 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-35516](https://nvd.nist.gov/vuln/detail/CVE-2021-35516) | | CVE-2021-35516 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-35516](https://nvd.nist.gov/vuln/detail/CVE-2021-35516) | | CVE-2021-35516 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-35516](https://nvd.nist.gov/vuln/detail/CVE-2021-35516) | | CVE-2021-35516 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-35516](https://nvd.nist.gov/vuln/detail/CVE-2021-35516) | | CVE-2021-35516 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-35516](https://nvd.nist.gov/vuln/detail/CVE-2021-35516) | | CVE-2021-35516 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-35516](https://nvd.nist.gov/vuln/detail/CVE-2021-35516) | | CVE-2021-35516 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-35516](https://nvd.nist.gov/vuln/detail/CVE-2021-35516) | | CVE-2021-35517 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-35517](https://nvd.nist.gov/vuln/detail/CVE-2021-35517) | | CVE-2021-35517 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-35517](https://nvd.nist.gov/vuln/detail/CVE-2021-35517) | | CVE-2021-35517 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-35517](https://nvd.nist.gov/vuln/detail/CVE-2021-35517) | | CVE-2021-35517 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-35517](https://nvd.nist.gov/vuln/detail/CVE-2021-35517) | | CVE-2021-35517 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-35517](https://nvd.nist.gov/vuln/detail/CVE-2021-35517) | | CVE-2021-36090 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-36090](https://nvd.nist.gov/vuln/detail/CVE-2021-36090) | | CVE-2021-36090 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-36090](https://nvd.nist.gov/vuln/detail/CVE-2021-36090) | | CVE-2021-36090 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-36090](https://nvd.nist.gov/vuln/detail/CVE-2021-36090) | | CVE-2021-36090 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-36090](https://nvd.nist.gov/vuln/detail/CVE-2021-36090) | | CVE-2021-36090 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-36090](https://nvd.nist.gov/vuln/detail/CVE-2021-36090) | | CVE-2021-37136 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-37136](https://nvd.nist.gov/vuln/detail/CVE-2021-37136) | | CVE-2021-37136 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-37136](https://nvd.nist.gov/vuln/detail/CVE-2021-37136) | | CVE-2021-37137 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-37137](https://nvd.nist.gov/vuln/detail/CVE-2021-37137) | | CVE-2021-37137 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-37137](https://nvd.nist.gov/vuln/detail/CVE-2021-37137) | | CVE-2021-37714 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-37714](https://nvd.nist.gov/vuln/detail/CVE-2021-37714) | | CVE-2021-37714 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-37714](https://nvd.nist.gov/vuln/detail/CVE-2021-37714) | | CVE-2021-37714 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-37714](https://nvd.nist.gov/vuln/detail/CVE-2021-37714) | | CVE-2021-37714 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-37714](https://nvd.nist.gov/vuln/detail/CVE-2021-37714) | | CVE-2021-46877 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2021-46877](https://nvd.nist.gov/vuln/detail/CVE-2021-46877) | | CVE-2022-24758 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-24758](https://nvd.nist.gov/vuln/detail/CVE-2022-24758) | | CVE-2022-25647 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-25647](https://nvd.nist.gov/vuln/detail/CVE-2022-25647) | | CVE-2022-25647 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-25647](https://nvd.nist.gov/vuln/detail/CVE-2022-25647) | | CVE-2022-25647 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-25647](https://nvd.nist.gov/vuln/detail/CVE-2022-25647) | | CVE-2022-25647 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-25647](https://nvd.nist.gov/vuln/detail/CVE-2022-25647) | | CVE-2022-25647 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-25647](https://nvd.nist.gov/vuln/detail/CVE-2022-25647) | | CVE-2022-25647 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-25647](https://nvd.nist.gov/vuln/detail/CVE-2022-25647) | | CVE-2022-25647 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-25647](https://nvd.nist.gov/vuln/detail/CVE-2022-25647) | | CVE-2022-25647 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-25647](https://nvd.nist.gov/vuln/detail/CVE-2022-25647) | | CVE-2022-25857 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-25857](https://nvd.nist.gov/vuln/detail/CVE-2022-25857) | | CVE-2022-25857 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-25857](https://nvd.nist.gov/vuln/detail/CVE-2022-25857) | | CVE-2022-25857 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-25857](https://nvd.nist.gov/vuln/detail/CVE-2022-25857) | | CVE-2022-25857 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-25857](https://nvd.nist.gov/vuln/detail/CVE-2022-25857) | | CVE-2022-31197 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-31197](https://nvd.nist.gov/vuln/detail/CVE-2022-31197) | | CVE-2022-3171 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-3171](https://nvd.nist.gov/vuln/detail/CVE-2022-3171) | | CVE-2022-3171 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-3171](https://nvd.nist.gov/vuln/detail/CVE-2022-3171) | | CVE-2022-3171 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-3171](https://nvd.nist.gov/vuln/detail/CVE-2022-3171) | | CVE-2022-3171 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-3171](https://nvd.nist.gov/vuln/detail/CVE-2022-3171) | | CVE-2022-3171 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-3171](https://nvd.nist.gov/vuln/detail/CVE-2022-3171) | | CVE-2022-3171 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-3171](https://nvd.nist.gov/vuln/detail/CVE-2022-3171) | | CVE-2022-3171 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-3171](https://nvd.nist.gov/vuln/detail/CVE-2022-3171) | | CVE-2022-3171 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-3171](https://nvd.nist.gov/vuln/detail/CVE-2022-3171) | | CVE-2022-3171 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-3171](https://nvd.nist.gov/vuln/detail/CVE-2022-3171) | | CVE-2022-34169 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-34169](https://nvd.nist.gov/vuln/detail/CVE-2022-34169) | | CVE-2022-3509 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-3509](https://nvd.nist.gov/vuln/detail/CVE-2022-3509) | | CVE-2022-3509 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-3509](https://nvd.nist.gov/vuln/detail/CVE-2022-3509) | | CVE-2022-3509 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-3509](https://nvd.nist.gov/vuln/detail/CVE-2022-3509) | | CVE-2022-3509 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-3509](https://nvd.nist.gov/vuln/detail/CVE-2022-3509) | | CVE-2022-3509 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-3509](https://nvd.nist.gov/vuln/detail/CVE-2022-3509) | | CVE-2022-3509 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-3509](https://nvd.nist.gov/vuln/detail/CVE-2022-3509) | | CVE-2022-3509 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-3509](https://nvd.nist.gov/vuln/detail/CVE-2022-3509) | | CVE-2022-3509 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-3509](https://nvd.nist.gov/vuln/detail/CVE-2022-3509) | | CVE-2022-3509 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-3509](https://nvd.nist.gov/vuln/detail/CVE-2022-3509) | | CVE-2022-3510 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-3510](https://nvd.nist.gov/vuln/detail/CVE-2022-3510) | | CVE-2022-3510 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-3510](https://nvd.nist.gov/vuln/detail/CVE-2022-3510) | | CVE-2022-3510 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-3510](https://nvd.nist.gov/vuln/detail/CVE-2022-3510) | | CVE-2022-3510 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-3510](https://nvd.nist.gov/vuln/detail/CVE-2022-3510) | | CVE-2022-3510 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-3510](https://nvd.nist.gov/vuln/detail/CVE-2022-3510) | | CVE-2022-3510 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-3510](https://nvd.nist.gov/vuln/detail/CVE-2022-3510) | | CVE-2022-3510 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-3510](https://nvd.nist.gov/vuln/detail/CVE-2022-3510) | | CVE-2022-3510 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-3510](https://nvd.nist.gov/vuln/detail/CVE-2022-3510) | | CVE-2022-3510 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-3510](https://nvd.nist.gov/vuln/detail/CVE-2022-3510) | | CVE-2022-36364 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-36364](https://nvd.nist.gov/vuln/detail/CVE-2022-36364) | | CVE-2022-37866 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-37866](https://nvd.nist.gov/vuln/detail/CVE-2022-37866) | | CVE-2022-37866 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-37866](https://nvd.nist.gov/vuln/detail/CVE-2022-37866) | | CVE-2022-40146 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-40146](https://nvd.nist.gov/vuln/detail/CVE-2022-40146) | | CVE-2022-40149 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-40149](https://nvd.nist.gov/vuln/detail/CVE-2022-40149) | | CVE-2022-40149 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-40149](https://nvd.nist.gov/vuln/detail/CVE-2022-40149) | | CVE-2022-40149 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-40149](https://nvd.nist.gov/vuln/detail/CVE-2022-40149) | | CVE-2022-40149 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-40149](https://nvd.nist.gov/vuln/detail/CVE-2022-40149) | | CVE-2022-40149 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-40149](https://nvd.nist.gov/vuln/detail/CVE-2022-40149) | | CVE-2022-40150 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-40150](https://nvd.nist.gov/vuln/detail/CVE-2022-40150) | | CVE-2022-40150 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-40150](https://nvd.nist.gov/vuln/detail/CVE-2022-40150) | | CVE-2022-40150 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-40150](https://nvd.nist.gov/vuln/detail/CVE-2022-40150) | | CVE-2022-40150 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-40150](https://nvd.nist.gov/vuln/detail/CVE-2022-40150) | | CVE-2022-40150 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-40150](https://nvd.nist.gov/vuln/detail/CVE-2022-40150) | | CVE-2022-40152 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-40152](https://nvd.nist.gov/vuln/detail/CVE-2022-40152) | | CVE-2022-40152 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-40152](https://nvd.nist.gov/vuln/detail/CVE-2022-40152) | | CVE-2022-41704 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-41704](https://nvd.nist.gov/vuln/detail/CVE-2022-41704) | | CVE-2022-42003 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-42003](https://nvd.nist.gov/vuln/detail/CVE-2022-42003) | | CVE-2022-42003 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-42003](https://nvd.nist.gov/vuln/detail/CVE-2022-42003) | | CVE-2022-42003 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-42003](https://nvd.nist.gov/vuln/detail/CVE-2022-42003) | | CVE-2022-42003 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-42003](https://nvd.nist.gov/vuln/detail/CVE-2022-42003) | | CVE-2022-42003 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-42003](https://nvd.nist.gov/vuln/detail/CVE-2022-42003) | | CVE-2022-42003 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-42003](https://nvd.nist.gov/vuln/detail/CVE-2022-42003) | | CVE-2022-42003 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-42003](https://nvd.nist.gov/vuln/detail/CVE-2022-42003) | | CVE-2022-42003 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-42003](https://nvd.nist.gov/vuln/detail/CVE-2022-42003) | | CVE-2022-42003 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-42003](https://nvd.nist.gov/vuln/detail/CVE-2022-42003) | | CVE-2022-42003 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-42003](https://nvd.nist.gov/vuln/detail/CVE-2022-42003) | | CVE-2022-42003 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-42003](https://nvd.nist.gov/vuln/detail/CVE-2022-42003) | | CVE-2022-42003 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-42003](https://nvd.nist.gov/vuln/detail/CVE-2022-42003) | | CVE-2022-42004 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-42004](https://nvd.nist.gov/vuln/detail/CVE-2022-42004) | | CVE-2022-42004 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-42004](https://nvd.nist.gov/vuln/detail/CVE-2022-42004) | | CVE-2022-42004 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-42004](https://nvd.nist.gov/vuln/detail/CVE-2022-42004) | | CVE-2022-42004 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-42004](https://nvd.nist.gov/vuln/detail/CVE-2022-42004) | | CVE-2022-42004 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-42004](https://nvd.nist.gov/vuln/detail/CVE-2022-42004) | | CVE-2022-42004 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-42004](https://nvd.nist.gov/vuln/detail/CVE-2022-42004) | | CVE-2022-42004 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-42004](https://nvd.nist.gov/vuln/detail/CVE-2022-42004) | | CVE-2022-42004 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-42004](https://nvd.nist.gov/vuln/detail/CVE-2022-42004) | | CVE-2022-42004 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-42004](https://nvd.nist.gov/vuln/detail/CVE-2022-42004) | | CVE-2022-42004 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-42004](https://nvd.nist.gov/vuln/detail/CVE-2022-42004) | | CVE-2022-4244 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-4244](https://nvd.nist.gov/vuln/detail/CVE-2022-4244) | | CVE-2022-42890 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-42890](https://nvd.nist.gov/vuln/detail/CVE-2022-42890) | | CVE-2022-44729 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-44729](https://nvd.nist.gov/vuln/detail/CVE-2022-44729) | | CVE-2022-44729 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-44729](https://nvd.nist.gov/vuln/detail/CVE-2022-44729) | | CVE-2022-44729 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-44729](https://nvd.nist.gov/vuln/detail/CVE-2022-44729) | | CVE-2022-45685 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-45685](https://nvd.nist.gov/vuln/detail/CVE-2022-45685) | | CVE-2022-45685 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-45685](https://nvd.nist.gov/vuln/detail/CVE-2022-45685) | | CVE-2022-45685 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-45685](https://nvd.nist.gov/vuln/detail/CVE-2022-45685) | | CVE-2022-45685 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-45685](https://nvd.nist.gov/vuln/detail/CVE-2022-45685) | | CVE-2022-45685 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-45685](https://nvd.nist.gov/vuln/detail/CVE-2022-45685) | | CVE-2022-45688 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-45688](https://nvd.nist.gov/vuln/detail/CVE-2022-45688) | | CVE-2022-45688 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-45688](https://nvd.nist.gov/vuln/detail/CVE-2022-45688) | | CVE-2022-45688 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-45688](https://nvd.nist.gov/vuln/detail/CVE-2022-45688) | | CVE-2022-45693 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-45693](https://nvd.nist.gov/vuln/detail/CVE-2022-45693) | | CVE-2022-45693 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-45693](https://nvd.nist.gov/vuln/detail/CVE-2022-45693) | | CVE-2022-45693 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-45693](https://nvd.nist.gov/vuln/detail/CVE-2022-45693) | | CVE-2022-45693 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-45693](https://nvd.nist.gov/vuln/detail/CVE-2022-45693) | | CVE-2022-45693 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-45693](https://nvd.nist.gov/vuln/detail/CVE-2022-45693) | | CVE-2022-46751 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-46751](https://nvd.nist.gov/vuln/detail/CVE-2022-46751) | | CVE-2022-46751 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-46751](https://nvd.nist.gov/vuln/detail/CVE-2022-46751) | | CVE-2022-46751 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2022-46751](https://nvd.nist.gov/vuln/detail/CVE-2022-46751) | | CVE-2023-1370 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-1370](https://nvd.nist.gov/vuln/detail/CVE-2023-1370) | | CVE-2023-1370 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-1370](https://nvd.nist.gov/vuln/detail/CVE-2023-1370) | | CVE-2023-1370 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-1370](https://nvd.nist.gov/vuln/detail/CVE-2023-1370) | | CVE-2023-1370 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-1370](https://nvd.nist.gov/vuln/detail/CVE-2023-1370) | | CVE-2023-1370 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-1370](https://nvd.nist.gov/vuln/detail/CVE-2023-1370) | | CVE-2023-1428 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-1428](https://nvd.nist.gov/vuln/detail/CVE-2023-1428) | | CVE-2023-1436 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-1436](https://nvd.nist.gov/vuln/detail/CVE-2023-1436) | | CVE-2023-1436 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-1436](https://nvd.nist.gov/vuln/detail/CVE-2023-1436) | | CVE-2023-1436 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-1436](https://nvd.nist.gov/vuln/detail/CVE-2023-1436) | | CVE-2023-1436 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-1436](https://nvd.nist.gov/vuln/detail/CVE-2023-1436) | | CVE-2023-1436 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-1436](https://nvd.nist.gov/vuln/detail/CVE-2023-1436) | | CVE-2023-20883 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-20883](https://nvd.nist.gov/vuln/detail/CVE-2023-20883) | | CVE-2023-24580 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-24580](https://nvd.nist.gov/vuln/detail/CVE-2023-24580) | | CVE-2023-25194 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-25194](https://nvd.nist.gov/vuln/detail/CVE-2023-25194) | | CVE-2023-25194 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-25194](https://nvd.nist.gov/vuln/detail/CVE-2023-25194) | | CVE-2023-25194 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-25194](https://nvd.nist.gov/vuln/detail/CVE-2023-25194) | | CVE-2023-2976 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-2976](https://nvd.nist.gov/vuln/detail/CVE-2023-2976) | | CVE-2023-2976 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-2976](https://nvd.nist.gov/vuln/detail/CVE-2023-2976) | | CVE-2023-2976 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-2976](https://nvd.nist.gov/vuln/detail/CVE-2023-2976) | | CVE-2023-2976 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-2976](https://nvd.nist.gov/vuln/detail/CVE-2023-2976) | | CVE-2023-2976 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-2976](https://nvd.nist.gov/vuln/detail/CVE-2023-2976) | | CVE-2023-2976 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-2976](https://nvd.nist.gov/vuln/detail/CVE-2023-2976) | | CVE-2023-2976 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-2976](https://nvd.nist.gov/vuln/detail/CVE-2023-2976) | | CVE-2023-2976 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-2976](https://nvd.nist.gov/vuln/detail/CVE-2023-2976) | | CVE-2023-2976 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-2976](https://nvd.nist.gov/vuln/detail/CVE-2023-2976) | | CVE-2023-2976 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-2976](https://nvd.nist.gov/vuln/detail/CVE-2023-2976) | | CVE-2023-2976 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-2976](https://nvd.nist.gov/vuln/detail/CVE-2023-2976) | | CVE-2023-2976 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-2976](https://nvd.nist.gov/vuln/detail/CVE-2023-2976) | | CVE-2023-2976 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-2976](https://nvd.nist.gov/vuln/detail/CVE-2023-2976) | | CVE-2023-2976 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-2976](https://nvd.nist.gov/vuln/detail/CVE-2023-2976) | | CVE-2023-2976 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-2976](https://nvd.nist.gov/vuln/detail/CVE-2023-2976) | | CVE-2023-2976 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-2976](https://nvd.nist.gov/vuln/detail/CVE-2023-2976) | | CVE-2023-2976 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-2976](https://nvd.nist.gov/vuln/detail/CVE-2023-2976) | | CVE-2023-2976 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-2976](https://nvd.nist.gov/vuln/detail/CVE-2023-2976) | | CVE-2023-2976 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-2976](https://nvd.nist.gov/vuln/detail/CVE-2023-2976) | | CVE-2023-2976 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-2976](https://nvd.nist.gov/vuln/detail/CVE-2023-2976) | | CVE-2023-2976 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-2976](https://nvd.nist.gov/vuln/detail/CVE-2023-2976) | | CVE-2023-2976 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-2976](https://nvd.nist.gov/vuln/detail/CVE-2023-2976) | | CVE-2023-31418 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-31418](https://nvd.nist.gov/vuln/detail/CVE-2023-31418) | | CVE-2023-32731 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-32731](https://nvd.nist.gov/vuln/detail/CVE-2023-32731) | | CVE-2023-34453 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-34453](https://nvd.nist.gov/vuln/detail/CVE-2023-34453) | | CVE-2023-34453 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-34453](https://nvd.nist.gov/vuln/detail/CVE-2023-34453) | | CVE-2023-34453 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-34453](https://nvd.nist.gov/vuln/detail/CVE-2023-34453) | | CVE-2023-34453 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-34453](https://nvd.nist.gov/vuln/detail/CVE-2023-34453) | | CVE-2023-34453 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-34453](https://nvd.nist.gov/vuln/detail/CVE-2023-34453) | | CVE-2023-34454 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-34454](https://nvd.nist.gov/vuln/detail/CVE-2023-34454) | | CVE-2023-34454 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-34454](https://nvd.nist.gov/vuln/detail/CVE-2023-34454) | | CVE-2023-34454 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-34454](https://nvd.nist.gov/vuln/detail/CVE-2023-34454) | | CVE-2023-34454 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-34454](https://nvd.nist.gov/vuln/detail/CVE-2023-34454) | | CVE-2023-34454 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-34454](https://nvd.nist.gov/vuln/detail/CVE-2023-34454) | | CVE-2023-34455 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-34455](https://nvd.nist.gov/vuln/detail/CVE-2023-34455) | | CVE-2023-34455 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-34455](https://nvd.nist.gov/vuln/detail/CVE-2023-34455) | | CVE-2023-34455 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-34455](https://nvd.nist.gov/vuln/detail/CVE-2023-34455) | | CVE-2023-34455 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-34455](https://nvd.nist.gov/vuln/detail/CVE-2023-34455) | | CVE-2023-34455 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-34455](https://nvd.nist.gov/vuln/detail/CVE-2023-34455) | | CVE-2023-34610 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-34610](https://nvd.nist.gov/vuln/detail/CVE-2023-34610) | | CVE-2023-36053 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-36053](https://nvd.nist.gov/vuln/detail/CVE-2023-36053) | | CVE-2023-3635 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-3635](https://nvd.nist.gov/vuln/detail/CVE-2023-3635) | | CVE-2023-3635 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-3635](https://nvd.nist.gov/vuln/detail/CVE-2023-3635) | | CVE-2023-3635 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-3635](https://nvd.nist.gov/vuln/detail/CVE-2023-3635) | | CVE-2023-3635 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-3635](https://nvd.nist.gov/vuln/detail/CVE-2023-3635) | | CVE-2023-3635 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-3635](https://nvd.nist.gov/vuln/detail/CVE-2023-3635) | | CVE-2023-36478 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-36478](https://nvd.nist.gov/vuln/detail/CVE-2023-36478) | | CVE-2023-36478 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-36478](https://nvd.nist.gov/vuln/detail/CVE-2023-36478) | | CVE-2023-39410 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-39410](https://nvd.nist.gov/vuln/detail/CVE-2023-39410) | | CVE-2023-39410 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-39410](https://nvd.nist.gov/vuln/detail/CVE-2023-39410) | | CVE-2023-39410 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-39410](https://nvd.nist.gov/vuln/detail/CVE-2023-39410) | | CVE-2023-39410 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-39410](https://nvd.nist.gov/vuln/detail/CVE-2023-39410) | | CVE-2023-39410 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-39410](https://nvd.nist.gov/vuln/detail/CVE-2023-39410) | | CVE-2023-39410 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-39410](https://nvd.nist.gov/vuln/detail/CVE-2023-39410) | | CVE-2023-39410 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-39410](https://nvd.nist.gov/vuln/detail/CVE-2023-39410) | | CVE-2023-41164 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-41164](https://nvd.nist.gov/vuln/detail/CVE-2023-41164) | | CVE-2023-43642 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-43642](https://nvd.nist.gov/vuln/detail/CVE-2023-43642) | | CVE-2023-43642 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-43642](https://nvd.nist.gov/vuln/detail/CVE-2023-43642) | | CVE-2023-43642 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-43642](https://nvd.nist.gov/vuln/detail/CVE-2023-43642) | | CVE-2023-43642 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-43642](https://nvd.nist.gov/vuln/detail/CVE-2023-43642) | | CVE-2023-43642 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-43642](https://nvd.nist.gov/vuln/detail/CVE-2023-43642) | | CVE-2023-43642 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-43642](https://nvd.nist.gov/vuln/detail/CVE-2023-43642) | | CVE-2023-43642 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-43642](https://nvd.nist.gov/vuln/detail/CVE-2023-43642) | | CVE-2023-43665 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-43665](https://nvd.nist.gov/vuln/detail/CVE-2023-43665) | | CVE-2023-46120 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-46120](https://nvd.nist.gov/vuln/detail/CVE-2023-46120) | | CVE-2023-46695 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-46695](https://nvd.nist.gov/vuln/detail/CVE-2023-46695) | | CVE-2023-4759 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-4759](https://nvd.nist.gov/vuln/detail/CVE-2023-4759) | | CVE-2023-5072 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-5072](https://nvd.nist.gov/vuln/detail/CVE-2023-5072) | | CVE-2023-5072 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-5072](https://nvd.nist.gov/vuln/detail/CVE-2023-5072) | | CVE-2023-5072 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-5072](https://nvd.nist.gov/vuln/detail/CVE-2023-5072) | | CVE-2023-6378 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-6378](https://nvd.nist.gov/vuln/detail/CVE-2023-6378) | | CVE-2023-6378 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-6378](https://nvd.nist.gov/vuln/detail/CVE-2023-6378) | | CVE-2023-6378 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-6378](https://nvd.nist.gov/vuln/detail/CVE-2023-6378) | | CVE-2023-6378 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-6378](https://nvd.nist.gov/vuln/detail/CVE-2023-6378) | | CVE-2023-6481 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-6481](https://nvd.nist.gov/vuln/detail/CVE-2023-6481) | | CVE-2023-6481 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-6481](https://nvd.nist.gov/vuln/detail/CVE-2023-6481) | | CVE-2023-6481 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-6481](https://nvd.nist.gov/vuln/detail/CVE-2023-6481) | | CVE-2023-6481 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2023-6481](https://nvd.nist.gov/vuln/detail/CVE-2023-6481) | | CVE-2024-21634 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-21634](https://nvd.nist.gov/vuln/detail/CVE-2024-21634) | | CVE-2024-24680 | HIGH | [https://nvd.nist.gov/vuln/detail/CVE-2024-24680](https://nvd.nist.gov/vuln/detail/CVE-2024-24680) | ## 7.2.0 release notes The following release notes include information for Amazon EMR release 7.2.0. Changes are relative to 7.2.0. **New features** + **Application upgrades** – Amazon EMR 7.2.0 application upgrades include Iceberg 1.5.0-amzn-0 and Delta 3.1.0. + Amazon EMR adds support so that you can use other applications such as HBase, Flink, and Hive with the Amazon S3 Express One Zone storage class. + This release adds the capability to read restored objects, so you can read Glacier objects from an S3 location with the `S3A` protocol. This capability works with Spark, Flink, and Hive. + Node labels with managed scaling - Use node labels with your instances based on market type or node type to improve automatic scaling from Amazon EMR. For more information, see [Using managed scaling in Amazon EMR](https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-managed-scaling.html). **Known issues** + Python 3.11 isn't supported with EMR Studio. **Changes, enhancements, and resolved issues** + This release fixes an issue in an on-cluster instance-state script that monitors health of the instance which perviously consumed excessive CPU and memory resources when there are a large number of threads and/or open file handles on the node. + Starting with Spark 3.3.1 (supported in EMR versions 6.10 and above), all executors in a decommissioning host are set to a new `ExecutorState`, called *DECOMMISSIONING* state. The executors being decommissioned cannot be used by Yarn to allocate tasks and thus it will request for new executors, if needed, for the tasks being executed. Thus, if you disable Spark DRA while using EMR Managed Scaling, EMR Auto Scaling, or any custom scaling mechanism on EMR-EC2 clusters, then Yarn may request maximum permissible executors for each job. In order to avoid this issue, leave the `spark.dynamicAllocation.enabled` property set to `TRUE` (which is the default) when you are using the above combination of features. In addition, you can also set minimum and maximum executor constraints by setting values for `spark.dynamicAllocation.maxExecutors` and `spark.dynamicAllocation.minExecutors` properties for your Spark jobs, to restrict the number of executors allocated during the job’s execution. + This release fixes a deadlock issue that can occur during internal step cleanup operations. This operation manages the life cycle of steps as they complete on the EMR cluster. This issue affects critical Amazon EMR operations, such as step operation and scaling. + This release resolves an issue where custom clusters with custom AMIs that have certain pre-existing log files can cause the Amazon EMR log management daemon to fail. + Amazon EMR 7.2.0 upgrades the Amazon EMR daemon responsible for cluster management and monitoring activities from AWS SDK v1 to v2. + When you launch a cluster with *the latest patch release* of Amazon EMR 5.36 or higher, 6.6 or higher, or 7.0 or higher, Amazon EMR uses the latest Amazon Linux 2023 or Amazon Linux 2 release for the default Amazon EMR AMI. For more information, see [Using the default Amazon Linux AMI for Amazon EMR](https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-default-ami.html). [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-720-release.html) ## 7.2.0 default Java versions Amazon EMR release 7.2 and higher ships with Amazon Corretto 17 (built on OpenJDK) by default for applications that support Corretto 17 (JDK 17), with the exception of Apache Livy. The following table shows the default Java versions for applications in Amazon EMR 7.2.0. If you want to change the default JVM on your cluster, follow the instructions in [Configure applications to use a specific Java Virtual Machine](configuring-java8.md) for each application that runs on the cluster. You can only use one Java runtime version for a cluster. Amazon EMR doesn't support running different nodes or applications on different runtime versions on the same cluster. | Application | Java / Amazon Corretto version (default is bold) | | --- | --- | | Delta | 17, 11, 8 | | Flink | 17, 11, 8 | | HBase | 11, 8 | | HCatalog | 17, 11, 8 | | Hadoop | 17, 11, 8 | | Hive | 17, 11, 8 | | Hudi | 17, 11, 8 | | Iceberg | 17, 11, 8 | | Livy | 17, 11, 8 | | Oozie | 17, 11, 8 | | Phoenix | 8 | | PrestoDB | 8 | | Spark | 17, 11, 8 | | Spark RAPIDS | 17, 11, 8 | | Sqoop | 8 | | Tez | 17, 11, 8 | | Trino | 21, 17 | | Zeppelin | 8 | | Pig | 8 | | Zookeeper | 8 | ## 7.2.0 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | adot-java-agent | 1.31.0 | A Java Agent that collects metrics from application daemons. | | delta | 3.1.0 | Delta lake is an open table format for huge analytic datasets | | emr-amazon-cloudwatch-agent | 1.300032.2-amzn-0 | An application that collects internal system-level metrics and custom application metrics from Amazon EC2 instances. | | emr-ddb | 5.4.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 3.11.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.15.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-notebook-env | 1.16.0 | Conda env for emr notebook which includes jupyter enterprise gateway | | emr-s3-dist-cp | 2.32.0 | Distributed copy application optimized for Amazon S3. | | emr-s3-select | 2.11.0 | EMR S3Select Connector | | emr-wal-cli | 1.2.2 | Cli used for emrwal list/deletion. | | emrfs | 2.63.0 | Amazon S3 connector for Hadoop ecosystem applications. | | flink-client | 1.18.1-amzn-1 | Apache Flink command line client scripts and applications. | | flink-jobmanager-config | 1.18.1-amzn-1 | Managing resources on EMR nodes for Apache Flink JobManager. | | hadoop-client | 3.3.6-amzn-4 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 3.3.6-amzn-4 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 3.3.6-amzn-4 | HDFS command-line client and library | | hadoop-hdfs-namenode | 3.3.6-amzn-4 | HDFS service for tracking file names and block locations. | | hadoop-hdfs-journalnode | 3.3.6-amzn-4 | HDFS service for managing the Hadoop filesystem journal on HA clusters. | | hadoop-httpfs-server | 3.3.6-amzn-4 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 3.3.6-amzn-4 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 3.3.6-amzn-4 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 3.3.6-amzn-4 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 3.3.6-amzn-4 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 3.3.6-amzn-4 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 2.4.17-amzn-6 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 2.4.17-amzn-6 | Service for serving one or more HBase regions. | | hbase-client | 2.4.17-amzn-6 | HBase command-line client. | | hbase-rest-server | 2.4.17-amzn-6 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 2.4.17-amzn-6 | Service providing a Thrift endpoint to HBase. | | hbase-operator-tools | 2.4.17-amzn-6 | Repair tool for Apache HBase clusters. | | hcatalog-client | 3.1.3-amzn-11 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 3.1.3-amzn-11 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 3.1.3-amzn-11 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 3.1.3-amzn-11 | Hive command line client. | | hive-hbase | 3.1.3-amzn-11 | Hive-hbase client. | | hive-metastore-server | 3.1.3-amzn-11 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server2 | 3.1.3-amzn-11 | Service for accepting Hive queries as web requests. | | hudi | 0.14.1-amzn-1 | Incremental processing framework to power data pipeline at low latency and high efficiency. | | hudi-presto | 0.14.1-amzn-1 | Bundle library for running Presto with Hudi. | | hudi-trino | 0.14.1-amzn-1 | Bundle library for running Trino with Hudi. | | hudi-spark | 0.14.1-amzn-1 | Bundle library for running Spark with Hudi. | | hue-server | 4.11.0 | Web application for analyzing data using Hadoop ecosystem applications | | iceberg | 1.5.0-amzn-0 | Apache Iceberg is an open table format for huge analytic datasets | | jupyterhub | 1.5.0 | Multi-user server for Jupyter notebooks | | livy-server | 0.8.0-incubating | REST interface for interacting with Apache Spark | | nginx | 1.12.1 | nginx [engine x] is an HTTP and reverse proxy server | | mxnet | 1.9.1 | A flexible, scalable, and efficient library for deep learning. | | mariadb-server | 5.5.68\$1 | MariaDB database server. | | nvidia-cuda | 11.8.0 | Nvidia drivers and Cuda toolkit | | oozie-client | 5.2.1 | Oozie command-line client. | | oozie-server | 5.2.1 | Service for accepting Oozie workflow requests. | | opencv | 4.7.0 | Open Source Computer Vision Library. | | phoenix-library | 5.1.3 | The phoenix libraries for server and client | | phoenix-connectors | 5.1.3 | Apache Phoenix-Connectors for Spark-3 | | phoenix-query-server | 6.0.0 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.285-amzn-0 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.285-amzn-0 | Service for executing pieces of a query. | | presto-client | 0.285-amzn-0 | Presto command-line client which is installed on an HA cluster's stand-by masters where Presto server is not started. | | trino-coordinator | 436-amzn-0 | Service for accepting queries and managing query execution among trino-workers. | | trino-worker | 436-amzn-0 | Service for executing pieces of a query. | | trino-client | 436-amzn-0 | Trino command-line client which is installed on an HA cluster's stand-by masters where Trino server is not started. | | pig-client | 0.17.0 | Pig command-line client. | | r | 4.3.2 | The R Project for Statistical Computing | | ranger-kms-server | 2.0.0 | Apache Ranger Key Management System | | spark-client | 3.5.1-amzn-0 | Spark command-line clients. | | spark-history-server | 3.5.1-amzn-0 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 3.5.1-amzn-0 | In-memory execution engine for YARN. | | spark-yarn-slave | 3.5.1-amzn-0 | Apache Spark libraries needed by YARN slaves. | | spark-rapids | 24.02.0-amzn-0 | Nvidia Spark RAPIDS plugin that accelerates Apache Spark with GPUs. | | sqoop-client | 1.4.7 | Apache Sqoop command-line client. | | tensorflow | 2.11.0 | TensorFlow open source software library for high performance numerical computation. | | tez-on-yarn | 0.10.2-amzn-9 | The tez YARN application and libraries. | | tez-on-worker | 0.10.2-amzn-9 | The tez YARN application and libraries for worker nodes. | | zeppelin-server | 0.10.1 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.9.1-amzn-1 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.9.1-amzn-1 | ZooKeeper command line client. | ## 7.2.0 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). Reconfiguration actions occur when you specify a configuration for instance groups in a running cluster. Amazon EMR only initiates reconfiguration actions for the classifications that you modify. For more information, see [Reconfigure an instance group in a running cluster](emr-configure-apps-running-cluster.md). **emr-7.2.0 classifications** | Classifications | Description | Reconfiguration Actions | | --- | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | Restarts the ResourceManager service. | | container-executor | Change values in Hadoop YARN's container-executor.cfg file. | Not available. | | container-log4j | Change values in Hadoop YARN's container-log4j.properties file. | Not available. | | core-site | Change values in Hadoop's core-site.xml file. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts Hadoop KMS, Ranger KMS, HiveServer2, Hive MetaStore, Hadoop Httpfs, and MapReduce-HistoryServer. | | docker-conf | Change docker related settings. | Not available. | | emrfs-site | Change EMRFS settings. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts HBaseRegionserver, HBaseMaster, HBaseThrift, HBaseRest, HiveServer2, Hive MetaStore, Hadoop Httpfs, and MapReduce-HistoryServer. | | flink-conf | Change flink-conf.yaml settings. | Restarts Flink history server. | | flink-log4j | Change Flink log4j.properties settings. | Restarts Flink history server. | | flink-log4j-session | Change Flink log4j-session.properties settings for Kubernetes/Yarn session. | Restarts Flink history server. | | flink-log4j-cli | Change Flink log4j-cli.properties settings. | Restarts Flink history server. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts PhoenixQueryserver, HiveServer2, Hive MetaStore, and MapReduce-HistoryServer. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | Restarts the Hadoop HDFS services SecondaryNamenode, Datanode, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts Hadoop KMS, Hadoop Httpfs, and MapReduce-HistoryServer. | | hadoop-ssl-server | Change hadoop ssl server configuration | Not available. | | hadoop-ssl-client | Change hadoop ssl client configuration | Not available. | | hbase | Amazon EMR-curated settings for Apache HBase. | Custom EMR specific property. Sets emrfs-site and hbase-site configs. See those for their associated restarts. | | hbase-env | Change values in HBase's environment. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbase.properties file. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | Not available. | | hbase-site | Change values in HBase's hbase-site.xml file. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. Additionally restarts Phoenix QueryServer. | | hdfs-encryption-zones | Configure HDFS encryption zones. | This classification should not be reconfigured. | | hdfs-env | Change values in the HDFS environment. | Restarts Hadoop HDFS services Namenode, Datanode, and ZKFC. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Additionally restarts Hadoop Httpfs. | | hcatalog-env | Change values in HCatalog's environment. | Restarts Hive HCatalog Server. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | Restarts Hive HCatalog Server. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | Restarts Hive HCatalog Server. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | Restarts Hive WebHCat server. | | hcatalog-webhcat-log4j2 | Change values in HCatalog WebHCat's log4j2.properties. | Restarts Hive WebHCat server. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | Restarts Hive WebHCat server. | | hive | Amazon EMR-curated settings for Apache Hive. | Sets configurations to launch Hive LLAP service. | | hive-beeline-log4j2 | Change values in Hive's beeline-log4j2.properties file. | Not available. | | hive-parquet-logging | Change values in Hive's parquet-logging.properties file. | Not available. | | hive-env | Change values in the Hive environment. | Restarts HiveServer2, HiveMetastore, and Hive HCatalog-Server. Runs Hive schemaTool CLI commands to verify hive-metastore. | | hive-exec-log4j2 | Change values in Hive's hive-exec-log4j2.properties file. | Not available. | | hive-llap-daemon-log4j2 | Change values in Hive's llap-daemon-log4j2.properties file. | Not available. | | hive-log4j2 | Change values in Hive's hive-log4j2.properties file. | Not available. | | hive-site | Change values in Hive's hive-site.xml file | Restarts HiveServer2, HiveMetastore, and Hive HCatalog-Server. Runs Hive schemaTool CLI commands to verify hive-metastore. Also restarts Oozie and Zeppelin. | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | Not available. | | hue-ini | Change values in Hue's ini file | Restarts Hue. Also activates Hue config override CLI commands to pick up new configurations. | | httpfs-env | Change values in the HTTPFS environment. | Restarts Hadoop Httpfs service. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | Restarts Hadoop Httpfs service. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | Not available. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | Restarts Hadoop-KMS service. | | hadoop-kms-java-home | Change Hadoop's KMS java home | Not available. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | Not available. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | Restarts Hadoop-KMS and Ranger-KMS service. | | hudi-env | Change values in the Hudi environment. | Not available. | | hudi-defaults | Change values in Hudi's hudi-defaults.conf file. | Not available. | | iceberg-defaults | Change values in Iceberg's iceberg-defaults.conf file. | Not available. | | delta-defaults | Change values in Delta's delta-defaults.conf file. | Not available. | | jupyter-notebook-conf | Change values in Jupyter Notebook's jupyter\$1notebook\$1config.py file. | Not available. | | jupyter-hub-conf | Change values in JupyterHubs's jupyterhub\$1config.py file. | Not available. | | jupyter-s3-conf | Configure Jupyter Notebook S3 persistence. | Not available. | | jupyter-sparkmagic-conf | Change values in Sparkmagic's config.json file. | Not available. | | livy-conf | Change values in Livy's livy.conf file. | Restarts Livy Server. | | livy-env | Change values in the Livy environment. | Restarts Livy Server. | | livy-log4j2 | Change Livy log4j2.properties settings. | Restarts Livy Server. | | mapred-env | Change values in the MapReduce application's environment. | Restarts Hadoop MapReduce-HistoryServer. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | Restarts Hadoop MapReduce-HistoryServer. | | oozie-env | Change values in Oozie's environment. | Restarts Oozie. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | Restarts Oozie. | | oozie-site | Change values in Oozie's oozie-site.xml file. | Restarts Oozie. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | Not available. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | Not available. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | Restarts Phoenix-QueryServer. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | Not available. | | pig-env | Change values in the Pig environment. | Not available. | | pig-properties | Change values in Pig's pig.properties file. | Restarts Oozie. | | pig-log4j | Change values in Pig's log4j.properties file. | Not available. | | presto-log | Change values in Presto's log.properties file. | Restarts Presto-Server (for PrestoDB) | | presto-config | Change values in Presto's config.properties file. | Restarts Presto-Server (for PrestoDB) | | presto-password-authenticator | Change values in Presto's password-authenticator.properties file. | Not available. | | presto-env | Change values in Presto's presto-env.sh file. | Restarts Presto-Server (for PrestoDB) | | presto-node | Change values in Presto's node.properties file. | Not available. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | Not available. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | Not available. | | presto-connector-hive | Change values in Presto's hive.properties file. | Restarts Presto-Server (for PrestoDB) | | presto-connector-jmx | Change values in Presto's jmx.properties file. | Not available. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | Not available. | | presto-connector-lakeformation | Change values in Presto's lakeformation.properties file. | Restarts Presto-Server (for PrestoDB) | | presto-connector-localfile | Change values in Presto's localfile.properties file. | Not available. | | presto-connector-memory | Change values in Presto's memory.properties file. | Not available. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | Not available. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | Not available. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | Not available. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | Not available. | | presto-connector-redis | Change values in Presto's redis.properties file. | Not available. | | presto-connector-redshift | Change values in Presto's redshift.properties file. | Not available. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | Not available. | | presto-connector-tpcds | Change values in Presto's tpcds.properties file. | Not available. | | trino-log | Change values in Trino's log.properties file. | Restarts Trino-Server (for Trino) | | trino-config | Change values in Trino's config.properties file. | Restarts Trino-Server (for Trino) | | trino-password-authenticator | Change values in Trino's password-authenticator.properties file. | Restarts Trino-Server (for Trino) | | trino-env | Change values in Trino's trino-env.sh file. | Restarts Trino-Server (for Trino) | | trino-node | Change values in Trino's node.properties file. | Not available. | | trino-connector-blackhole | Change values in Trino's blackhole.properties file. | Not available. | | trino-connector-cassandra | Change values in Trino's cassandra.properties file. | Not available. | | trino-connector-delta | Change values in Trino's delta.properties file. | Restarts Trino-Server (for Trino) | | trino-connector-hive | Change values in Trino's hive.properties file. | Restarts Trino-Server (for Trino) | | trino-exchange-manager | Change values in Trino's exchange-manager.properties file. | Restarts Trino-Server (for Trino) | | trino-connector-iceberg | Change values in Trino's iceberg.properties file. | Restarts Trino-Server (for Trino) | | trino-connector-hudi | Change values in Trino's hudi.properties file. | Restarts Trino-Server (for Trino) | | trino-connector-jmx | Change values in Trino's jmx.properties file. | Not available. | | trino-connector-kafka | Change values in Trino's kafka.properties file. | Not available. | | trino-connector-localfile | Change values in Trino's localfile.properties file. | Not available. | | trino-connector-memory | Change values in Trino's memory.properties file. | Not available. | | trino-connector-mongodb | Change values in Trino's mongodb.properties file. | Not available. | | trino-connector-mysql | Change values in Trino's mysql.properties file. | Not available. | | trino-connector-postgresql | Change values in Trino's postgresql.properties file. | Not available. | | trino-connector-raptor | Change values in Trino's raptor.properties file. | Not available. | | trino-connector-redis | Change values in Trino's redis.properties file. | Not available. | | trino-connector-redshift | Change values in Trino's redshift.properties file. | Not available. | | trino-connector-tpch | Change values in Trino's tpch.properties file. | Not available. | | trino-connector-tpcds | Change values in Trino's tpcds.properties file. | Not available. | | ranger-kms-dbks-site | Change values in dbks-site.xml file of Ranger KMS. | Restarts Ranger KMS Server. | | ranger-kms-site | Change values in ranger-kms-site.xml file of Ranger KMS. | Restarts Ranger KMS Server. | | ranger-kms-env | Change values in the Ranger KMS environment. | Restarts Ranger KMS Server. | | ranger-kms-logback | Change values in kms-logback.xml file of Ranger KMS. | Not available. | | ranger-kms-db-ca | Change values for CA file on S3 for MySQL SSL connection with Ranger KMS. | Not available. | | spark | Amazon EMR-curated settings for Apache Spark. | This property modifies spark-defaults. See actions there. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | Restarts Spark history server and Spark thrift server. | | spark-env | Change values in the Spark environment. | Restarts Spark history server and Spark thrift server. | | spark-hive-site | Change values in Spark's hive-site.xml file | Not available. | | spark-log4j2 | Change values in Spark's log4j2.properties file. | Restarts Spark history server and Spark thrift server. | | spark-metrics | Change values in Spark's metrics.properties file. | Restarts Spark history server and Spark thrift server. | | sqoop-env | Change values in Sqoop's environment. | Not available. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | Not available. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | Not available. | | tez-site | Change values in Tez's tez-site.xml file. | Restart Oozie and HiveServer2. | | yarn-env | Change values in the YARN environment. | Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts MapReduce-HistoryServer. | | yarn-site | Change values in YARN's yarn-site.xml file. | Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts Livy Server and MapReduce-HistoryServer. | | zeppelin-env | Change values in the Zeppelin environment. | Restarts Zeppelin. | | zeppelin-site | Change configuration settings in zeppelin-site.xml. | Restarts Zeppelin. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | Restarts Zookeeper server. | | zookeeper-logback | Change values in ZooKeeper's logback.xml file. | Restarts Zookeeper server. | | emr-metrics | Change emr metric settings for this node. | Restarts the CloudWatchAgent service. | ## 7.2.0 change log **Change log for 7.2.0 release and release notes** | Date | Event | Description | | --- | --- | --- | | 2025-09-03 | Docs revision | Amazon EMR 7.2.0 release note added | | 2024-07-25 | Docs publication | Amazon EMR 7.2.0 release notes first published | | 2024-07-16 | Initial release | Amazon EMR 7.2.0 first deployed to initial commercial Regions | # Amazon EMR release 7.1.0 ## 7.1.0 application versions This release includes the following applications: [https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/Install-CloudWatch-Agent.html](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/Install-CloudWatch-Agent.html), [https://delta.io/](https://delta.io/), [https://flink.apache.org/](https://flink.apache.org/), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [https://hudi.apache.org](https://hudi.apache.org), [http://gethue.com/](http://gethue.com/), [https://iceberg.apache.org/](https://iceberg.apache.org/), [https://jupyter-enterprise-gateway.readthedocs.io/en/latest/](https://jupyter-enterprise-gateway.readthedocs.io/en/latest/), [https://jupyterhub.readthedocs.io/en/latest/#](https://jupyterhub.readthedocs.io/en/latest/#), [https://livy.incubator.apache.org/](https://livy.incubator.apache.org/), [https://mxnet.incubator.apache.org/](https://mxnet.incubator.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://www.tensorflow.org/](https://www.tensorflow.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://trino.io/](https://trino.io/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-7.3.0 | emr-7.2.0 | emr-7.1.0 | emr-7.0.0 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 2.25.70, 1.12.747 | 2.23.18, 1.12.705 | 2.23.18, 1.12.656 | 2.20.160-amzn-0, 1.12.569 | | Python | 3.9, 3.11 | 3.9, 3.11 | 3.9, 3.11 | 3.9 | | Scala | 2.12.18 | 2.12.17 | 2.12.17 | 2.12.17 | | AmazonCloudWatchAgent | 1.300032.2-amzn-0 | 1.300032.2-amzn-0 | 1.300032.2-amzn-0 | 1.300031.1-amzn-0 | | Delta | 3.2.0-amzn-0 | 3.1.0 | 3.0.0 | 3.0.0 | | Flink | 1.18.1-amzn-2 | 1.18.1-amzn-1 | 1.18.1-amzn-0 | 1.18.0-amzn-0 | | Ganglia | - | - | - | - | | HBase | 2.4.17-amzn-7 | 2.4.17-amzn-6 | 2.4.17-amzn-5 | 2.4.17-amzn-4 | | HCatalog | 3.1.3-amzn-12 | 3.1.3-amzn-11 | 3.1.3-amzn-10 | 3.1.3-amzn-9 | | Hadoop | 3.3.6-amzn-5 | 3.3.6-amzn-4 | 3.3.6-amzn-3 | 3.3.6-amzn-2 | | Hive | 3.1.3-amzn-12 | 3.1.3-amzn-11 | 3.1.3-amzn-10 | 3.1.3-amzn-9 | | Hudi | 0.15.0-amzn-0 | 0.14.1-amzn-1 | 0.14.1-amzn-0 | 0.14.0-amzn-1 | | Hue | 4.11.0 | 4.11.0 | 4.11.0 | 4.11.0 | | Iceberg | 1.5.2-amzn-0 | 1.5.0-amzn-0 | 1.4.3-amzn-0 | 1.4.2-amzn-0 | | JupyterEnterpriseGateway | 2.6.0 | 2.6.0 | 2.6.0 | 2.6.0 | | JupyterHub | 1.5.0 | 1.5.0 | 1.5.0 | 1.5.0 | | Livy | 0.8.0-incubating | 0.8.0-incubating | 0.8.0-incubating | 0.7.1-incubating | | MXNet | - | 1.9.1 | 1.9.1 | 1.9.1 | | Mahout | - | - | - | - | | Oozie | 5.2.1 | 5.2.1 | 5.2.1 | 5.2.1 | | Phoenix | 5.1.3 | 5.1.3 | 5.1.3 | 5.1.3 | | Pig | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | | Presto | 0.285-amzn-1 | 0.285-amzn-0 | 0.284-amzn-0 | 0.283-amzn-1 | | Spark | 3.5.1-amzn-1 | 3.5.1-amzn-0 | 3.5.0-amzn-1 | 3.5.0-amzn-0 | | Sqoop | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | | TensorFlow | 2.16.1 | 2.11.0 | 2.11.0 | 2.11.0 | | Tez | 0.10.2-amzn-10 | 0.10.2-amzn-9 | 0.10.2-amzn-8 | 0.10.2-amzn-7 | | Trino (PrestoSQL) | 442-amzn-0 | 436-amzn-0 | 435-amzn-0 | 426-amzn-1 | | Zeppelin | 0.11.1 | 0.10.1 | 0.10.1 | 0.10.1 | | ZooKeeper | 3.9.1-amzn-1 | 3.9.1-amzn-1 | 3.9.1-amzn-0 | 3.5.10 | ## 7.1.0 release notes The following release notes include information for Amazon EMR release 7.1.0. Changes are relative to 7.1.0. **New features** + **Application upgrades** – Amazon EMR 7.1.0 application upgrades include Livy 0.8.0, Trino 435, and ZooKeeper 3.9.1. + [https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-plan-node-replacement.html](https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-plan-node-replacement.html) – With Amazon EMR 7.1.0 and higher, unhealthy node replacement is enabled by default, so Amazon EMR will gracefully replace your unhealthy nodes. To avoid affecting your existing workflows on Amazon EMR releases 7.0.0 and lower, unhealthy node replacement is disabled if you enabled termination protection in your cluster. + [https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-AmazonCloudWatchAgent.html](https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-AmazonCloudWatchAgent.html) – Configure the CloudWatch agent to use additional system metrics, add application metrics, and change metrics destination with the Amazon EMR configuration API. **Known issues** + The application JupyterEnterpriseGateway (JEG) does not support Python 3.11 in this release. + An on-cluster instance-state script that monitors health of the instance can consume excessive CPU and memory resources when there are a large number of threads and/or open file handles on the node. **Changes, enhancements, and resolved issues** + Starting with Spark 3.3.1 (supported in EMR versions 6.10 and above), all executors in a decommissioning host are set to a new `ExecutorState`, called *DECOMMISSIONING* state. The executors being decommissioned cannot be used by Yarn to allocate tasks and thus it will request for new executors, if needed, for the tasks being executed. Thus, if you disable Spark DRA while using EMR Managed Scaling, EMR Auto Scaling, or any custom scaling mechanism on EMR-EC2 clusters, then Yarn may request maximum permissible executors for each job. In order to avoid this issue, leave the `spark.dynamicAllocation.enabled` property set to `TRUE` (which is the default) when you are using the above combination of features. In addition, you can also set minimum and maximum executor constraints by setting values for `spark.dynamicAllocation.maxExecutors` and `spark.dynamicAllocation.minExecutors` properties for your Spark jobs, to restrict the number of executors allocated during the job’s execution. + While Amazon EMR 7.1.0 supports Python 3.9 by default, Livy 0.8.0 and Spark in Amazon EMR 7.1.0 support Python 3.11. + This release fixes the issue of needing to run each line one at a time when using PySpark with Python version 3.11. + **Zeppelin upgrade** – Amazon EMR 7.1.0 includes an upgrade of Zeppelin to the AWS SDK for Java v2. This upgrade enables a Zeppelin S3 Notebook to accept a custom encryption materials provider. The AWS SDK for Java v2 removes the EncryptionMaterialsProvider interface. When you upgrade to Amazon EMR 7.1.0, you must implement the [Keyring interface](https://github.com/aws/amazon-s3-encryption-client-java/blob/main/src/main/java/software/amazon/encryption/s3/materials/Keyring.java) if you want to use custom encryption. For an example of how to implement the Keyring interface, see [ KmsKeyring.java](https://github.com/aws/amazon-s3-encryption-client-java/blob/a5d714fe27f97793997e41745939ef7da8a22167/src/main/java/software/amazon/encryption/s3/materials/KmsKeyring.java#L36). + When upgrading to Amazon EMR release 7.1.0, change your custom key provider for local disk encryption to generate keys using the AES algorithm of AES/GCM/NoPadding. If you don't update the algorithm, cluster creation might fail with the error `Local disk encryption failed on master instance (i-123456789) due to internal error.` For more information about creating a custom key provider, see [Creating a custom key provider](https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-encryption-enable.html#emr-custom-keys). + Amazon EMR 7.1.0 improves the resiliency of a node under low disk space conditions by improving log truncation logic for files with open file handles. + This release enhances the encoding and decoding logic to minimize risk of data corruption and node failure with Amazon EMR daemons read and write files when restarting a node. + When you launch a cluster with *the latest patch release* of Amazon EMR 5.36 or higher, 6.6 or higher, or 7.0 or higher, Amazon EMR uses the latest Amazon Linux 2023 or Amazon Linux 2 release for the default Amazon EMR AMI. For more information, see [Using the default Amazon Linux AMI for Amazon EMR](https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-default-ami.html). [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-710-release.html) ## 7.1.0 default Java versions Amazon EMR release 7.1 and higher ships with Amazon Corretto 17 (built on OpenJDK) by default for applications that support Corretto 17 (JDK 17), with the exception of Apache Livy. The following table shows the default Java versions for applications in Amazon EMR 7.1.0. If you want to change the default JVM on your cluster, follow the instructions in [Configure applications to use a specific Java Virtual Machine](configuring-java8.md) for each application that runs on the cluster. You can only use one Java runtime version for a cluster. Amazon EMR doesn't support running different nodes or applications on different runtime versions on the same cluster. | Application | Java / Amazon Corretto version (default is bold) | | --- | --- | | Delta | 17, 11, 8 | | Flink | 17, 11, 8 | | HBase | 11, 8 | | HCatalog | 17, 11, 8 | | Hadoop | 17, 11, 8 | | Hive | 17, 11, 8 | | Hudi | 17, 11, 8 | | Iceberg | 17, 11, 8 | | Livy | 17, 11, 8 | | Oozie | 17, 11, 8 | | Phoenix | 8 | | PrestoDB | 8 | | Spark | 17, 11, 8 | | Spark RAPIDS | 17, 11, 8 | | Sqoop | 8 | | Tez | 17, 11, 8 | | Trino | 17 | | Zeppelin | 8 | | Pig | 8 | | Zookeeper | 8 | ## 7.1.0 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | adot-java-agent | 1.31.0 | A Java Agent that collects metrics from application daemons. | | delta | 3.0.0 | Delta lake is an open table format for huge analytic datasets | | emr-amazon-cloudwatch-agent | 1.300032.2-amzn-0 | An application that collects internal system-level metrics and custom application metrics from Amazon EC2 instances. | | emr-ddb | 5.3.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 3.10.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.14.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-notebook-env | 1.7.0 | Conda env for emr notebook which includes jupyter enterprise gateway | | emr-s3-dist-cp | 2.31.0 | Distributed copy application optimized for Amazon S3. | | emr-s3-select | 2.10.0 | EMR S3Select Connector | | emr-wal-cli | 1.2.2 | Cli used for emrwal list/deletion. | | emrfs | 2.62.0 | Amazon S3 connector for Hadoop ecosystem applications. | | flink-client | 1.18.1-amzn-0 | Apache Flink command line client scripts and applications. | | flink-jobmanager-config | 1.18.1-amzn-0 | Managing resources on EMR nodes for Apache Flink JobManager. | | hadoop-client | 3.3.6-amzn-3 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 3.3.6-amzn-3 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 3.3.6-amzn-3 | HDFS command-line client and library | | hadoop-hdfs-namenode | 3.3.6-amzn-3 | HDFS service for tracking file names and block locations. | | hadoop-hdfs-journalnode | 3.3.6-amzn-3 | HDFS service for managing the Hadoop filesystem journal on HA clusters. | | hadoop-httpfs-server | 3.3.6-amzn-3 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 3.3.6-amzn-3 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 3.3.6-amzn-3 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 3.3.6-amzn-3 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 3.3.6-amzn-3 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 3.3.6-amzn-3 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 2.4.17-amzn-5 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 2.4.17-amzn-5 | Service for serving one or more HBase regions. | | hbase-client | 2.4.17-amzn-5 | HBase command-line client. | | hbase-rest-server | 2.4.17-amzn-5 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 2.4.17-amzn-5 | Service providing a Thrift endpoint to HBase. | | hbase-operator-tools | 2.4.17-amzn-5 | Repair tool for Apache HBase clusters. | | hcatalog-client | 3.1.3-amzn-10 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 3.1.3-amzn-10 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 3.1.3-amzn-10 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 3.1.3-amzn-10 | Hive command line client. | | hive-hbase | 3.1.3-amzn-10 | Hive-hbase client. | | hive-metastore-server | 3.1.3-amzn-10 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server2 | 3.1.3-amzn-10 | Service for accepting Hive queries as web requests. | | hudi | 0.14.1-amzn-0 | Incremental processing framework to power data pipeline at low latency and high efficiency. | | hudi-presto | 0.14.1-amzn-0 | Bundle library for running Presto with Hudi. | | hudi-trino | 0.14.1-amzn-0 | Bundle library for running Trino with Hudi. | | hudi-spark | 0.14.1-amzn-0 | Bundle library for running Spark with Hudi. | | hue-server | 4.11.0 | Web application for analyzing data using Hadoop ecosystem applications | | iceberg | 1.4.3-amzn-0 | Apache Iceberg is an open table format for huge analytic datasets | | jupyterhub | 1.5.0 | Multi-user server for Jupyter notebooks | | livy-server | 0.8.0-incubating | REST interface for interacting with Apache Spark | | nginx | 1.12.1 | nginx [engine x] is an HTTP and reverse proxy server | | mxnet | 1.9.1 | A flexible, scalable, and efficient library for deep learning. | | mariadb-server | 5.5.68\$1 | MariaDB database server. | | nvidia-cuda | 11.8.0 | Nvidia drivers and Cuda toolkit | | oozie-client | 5.2.1 | Oozie command-line client. | | oozie-server | 5.2.1 | Service for accepting Oozie workflow requests. | | opencv | 4.7.0 | Open Source Computer Vision Library. | | phoenix-library | 5.1.3 | The phoenix libraries for server and client | | phoenix-connectors | 5.1.3 | Apache Phoenix-Connectors for Spark-3 | | phoenix-query-server | 6.0.0 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.284-amzn-0 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.284-amzn-0 | Service for executing pieces of a query. | | presto-client | 0.284-amzn-0 | Presto command-line client which is installed on an HA cluster's stand-by masters where Presto server is not started. | | trino-coordinator | 435-amzn-0 | Service for accepting queries and managing query execution among trino-workers. | | trino-worker | 435-amzn-0 | Service for executing pieces of a query. | | trino-client | 435-amzn-0 | Trino command-line client which is installed on an HA cluster's stand-by masters where Trino server is not started. | | pig-client | 0.17.0 | Pig command-line client. | | r | 4.3.2 | The R Project for Statistical Computing | | ranger-kms-server | 2.0.0 | Apache Ranger Key Management System | | spark-client | 3.5.0-amzn-1 | Spark command-line clients. | | spark-history-server | 3.5.0-amzn-1 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 3.5.0-amzn-1 | In-memory execution engine for YARN. | | spark-yarn-slave | 3.5.0-amzn-1 | Apache Spark libraries needed by YARN slaves. | | spark-rapids | 23.10.0-amzn-1 | Nvidia Spark RAPIDS plugin that accelerates Apache Spark with GPUs. | | sqoop-client | 1.4.7 | Apache Sqoop command-line client. | | tensorflow | 2.11.0 | TensorFlow open source software library for high performance numerical computation. | | tez-on-yarn | 0.10.2-amzn-8 | The tez YARN application and libraries. | | tez-on-worker | 0.10.2-amzn-8 | The tez YARN application and libraries for worker nodes. | | zeppelin-server | 0.10.1 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.9.1-amzn-0 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.9.1-amzn-0 | ZooKeeper command line client. | ## 7.1.0 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). Reconfiguration actions occur when you specify a configuration for instance groups in a running cluster. Amazon EMR only initiates reconfiguration actions for the classifications that you modify. For more information, see [Reconfigure an instance group in a running cluster](emr-configure-apps-running-cluster.md). **emr-7.1.0 classifications** | Classifications | Description | Reconfiguration Actions | | --- | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | Restarts the ResourceManager service. | | container-executor | Change values in Hadoop YARN's container-executor.cfg file. | Not available. | | container-log4j | Change values in Hadoop YARN's container-log4j.properties file. | Not available. | | core-site | Change values in Hadoop's core-site.xml file. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts Hadoop KMS, Ranger KMS, HiveServer2, Hive MetaStore, Hadoop Httpfs, and MapReduce-HistoryServer. | | docker-conf | Change docker related settings. | Not available. | | emrfs-site | Change EMRFS settings. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts HBaseRegionserver, HBaseMaster, HBaseThrift, HBaseRest, HiveServer2, Hive MetaStore, Hadoop Httpfs, and MapReduce-HistoryServer. | | flink-conf | Change flink-conf.yaml settings. | Restarts Flink history server. | | flink-log4j | Change Flink log4j.properties settings. | Restarts Flink history server. | | flink-log4j-session | Change Flink log4j-session.properties settings for Kubernetes/Yarn session. | Restarts Flink history server. | | flink-log4j-cli | Change Flink log4j-cli.properties settings. | Restarts Flink history server. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts PhoenixQueryserver, HiveServer2, Hive MetaStore, and MapReduce-HistoryServer. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | Restarts the Hadoop HDFS services SecondaryNamenode, Datanode, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts Hadoop KMS, Hadoop Httpfs, and MapReduce-HistoryServer. | | hadoop-ssl-server | Change hadoop ssl server configuration | Not available. | | hadoop-ssl-client | Change hadoop ssl client configuration | Not available. | | hbase | Amazon EMR-curated settings for Apache HBase. | Custom EMR specific property. Sets emrfs-site and hbase-site configs. See those for their associated restarts. | | hbase-env | Change values in HBase's environment. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbase.properties file. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | Not available. | | hbase-site | Change values in HBase's hbase-site.xml file. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. Additionally restarts Phoenix QueryServer. | | hdfs-encryption-zones | Configure HDFS encryption zones. | This classification should not be reconfigured. | | hdfs-env | Change values in the HDFS environment. | Restarts Hadoop HDFS services Namenode, Datanode, and ZKFC. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Additionally restarts Hadoop Httpfs. | | hcatalog-env | Change values in HCatalog's environment. | Restarts Hive HCatalog Server. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | Restarts Hive HCatalog Server. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | Restarts Hive HCatalog Server. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | Restarts Hive WebHCat server. | | hcatalog-webhcat-log4j2 | Change values in HCatalog WebHCat's log4j2.properties. | Restarts Hive WebHCat server. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | Restarts Hive WebHCat server. | | hive | Amazon EMR-curated settings for Apache Hive. | Sets configurations to launch Hive LLAP service. | | hive-beeline-log4j2 | Change values in Hive's beeline-log4j2.properties file. | Not available. | | hive-parquet-logging | Change values in Hive's parquet-logging.properties file. | Not available. | | hive-env | Change values in the Hive environment. | Restarts HiveServer2, HiveMetastore, and Hive HCatalog-Server. Runs Hive schemaTool CLI commands to verify hive-metastore. | | hive-exec-log4j2 | Change values in Hive's hive-exec-log4j2.properties file. | Not available. | | hive-llap-daemon-log4j2 | Change values in Hive's llap-daemon-log4j2.properties file. | Not available. | | hive-log4j2 | Change values in Hive's hive-log4j2.properties file. | Not available. | | hive-site | Change values in Hive's hive-site.xml file | Restarts HiveServer2, HiveMetastore, and Hive HCatalog-Server. Runs Hive schemaTool CLI commands to verify hive-metastore. Also restarts Oozie and Zeppelin. | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | Not available. | | hue-ini | Change values in Hue's ini file | Restarts Hue. Also activates Hue config override CLI commands to pick up new configurations. | | httpfs-env | Change values in the HTTPFS environment. | Restarts Hadoop Httpfs service. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | Restarts Hadoop Httpfs service. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | Not available. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | Restarts Hadoop-KMS service. | | hadoop-kms-java-home | Change Hadoop's KMS java home | Not available. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | Not available. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | Restarts Hadoop-KMS and Ranger-KMS service. | | hudi-env | Change values in the Hudi environment. | Not available. | | hudi-defaults | Change values in Hudi's hudi-defaults.conf file. | Not available. | | iceberg-defaults | Change values in Iceberg's iceberg-defaults.conf file. | Not available. | | delta-defaults | Change values in Delta's delta-defaults.conf file. | Not available. | | jupyter-notebook-conf | Change values in Jupyter Notebook's jupyter\$1notebook\$1config.py file. | Not available. | | jupyter-hub-conf | Change values in JupyterHubs's jupyterhub\$1config.py file. | Not available. | | jupyter-s3-conf | Configure Jupyter Notebook S3 persistence. | Not available. | | jupyter-sparkmagic-conf | Change values in Sparkmagic's config.json file. | Not available. | | livy-conf | Change values in Livy's livy.conf file. | Restarts Livy Server. | | livy-env | Change values in the Livy environment. | Restarts Livy Server. | | livy-log4j2 | Change Livy log4j2.properties settings. | Restarts Livy Server. | | mapred-env | Change values in the MapReduce application's environment. | Restarts Hadoop MapReduce-HistoryServer. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | Restarts Hadoop MapReduce-HistoryServer. | | oozie-env | Change values in Oozie's environment. | Restarts Oozie. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | Restarts Oozie. | | oozie-site | Change values in Oozie's oozie-site.xml file. | Restarts Oozie. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | Not available. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | Not available. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | Restarts Phoenix-QueryServer. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | Not available. | | pig-env | Change values in the Pig environment. | Not available. | | pig-properties | Change values in Pig's pig.properties file. | Restarts Oozie. | | pig-log4j | Change values in Pig's log4j.properties file. | Not available. | | presto-log | Change values in Presto's log.properties file. | Restarts Presto-Server (for PrestoDB) | | presto-config | Change values in Presto's config.properties file. | Restarts Presto-Server (for PrestoDB) | | presto-password-authenticator | Change values in Presto's password-authenticator.properties file. | Not available. | | presto-env | Change values in Presto's presto-env.sh file. | Restarts Presto-Server (for PrestoDB) | | presto-node | Change values in Presto's node.properties file. | Not available. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | Not available. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | Not available. | | presto-connector-hive | Change values in Presto's hive.properties file. | Restarts Presto-Server (for PrestoDB) | | presto-connector-jmx | Change values in Presto's jmx.properties file. | Not available. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | Not available. | | presto-connector-lakeformation | Change values in Presto's lakeformation.properties file. | Restarts Presto-Server (for PrestoDB) | | presto-connector-localfile | Change values in Presto's localfile.properties file. | Not available. | | presto-connector-memory | Change values in Presto's memory.properties file. | Not available. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | Not available. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | Not available. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | Not available. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | Not available. | | presto-connector-redis | Change values in Presto's redis.properties file. | Not available. | | presto-connector-redshift | Change values in Presto's redshift.properties file. | Not available. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | Not available. | | presto-connector-tpcds | Change values in Presto's tpcds.properties file. | Not available. | | trino-log | Change values in Trino's log.properties file. | Restarts Trino-Server (for Trino) | | trino-config | Change values in Trino's config.properties file. | Restarts Trino-Server (for Trino) | | trino-password-authenticator | Change values in Trino's password-authenticator.properties file. | Restarts Trino-Server (for Trino) | | trino-env | Change values in Trino's trino-env.sh file. | Restarts Trino-Server (for Trino) | | trino-node | Change values in Trino's node.properties file. | Not available. | | trino-connector-blackhole | Change values in Trino's blackhole.properties file. | Not available. | | trino-connector-cassandra | Change values in Trino's cassandra.properties file. | Not available. | | trino-connector-delta | Change values in Trino's delta.properties file. | Restarts Trino-Server (for Trino) | | trino-connector-hive | Change values in Trino's hive.properties file. | Restarts Trino-Server (for Trino) | | trino-exchange-manager | Change values in Trino's exchange-manager.properties file. | Restarts Trino-Server (for Trino) | | trino-connector-iceberg | Change values in Trino's iceberg.properties file. | Restarts Trino-Server (for Trino) | | trino-connector-hudi | Change values in Trino's hudi.properties file. | Restarts Trino-Server (for Trino) | | trino-connector-jmx | Change values in Trino's jmx.properties file. | Not available. | | trino-connector-kafka | Change values in Trino's kafka.properties file. | Not available. | | trino-connector-localfile | Change values in Trino's localfile.properties file. | Not available. | | trino-connector-memory | Change values in Trino's memory.properties file. | Not available. | | trino-connector-mongodb | Change values in Trino's mongodb.properties file. | Not available. | | trino-connector-mysql | Change values in Trino's mysql.properties file. | Not available. | | trino-connector-postgresql | Change values in Trino's postgresql.properties file. | Not available. | | trino-connector-raptor | Change values in Trino's raptor.properties file. | Not available. | | trino-connector-redis | Change values in Trino's redis.properties file. | Not available. | | trino-connector-redshift | Change values in Trino's redshift.properties file. | Not available. | | trino-connector-tpch | Change values in Trino's tpch.properties file. | Not available. | | trino-connector-tpcds | Change values in Trino's tpcds.properties file. | Not available. | | ranger-kms-dbks-site | Change values in dbks-site.xml file of Ranger KMS. | Restarts Ranger KMS Server. | | ranger-kms-site | Change values in ranger-kms-site.xml file of Ranger KMS. | Restarts Ranger KMS Server. | | ranger-kms-env | Change values in the Ranger KMS environment. | Restarts Ranger KMS Server. | | ranger-kms-logback | Change values in kms-logback.xml file of Ranger KMS. | Not available. | | ranger-kms-db-ca | Change values for CA file on S3 for MySQL SSL connection with Ranger KMS. | Not available. | | spark | Amazon EMR-curated settings for Apache Spark. | This property modifies spark-defaults. See actions there. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | Restarts Spark history server and Spark thrift server. | | spark-env | Change values in the Spark environment. | Restarts Spark history server and Spark thrift server. | | spark-hive-site | Change values in Spark's hive-site.xml file | Not available. | | spark-log4j2 | Change values in Spark's log4j2.properties file. | Restarts Spark history server and Spark thrift server. | | spark-metrics | Change values in Spark's metrics.properties file. | Restarts Spark history server and Spark thrift server. | | sqoop-env | Change values in Sqoop's environment. | Not available. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | Not available. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | Not available. | | tez-site | Change values in Tez's tez-site.xml file. | Restart Oozie and HiveServer2. | | yarn-env | Change values in the YARN environment. | Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts MapReduce-HistoryServer. | | yarn-site | Change values in YARN's yarn-site.xml file. | Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts Livy Server and MapReduce-HistoryServer. | | zeppelin-env | Change values in the Zeppelin environment. | Restarts Zeppelin. | | zeppelin-site | Change configuration settings in zeppelin-site.xml. | Restarts Zeppelin. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | Restarts Zookeeper server. | | zookeeper-logback | Change values in ZooKeeper's logback.xml file. | Restarts Zookeeper server. | | emr-metrics | Change emr metric settings for this node. | Restarts the CloudWatchAgent service. | ## 7.1.0 change log **Change log for 7.1.0 release and release notes** | Date | Event | Description | | --- | --- | --- | | 2025-09-03 | Docs revision | Amazon EMR 7.1.0 release notes added known issue | | 2024-05-10 | Docs publication | Amazon EMR 7.1.0 release notes first published | | 2024-05-10 | Docs publication | Amazon EMR 7.1.0 release notes first published | | 2024-05-08 | Deployment complete | Amazon EMR 7.1.0 fully deployed to all [supported Regions](https://aws.amazon.com/about-aws/global-infrastructure/regional-product-services/) | | 2024-04-23 | Initial release | Amazon EMR 7.1.0 first deployed to initial commercial Regions | # Amazon EMR release 7.0.0 ## 7.0.0 application versions This release includes the following applications: [https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/Install-CloudWatch-Agent.html](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/Install-CloudWatch-Agent.html), [https://delta.io/](https://delta.io/), [https://flink.apache.org/](https://flink.apache.org/), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [https://hudi.apache.org](https://hudi.apache.org), [http://gethue.com/](http://gethue.com/), [https://iceberg.apache.org/](https://iceberg.apache.org/), [https://jupyter-enterprise-gateway.readthedocs.io/en/latest/](https://jupyter-enterprise-gateway.readthedocs.io/en/latest/), [https://jupyterhub.readthedocs.io/en/latest/#](https://jupyterhub.readthedocs.io/en/latest/#), [https://livy.incubator.apache.org/](https://livy.incubator.apache.org/), [https://mxnet.incubator.apache.org/](https://mxnet.incubator.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://www.tensorflow.org/](https://www.tensorflow.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://trino.io/](https://trino.io/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-7.3.0 | emr-7.2.0 | emr-7.1.0 | emr-7.0.0 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 2.25.70, 1.12.747 | 2.23.18, 1.12.705 | 2.23.18, 1.12.656 | 2.20.160-amzn-0, 1.12.569 | | Python | 3.9, 3.11 | 3.9, 3.11 | 3.9, 3.11 | 3.9 | | Scala | 2.12.18 | 2.12.17 | 2.12.17 | 2.12.17 | | AmazonCloudWatchAgent | 1.300032.2-amzn-0 | 1.300032.2-amzn-0 | 1.300032.2-amzn-0 | 1.300031.1-amzn-0 | | Delta | 3.2.0-amzn-0 | 3.1.0 | 3.0.0 | 3.0.0 | | Flink | 1.18.1-amzn-2 | 1.18.1-amzn-1 | 1.18.1-amzn-0 | 1.18.0-amzn-0 | | Ganglia | - | - | - | - | | HBase | 2.4.17-amzn-7 | 2.4.17-amzn-6 | 2.4.17-amzn-5 | 2.4.17-amzn-4 | | HCatalog | 3.1.3-amzn-12 | 3.1.3-amzn-11 | 3.1.3-amzn-10 | 3.1.3-amzn-9 | | Hadoop | 3.3.6-amzn-5 | 3.3.6-amzn-4 | 3.3.6-amzn-3 | 3.3.6-amzn-2 | | Hive | 3.1.3-amzn-12 | 3.1.3-amzn-11 | 3.1.3-amzn-10 | 3.1.3-amzn-9 | | Hudi | 0.15.0-amzn-0 | 0.14.1-amzn-1 | 0.14.1-amzn-0 | 0.14.0-amzn-1 | | Hue | 4.11.0 | 4.11.0 | 4.11.0 | 4.11.0 | | Iceberg | 1.5.2-amzn-0 | 1.5.0-amzn-0 | 1.4.3-amzn-0 | 1.4.2-amzn-0 | | JupyterEnterpriseGateway | 2.6.0 | 2.6.0 | 2.6.0 | 2.6.0 | | JupyterHub | 1.5.0 | 1.5.0 | 1.5.0 | 1.5.0 | | Livy | 0.8.0-incubating | 0.8.0-incubating | 0.8.0-incubating | 0.7.1-incubating | | MXNet | - | 1.9.1 | 1.9.1 | 1.9.1 | | Mahout | - | - | - | - | | Oozie | 5.2.1 | 5.2.1 | 5.2.1 | 5.2.1 | | Phoenix | 5.1.3 | 5.1.3 | 5.1.3 | 5.1.3 | | Pig | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | | Presto | 0.285-amzn-1 | 0.285-amzn-0 | 0.284-amzn-0 | 0.283-amzn-1 | | Spark | 3.5.1-amzn-1 | 3.5.1-amzn-0 | 3.5.0-amzn-1 | 3.5.0-amzn-0 | | Sqoop | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | | TensorFlow | 2.16.1 | 2.11.0 | 2.11.0 | 2.11.0 | | Tez | 0.10.2-amzn-10 | 0.10.2-amzn-9 | 0.10.2-amzn-8 | 0.10.2-amzn-7 | | Trino (PrestoSQL) | 442-amzn-0 | 436-amzn-0 | 435-amzn-0 | 426-amzn-1 | | Zeppelin | 0.11.1 | 0.10.1 | 0.10.1 | 0.10.1 | | ZooKeeper | 3.9.1-amzn-1 | 3.9.1-amzn-1 | 3.9.1-amzn-0 | 3.5.10 | ## 7.0.0 release notes The following release notes include information for Amazon EMR release 7.0.0. Changes are relative to 6.15.0. **New features** + **Application upgrades** – Amazon EMR 7.0.0 application upgrades include Python 3.9, Spark 3.5, Flink 1.18, and Delta 3.0. This release also adds support for the [Amazon CloudWatch agent](emr-AmazonCloudWatchAgent.md) application and removes support for Ganglia. + **Amazon Corretto 17** – Amazon EMR releases 7.0 and higher ship with Amazon Corretto 17 (built on OpenJDK) by default for applications that support Corretto 17 (JDK 17), with the exception of Apache Livy. For more information about the supported JDK versions for applications in this release, see [7.0.0 default Java versions](#emr-700-jdk). + **Amazon Linux 2023** – With the 7.0 release, Amazon EMR clusters now run on AL2023 by default. For information on how this affects the default AMI version, see [Software update considerations](https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-default-ami.html#emr-default-ami-managing) in the *Amazon EMR Management Guide*. Also note that AL2023 removed Python 2.7, so any components that require Python should now be written with Python 3. + **S3 on Outposts with `s3a`** – Amazon EMR now supports Amazon S3 on Outposts buckets with the `s3a` file system. For more information about S3 on Outposts, see [What is S3 on Outposts?](https://docs.aws.amazon.com/AmazonS3/latest/userguide/S3onOutposts.html) in the *Amazon Simple Storage Service User Guide*. **Known issues** + Note that you can't run more than one line at a time if you are using PySpark with Python versions 3.10 or later. You must run each line one at a time. + An on-cluster instance-state script that monitors health of the instance can consume excessive CPU and memory resources when there are a large number of threads and/or open file handles on the node. **Changes, enhancements, and resolved issues** + Starting with Spark 3.3.1 (supported in EMR versions 6.10 and above), all executors in a decommissioning host are set to a new `ExecutorState`, called *DECOMMISSIONING* state. The executors being decommissioned cannot be used by Yarn to allocate tasks and thus it will request for new executors, if needed, for the tasks being executed. Thus, if you disable Spark DRA while using EMR Managed Scaling, EMR Auto Scaling, or any custom scaling mechanism on EMR-EC2 clusters, then Yarn may request maximum permissible executors for each job. In order to avoid this issue, leave the `spark.dynamicAllocation.enabled` property set to `TRUE` (which is the default) when you are using the above combination of features. In addition, you can also set minimum and maximum executor constraints by setting values for `spark.dynamicAllocation.maxExecutors` and `spark.dynamicAllocation.minExecutors` properties for your Spark jobs, to restrict the number of executors allocated during the job’s execution. + To maintain the state of all the instances in a cluster, Amazon EMR integrates with Apache YARN, Apache HDFS, and Kerberos. With 7.0, we enhanced these integrations to reliably remove the state of instances that are terminated due to scaling or other operations. This is particularly important for long-lived clusters that have managed scaling enabled, as they scale frequently and accumulate thousands of terminated instances over their lifetimes. + This release improves the Kerberos configuration to only include support for AES-based ciphers. Kerberos KDC with non-AES based ciphers are no longer supported with EMR clusters that run on Amazon EMR releases 7.0.0 and higher. AES-based ciphers offer the strongest security for your clusters. + As part of the AWS SDK 2.x migration, Amazon EMR 7.0 includes an update to the Spark Kinesis connector for compatibility. This update isn't available in the community version of Apache Spark. If you use the Spark Kinesis connector from an Amazon EMR release lower than 7.0, you must migrate your application codes to run on SDK 2.x before you can migrate your workloads to Amazon EMR 7.0. For more information, see [Migrating Spark Kinesis connector to SDK 2.x for Amazon EMR 7.0](migrating-spark-kinesis.md). + The `aws-sagemaker-spark-sdk` component is not available from Amazon EMR 7.x onwards. + When you launch a cluster with *the latest patch release* of Amazon EMR 5.36 or higher, 6.6 or higher, or 7.0 or higher, Amazon EMR uses the latest Amazon Linux 2023 or Amazon Linux 2 release for the default Amazon EMR AMI. For more information, see [Using the default Amazon Linux AMI for Amazon EMR](https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-default-ami.html). [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-700-release.html) ## 7.0.0 default Java versions Amazon EMR release 7.0 and higher ships with Amazon Corretto 17 (built on OpenJDK) by default for applications that support Corretto 17 (JDK 17), with the exception of Apache Livy. The following table shows the default Java versions for applications in Amazon EMR 7.0.0. If you want to change the default JVM on your cluster, follow the instructions in [Configure applications to use a specific Java Virtual Machine](configuring-java8.md) for each application that runs on the cluster. You can only use one Java runtime version for a cluster. Amazon EMR doesn't support running different nodes or applications on different runtime versions on the same cluster. | Application | Java / Amazon Corretto version (default is bold) | | --- | --- | | Delta | 17, 11, 8 | | Flink | 11, 8 | | HBase | 11, 8 | | HCatalog | 17, 11, 8 | | Hadoop | 17, 11, 8 | | Hive | 17, 11, 8 | | Hudi | 17, 11, 8 | | Iceberg | 17, 11, 8 | | Livy | 17, 11, 8 | | Oozie | 17, 11, 8 | | Phoenix | 8 | | PrestoDB | 8 | | Spark | 17, 11, 8 | | Spark RAPIDS | 17, 11, 8 | | Sqoop | 8 | | Tez | 17, 11, 8 | | Trino | 17 | | Zeppelin | 8 | | Pig | 8 | | Zookeeper | 8 | ## 7.0.0 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | adot-java-agent | 1.31.0 | A Java Agent that collects metrics from application daemons. | | delta | 3.0.0 | Delta lake is an open table format for huge analytic datasets | | delta-standalone-connectors | 0.6.0 | Delta Connectors provide different runtimes to integrate Delta Lake with engines like Flink, Hive and Presto. | | emr-amazon-cloudwatch-agent | 1.300031.1-amzn-0 | An application that collects internal system-level metrics and custom application metrics from Amazon EC2 instances. | | emr-ddb | 5.2.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 3.9.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.13.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-notebook-env | 1.7.0 | Conda env for emr notebook which includes jupyter enterprise gateway | | emr-s3-dist-cp | 2.30.0 | Distributed copy application optimized for Amazon S3. | | emr-s3-select | 2.9.0 | EMR S3Select Connector | | emr-wal-cli | 1.2.1 | Cli used for emrwal list/deletion. | | emrfs | 2.61.0 | Amazon S3 connector for Hadoop ecosystem applications. | | flink-client | 1.18.0-amzn-0 | Apache Flink command line client scripts and applications. | | flink-jobmanager-config | 1.18.0-amzn-0 | Managing resources on EMR nodes for Apache Flink JobManager. | | hadoop-client | 3.3.6-amzn-2 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 3.3.6-amzn-2 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 3.3.6-amzn-2 | HDFS command-line client and library | | hadoop-hdfs-namenode | 3.3.6-amzn-2 | HDFS service for tracking file names and block locations. | | hadoop-hdfs-journalnode | 3.3.6-amzn-2 | HDFS service for managing the Hadoop filesystem journal on HA clusters. | | hadoop-httpfs-server | 3.3.6-amzn-2 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 3.3.6-amzn-2 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 3.3.6-amzn-2 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 3.3.6-amzn-2 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 3.3.6-amzn-2 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 3.3.6-amzn-2 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 2.4.17-amzn-4 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 2.4.17-amzn-4 | Service for serving one or more HBase regions. | | hbase-client | 2.4.17-amzn-4 | HBase command-line client. | | hbase-rest-server | 2.4.17-amzn-4 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 2.4.17-amzn-4 | Service providing a Thrift endpoint to HBase. | | hbase-operator-tools | 2.4.17-amzn-4 | Repair tool for Apache HBase clusters. | | hcatalog-client | 3.1.3-amzn-9 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 3.1.3-amzn-9 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 3.1.3-amzn-9 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 3.1.3-amzn-9 | Hive command line client. | | hive-hbase | 3.1.3-amzn-9 | Hive-hbase client. | | hive-metastore-server | 3.1.3-amzn-9 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server2 | 3.1.3-amzn-9 | Service for accepting Hive queries as web requests. | | hudi | 0.14.0-amzn-1 | Incremental processing framework to power data pipeline at low latency and high efficiency. | | hudi-presto | 0.14.0-amzn-1 | Bundle library for running Presto with Hudi. | | hudi-trino | 0.14.0-amzn-1 | Bundle library for running Trino with Hudi. | | hudi-spark | 0.14.0-amzn-1 | Bundle library for running Spark with Hudi. | | hue-server | 4.11.0 | Web application for analyzing data using Hadoop ecosystem applications | | iceberg | 1.4.2-amzn-0 | Apache Iceberg is an open table format for huge analytic datasets | | jupyterhub | 1.5.0 | Multi-user server for Jupyter notebooks | | livy-server | 0.7.1-incubating | REST interface for interacting with Apache Spark | | nginx | 1.12.1 | nginx [engine x] is an HTTP and reverse proxy server | | mxnet | 1.9.1 | A flexible, scalable, and efficient library for deep learning. | | mariadb-server | 5.5.68\$1 | MariaDB database server. | | nvidia-cuda | 11.8.0 | Nvidia drivers and Cuda toolkit | | oozie-client | 5.2.1 | Oozie command-line client. | | oozie-server | 5.2.1 | Service for accepting Oozie workflow requests. | | opencv | 4.7.0 | Open Source Computer Vision Library. | | phoenix-library | 5.1.3 | The phoenix libraries for server and client | | phoenix-connectors | 5.1.3 | Apache Phoenix-Connectors for Spark-3 | | phoenix-query-server | 6.0.0 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.283-amzn-1 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.283-amzn-1 | Service for executing pieces of a query. | | presto-client | 0.283-amzn-1 | Presto command-line client which is installed on an HA cluster's stand-by masters where Presto server is not started. | | trino-coordinator | 426-amzn-1 | Service for accepting queries and managing query execution among trino-workers. | | trino-worker | 426-amzn-1 | Service for executing pieces of a query. | | trino-client | 426-amzn-1 | Trino command-line client which is installed on an HA cluster's stand-by masters where Trino server is not started. | | pig-client | 0.17.0 | Pig command-line client. | | r | 4.3.2 | The R Project for Statistical Computing | | ranger-kms-server | 2.0.0 | Apache Ranger Key Management System | | spark-client | 3.5.0-amzn-0 | Spark command-line clients. | | spark-history-server | 3.5.0-amzn-0 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 3.5.0-amzn-0 | In-memory execution engine for YARN. | | spark-yarn-slave | 3.5.0-amzn-0 | Apache Spark libraries needed by YARN slaves. | | spark-rapids | 23.10.0-amzn-0 | Nvidia Spark RAPIDS plugin that accelerates Apache Spark with GPUs. | | sqoop-client | 1.4.7 | Apache Sqoop command-line client. | | tensorflow | 2.11.0 | TensorFlow open source software library for high performance numerical computation. | | tez-on-yarn | 0.10.2-amzn-7 | The tez YARN application and libraries. | | tez-on-worker | 0.10.2-amzn-7 | The tez YARN application and libraries for worker nodes. | | webserver | 2.4.58 | Apache HTTP server. | | zeppelin-server | 0.10.1 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.5.10 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.5.10 | ZooKeeper command line client. | ## 7.0.0 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). Reconfiguration actions occur when you specify a configuration for instance groups in a running cluster. Amazon EMR only initiates reconfiguration actions for the classifications that you modify. For more information, see [Reconfigure an instance group in a running cluster](emr-configure-apps-running-cluster.md). **emr-7.0.0 classifications** | Classifications | Description | Reconfiguration Actions | | --- | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | Restarts the ResourceManager service. | | container-executor | Change values in Hadoop YARN's container-executor.cfg file. | Not available. | | container-log4j | Change values in Hadoop YARN's container-log4j.properties file. | Not available. | | core-site | Change values in Hadoop's core-site.xml file. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts Hadoop KMS, Ranger KMS, HiveServer2, Hive MetaStore, Hadoop Httpfs, and MapReduce-HistoryServer. | | docker-conf | Change docker related settings. | Not available. | | emrfs-site | Change EMRFS settings. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts HBaseRegionserver, HBaseMaster, HBaseThrift, HBaseRest, HiveServer2, Hive MetaStore, Hadoop Httpfs, and MapReduce-HistoryServer. | | flink-conf | Change flink-conf.yaml settings. | Restarts Flink history server. | | flink-log4j | Change Flink log4j.properties settings. | Restarts Flink history server. | | flink-log4j-session | Change Flink log4j-session.properties settings for Kubernetes/Yarn session. | Restarts Flink history server. | | flink-log4j-cli | Change Flink log4j-cli.properties settings. | Restarts Flink history server. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts PhoenixQueryserver, HiveServer2, Hive MetaStore, and MapReduce-HistoryServer. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | Restarts the Hadoop HDFS services SecondaryNamenode, Datanode, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts Hadoop KMS, Hadoop Httpfs, and MapReduce-HistoryServer. | | hadoop-ssl-server | Change hadoop ssl server configuration | Not available. | | hadoop-ssl-client | Change hadoop ssl client configuration | Not available. | | hbase | Amazon EMR-curated settings for Apache HBase. | Custom EMR specific property. Sets emrfs-site and hbase-site configs. See those for their associated restarts. | | hbase-env | Change values in HBase's environment. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbase.properties file. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | Not available. | | hbase-site | Change values in HBase's hbase-site.xml file. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. Additionally restarts Phoenix QueryServer. | | hdfs-encryption-zones | Configure HDFS encryption zones. | This classification should not be reconfigured. | | hdfs-env | Change values in the HDFS environment. | Restarts Hadoop HDFS services Namenode, Datanode, and ZKFC. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Additionally restarts Hadoop Httpfs. | | hcatalog-env | Change values in HCatalog's environment. | Restarts Hive HCatalog Server. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | Restarts Hive HCatalog Server. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | Restarts Hive HCatalog Server. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | Restarts Hive WebHCat server. | | hcatalog-webhcat-log4j2 | Change values in HCatalog WebHCat's log4j2.properties. | Restarts Hive WebHCat server. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | Restarts Hive WebHCat server. | | hive | Amazon EMR-curated settings for Apache Hive. | Sets configurations to launch Hive LLAP service. | | hive-beeline-log4j2 | Change values in Hive's beeline-log4j2.properties file. | Not available. | | hive-parquet-logging | Change values in Hive's parquet-logging.properties file. | Not available. | | hive-env | Change values in the Hive environment. | Restarts HiveServer2, HiveMetastore, and Hive HCatalog-Server. Runs Hive schemaTool CLI commands to verify hive-metastore. | | hive-exec-log4j2 | Change values in Hive's hive-exec-log4j2.properties file. | Not available. | | hive-llap-daemon-log4j2 | Change values in Hive's llap-daemon-log4j2.properties file. | Not available. | | hive-log4j2 | Change values in Hive's hive-log4j2.properties file. | Not available. | | hive-site | Change values in Hive's hive-site.xml file | Restarts HiveServer2, HiveMetastore, and Hive HCatalog-Server. Runs Hive schemaTool CLI commands to verify hive-metastore. Also restarts Oozie and Zeppelin. | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | Not available. | | hue-ini | Change values in Hue's ini file | Restarts Hue. Also activates Hue config override CLI commands to pick up new configurations. | | httpfs-env | Change values in the HTTPFS environment. | Restarts Hadoop Httpfs service. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | Restarts Hadoop Httpfs service. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | Not available. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | Restarts Hadoop-KMS service. | | hadoop-kms-java-home | Change Hadoop's KMS java home | Not available. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | Not available. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | Restarts Hadoop-KMS and Ranger-KMS service. | | hudi-env | Change values in the Hudi environment. | Not available. | | hudi-defaults | Change values in Hudi's hudi-defaults.conf file. | Not available. | | iceberg-defaults | Change values in Iceberg's iceberg-defaults.conf file. | Not available. | | delta-defaults | Change values in Delta's delta-defaults.conf file. | Not available. | | jupyter-notebook-conf | Change values in Jupyter Notebook's jupyter\$1notebook\$1config.py file. | Not available. | | jupyter-hub-conf | Change values in JupyterHubs's jupyterhub\$1config.py file. | Not available. | | jupyter-s3-conf | Configure Jupyter Notebook S3 persistence. | Not available. | | jupyter-sparkmagic-conf | Change values in Sparkmagic's config.json file. | Not available. | | livy-conf | Change values in Livy's livy.conf file. | Restarts Livy Server. | | livy-env | Change values in the Livy environment. | Restarts Livy Server. | | livy-log4j2 | Change Livy log4j2.properties settings. | Restarts Livy Server. | | mapred-env | Change values in the MapReduce application's environment. | Restarts Hadoop MapReduce-HistoryServer. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | Restarts Hadoop MapReduce-HistoryServer. | | oozie-env | Change values in Oozie's environment. | Restarts Oozie. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | Restarts Oozie. | | oozie-site | Change values in Oozie's oozie-site.xml file. | Restarts Oozie. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | Not available. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | Not available. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | Restarts Phoenix-QueryServer. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | Not available. | | pig-env | Change values in the Pig environment. | Not available. | | pig-properties | Change values in Pig's pig.properties file. | Restarts Oozie. | | pig-log4j | Change values in Pig's log4j.properties file. | Not available. | | presto-log | Change values in Presto's log.properties file. | Restarts Presto-Server (for PrestoDB) | | presto-config | Change values in Presto's config.properties file. | Restarts Presto-Server (for PrestoDB) | | presto-password-authenticator | Change values in Presto's password-authenticator.properties file. | Not available. | | presto-env | Change values in Presto's presto-env.sh file. | Restarts Presto-Server (for PrestoDB) | | presto-node | Change values in Presto's node.properties file. | Not available. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | Not available. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | Not available. | | presto-connector-hive | Change values in Presto's hive.properties file. | Restarts Presto-Server (for PrestoDB) | | presto-connector-jmx | Change values in Presto's jmx.properties file. | Not available. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | Not available. | | presto-connector-lakeformation | Change values in Presto's lakeformation.properties file. | Restarts Presto-Server (for PrestoDB) | | presto-connector-localfile | Change values in Presto's localfile.properties file. | Not available. | | presto-connector-memory | Change values in Presto's memory.properties file. | Not available. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | Not available. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | Not available. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | Not available. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | Not available. | | presto-connector-redis | Change values in Presto's redis.properties file. | Not available. | | presto-connector-redshift | Change values in Presto's redshift.properties file. | Not available. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | Not available. | | presto-connector-tpcds | Change values in Presto's tpcds.properties file. | Not available. | | trino-log | Change values in Trino's log.properties file. | Restarts Trino-Server (for Trino) | | trino-config | Change values in Trino's config.properties file. | Restarts Trino-Server (for Trino) | | trino-password-authenticator | Change values in Trino's password-authenticator.properties file. | Restarts Trino-Server (for Trino) | | trino-env | Change values in Trino's trino-env.sh file. | Restarts Trino-Server (for Trino) | | trino-node | Change values in Trino's node.properties file. | Not available. | | trino-connector-blackhole | Change values in Trino's blackhole.properties file. | Not available. | | trino-connector-cassandra | Change values in Trino's cassandra.properties file. | Not available. | | trino-connector-delta | Change values in Trino's delta.properties file. | Restarts Trino-Server (for Trino) | | trino-connector-hive | Change values in Trino's hive.properties file. | Restarts Trino-Server (for Trino) | | trino-exchange-manager | Change values in Trino's exchange-manager.properties file. | Restarts Trino-Server (for Trino) | | trino-connector-iceberg | Change values in Trino's iceberg.properties file. | Restarts Trino-Server (for Trino) | | trino-connector-hudi | Change values in Trino's hudi.properties file. | Restarts Trino-Server (for Trino) | | trino-connector-jmx | Change values in Trino's jmx.properties file. | Not available. | | trino-connector-kafka | Change values in Trino's kafka.properties file. | Not available. | | trino-connector-localfile | Change values in Trino's localfile.properties file. | Not available. | | trino-connector-memory | Change values in Trino's memory.properties file. | Not available. | | trino-connector-mongodb | Change values in Trino's mongodb.properties file. | Not available. | | trino-connector-mysql | Change values in Trino's mysql.properties file. | Not available. | | trino-connector-postgresql | Change values in Trino's postgresql.properties file. | Not available. | | trino-connector-raptor | Change values in Trino's raptor.properties file. | Not available. | | trino-connector-redis | Change values in Trino's redis.properties file. | Not available. | | trino-connector-redshift | Change values in Trino's redshift.properties file. | Not available. | | trino-connector-tpch | Change values in Trino's tpch.properties file. | Not available. | | trino-connector-tpcds | Change values in Trino's tpcds.properties file. | Not available. | | ranger-kms-dbks-site | Change values in dbks-site.xml file of Ranger KMS. | Restarts Ranger KMS Server. | | ranger-kms-site | Change values in ranger-kms-site.xml file of Ranger KMS. | Restarts Ranger KMS Server. | | ranger-kms-env | Change values in the Ranger KMS environment. | Restarts Ranger KMS Server. | | ranger-kms-logback | Change values in kms-logback.xml file of Ranger KMS. | Not available. | | ranger-kms-db-ca | Change values for CA file on S3 for MySQL SSL connection with Ranger KMS. | Not available. | | spark | Amazon EMR-curated settings for Apache Spark. | This property modifies spark-defaults. See actions there. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | Restarts Spark history server and Spark thrift server. | | spark-env | Change values in the Spark environment. | Restarts Spark history server and Spark thrift server. | | spark-hive-site | Change values in Spark's hive-site.xml file | Not available. | | spark-log4j2 | Change values in Spark's log4j2.properties file. | Restarts Spark history server and Spark thrift server. | | spark-metrics | Change values in Spark's metrics.properties file. | Restarts Spark history server and Spark thrift server. | | sqoop-env | Change values in Sqoop's environment. | Not available. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | Not available. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | Not available. | | tez-site | Change values in Tez's tez-site.xml file. | Restart Oozie and HiveServer2. | | yarn-env | Change values in the YARN environment. | Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts MapReduce-HistoryServer. | | yarn-site | Change values in YARN's yarn-site.xml file. | Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts Livy Server and MapReduce-HistoryServer. | | zeppelin-env | Change values in the Zeppelin environment. | Restarts Zeppelin. | | zeppelin-site | Change configuration settings in zeppelin-site.xml. | Restarts Zeppelin. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | Restarts Zookeeper server. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | Restarts Zookeeper server. | ## 7.0.0 change log **Change log for 7.0.0 release and release notes** | Date | Event | Description | | --- | --- | --- | | 2025-09-03 | Docs revision | Amazon EMR 7.0.0 release notes added known issue | | 2023-12-29 | Deployment complete | Amazon EMR 7.0.0 fully deployed to all [supported Regions](https://aws.amazon.com/about-aws/global-infrastructure/regional-product-services/) | | 2023-12-28 | Docs publication | Amazon EMR 7.0.0 release notes first published | | 2023-12-19 | Initial release | Amazon EMR 7.0.0 first deployed to initial commercial Regions | # Amazon EMR 6.x release versions This section contains application versions, release notes, component versions, and configuration classifications available in each Amazon EMR 6.x release version. When you launch a cluster, you can choose from multiple releases of Amazon EMR. This allows you to test and use application versions that fit your compatibility requirements. You specify the release number with the *release label*. Release labels are in the form `emr-x.x.x`. For example, `emr-7.12.0`. New Amazon EMR releases are made available in different Regions over a period of several days, beginning with the first Region on the initial release date. The latest release version may not be available in your Region during this period. For a comprehensive table of application versions in every Amazon EMR 6.x release, see [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md). **Topics** + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Amazon EMR release 6.15.0](emr-6150-release.md) + [Amazon EMR release 6.14.0](emr-6140-release.md) + [Amazon EMR release 6.13.0](emr-6130-release.md) + [Amazon EMR release 6.12.0](emr-6120-release.md) + [Amazon EMR release 6.11.1](emr-6111-release.md) + [Amazon EMR release 6.11.0](emr-6110-release.md) + [Amazon EMR release 6.10.1](emr-6101-release.md) + [Amazon EMR release 6.10.0](emr-6100-release.md) + [Amazon EMR release 6.9.1](emr-691-release.md) + [Amazon EMR release 6.9.0](emr-690-release.md) + [Amazon EMR release 6.8.1](emr-681-release.md) + [Amazon EMR release 6.8.0](emr-680-release.md) + [Amazon EMR release 6.7.0](emr-670-release.md) + [Amazon EMR release 6.6.0](emr-660-release.md) + [Amazon EMR release 6.5.0](emr-650-release.md) + [Amazon EMR release 6.4.0](emr-640-release.md) + [Amazon EMR release 6.3.1](emr-631-release.md) + [Amazon EMR release 6.3.0](emr-630-release.md) + [Amazon EMR release 6.2.1](emr-621-release.md) + [Amazon EMR release 6.2.0](emr-620-release.md) + [Amazon EMR release 6.1.1](emr-611-release.md) + [Amazon EMR release 6.1.0](emr-610-release.md) + [Amazon EMR release 6.0.1](emr-601-release.md) + [Amazon EMR release 6.0.0](emr-600-release.md) # Application versions in Amazon EMR 6.x releases The following table lists the application versions that are available in each Amazon EMR 6.x release. **Application version information** | | emr-6.15.0 | emr-6.14.0 | emr-6.13.0 | emr-6.12.0 | emr-6.11.1 | emr-6.11.0 | emr-6.10.1 | emr-6.10.0 | emr-6.9.1 | emr-6.9.0 | emr-6.8.1 | emr-6.8.0 | emr-6.7.0 | emr-6.6.0 | emr-6.5.0 | emr-6.4.0 | emr-6.3.1 | emr-6.3.0 | emr-6.2.1 | emr-6.2.0 | emr-6.1.1 | emr-6.1.0 | emr-6.0.1 | emr-6.0.0 | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | AWS SDK for Java | 2.20.160-amzn-0, 1.12.569 | 1.12.543 | 1.12.513 | 1.12.490 | 1.12.446 | 1.12.446 | 1.12.397 | 1.12.397 | 1.12.170 | 1.12.170 | 1.12.170 | 1.12.170 | 1.12.170 | 1.12.170 | 1.12.31 | 1.12.31 | 1.11.977 | 1.11.977 | 1.11.880 | 1.11.880 | 1.11.828 | 1.11.828 | 1.11.711 | 1.11.711 | | Python | 2.7, 3.7 | 2.7, 3.7 | 2.7, 3.7 | 2.7, 3.7 | 2.7, 3.7 | 2.7, 3.7 | 2.7, 3.7 | 2.7, 3.7 | 2.7, 3.7 | 2.7, 3.7 | 2.7, 3.7 | 2.7, 3.7 | 2.7, 3.7 | 2.7, 3.7 | 2.7, 3.7 | 2.7, 3.7 | 2.7, 3.7 | 2.7, 3.7 | 2.7, 3.7 | 2.7, 3.7 | 2.7, 3.7 | 2.7, 3.7 | 2.7, 3.7 | 2.7, 3.7 | | Scala | 2.12.15 | 2.12.15 | 2.12.15 | 2.12.15 | 2.12.15 | 2.12.15 | 2.12.15 | 2.12.15 | 2.12.15 | 2.12.15 | 2.12.15 | 2.12.15 | 2.12.15 | 2.12.10 | 2.12.10 | 2.12.10 | 2.12.10 | 2.12.10 | 2.12.10 | 2.12.10 | 2.12.10 | 2.12.10 | 2.12.10 | 2.11.12 | | AmazonCloudWatchAgent | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | Delta | 2.4.0 | 2.4.0 | 2.4.0 | 2.4.0 | 2.2.0 | 2.2.0 | 2.2.0 | 2.2.0 | 2.1.0 | 2.1.0 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | Flink | 1.17.1-amzn-1 | 1.17.1-amzn-0 | 1.17.0 | 1.17.0 | 1.16.0 | 1.16.0 | 1.16.0 | 1.16.0 | 1.15.2 | 1.15.2 | 1.15.1 | 1.15.1 | 1.14.2 | 1.14.2 | 1.14.0 | 1.13.1 | 1.12.1 | 1.12.1 | 1.11.2 | 1.11.2 | 1.11.0 | 1.11.0 | - | - | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 2.4.17-amzn-3 | 2.4.17-amzn-2 | 2.4.17-amzn-1 | 2.4.17-amzn-0 | 2.4.15-amzn-1.1 | 2.4.15-amzn-1 | 2.4.15-amzn-0.1 | 2.4.15-amzn-0 | 2.4.13-amzn-0.1 | 2.4.13-amzn-0 | 2.4.12-amzn-0.1 | 2.4.12-amzn-0 | 2.4.4-amzn-3 | 2.4.4-amzn-2 | 2.4.4-amzn-1 | 2.4.4-amzn-0 | 2.2.6-amzn-1 | 2.2.6-amzn-1 | 2.2.6-amzn-0 | 2.2.6-amzn-0 | 2.2.5 | 2.2.5 | 2.2.3 | 2.2.3 | | HCatalog | 3.1.3-amzn-8 | 3.1.3-amzn-7 | 3.1.3-amzn-6 | 3.1.3-amzn-5 | 3.1.3-amzn-4.1 | 3.1.3-amzn-4 | 3.1.3-amzn-3.1 | 3.1.3-amzn-3 | 3.1.3-amzn-2.1 | 3.1.3-amzn-2 | 3.1.3-amzn-1.1 | 3.1.3-amzn-1 | 3.1.3-amzn-0 | 3.1.2-amzn-7 | 3.1.2-amzn-6 | 3.1.2-amzn-5 | 3.1.2-amzn-4 | 3.1.2-amzn-4 | 3.1.2-amzn-3 | 3.1.2-amzn-3 | 3.1.2-amzn-2 | 3.1.2-amzn-2 | 3.1.2-amzn-0 | 3.1.2-amzn-0 | | Hadoop | 3.3.6-amzn-1 | 3.3.3-amzn-6 | 3.3.3-amzn-5 | 3.3.3-amzn-4 | 3.3.3-amzn-3.1 | 3.3.3-amzn-3 | 3.3.3-amzn-2.1 | 3.3.3-amzn-2 | 3.3.3-amzn-1.1 | 3.3.3-amzn-1 | 3.2.1-amzn-8.1 | 3.2.1-amzn-8 | 3.2.1-amzn-7 | 3.2.1-amzn-6 | 3.2.1-amzn-5 | 3.2.1-amzn-4 | 3.2.1-amzn-3.1 | 3.2.1-amzn-3 | 3.2.1-amzn-2.1 | 3.2.1-amzn-2 | 3.2.1-amzn-1.1 | 3.2.1-amzn-1 | 3.2.1-amzn-0.1 | 3.2.1-amzn-0 | | Hive | 3.1.3-amzn-8 | 3.1.3-amzn-7 | 3.1.3-amzn-6 | 3.1.3-amzn-5 | 3.1.3-amzn-4.1 | 3.1.3-amzn-4 | 3.1.3-amzn-3.1 | 3.1.3-amzn-3 | 3.1.3-amzn-2.1 | 3.1.3-amzn-2 | 3.1.3-amzn-1.1 | 3.1.3-amzn-1 | 3.1.3-amzn-0 | 3.1.2-amzn-7 | 3.1.2-amzn-6 | 3.1.2-amzn-5 | 3.1.2-amzn-4 | 3.1.2-amzn-4 | 3.1.2-amzn-3 | 3.1.2-amzn-3 | 3.1.2-amzn-2 | 3.1.2-amzn-2 | 3.1.2-amzn-0 | 3.1.2-amzn-0 | | Hudi | 0.14.0-amzn-0 | 0.13.1-amzn-2 | 0.13.1-amzn-1 | 0.13.1-amzn-0 | 0.13.0-amzn-0 | 0.13.0-amzn-0 | 0.12.2-amzn-0 | 0.12.2-amzn-0 | 0.12.1-amzn-0 | 0.12.1-amzn-0 | 0.11.1-amzn-0 | 0.11.1-amzn-0 | 0.11.0-amzn-0 | 0.10.1-amzn-0 | 0.9.0-amzn-1 | 0.8.0-amzn-0 | 0.7.0-amzn-0 | 0.7.0-amzn-0 | 0.6.0-amzn-1 | 0.6.0-amzn-1 | 0.5.2-incubating-amzn-2 | 0.5.2-incubating-amzn-2 | 0.5.0-incubating-amzn-1 | 0.5.0-incubating-amzn-1 | | Hue | 4.11.0 | 4.11.0 | 4.11.0 | 4.11.0 | 4.11.0 | 4.11.0 | 4.10.0 | 4.10.0 | 4.10.0 | 4.10.0 | 4.10.0 | 4.10.0 | 4.10.0 | 4.10.0 | 4.9.0 | 4.9.0 | 4.9.0 | 4.9.0 | 4.8.0 | 4.8.0 | 4.7.1 | 4.7.1 | 4.4.0 | 4.4.0 | | Iceberg | 1.4.0-amzn-0 | 1.3.1-amzn-0 | 1.3.0-amzn-1 | 1.3.0-amzn-0 | 1.2.0-amzn-0 | 1.2.0-amzn-0 | 1.1.0-amzn-0 | 1.1.0-amzn-0 | 0.14.1-amzn-0 | 0.14.1-amzn-0 | 0.14.0-amzn-0 | 0.14.0-amzn-0 | 0.13.1-amzn-0 | 0.13.1 | 0.12.0 | - | - | - | - | - | - | - | - | - | | JupyterEnterpriseGateway | 2.6.0 | 2.6.0 | 2.6.0 | 2.6.0 | 2.6.0 | 2.6.0 | 2.6.0 | 2.6.0 | 2.6.0 | 2.6.0 | 2.1.0 | 2.1.0 | 2.1.0 | 2.1.0 | 2.1.0 | 2.1.0 | 2.1.0 | 2.1.0 | 2.1.0 | 2.1.0 | - | - | - | - | | JupyterHub | 1.5.0 | 1.5.0 | 1.5.0 | 1.4.1 | 1.4.1 | 1.4.1 | 1.5.0 | 1.5.0 | 1.4.1 | 1.4.1 | 1.4.1 | 1.4.1 | 1.4.1 | 1.4.1 | 1.4.1 | 1.4.1 | 1.2.2 | 1.2.2 | 1.1.0 | 1.1.0 | 1.1.0 | 1.1.0 | 1.0.0 | 1.0.0 | | Livy | 0.7.1-incubating | 0.7.1-incubating | 0.7.1-incubating | 0.7.1-incubating | 0.7.1-incubating | 0.7.1-incubating | 0.7.1-incubating | 0.7.1-incubating | 0.7.1-incubating | 0.7.1-incubating | 0.7.1-incubating | 0.7.1-incubating | 0.7.1-incubating | 0.7.1-incubating | 0.7.1-incubating | 0.7.1-incubating | 0.7.0-incubating | 0.7.0-incubating | 0.7.0-incubating | 0.7.0-incubating | 0.7.0-incubating | 0.7.0-incubating | 0.6.0-incubating | 0.6.0-incubating | | MXNet | 1.9.1 | 1.9.1 | 1.9.1 | 1.9.1 | 1.9.1 | 1.9.1 | 1.9.1 | 1.9.1 | 1.9.1 | 1.9.1 | 1.9.1 | 1.9.1 | 1.8.0 | 1.8.0 | 1.8.0 | 1.8.0 | 1.7.0 | 1.7.0 | 1.7.0 | 1.7.0 | 1.6.0 | 1.6.0 | 1.5.1 | 1.5.1 | | Mahout | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | Oozie | 5.2.1 | 5.2.1 | 5.2.1 | 5.2.1 | 5.2.1 | 5.2.1 | 5.2.1 | 5.2.1 | 5.2.1 | 5.2.1 | 5.2.1 | 5.2.1 | 5.2.1 | 5.2.1 | 5.2.1 | 5.2.1 | 5.2.1 | 5.2.1 | 5.2.0 | 5.2.0 | 5.2.0 | 5.2.0 | 5.1.0 | 5.1.0 | | Phoenix | 5.1.3 | 5.1.3 | 5.1.3 | 5.1.3 | 5.1.2 | 5.1.2 | 5.1.2 | 5.1.2 | 5.1.2 | 5.1.2 | 5.1.2 | 5.1.2 | 5.1.2 | 5.1.2 | 5.1.2 | 5.1.2 | 5.0.0-HBase-2.0 | 5.0.0-HBase-2.0 | 5.0.0-HBase-2.0 | 5.0.0-HBase-2.0 | 5.0.0-HBase-2.0 | 5.0.0-HBase-2.0 | 5.0.0-HBase-2.0 | 5.0.0-HBase-2.0 | | Pig | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | - | - | | Presto | 0.283-amzn-0 | 0.281-amzn-2 | 0.281-amzn-1 | 0.281-amzn-0 | 0.279-amzn-0 | 0.279-amzn-0 | 0.278.1-amzn-0 | 0.278.1-amzn-0 | 0.276-amzn-0 | 0.276-amzn-0 | 0.273.3-amzn-0 | 0.273.3-amzn-0 | 0.272-amzn-0 | 0.267-amzn-0 | 0.261-amzn-0 | 0.254.1-amzn-0 | 0.245.1-amzn-0 | 0.245.1-amzn-0 | 0.238.3-amzn-1 | 0.238.3-amzn-1 | 0.232 | 0.232 | 0.230 | 0.230 | | Spark | 3.4.1-amzn-2 | 3.4.1-amzn-1 | 3.4.1-amzn-0 | 3.4.0-amzn-0 | 3.3.2-amzn-0.1 | 3.3.2-amzn-0 | 3.3.1-amzn-0.1 | 3.3.1-amzn-0 | 3.3.0-amzn-1.1 | 3.3.0-amzn-1 | 3.3.0-amzn-0.1 | 3.3.0-amzn-0 | 3.2.1-amzn-0 | 3.2.0-amzn-0 | 3.1.2-amzn-1 | 3.1.2-amzn-0 | 3.1.1-amzn-0.1 | 3.1.1-amzn-0 | 3.0.1-amzn-0.1 | 3.0.1-amzn-0 | 3.0.0-amzn-0.1 | 3.0.0-amzn-0 | 2.4.4 | 2.4.4 | | Sqoop | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | - | - | | TensorFlow | 2.11.0 | 2.11.0 | 2.11.0 | 2.11.0 | 2.11.0 | 2.11.0 | 2.11.0 | 2.11.0 | 2.10.0 | 2.10.0 | 2.9.1 | 2.9.1 | 2.4.1 | 2.4.1 | 2.4.1 | 2.4.1 | 2.4.1 | 2.4.1 | 2.3.1 | 2.3.1 | 2.1.0 | 2.1.0 | 1.14.0 | 1.14.0 | | Tez | 0.10.2-amzn-6 | 0.10.2-amzn-5 | 0.10.2-amzn-4 | 0.10.2-amzn-3 | 0.10.2-amzn-2.1 | 0.10.2-amzn-2 | 0.10.2-amzn-1.1 | 0.10.2-amzn-1 | 0.10.2-amzn-0.1 | 0.10.2-amzn-0 | 0.9.2 | 0.9.2 | 0.9.2 | 0.9.2 | 0.9.2 | 0.9.2 | 0.9.2 | 0.9.2 | 0.9.2 | 0.9.2 | 0.9.2 | 0.9.2 | 0.9.2 | 0.9.2 | | Trino (PrestoSQL) | 426-amzn-0 | 422-amzn-0 | 414-amzn-1 | 414-amzn-0 | 410-amzn-0 | 410-amzn-0 | 403-amzn-0 | 403-amzn-0 | 398-amzn-0 | 398-amzn-0 | 388-amzn-0 | 388-amzn-0 | 378-amzn-0 | 367-amzn-0 | 360 | 359 | 350 | 350 | 343 | 343 | 338 | 338 | - | - | | Zeppelin | 0.10.1 | 0.10.1 | 0.10.1 | 0.10.1 | 0.10.1 | 0.10.1 | 0.10.1 | 0.10.1 | 0.10.1 | 0.10.1 | 0.10.1 | 0.10.1 | 0.10.0 | 0.10.0 | 0.10.0 | 0.9.0 | 0.9.0 | 0.9.0 | 0.9.0-preview1 | 0.9.0-preview1 | 0.9.0-preview1 | 0.9.0-preview1 | 0.9.0-SNAPSHOT | 0.9.0-SNAPSHOT | | ZooKeeper | 3.5.10 | 3.5.10 | 3.5.10 | 3.5.10 | 3.5.10 | 3.5.10 | 3.5.10 | 3.5.10 | 3.5.10 | 3.5.10 | 3.5.10 | 3.5.10 | 3.5.7 | 3.5.7 | 3.5.7 | 3.5.7 | 3.4.14 | 3.4.14 | 3.4.14 | 3.4.14 | 3.4.14 | 3.4.14 | 3.4.14 | 3.4.14 | # Amazon EMR release 6.15.0 ## 6.15.0 application versions This release includes the following applications: [https://delta.io/](https://delta.io/), [https://flink.apache.org/](https://flink.apache.org/), [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [https://hudi.apache.org](https://hudi.apache.org), [http://gethue.com/](http://gethue.com/), [https://iceberg.apache.org/](https://iceberg.apache.org/), [https://jupyter-enterprise-gateway.readthedocs.io/en/latest/](https://jupyter-enterprise-gateway.readthedocs.io/en/latest/), [https://jupyterhub.readthedocs.io/en/latest/#](https://jupyterhub.readthedocs.io/en/latest/#), [https://livy.incubator.apache.org/](https://livy.incubator.apache.org/), [https://mxnet.incubator.apache.org/](https://mxnet.incubator.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://www.tensorflow.org/](https://www.tensorflow.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://trino.io/](https://trino.io/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-6.15.0 | emr-6.14.0 | emr-6.13.0 | emr-6.12.0 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 2.20.160-amzn-0, 1.12.569 | 1.12.543 | 1.12.513 | 1.12.490 | | Python | 2.7, 3.7 | 2.7, 3.7 | 2.7, 3.7 | 2.7, 3.7 | | Scala | 2.12.15 | 2.12.15 | 2.12.15 | 2.12.15 | | AmazonCloudWatchAgent | - | - | - | - | | Delta | 2.4.0 | 2.4.0 | 2.4.0 | 2.4.0 | | Flink | 1.17.1-amzn-1 | 1.17.1-amzn-0 | 1.17.0 | 1.17.0 | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 2.4.17-amzn-3 | 2.4.17-amzn-2 | 2.4.17-amzn-1 | 2.4.17-amzn-0 | | HCatalog | 3.1.3-amzn-8 | 3.1.3-amzn-7 | 3.1.3-amzn-6 | 3.1.3-amzn-5 | | Hadoop | 3.3.6-amzn-1 | 3.3.3-amzn-6 | 3.3.3-amzn-5 | 3.3.3-amzn-4 | | Hive | 3.1.3-amzn-8 | 3.1.3-amzn-7 | 3.1.3-amzn-6 | 3.1.3-amzn-5 | | Hudi | 0.14.0-amzn-0 | 0.13.1-amzn-2 | 0.13.1-amzn-1 | 0.13.1-amzn-0 | | Hue | 4.11.0 | 4.11.0 | 4.11.0 | 4.11.0 | | Iceberg | 1.4.0-amzn-0 | 1.3.1-amzn-0 | 1.3.0-amzn-1 | 1.3.0-amzn-0 | | JupyterEnterpriseGateway | 2.6.0 | 2.6.0 | 2.6.0 | 2.6.0 | | JupyterHub | 1.5.0 | 1.5.0 | 1.5.0 | 1.4.1 | | Livy | 0.7.1-incubating | 0.7.1-incubating | 0.7.1-incubating | 0.7.1-incubating | | MXNet | 1.9.1 | 1.9.1 | 1.9.1 | 1.9.1 | | Mahout | - | - | - | - | | Oozie | 5.2.1 | 5.2.1 | 5.2.1 | 5.2.1 | | Phoenix | 5.1.3 | 5.1.3 | 5.1.3 | 5.1.3 | | Pig | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | | Presto | 0.283-amzn-0 | 0.281-amzn-2 | 0.281-amzn-1 | 0.281-amzn-0 | | Spark | 3.4.1-amzn-2 | 3.4.1-amzn-1 | 3.4.1-amzn-0 | 3.4.0-amzn-0 | | Sqoop | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | | TensorFlow | 2.11.0 | 2.11.0 | 2.11.0 | 2.11.0 | | Tez | 0.10.2-amzn-6 | 0.10.2-amzn-5 | 0.10.2-amzn-4 | 0.10.2-amzn-3 | | Trino (PrestoSQL) | 426-amzn-0 | 422-amzn-0 | 414-amzn-1 | 414-amzn-0 | | Zeppelin | 0.10.1 | 0.10.1 | 0.10.1 | 0.10.1 | | ZooKeeper | 3.5.10 | 3.5.10 | 3.5.10 | 3.5.10 | ## 6.15.0 release notes The following release notes include information for Amazon EMR release 6.15.0. Changes are relative to 6.14.0. For information on the release timeline, see the [6.15.0 change log](#6150-changelog). **New features** + **Application upgrades** – Amazon EMR 6.15.0 application upgrades include Apache Hadoop 3.3.6, Apache Hudi 0.14.0-amzn-0, Iceberg 1.4.0-amzn-0, and Trino 426. + **[Faster launches for EMR clusters that run on EC2](https://aws.amazon.com/about-aws/whats-new/2023/11/amazon-emr-ec2-clusters-5-minutes-less/)** – It's now up to 35% faster to launch an Amazon EMR on EC2 cluster. With this improvement, most customers can launch their clusters in 5 minutes or less. + **[CodeWhisperer for EMR Studio](https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-studio-codewhisperer.html)** – You can now use Amazon CodeWhisperer with Amazon EMR Studio to get real-time recommendations as you write code in JupyterLab. CodeWhisperer can complete your comments, finish single lines of code, make line-by-line recommendations, and generate fully-formed functions. + **[Faster job restart times with Flink](https://docs.aws.amazon.com/emr/latest/ReleaseGuide/flink-restart.html)** – With Amazon EMR 6.15.0 and higher, several new mechanisms are available for Apache Flink to improve the job restart time during task recovery or scaling operations. This optimizes the speed of recovery and restart of execution graphs to improve job stability. + **[Table-level and fine-grained access control for open-table formats](https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-lf-enable.html)** – With Amazon EMR 6.15.0 and higher, when you run Spark jobs on Amazon EMR on EC2 clusters that access data in the AWS Glue Data Catalog, you can use AWS Lake Formation to apply table, row, column, and cell level permissions on Hudi, Iceberg, or Delta Lake based tables. + **Hadoop upgrade** – Amazon EMR 6.15.0 includes an upgrade of Apache Hadoop to version 3.3.6. Hadoop 3.3.6 was the latest version at the time of the Amazon EMR 6.15 deployment, released by Apache in June 2023. Prior releases of Amazon EMR (6.9.0 to 6.14.x) used Hadoop 3.3.3. The upgrade includes hundreds of improvements and fixes, and features that include reconfigurable datanode parameters, `DFSAdmin` option to initiate bulk reconfiguration operations on all live datanodes, and a vectored API that allows seek-heavy readers to specify multiple ranges to read. Hadoop 3.3.6 also adds support for HDFS APIs and semantics for its write-ahead log (WAL), so that HBase can run on other storage system implementations. For more information, see the changelogs for versions [3.3.4](https://hadoop.apache.org/docs/r3.3.4/hadoop-project-dist/hadoop-common/release/3.3.4/CHANGELOG.3.3.4.html), [3.3.5](https://hadoop.apache.org/docs/r3.3.5/hadoop-project-dist/hadoop-common/release/3.3.5/CHANGELOG.3.3.5.html), and [3.3.6](https://hadoop.apache.org/docs/r3.3.6/hadoop-project-dist/hadoop-common/release/3.3.6/CHANGELOG.3.3.6.html) in the *Apache Hadoop documentation*. + **Support for AWS SDK for Java, version 2** - Amazon EMR 6.15.0 applications can use AWS SDK for Java versions [1.12.569](https://github.com/aws/aws-sdk-java/tree/1.12.569) or [2.20.160](https://github.com/aws/aws-sdk-java-v2/tree/2.20.160) if the application supports v2. The AWS SDK for Java 2.x is a major rewrite of the version 1.x code base. It’s built on top of Java 8\$1 and adds several frequently requested features. These include support for non-blocking I/O, and the ability to plug in a different HTTP implementation at runtime. For more information, including a **Migration Guide from SDK for Java v1 to v2**, see the [AWS SDK for Java, version 2](https://docs.aws.amazon.com/sdk-for-java) guide. **Known issues** + An on-cluster instance-state script that monitors health of the instance can consume excessive CPU and memory resources when there are a large number of threads and/or open file handles on the node. **Changes, enhancements, and resolved issues** + Starting with Spark 3.3.1 (supported in EMR versions 6.10 and above), all executors in a decommissioning host are set to a new `ExecutorState`, called *DECOMMISSIONING* state. The executors being decommissioned cannot be used by Yarn to allocate tasks and thus it will request for new executors, if needed, for the tasks being executed. Thus, if you disable Spark DRA while using EMR Managed Scaling, EMR Auto Scaling, or any custom scaling mechanism on EMR-EC2 clusters, then Yarn may request maximum permissible executors for each job. In order to avoid this issue, leave the `spark.dynamicAllocation.enabled` property set to `TRUE` (which is the default) when you are using the above combination of features. In addition, you can also set minimum and maximum executor constraints by setting values for `spark.dynamicAllocation.maxExecutors` and `spark.dynamicAllocation.minExecutors` properties for your Spark jobs, to restrict the number of executors allocated during the job’s execution. + To improve your high-availability EMR clusters, this release enables connectivity to Amazon EMR daemons on local host that use IPv6 endpoints. + This release enables TLS 1.2 for communication with ZooKeeper provisioned on all the primary nodes of your high-availability cluster. + This release improves the management of ZooKeeper transaction log files that are maintained on primary nodes to minimize scenarios where the log files grow out of bounds and interrupt cluster operations. + This release makes intra-node communication more resilient for high-availability EMR clusters. This improvement reduces the chance of bootstrap action failures or cluster start failures. + Tez in Amazon EMR 6.15.0 introduces configurations that you can specify to asynchronously open the input splits in a Tez grouped split. This results in faster performance of read queries when there are a large number of input splits in a single Tez grouped split. For more information, see [Tez asynchronous split opening](https://docs.aws.amazon.com/emr/latest/ReleaseGuide/tez-configure.html#tez-configure-async). + When you launch a cluster with *the latest patch release* of Amazon EMR 5.36 or higher, 6.6 or higher, or 7.0 or higher, Amazon EMR uses the latest Amazon Linux 2023 or Amazon Linux 2 release for the default Amazon EMR AMI. For more information, see [Using the default Amazon Linux AMI for Amazon EMR](https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-default-ami.html). [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-6150-release.html) ## 6.15.0 default Java versions Amazon EMR releases 6.12.0 and higher support all applications with Amazon Corretto 8 by default, except for Trino. For Trino, Amazon EMR supports Amazon Corretto 17 by default starting with Amazon EMR release 6.9.0. Amazon EMR also supports some applications with Amazon Corretto 11 and 17. Those applications are listed in the following table. If you want to change the default JVM on your cluster, follow the instructions in [Configure applications to use a specific Java Virtual Machine](configuring-java8.md) for each application that runs on the cluster. You can only use one Java runtime version for a cluster. Amazon EMR doesn't support running different nodes or applications on different runtime versions on the same cluster. While Amazon EMR supports both Amazon Corretto 11 and 17 on Apache Spark, Apache Hadoop, and Apache Hive, performance might regress for some workloads when you use these versions of Corretto. We recommend that you test your workloads before you change defaults. The following table shows the default Java versions for applications in Amazon EMR 6.15.0: | Application | Java / Amazon Corretto version (default is bold) | | --- | --- | | Delta | 17, 11, 8 | | Flink | 11, 8 | | Ganglia | 8 | | HBase | 11, 8 | | HCatalog | 17, 11, 8 | | Hadoop | 17, 11, 8 | | Hive | 17, 11, 8 | | Hudi | 17, 11, 8 | | Iceberg | 17, 11, 8 | | Livy | 17, 11, 8 | | Oozie | 17, 11, 8 | | Phoenix | 8 | | PrestoDB | 8 | | Spark | 17, 11, 8 | | Spark RAPIDS | 17, 11, 8 | | Sqoop | 8 | | Tez | 17, 11, 8 | | Trino | 17 | | Zeppelin | 8 | | Pig | 8 | | Zookeeper | 8 | ## 6.15.0 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | aws-sagemaker-spark-sdk | 1.4.2 | Amazon SageMaker Spark SDK | | delta | 2.4.0 | Delta lake is an open table format for huge analytic datasets | | delta-standalone-connectors | 0.6.0 | Delta Connectors provide different runtimes to integrate Delta Lake with engines like Flink, Hive and Presto. | | emr-ddb | 5.2.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 3.8.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.12.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-notebook-env | 1.7.0 | Conda env for emr notebook which includes jupyter enterprise gateway | | emr-s3-dist-cp | 2.29.0 | Distributed copy application optimized for Amazon S3. | | emr-s3-select | 2.8.0 | EMR S3Select Connector | | emr-wal-cli | 1.2.0 | Cli used for emrwal list/deletion. | | emrfs | 2.60.0 | Amazon S3 connector for Hadoop ecosystem applications. | | flink-client | 1.17.1-amzn-1 | Apache Flink command line client scripts and applications. | | flink-jobmanager-config | 1.17.1-amzn-1 | Managing resources on EMR nodes for Apache Flink JobManager. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 3.3.6-amzn-1 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 3.3.6-amzn-1 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 3.3.6-amzn-1 | HDFS command-line client and library | | hadoop-hdfs-namenode | 3.3.6-amzn-1 | HDFS service for tracking file names and block locations. | | hadoop-hdfs-journalnode | 3.3.6-amzn-1 | HDFS service for managing the Hadoop filesystem journal on HA clusters. | | hadoop-httpfs-server | 3.3.6-amzn-1 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 3.3.6-amzn-1 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 3.3.6-amzn-1 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 3.3.6-amzn-1 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 3.3.6-amzn-1 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 3.3.6-amzn-1 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 2.4.17-amzn-3 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 2.4.17-amzn-3 | Service for serving one or more HBase regions. | | hbase-client | 2.4.17-amzn-3 | HBase command-line client. | | hbase-rest-server | 2.4.17-amzn-3 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 2.4.17-amzn-3 | Service providing a Thrift endpoint to HBase. | | hbase-operator-tools | 2.4.17-amzn-3 | Repair tool for Apache HBase clusters. | | hcatalog-client | 3.1.3-amzn-8 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 3.1.3-amzn-8 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 3.1.3-amzn-8 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 3.1.3-amzn-8 | Hive command line client. | | hive-hbase | 3.1.3-amzn-8 | Hive-hbase client. | | hive-metastore-server | 3.1.3-amzn-8 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server2 | 3.1.3-amzn-8 | Service for accepting Hive queries as web requests. | | hudi | 0.14.0-amzn-0 | Incremental processing framework to power data pipeline at low latency and high efficiency. | | hudi-presto | 0.14.0-amzn-0 | Bundle library for running Presto with Hudi. | | hudi-trino | 0.14.0-amzn-0 | Bundle library for running Trino with Hudi. | | hudi-spark | 0.14.0-amzn-0 | Bundle library for running Spark with Hudi. | | hue-server | 4.11.0 | Web application for analyzing data using Hadoop ecosystem applications | | iceberg | 1.4.0-amzn-0 | Apache Iceberg is an open table format for huge analytic datasets | | jupyterhub | 1.5.0 | Multi-user server for Jupyter notebooks | | livy-server | 0.7.1-incubating | REST interface for interacting with Apache Spark | | nginx | 1.12.1 | nginx [engine x] is an HTTP and reverse proxy server | | mxnet | 1.9.1 | A flexible, scalable, and efficient library for deep learning. | | mariadb-server | 5.5.68\$1 | MariaDB database server. | | nvidia-cuda | 11.8.0 | Nvidia drivers and Cuda toolkit | | oozie-client | 5.2.1 | Oozie command-line client. | | oozie-server | 5.2.1 | Service for accepting Oozie workflow requests. | | opencv | 4.7.0 | Open Source Computer Vision Library. | | phoenix-library | 5.1.3 | The phoenix libraries for server and client | | phoenix-connectors | 5.1.3 | Apache Phoenix-Connectors for Spark-3 | | phoenix-query-server | 6.0.0 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.283-amzn-0 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.283-amzn-0 | Service for executing pieces of a query. | | presto-client | 0.283-amzn-0 | Presto command-line client which is installed on an HA cluster's stand-by masters where Presto server is not started. | | trino-coordinator | 426-amzn-0 | Service for accepting queries and managing query execution among trino-workers. | | trino-worker | 426-amzn-0 | Service for executing pieces of a query. | | trino-client | 426-amzn-0 | Trino command-line client which is installed on an HA cluster's stand-by masters where Trino server is not started. | | pig-client | 0.17.0 | Pig command-line client. | | r | 4.0.2 | The R Project for Statistical Computing | | ranger-kms-server | 2.0.0 | Apache Ranger Key Management System | | spark-client | 3.4.1-amzn-2 | Spark command-line clients. | | spark-history-server | 3.4.1-amzn-2 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 3.4.1-amzn-2 | In-memory execution engine for YARN. | | spark-yarn-slave | 3.4.1-amzn-2 | Apache Spark libraries needed by YARN slaves. | | spark-rapids | 23.08.1-amzn-0 | Nvidia Spark RAPIDS plugin that accelerates Apache Spark with GPUs. | | sqoop-client | 1.4.7 | Apache Sqoop command-line client. | | tensorflow | 2.11.0 | TensorFlow open source software library for high performance numerical computation. | | tez-on-yarn | 0.10.2-amzn-6 | The tez YARN application and libraries. | | tez-on-worker | 0.10.2-amzn-6 | The tez YARN application and libraries for worker nodes. | | webserver | 2.4.41\$1 | Apache HTTP server. | | zeppelin-server | 0.10.1 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.5.10 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.5.10 | ZooKeeper command line client. | ## 6.15.0 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). Reconfiguration actions occur when you specify a configuration for instance groups in a running cluster. Amazon EMR only initiates reconfiguration actions for the classifications that you modify. For more information, see [Reconfigure an instance group in a running cluster](emr-configure-apps-running-cluster.md). **emr-6.15.0 classifications** | Classifications | Description | Reconfiguration Actions | | --- | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | Restarts the ResourceManager service. | | container-executor | Change values in Hadoop YARN's container-executor.cfg file. | Not available. | | container-log4j | Change values in Hadoop YARN's container-log4j.properties file. | Not available. | | core-site | Change values in Hadoop's core-site.xml file. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts Hadoop KMS, Ranger KMS, HiveServer2, Hive MetaStore, Hadoop Httpfs, and MapReduce-HistoryServer. | | docker-conf | Change docker related settings. | Not available. | | emrfs-site | Change EMRFS settings. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts HBaseRegionserver, HBaseMaster, HBaseThrift, HBaseRest, HiveServer2, Hive MetaStore, Hadoop Httpfs, and MapReduce-HistoryServer. | | flink-conf | Change flink-conf.yaml settings. | Restarts Flink history server. | | flink-log4j | Change Flink log4j.properties settings. | Restarts Flink history server. | | flink-log4j-session | Change Flink log4j-session.properties settings for Kubernetes/Yarn session. | Restarts Flink history server. | | flink-log4j-cli | Change Flink log4j-cli.properties settings. | Restarts Flink history server. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts PhoenixQueryserver, HiveServer2, Hive MetaStore, and MapReduce-HistoryServer. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | Restarts the Hadoop HDFS services SecondaryNamenode, Datanode, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts Hadoop KMS, Hadoop Httpfs, and MapReduce-HistoryServer. | | hadoop-ssl-server | Change hadoop ssl server configuration | Not available. | | hadoop-ssl-client | Change hadoop ssl client configuration | Not available. | | hbase | Amazon EMR-curated settings for Apache HBase. | Custom EMR specific property. Sets emrfs-site and hbase-site configs. See those for their associated restarts. | | hbase-env | Change values in HBase's environment. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbase.properties file. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | Not available. | | hbase-site | Change values in HBase's hbase-site.xml file. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. Additionally restarts Phoenix QueryServer. | | hdfs-encryption-zones | Configure HDFS encryption zones. | This classification should not be reconfigured. | | hdfs-env | Change values in the HDFS environment. | Restarts Hadoop HDFS services Namenode, Datanode, and ZKFC. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Additionally restarts Hadoop Httpfs. | | hcatalog-env | Change values in HCatalog's environment. | Restarts Hive HCatalog Server. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | Restarts Hive HCatalog Server. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | Restarts Hive HCatalog Server. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | Restarts Hive WebHCat server. | | hcatalog-webhcat-log4j2 | Change values in HCatalog WebHCat's log4j2.properties. | Restarts Hive WebHCat server. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | Restarts Hive WebHCat server. | | hive | Amazon EMR-curated settings for Apache Hive. | Sets configurations to launch Hive LLAP service. | | hive-beeline-log4j2 | Change values in Hive's beeline-log4j2.properties file. | Not available. | | hive-parquet-logging | Change values in Hive's parquet-logging.properties file. | Not available. | | hive-env | Change values in the Hive environment. | Restarts HiveServer2, HiveMetastore, and Hive HCatalog-Server. Runs Hive schemaTool CLI commands to verify hive-metastore. | | hive-exec-log4j2 | Change values in Hive's hive-exec-log4j2.properties file. | Not available. | | hive-llap-daemon-log4j2 | Change values in Hive's llap-daemon-log4j2.properties file. | Not available. | | hive-log4j2 | Change values in Hive's hive-log4j2.properties file. | Not available. | | hive-site | Change values in Hive's hive-site.xml file | Restarts HiveServer2, HiveMetastore, and Hive HCatalog-Server. Runs Hive schemaTool CLI commands to verify hive-metastore. Also restarts Oozie and Zeppelin. | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | Not available. | | hue-ini | Change values in Hue's ini file | Restarts Hue. Also activates Hue config override CLI commands to pick up new configurations. | | httpfs-env | Change values in the HTTPFS environment. | Restarts Hadoop Httpfs service. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | Restarts Hadoop Httpfs service. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | Not available. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | Restarts Hadoop-KMS service. | | hadoop-kms-java-home | Change Hadoop's KMS java home | Not available. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | Not available. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | Restarts Hadoop-KMS and Ranger-KMS service. | | hudi-env | Change values in the Hudi environment. | Not available. | | hudi-defaults | Change values in Hudi's hudi-defaults.conf file. | Not available. | | iceberg-defaults | Change values in Iceberg's iceberg-defaults.conf file. | Not available. | | delta-defaults | Change values in Delta's delta-defaults.conf file. | Not available. | | jupyter-notebook-conf | Change values in Jupyter Notebook's jupyter\$1notebook\$1config.py file. | Not available. | | jupyter-hub-conf | Change values in JupyterHubs's jupyterhub\$1config.py file. | Not available. | | jupyter-s3-conf | Configure Jupyter Notebook S3 persistence. | Not available. | | jupyter-sparkmagic-conf | Change values in Sparkmagic's config.json file. | Not available. | | livy-conf | Change values in Livy's livy.conf file. | Restarts Livy Server. | | livy-env | Change values in the Livy environment. | Restarts Livy Server. | | livy-log4j2 | Change Livy log4j2.properties settings. | Restarts Livy Server. | | mapred-env | Change values in the MapReduce application's environment. | Restarts Hadoop MapReduce-HistoryServer. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | Restarts Hadoop MapReduce-HistoryServer. | | oozie-env | Change values in Oozie's environment. | Restarts Oozie. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | Restarts Oozie. | | oozie-site | Change values in Oozie's oozie-site.xml file. | Restarts Oozie. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | Not available. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | Not available. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | Restarts Phoenix-QueryServer. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | Not available. | | pig-env | Change values in the Pig environment. | Not available. | | pig-properties | Change values in Pig's pig.properties file. | Restarts Oozie. | | pig-log4j | Change values in Pig's log4j.properties file. | Not available. | | presto-log | Change values in Presto's log.properties file. | Restarts Presto-Server (for PrestoDB) | | presto-config | Change values in Presto's config.properties file. | Restarts Presto-Server (for PrestoDB) | | presto-password-authenticator | Change values in Presto's password-authenticator.properties file. | Not available. | | presto-env | Change values in Presto's presto-env.sh file. | Restarts Presto-Server (for PrestoDB) | | presto-node | Change values in Presto's node.properties file. | Not available. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | Not available. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | Not available. | | presto-connector-hive | Change values in Presto's hive.properties file. | Restarts Presto-Server (for PrestoDB) | | presto-connector-jmx | Change values in Presto's jmx.properties file. | Not available. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | Not available. | | presto-connector-lakeformation | Change values in Presto's lakeformation.properties file. | Restarts Presto-Server (for PrestoDB) | | presto-connector-localfile | Change values in Presto's localfile.properties file. | Not available. | | presto-connector-memory | Change values in Presto's memory.properties file. | Not available. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | Not available. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | Not available. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | Not available. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | Not available. | | presto-connector-redis | Change values in Presto's redis.properties file. | Not available. | | presto-connector-redshift | Change values in Presto's redshift.properties file. | Not available. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | Not available. | | presto-connector-tpcds | Change values in Presto's tpcds.properties file. | Not available. | | trino-log | Change values in Trino's log.properties file. | Restarts Trino-Server (for Trino) | | trino-config | Change values in Trino's config.properties file. | Restarts Trino-Server (for Trino) | | trino-password-authenticator | Change values in Trino's password-authenticator.properties file. | Restarts Trino-Server (for Trino) | | trino-env | Change values in Trino's trino-env.sh file. | Restarts Trino-Server (for Trino) | | trino-node | Change values in Trino's node.properties file. | Not available. | | trino-connector-blackhole | Change values in Trino's blackhole.properties file. | Not available. | | trino-connector-cassandra | Change values in Trino's cassandra.properties file. | Not available. | | trino-connector-delta | Change values in Trino's delta.properties file. | Restarts Trino-Server (for Trino) | | trino-connector-hive | Change values in Trino's hive.properties file. | Restarts Trino-Server (for Trino) | | trino-exchange-manager | Change values in Trino's exchange-manager.properties file. | Restarts Trino-Server (for Trino) | | trino-connector-iceberg | Change values in Trino's iceberg.properties file. | Restarts Trino-Server (for Trino) | | trino-connector-hudi | Change values in Trino's hudi.properties file. | Restarts Trino-Server (for Trino) | | trino-connector-jmx | Change values in Trino's jmx.properties file. | Not available. | | trino-connector-kafka | Change values in Trino's kafka.properties file. | Not available. | | trino-connector-localfile | Change values in Trino's localfile.properties file. | Not available. | | trino-connector-memory | Change values in Trino's memory.properties file. | Not available. | | trino-connector-mongodb | Change values in Trino's mongodb.properties file. | Not available. | | trino-connector-mysql | Change values in Trino's mysql.properties file. | Not available. | | trino-connector-postgresql | Change values in Trino's postgresql.properties file. | Not available. | | trino-connector-raptor | Change values in Trino's raptor.properties file. | Not available. | | trino-connector-redis | Change values in Trino's redis.properties file. | Not available. | | trino-connector-redshift | Change values in Trino's redshift.properties file. | Not available. | | trino-connector-tpch | Change values in Trino's tpch.properties file. | Not available. | | trino-connector-tpcds | Change values in Trino's tpcds.properties file. | Not available. | | ranger-kms-dbks-site | Change values in dbks-site.xml file of Ranger KMS. | Restarts Ranger KMS Server. | | ranger-kms-site | Change values in ranger-kms-site.xml file of Ranger KMS. | Restarts Ranger KMS Server. | | ranger-kms-env | Change values in the Ranger KMS environment. | Restarts Ranger KMS Server. | | ranger-kms-logback | Change values in kms-logback.xml file of Ranger KMS. | Not available. | | ranger-kms-db-ca | Change values for CA file on S3 for MySQL SSL connection with Ranger KMS. | Not available. | | spark | Amazon EMR-curated settings for Apache Spark. | This property modifies spark-defaults. See actions there. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | Restarts Spark history server and Spark thrift server. | | spark-env | Change values in the Spark environment. | Restarts Spark history server and Spark thrift server. | | spark-hive-site | Change values in Spark's hive-site.xml file | Not available. | | spark-log4j2 | Change values in Spark's log4j2.properties file. | Restarts Spark history server and Spark thrift server. | | spark-metrics | Change values in Spark's metrics.properties file. | Restarts Spark history server and Spark thrift server. | | sqoop-env | Change values in Sqoop's environment. | Not available. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | Not available. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | Not available. | | tez-site | Change values in Tez's tez-site.xml file. | Restart Oozie and HiveServer2. | | yarn-env | Change values in the YARN environment. | Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts MapReduce-HistoryServer. | | yarn-site | Change values in YARN's yarn-site.xml file. | Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts Livy Server and MapReduce-HistoryServer. | | zeppelin-env | Change values in the Zeppelin environment. | Restarts Zeppelin. | | zeppelin-site | Change configuration settings in zeppelin-site.xml. | Restarts Zeppelin. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | Restarts Zookeeper server. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | Restarts Zookeeper server. | ## 6.15.0 change log **Change log for 6.15.0 release and release notes** | Date | Event | Description | | --- | --- | --- | | 2025-09-03 | Docs revision | Amazon EMR 6.15.0 release notes added known issue | | 2023-11-17 | Docs publication | Amazon EMR 6.15.0 release notes first published | | 2023-11-17 | Deployment complete | Amazon EMR 6.15.0 fully deployed to all [supported Regions](https://aws.amazon.com/about-aws/global-infrastructure/regional-product-services/) | | 2023-11-13 | Initial release | Amazon EMR 6.15.0 first deployed to initial commercial Regions | # Amazon EMR release 6.14.0 ## 6.14.0 application versions This release includes the following applications: [https://delta.io/](https://delta.io/), [https://flink.apache.org/](https://flink.apache.org/), [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [https://hudi.apache.org](https://hudi.apache.org), [http://gethue.com/](http://gethue.com/), [https://iceberg.apache.org/](https://iceberg.apache.org/), [https://jupyter-enterprise-gateway.readthedocs.io/en/latest/](https://jupyter-enterprise-gateway.readthedocs.io/en/latest/), [https://jupyterhub.readthedocs.io/en/latest/#](https://jupyterhub.readthedocs.io/en/latest/#), [https://livy.incubator.apache.org/](https://livy.incubator.apache.org/), [https://mxnet.incubator.apache.org/](https://mxnet.incubator.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://www.tensorflow.org/](https://www.tensorflow.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://trino.io/](https://trino.io/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-6.14.0 | emr-6.13.0 | emr-6.12.0 | emr-6.11.1 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.12.543 | 1.12.513 | 1.12.490 | 1.12.446 | | Python | 2.7, 3.7 | 2.7, 3.7 | 2.7, 3.7 | 2.7, 3.7 | | Scala | 2.12.15 | 2.12.15 | 2.12.15 | 2.12.15 | | AmazonCloudWatchAgent | - | - | - | - | | Delta | 2.4.0 | 2.4.0 | 2.4.0 | 2.2.0 | | Flink | 1.17.1-amzn-0 | 1.17.0 | 1.17.0 | 1.16.0 | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 2.4.17-amzn-2 | 2.4.17-amzn-1 | 2.4.17-amzn-0 | 2.4.15-amzn-1.1 | | HCatalog | 3.1.3-amzn-7 | 3.1.3-amzn-6 | 3.1.3-amzn-5 | 3.1.3-amzn-4.1 | | Hadoop | 3.3.3-amzn-6 | 3.3.3-amzn-5 | 3.3.3-amzn-4 | 3.3.3-amzn-3.1 | | Hive | 3.1.3-amzn-7 | 3.1.3-amzn-6 | 3.1.3-amzn-5 | 3.1.3-amzn-4.1 | | Hudi | 0.13.1-amzn-2 | 0.13.1-amzn-1 | 0.13.1-amzn-0 | 0.13.0-amzn-0 | | Hue | 4.11.0 | 4.11.0 | 4.11.0 | 4.11.0 | | Iceberg | 1.3.1-amzn-0 | 1.3.0-amzn-1 | 1.3.0-amzn-0 | 1.2.0-amzn-0 | | JupyterEnterpriseGateway | 2.6.0 | 2.6.0 | 2.6.0 | 2.6.0 | | JupyterHub | 1.5.0 | 1.5.0 | 1.4.1 | 1.4.1 | | Livy | 0.7.1-incubating | 0.7.1-incubating | 0.7.1-incubating | 0.7.1-incubating | | MXNet | 1.9.1 | 1.9.1 | 1.9.1 | 1.9.1 | | Mahout | - | - | - | - | | Oozie | 5.2.1 | 5.2.1 | 5.2.1 | 5.2.1 | | Phoenix | 5.1.3 | 5.1.3 | 5.1.3 | 5.1.2 | | Pig | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | | Presto | 0.281-amzn-2 | 0.281-amzn-1 | 0.281-amzn-0 | 0.279-amzn-0 | | Spark | 3.4.1-amzn-1 | 3.4.1-amzn-0 | 3.4.0-amzn-0 | 3.3.2-amzn-0.1 | | Sqoop | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | | TensorFlow | 2.11.0 | 2.11.0 | 2.11.0 | 2.11.0 | | Tez | 0.10.2-amzn-5 | 0.10.2-amzn-4 | 0.10.2-amzn-3 | 0.10.2-amzn-2.1 | | Trino (PrestoSQL) | 422-amzn-0 | 414-amzn-1 | 414-amzn-0 | 410-amzn-0 | | Zeppelin | 0.10.1 | 0.10.1 | 0.10.1 | 0.10.1 | | ZooKeeper | 3.5.10 | 3.5.10 | 3.5.10 | 3.5.10 | ## 6.14.0 release notes The following release notes include information for Amazon EMR release 6.14.0. Changes are relative to 6.13.0. For information on the release timeline, see the [6.14.0 change log](#6140-changelog). **New features** + Amazon EMR 6.14.0 supports Apache Spark 3.4.1, Apache Spark RAPIDS 23.06.0-amzn-2, Flink 1.17.1, Iceberg 1.3.1, and Trino 422. + [Amazon EMR managed scaling](https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-managed-scaling.html) is now available in the `ap-southeast-3` Asia Pacific (Jakarta) Region for clusters that you create with Amazon EMR 6.14.0 and higher. **Known issues** + An on-cluster instance-state script that monitors health of the instance can consume excessive CPU and memory resources when there are a large number of threads and/or open file handles on the node. **Changes, enhancements, and resolved issues** + Starting with Spark 3.3.1 (supported in EMR versions 6.10 and above), all executors in a decommissioning host are set to a new `ExecutorState`, called *DECOMMISSIONING* state. The executors being decommissioned cannot be used by Yarn to allocate tasks and thus it will request for new executors, if needed, for the tasks being executed. Thus, if you disable Spark DRA while using EMR Managed Scaling, EMR Auto Scaling, or any custom scaling mechanism on EMR-EC2 clusters, then Yarn may request maximum permissible executors for each job. In order to avoid this issue, leave the `spark.dynamicAllocation.enabled` property set to `TRUE` (which is the default) when you are using the above combination of features. In addition, you can also set minimum and maximum executor constraints by setting values for `spark.dynamicAllocation.maxExecutors` and `spark.dynamicAllocation.minExecutors` properties for your Spark jobs, to restrict the number of executors allocated during the job’s execution. + The 6.14.0 release optimizes log management with Amazon EMR running on Amazon EC2. As a result, you might see a slight reduction in storage costs for your cluster logs. + The 6.14.0 release improves the scaling workflow to account for different core instances that have a substantial variation in size for their Amazon EBS volumes. This improvement applies to core nodes only; scale-down operations for task nodes aren’t affected. + The 6.14.0 release improves the way that Amazon EMR interacts with open-source applications such as Apache Hadoop YARN ResourceManager and HDFS NameNode. This improvement reduces the risk of operational delays with cluster scaling, and mitigates startup failures that occur due to connectivity issues with the open-source applications. + The 6.14.0 release optimizes application installation at cluster launch. This improves the cluster startup times for certain combinations of Amazon EMR applications. + The 6.14.0 release fixes an issue where cluster scale-down operations might stall when a cluster that's running in a VPC with a custom domain encounters a core or task node restart. + When you launch a cluster with *the latest patch release* of Amazon EMR 5.36 or higher, 6.6 or higher, or 7.0 or higher, Amazon EMR uses the latest Amazon Linux 2023 or Amazon Linux 2 release for the default Amazon EMR AMI. For more information, see [Using the default Amazon Linux AMI for Amazon EMR](https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-default-ami.html). [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-6140-release.html) ## 6.14.0 default Java versions Amazon EMR releases 6.12.0 and higher support all applications with Amazon Corretto 8 by default, except for Trino. For Trino, Amazon EMR supports Amazon Corretto 17 by default starting with Amazon EMR release 6.9.0. Amazon EMR also supports some applications with Amazon Corretto 11 and 17. Those applications are listed in the following table. If you want to change the default JVM on your cluster, follow the instructions in [Configure applications to use a specific Java Virtual Machine](configuring-java8.md) for each application that runs on the cluster. You can only use one Java runtime version for a cluster. Amazon EMR doesn't support running different nodes or applications on different runtime versions on the same cluster. While Amazon EMR supports both Amazon Corretto 11 and 17 on Apache Spark, Apache Hadoop, and Apache Hive, performance might regress for some workloads when you use these versions of Corretto. We recommend that you test your workloads before you change defaults. The following table shows the default Java versions for applications in Amazon EMR 6.14.0: | Application | Java / Amazon Corretto version (default is bold) | | --- | --- | | Delta | 17, 11, 8 | | Flink | 11, 8 | | Ganglia | 8 | | HBase | 11, 8 | | HCatalog | 17, 11, 8 | | Hadoop | 17, 11, 8 | | Hive | 17, 11, 8 | | Hudi | 17, 11, 8 | | Iceberg | 17, 11, 8 | | Livy | 17, 11, 8 | | Oozie | 17, 11, 8 | | Phoenix | 8 | | PrestoDB | 8 | | Spark | 17, 11, 8 | | Spark RAPIDS | 17, 11, 8 | | Sqoop | 8 | | Tez | 17, 11, 8 | | Trino | 17 | | Zeppelin | 8 | | Pig | 8 | | Zookeeper | 8 | ## 6.14.0 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | aws-sagemaker-spark-sdk | 1.4.2 | Amazon SageMaker Spark SDK | | delta | 2.4.0 | Delta lake is an open table format for huge analytic datasets | | delta-standalone-connectors | 0.6.0 | Delta Connectors provide different runtimes to integrate Delta Lake with engines like Flink, Hive and Presto. | | emr-ddb | 5.1.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 3.7.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.11.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-notebook-env | 1.7.0 | Conda env for emr notebook which includes jupyter enterprise gateway | | emr-s3-dist-cp | 2.28.0 | Distributed copy application optimized for Amazon S3. | | emr-s3-select | 2.7.0 | EMR S3Select Connector | | emr-wal-cli | 1.1.0 | Cli used for emrwal list/deletion. | | emrfs | 2.59.0 | Amazon S3 connector for Hadoop ecosystem applications. | | flink-client | 1.17.1-amzn-0 | Apache Flink command line client scripts and applications. | | flink-jobmanager-config | 1.17.1-amzn-0 | Managing resources on EMR nodes for Apache Flink JobManager. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 3.3.3-amzn-6 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 3.3.3-amzn-6 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 3.3.3-amzn-6 | HDFS command-line client and library | | hadoop-hdfs-namenode | 3.3.3-amzn-6 | HDFS service for tracking file names and block locations. | | hadoop-hdfs-journalnode | 3.3.3-amzn-6 | HDFS service for managing the Hadoop filesystem journal on HA clusters. | | hadoop-httpfs-server | 3.3.3-amzn-6 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 3.3.3-amzn-6 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 3.3.3-amzn-6 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 3.3.3-amzn-6 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 3.3.3-amzn-6 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 3.3.3-amzn-6 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 2.4.17-amzn-2 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 2.4.17-amzn-2 | Service for serving one or more HBase regions. | | hbase-client | 2.4.17-amzn-2 | HBase command-line client. | | hbase-rest-server | 2.4.17-amzn-2 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 2.4.17-amzn-2 | Service providing a Thrift endpoint to HBase. | | hbase-operator-tools | 2.4.17-amzn-2 | Repair tool for Apache HBase clusters. | | hcatalog-client | 3.1.3-amzn-7 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 3.1.3-amzn-7 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 3.1.3-amzn-7 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 3.1.3-amzn-7 | Hive command line client. | | hive-hbase | 3.1.3-amzn-7 | Hive-hbase client. | | hive-metastore-server | 3.1.3-amzn-7 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server2 | 3.1.3-amzn-7 | Service for accepting Hive queries as web requests. | | hudi | 0.13.1-amzn-2 | Incremental processing framework to power data pipeline at low latency and high efficiency. | | hudi-presto | 0.13.1-amzn-2 | Bundle library for running Presto with Hudi. | | hudi-trino | 0.13.1-amzn-2 | Bundle library for running Trino with Hudi. | | hudi-spark | 0.13.1-amzn-2 | Bundle library for running Spark with Hudi. | | hue-server | 4.11.0 | Web application for analyzing data using Hadoop ecosystem applications | | iceberg | 1.3.1-amzn-0 | Apache Iceberg is an open table format for huge analytic datasets | | jupyterhub | 1.5.0 | Multi-user server for Jupyter notebooks | | livy-server | 0.7.1-incubating | REST interface for interacting with Apache Spark | | nginx | 1.12.1 | nginx [engine x] is an HTTP and reverse proxy server | | mxnet | 1.9.1 | A flexible, scalable, and efficient library for deep learning. | | mariadb-server | 5.5.68\$1 | MariaDB database server. | | nvidia-cuda | 11.8.0 | Nvidia drivers and Cuda toolkit | | oozie-client | 5.2.1 | Oozie command-line client. | | oozie-server | 5.2.1 | Service for accepting Oozie workflow requests. | | opencv | 4.7.0 | Open Source Computer Vision Library. | | phoenix-library | 5.1.3 | The phoenix libraries for server and client | | phoenix-connectors | 5.1.3 | Apache Phoenix-Connectors for Spark-3 | | phoenix-query-server | 6.0.0 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.281-amzn-2 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.281-amzn-2 | Service for executing pieces of a query. | | presto-client | 0.281-amzn-2 | Presto command-line client which is installed on an HA cluster's stand-by masters where Presto server is not started. | | trino-coordinator | 422-amzn-0 | Service for accepting queries and managing query execution among trino-workers. | | trino-worker | 422-amzn-0 | Service for executing pieces of a query. | | trino-client | 422-amzn-0 | Trino command-line client which is installed on an HA cluster's stand-by masters where Trino server is not started. | | pig-client | 0.17.0 | Pig command-line client. | | r | 4.0.2 | The R Project for Statistical Computing | | ranger-kms-server | 2.0.0 | Apache Ranger Key Management System | | spark-client | 3.4.1-amzn-1 | Spark command-line clients. | | spark-history-server | 3.4.1-amzn-1 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 3.4.1-amzn-1 | In-memory execution engine for YARN. | | spark-yarn-slave | 3.4.1-amzn-1 | Apache Spark libraries needed by YARN slaves. | | spark-rapids | 23.06.0-amzn-2 | Nvidia Spark RAPIDS plugin that accelerates Apache Spark with GPUs. | | sqoop-client | 1.4.7 | Apache Sqoop command-line client. | | tensorflow | 2.11.0 | TensorFlow open source software library for high performance numerical computation. | | tez-on-yarn | 0.10.2-amzn-5 | The tez YARN application and libraries. | | tez-on-worker | 0.10.2-amzn-5 | The tez YARN application and libraries for worker nodes. | | webserver | 2.4.41\$1 | Apache HTTP server. | | zeppelin-server | 0.10.1 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.5.10 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.5.10 | ZooKeeper command line client. | ## 6.14.0 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). Reconfiguration actions occur when you specify a configuration for instance groups in a running cluster. Amazon EMR only initiates reconfiguration actions for the classifications that you modify. For more information, see [Reconfigure an instance group in a running cluster](emr-configure-apps-running-cluster.md). **emr-6.14.0 classifications** | Classifications | Description | Reconfiguration Actions | | --- | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | Restarts the ResourceManager service. | | container-executor | Change values in Hadoop YARN's container-executor.cfg file. | Not available. | | container-log4j | Change values in Hadoop YARN's container-log4j.properties file. | Not available. | | core-site | Change values in Hadoop's core-site.xml file. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts Hadoop KMS, Ranger KMS, HiveServer2, Hive MetaStore, Hadoop Httpfs, and MapReduce-HistoryServer. | | docker-conf | Change docker related settings. | Not available. | | emrfs-site | Change EMRFS settings. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts HBaseRegionserver, HBaseMaster, HBaseThrift, HBaseRest, HiveServer2, Hive MetaStore, Hadoop Httpfs, and MapReduce-HistoryServer. | | flink-conf | Change flink-conf.yaml settings. | Restarts Flink history server. | | flink-log4j | Change Flink log4j.properties settings. | Restarts Flink history server. | | flink-log4j-session | Change Flink log4j-session.properties settings for Kubernetes/Yarn session. | Restarts Flink history server. | | flink-log4j-cli | Change Flink log4j-cli.properties settings. | Restarts Flink history server. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts PhoenixQueryserver, HiveServer2, Hive MetaStore, and MapReduce-HistoryServer. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | Restarts the Hadoop HDFS services SecondaryNamenode, Datanode, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts Hadoop KMS, Hadoop Httpfs, and MapReduce-HistoryServer. | | hadoop-ssl-server | Change hadoop ssl server configuration | Not available. | | hadoop-ssl-client | Change hadoop ssl client configuration | Not available. | | hbase | Amazon EMR-curated settings for Apache HBase. | Custom EMR specific property. Sets emrfs-site and hbase-site configs. See those for their associated restarts. | | hbase-env | Change values in HBase's environment. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbase.properties file. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | Not available. | | hbase-site | Change values in HBase's hbase-site.xml file. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. Additionally restarts Phoenix QueryServer. | | hdfs-encryption-zones | Configure HDFS encryption zones. | This classification should not be reconfigured. | | hdfs-env | Change values in the HDFS environment. | Restarts Hadoop HDFS services Namenode, Datanode, and ZKFC. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Additionally restarts Hadoop Httpfs. | | hcatalog-env | Change values in HCatalog's environment. | Restarts Hive HCatalog Server. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | Restarts Hive HCatalog Server. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | Restarts Hive HCatalog Server. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | Restarts Hive WebHCat server. | | hcatalog-webhcat-log4j2 | Change values in HCatalog WebHCat's log4j2.properties. | Restarts Hive WebHCat server. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | Restarts Hive WebHCat server. | | hive | Amazon EMR-curated settings for Apache Hive. | Sets configurations to launch Hive LLAP service. | | hive-beeline-log4j2 | Change values in Hive's beeline-log4j2.properties file. | Not available. | | hive-parquet-logging | Change values in Hive's parquet-logging.properties file. | Not available. | | hive-env | Change values in the Hive environment. | Restarts HiveServer2, HiveMetastore, and Hive HCatalog-Server. Runs Hive schemaTool CLI commands to verify hive-metastore. | | hive-exec-log4j2 | Change values in Hive's hive-exec-log4j2.properties file. | Not available. | | hive-llap-daemon-log4j2 | Change values in Hive's llap-daemon-log4j2.properties file. | Not available. | | hive-log4j2 | Change values in Hive's hive-log4j2.properties file. | Not available. | | hive-site | Change values in Hive's hive-site.xml file | Restarts HiveServer2, HiveMetastore, and Hive HCatalog-Server. Runs Hive schemaTool CLI commands to verify hive-metastore. Also restarts Oozie and Zeppelin. | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | Not available. | | hue-ini | Change values in Hue's ini file | Restarts Hue. Also activates Hue config override CLI commands to pick up new configurations. | | httpfs-env | Change values in the HTTPFS environment. | Restarts Hadoop Httpfs service. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | Restarts Hadoop Httpfs service. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | Not available. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | Restarts Hadoop-KMS service. | | hadoop-kms-java-home | Change Hadoop's KMS java home | Not available. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | Not available. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | Restarts Hadoop-KMS and Ranger-KMS service. | | hudi-env | Change values in the Hudi environment. | Not available. | | hudi-defaults | Change values in Hudi's hudi-defaults.conf file. | Not available. | | iceberg-defaults | Change values in Iceberg's iceberg-defaults.conf file. | Not available. | | delta-defaults | Change values in Delta's delta-defaults.conf file. | Not available. | | jupyter-notebook-conf | Change values in Jupyter Notebook's jupyter\$1notebook\$1config.py file. | Not available. | | jupyter-hub-conf | Change values in JupyterHubs's jupyterhub\$1config.py file. | Not available. | | jupyter-s3-conf | Configure Jupyter Notebook S3 persistence. | Not available. | | jupyter-sparkmagic-conf | Change values in Sparkmagic's config.json file. | Not available. | | livy-conf | Change values in Livy's livy.conf file. | Restarts Livy Server. | | livy-env | Change values in the Livy environment. | Restarts Livy Server. | | livy-log4j2 | Change Livy log4j2.properties settings. | Restarts Livy Server. | | mapred-env | Change values in the MapReduce application's environment. | Restarts Hadoop MapReduce-HistoryServer. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | Restarts Hadoop MapReduce-HistoryServer. | | oozie-env | Change values in Oozie's environment. | Restarts Oozie. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | Restarts Oozie. | | oozie-site | Change values in Oozie's oozie-site.xml file. | Restarts Oozie. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | Not available. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | Not available. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | Restarts Phoenix-QueryServer. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | Not available. | | pig-env | Change values in the Pig environment. | Not available. | | pig-properties | Change values in Pig's pig.properties file. | Restarts Oozie. | | pig-log4j | Change values in Pig's log4j.properties file. | Not available. | | presto-log | Change values in Presto's log.properties file. | Restarts Presto-Server (for PrestoDB) | | presto-config | Change values in Presto's config.properties file. | Restarts Presto-Server (for PrestoDB) | | presto-password-authenticator | Change values in Presto's password-authenticator.properties file. | Not available. | | presto-env | Change values in Presto's presto-env.sh file. | Restarts Presto-Server (for PrestoDB) | | presto-node | Change values in Presto's node.properties file. | Not available. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | Not available. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | Not available. | | presto-connector-hive | Change values in Presto's hive.properties file. | Restarts Presto-Server (for PrestoDB) | | presto-connector-jmx | Change values in Presto's jmx.properties file. | Not available. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | Not available. | | presto-connector-lakeformation | Change values in Presto's lakeformation.properties file. | Restarts Presto-Server (for PrestoDB) | | presto-connector-localfile | Change values in Presto's localfile.properties file. | Not available. | | presto-connector-memory | Change values in Presto's memory.properties file. | Not available. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | Not available. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | Not available. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | Not available. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | Not available. | | presto-connector-redis | Change values in Presto's redis.properties file. | Not available. | | presto-connector-redshift | Change values in Presto's redshift.properties file. | Not available. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | Not available. | | presto-connector-tpcds | Change values in Presto's tpcds.properties file. | Not available. | | trino-log | Change values in Trino's log.properties file. | Restarts Trino-Server (for Trino) | | trino-config | Change values in Trino's config.properties file. | Restarts Trino-Server (for Trino) | | trino-password-authenticator | Change values in Trino's password-authenticator.properties file. | Restarts Trino-Server (for Trino) | | trino-env | Change values in Trino's trino-env.sh file. | Restarts Trino-Server (for Trino) | | trino-node | Change values in Trino's node.properties file. | Not available. | | trino-connector-blackhole | Change values in Trino's blackhole.properties file. | Not available. | | trino-connector-cassandra | Change values in Trino's cassandra.properties file. | Not available. | | trino-connector-delta | Change values in Trino's delta.properties file. | Restarts Trino-Server (for Trino) | | trino-connector-hive | Change values in Trino's hive.properties file. | Restarts Trino-Server (for Trino) | | trino-exchange-manager | Change values in Trino's exchange-manager.properties file. | Restarts Trino-Server (for Trino) | | trino-connector-iceberg | Change values in Trino's iceberg.properties file. | Restarts Trino-Server (for Trino) | | trino-connector-hudi | Change values in Trino's hudi.properties file. | Restarts Trino-Server (for Trino) | | trino-connector-jmx | Change values in Trino's jmx.properties file. | Not available. | | trino-connector-kafka | Change values in Trino's kafka.properties file. | Not available. | | trino-connector-localfile | Change values in Trino's localfile.properties file. | Not available. | | trino-connector-memory | Change values in Trino's memory.properties file. | Not available. | | trino-connector-mongodb | Change values in Trino's mongodb.properties file. | Not available. | | trino-connector-mysql | Change values in Trino's mysql.properties file. | Not available. | | trino-connector-postgresql | Change values in Trino's postgresql.properties file. | Not available. | | trino-connector-raptor | Change values in Trino's raptor.properties file. | Not available. | | trino-connector-redis | Change values in Trino's redis.properties file. | Not available. | | trino-connector-redshift | Change values in Trino's redshift.properties file. | Not available. | | trino-connector-tpch | Change values in Trino's tpch.properties file. | Not available. | | trino-connector-tpcds | Change values in Trino's tpcds.properties file. | Not available. | | ranger-kms-dbks-site | Change values in dbks-site.xml file of Ranger KMS. | Restarts Ranger KMS Server. | | ranger-kms-site | Change values in ranger-kms-site.xml file of Ranger KMS. | Restarts Ranger KMS Server. | | ranger-kms-env | Change values in the Ranger KMS environment. | Restarts Ranger KMS Server. | | ranger-kms-logback | Change values in kms-logback.xml file of Ranger KMS. | Not available. | | ranger-kms-db-ca | Change values for CA file on S3 for MySQL SSL connection with Ranger KMS. | Not available. | | spark | Amazon EMR-curated settings for Apache Spark. | This property modifies spark-defaults. See actions there. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | Restarts Spark history server and Spark thrift server. | | spark-env | Change values in the Spark environment. | Restarts Spark history server and Spark thrift server. | | spark-hive-site | Change values in Spark's hive-site.xml file | Not available. | | spark-log4j2 | Change values in Spark's log4j2.properties file. | Restarts Spark history server and Spark thrift server. | | spark-metrics | Change values in Spark's metrics.properties file. | Restarts Spark history server and Spark thrift server. | | sqoop-env | Change values in Sqoop's environment. | Not available. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | Not available. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | Not available. | | tez-site | Change values in Tez's tez-site.xml file. | Restart Oozie and HiveServer2. | | yarn-env | Change values in the YARN environment. | Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts MapReduce-HistoryServer. | | yarn-site | Change values in YARN's yarn-site.xml file. | Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts Livy Server and MapReduce-HistoryServer. | | zeppelin-env | Change values in the Zeppelin environment. | Restarts Zeppelin. | | zeppelin-site | Change configuration settings in zeppelin-site.xml. | Restarts Zeppelin. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | Restarts Zookeeper server. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | Restarts Zookeeper server. | ## 6.14.0 change log **Change log for 6.14.0 release and release notes** | Date | Event | Description | | --- | --- | --- | | 2025-09-03 | Docs revision | Amazon EMR 6.14.0 release notes added known issue | | *2023-11-02* | Deployment complete | Amazon EMR 6.14.0 fully deployed to all [supported Regions](https://aws.amazon.com/about-aws/global-infrastructure/regional-product-services/) | | 2023-10-10 | Docs publication | Amazon EMR 6.14.0 release notes first published | | 2023-10-04 | Initial release | Amazon EMR 6.14.0 first deployed to initial commercial Regions | # Amazon EMR release 6.13.0 ## 6.13.0 application versions This release includes the following applications: [https://delta.io/](https://delta.io/), [https://flink.apache.org/](https://flink.apache.org/), [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [https://hudi.apache.org](https://hudi.apache.org), [http://gethue.com/](http://gethue.com/), [https://iceberg.apache.org/](https://iceberg.apache.org/), [https://jupyter-enterprise-gateway.readthedocs.io/en/latest/](https://jupyter-enterprise-gateway.readthedocs.io/en/latest/), [https://jupyterhub.readthedocs.io/en/latest/#](https://jupyterhub.readthedocs.io/en/latest/#), [https://livy.incubator.apache.org/](https://livy.incubator.apache.org/), [https://mxnet.incubator.apache.org/](https://mxnet.incubator.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://www.tensorflow.org/](https://www.tensorflow.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://trino.io/](https://trino.io/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-6.13.0 | emr-6.12.0 | emr-6.11.1 | emr-6.11.0 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.12.513 | 1.12.490 | 1.12.446 | 1.12.446 | | Python | 2.7, 3.7 | 2.7, 3.7 | 2.7, 3.7 | 2.7, 3.7 | | Scala | 2.12.15 | 2.12.15 | 2.12.15 | 2.12.15 | | AmazonCloudWatchAgent | - | - | - | - | | Delta | 2.4.0 | 2.4.0 | 2.2.0 | 2.2.0 | | Flink | 1.17.0 | 1.17.0 | 1.16.0 | 1.16.0 | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 2.4.17-amzn-1 | 2.4.17-amzn-0 | 2.4.15-amzn-1.1 | 2.4.15-amzn-1 | | HCatalog | 3.1.3-amzn-6 | 3.1.3-amzn-5 | 3.1.3-amzn-4.1 | 3.1.3-amzn-4 | | Hadoop | 3.3.3-amzn-5 | 3.3.3-amzn-4 | 3.3.3-amzn-3.1 | 3.3.3-amzn-3 | | Hive | 3.1.3-amzn-6 | 3.1.3-amzn-5 | 3.1.3-amzn-4.1 | 3.1.3-amzn-4 | | Hudi | 0.13.1-amzn-1 | 0.13.1-amzn-0 | 0.13.0-amzn-0 | 0.13.0-amzn-0 | | Hue | 4.11.0 | 4.11.0 | 4.11.0 | 4.11.0 | | Iceberg | 1.3.0-amzn-1 | 1.3.0-amzn-0 | 1.2.0-amzn-0 | 1.2.0-amzn-0 | | JupyterEnterpriseGateway | 2.6.0 | 2.6.0 | 2.6.0 | 2.6.0 | | JupyterHub | 1.5.0 | 1.4.1 | 1.4.1 | 1.4.1 | | Livy | 0.7.1-incubating | 0.7.1-incubating | 0.7.1-incubating | 0.7.1-incubating | | MXNet | 1.9.1 | 1.9.1 | 1.9.1 | 1.9.1 | | Mahout | - | - | - | - | | Oozie | 5.2.1 | 5.2.1 | 5.2.1 | 5.2.1 | | Phoenix | 5.1.3 | 5.1.3 | 5.1.2 | 5.1.2 | | Pig | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | | Presto | 0.281-amzn-1 | 0.281-amzn-0 | 0.279-amzn-0 | 0.279-amzn-0 | | Spark | 3.4.1-amzn-0 | 3.4.0-amzn-0 | 3.3.2-amzn-0.1 | 3.3.2-amzn-0 | | Sqoop | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | | TensorFlow | 2.11.0 | 2.11.0 | 2.11.0 | 2.11.0 | | Tez | 0.10.2-amzn-4 | 0.10.2-amzn-3 | 0.10.2-amzn-2.1 | 0.10.2-amzn-2 | | Trino (PrestoSQL) | 414-amzn-1 | 414-amzn-0 | 410-amzn-0 | 410-amzn-0 | | Zeppelin | 0.10.1 | 0.10.1 | 0.10.1 | 0.10.1 | | ZooKeeper | 3.5.10 | 3.5.10 | 3.5.10 | 3.5.10 | ## 6.13.0 release notes The following release notes include information for Amazon EMR release 6.13.0. Changes are relative to 6.12.0. For information on the release timeline, see the [6.13.0 change log](#6130-changelog). **New features** + Amazon EMR 6.13.0 supports Apache Spark 3.4.1, Apache Spark RAPIDS 23.06.0-amzn-1, CUDA Toolkit 11.8.0, and JupyterHub 1.5.0. **Known issues** + An on-cluster instance-state script that monitors health of the instance can consume excessive CPU and memory resources when there are a large number of threads and/or open file handles on the node. **Changes, enhancements, and resolved issues** + Starting with Spark 3.3.1 (supported in EMR versions 6.10 and above), all executors in a decommissioning host are set to a new `ExecutorState`, called *DECOMMISSIONING* state. The executors being decommissioned cannot be used by Yarn to allocate tasks and thus it will request for new executors, if needed, for the tasks being executed. Thus, if you disable Spark DRA while using EMR Managed Scaling, EMR Auto Scaling, or any custom scaling mechanism on EMR-EC2 clusters, then Yarn may request maximum permissible executors for each job. In order to avoid this issue, leave the `spark.dynamicAllocation.enabled` property set to `TRUE` (which is the default) when you are using the above combination of features. In addition, you can also set minimum and maximum executor constraints by setting values for `spark.dynamicAllocation.maxExecutors` and `spark.dynamicAllocation.minExecutors` properties for your Spark jobs, to restrict the number of executors allocated during the job’s execution. + The 6.13.0 release improves the Amazon EMR log management daemon to ensure that all logs are uploaded at a regular cadence to Amazon S3 when a cluster termination command is issued. This facilitates faster cluster terminations. + The 6.13.0 release enhances Amazon EMR log management capabilities to ensure consistent and timely upload of all log files to Amazon S3. This especially benefits long-running EMR clusters. + When you launch a cluster with *the latest patch release* of Amazon EMR 5.36 or higher, 6.6 or higher, or 7.0 or higher, Amazon EMR uses the latest Amazon Linux 2023 or Amazon Linux 2 release for the default Amazon EMR AMI. For more information, see [Using the default Amazon Linux AMI for Amazon EMR](https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-default-ami.html). [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-6130-release.html) ## 6.13.0 default Java versions Amazon EMR releases 6.12.0 and higher support all applications with Amazon Corretto 8 by default, except for Trino. For Trino, Amazon EMR supports Amazon Corretto 17 by default starting with Amazon EMR release 6.9.0. Amazon EMR also supports some applications with Amazon Corretto 11 and 17. Those applications are listed in the following table. If you want to change the default JVM on your cluster, follow the instructions in [Configure applications to use a specific Java Virtual Machine](configuring-java8.md) for each application that runs on the cluster. You can only use one Java runtime version for a cluster. Amazon EMR doesn't support running different nodes or applications on different runtime versions on the same cluster. While Amazon EMR supports both Amazon Corretto 11 and 17 on Apache Spark, Apache Hadoop, and Apache Hive, performance might regress for some workloads when you use these versions of Corretto. We recommend that you test your workloads before you change defaults. The following table shows the default Java versions for applications in Amazon EMR 6.13.0: | Application | Java / Amazon Corretto version (default is bold) | | --- | --- | | Delta | 17, 11, 8 | | Flink | 11, 8 | | Ganglia | 8 | | HBase | 11, 8 | | HCatalog | 17, 11, 8 | | Hadoop | 17, 11, 8 | | Hive | 17, 11, 8 | | Hudi | 17, 11, 8 | | Iceberg | 17, 11, 8 | | Livy | 17, 11, 8 | | Oozie | 17, 11, 8 | | Phoenix | 8 | | PrestoDB | 8 | | Spark | 17, 11, 8 | | Spark RAPIDS | 17, 11, 8 | | Sqoop | 8 | | Tez | 17, 11, 8 | | Trino | 17 | | Zeppelin | 8 | | Pig | 8 | | Zookeeper | 8 | ## 6.13.0 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | aws-sagemaker-spark-sdk | 1.4.2 | Amazon SageMaker Spark SDK | | delta | 2.4.0 | Delta lake is an open table format for huge analytic datasets | | delta-standalone-connectors | 0.6.0 | Delta Connectors provide different runtimes to integrate Delta Lake with engines like Flink, Hive and Presto. | | emr-ddb | 5.1.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 3.6.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.10.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-notebook-env | 1.7.0 | Conda env for emr notebook which includes jupyter enterprise gateway | | emr-s3-dist-cp | 2.27.0 | Distributed copy application optimized for Amazon S3. | | emr-s3-select | 2.6.0 | EMR S3Select Connector | | emr-wal-cli | 1.1.0 | Cli used for emrwal list/deletion. | | emrfs | 2.58.0 | Amazon S3 connector for Hadoop ecosystem applications. | | flink-client | 1.17.0 | Apache Flink command line client scripts and applications. | | flink-jobmanager-config | 1.17.0 | Managing resources on EMR nodes for Apache Flink JobManager. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 3.3.3-amzn-5 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 3.3.3-amzn-5 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 3.3.3-amzn-5 | HDFS command-line client and library | | hadoop-hdfs-namenode | 3.3.3-amzn-5 | HDFS service for tracking file names and block locations. | | hadoop-hdfs-journalnode | 3.3.3-amzn-5 | HDFS service for managing the Hadoop filesystem journal on HA clusters. | | hadoop-httpfs-server | 3.3.3-amzn-5 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 3.3.3-amzn-5 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 3.3.3-amzn-5 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 3.3.3-amzn-5 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 3.3.3-amzn-5 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 3.3.3-amzn-5 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 2.4.17-amzn-1 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 2.4.17-amzn-1 | Service for serving one or more HBase regions. | | hbase-client | 2.4.17-amzn-1 | HBase command-line client. | | hbase-rest-server | 2.4.17-amzn-1 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 2.4.17-amzn-1 | Service providing a Thrift endpoint to HBase. | | hbase-operator-tools | 2.4.17-amzn-1 | Repair tool for Apache HBase clusters. | | hcatalog-client | 3.1.3-amzn-6 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 3.1.3-amzn-6 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 3.1.3-amzn-6 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 3.1.3-amzn-6 | Hive command line client. | | hive-hbase | 3.1.3-amzn-6 | Hive-hbase client. | | hive-metastore-server | 3.1.3-amzn-6 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server2 | 3.1.3-amzn-6 | Service for accepting Hive queries as web requests. | | hudi | 0.13.1-amzn-1 | Incremental processing framework to power data pipeline at low latency and high efficiency. | | hudi-presto | 0.13.1-amzn-1 | Bundle library for running Presto with Hudi. | | hudi-trino | 0.13.1-amzn-1 | Bundle library for running Trino with Hudi. | | hudi-spark | 0.13.1-amzn-1 | Bundle library for running Spark with Hudi. | | hue-server | 4.11.0 | Web application for analyzing data using Hadoop ecosystem applications | | iceberg | 1.3.0-amzn-1 | Apache Iceberg is an open table format for huge analytic datasets | | jupyterhub | 1.5.0 | Multi-user server for Jupyter notebooks | | livy-server | 0.7.1-incubating | REST interface for interacting with Apache Spark | | nginx | 1.12.1 | nginx [engine x] is an HTTP and reverse proxy server | | mxnet | 1.9.1 | A flexible, scalable, and efficient library for deep learning. | | mariadb-server | 5.5.68\$1 | MariaDB database server. | | nvidia-cuda | 11.8.0 | Nvidia drivers and Cuda toolkit | | oozie-client | 5.2.1 | Oozie command-line client. | | oozie-server | 5.2.1 | Service for accepting Oozie workflow requests. | | opencv | 4.7.0 | Open Source Computer Vision Library. | | phoenix-library | 5.1.3 | The phoenix libraries for server and client | | phoenix-connectors | 5.1.3 | Apache Phoenix-Connectors for Spark-3 | | phoenix-query-server | 6.0.0 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.281-amzn-1 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.281-amzn-1 | Service for executing pieces of a query. | | presto-client | 0.281-amzn-1 | Presto command-line client which is installed on an HA cluster's stand-by masters where Presto server is not started. | | trino-coordinator | 414-amzn-1 | Service for accepting queries and managing query execution among trino-workers. | | trino-worker | 414-amzn-1 | Service for executing pieces of a query. | | trino-client | 414-amzn-1 | Trino command-line client which is installed on an HA cluster's stand-by masters where Trino server is not started. | | pig-client | 0.17.0 | Pig command-line client. | | r | 4.0.2 | The R Project for Statistical Computing | | ranger-kms-server | 2.0.0 | Apache Ranger Key Management System | | spark-client | 3.4.1-amzn-0 | Spark command-line clients. | | spark-history-server | 3.4.1-amzn-0 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 3.4.1-amzn-0 | In-memory execution engine for YARN. | | spark-yarn-slave | 3.4.1-amzn-0 | Apache Spark libraries needed by YARN slaves. | | spark-rapids | 23.06.0-amzn-1 | Nvidia Spark RAPIDS plugin that accelerates Apache Spark with GPUs. | | sqoop-client | 1.4.7 | Apache Sqoop command-line client. | | tensorflow | 2.11.0 | TensorFlow open source software library for high performance numerical computation. | | tez-on-yarn | 0.10.2-amzn-4 | The tez YARN application and libraries. | | tez-on-worker | 0.10.2-amzn-4 | The tez YARN application and libraries for worker nodes. | | webserver | 2.4.41\$1 | Apache HTTP server. | | zeppelin-server | 0.10.1 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.5.10 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.5.10 | ZooKeeper command line client. | ## 6.13.0 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). Reconfiguration actions occur when you specify a configuration for instance groups in a running cluster. Amazon EMR only initiates reconfiguration actions for the classifications that you modify. For more information, see [Reconfigure an instance group in a running cluster](emr-configure-apps-running-cluster.md). **emr-6.13.0 classifications** | Classifications | Description | Reconfiguration Actions | | --- | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | Restarts the ResourceManager service. | | container-executor | Change values in Hadoop YARN's container-executor.cfg file. | Not available. | | container-log4j | Change values in Hadoop YARN's container-log4j.properties file. | Not available. | | core-site | Change values in Hadoop's core-site.xml file. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts Hadoop KMS, Ranger KMS, HiveServer2, Hive MetaStore, Hadoop Httpfs, and MapReduce-HistoryServer. | | docker-conf | Change docker related settings. | Not available. | | emrfs-site | Change EMRFS settings. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts HBaseRegionserver, HBaseMaster, HBaseThrift, HBaseRest, HiveServer2, Hive MetaStore, Hadoop Httpfs, and MapReduce-HistoryServer. | | flink-conf | Change flink-conf.yaml settings. | Restarts Flink history server. | | flink-log4j | Change Flink log4j.properties settings. | Restarts Flink history server. | | flink-log4j-session | Change Flink log4j-session.properties settings for Kubernetes/Yarn session. | Restarts Flink history server. | | flink-log4j-cli | Change Flink log4j-cli.properties settings. | Restarts Flink history server. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts PhoenixQueryserver, HiveServer2, Hive MetaStore, and MapReduce-HistoryServer. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | Restarts the Hadoop HDFS services SecondaryNamenode, Datanode, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts Hadoop KMS, Hadoop Httpfs, and MapReduce-HistoryServer. | | hadoop-ssl-server | Change hadoop ssl server configuration | Not available. | | hadoop-ssl-client | Change hadoop ssl client configuration | Not available. | | hbase | Amazon EMR-curated settings for Apache HBase. | Custom EMR specific property. Sets emrfs-site and hbase-site configs. See those for their associated restarts. | | hbase-env | Change values in HBase's environment. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbase.properties file. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | Not available. | | hbase-site | Change values in HBase's hbase-site.xml file. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. Additionally restarts Phoenix QueryServer. | | hdfs-encryption-zones | Configure HDFS encryption zones. | This classification should not be reconfigured. | | hdfs-env | Change values in the HDFS environment. | Restarts Hadoop HDFS services Namenode, Datanode, and ZKFC. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Additionally restarts Hadoop Httpfs. | | hcatalog-env | Change values in HCatalog's environment. | Restarts Hive HCatalog Server. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | Restarts Hive HCatalog Server. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | Restarts Hive HCatalog Server. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | Restarts Hive WebHCat server. | | hcatalog-webhcat-log4j2 | Change values in HCatalog WebHCat's log4j2.properties. | Restarts Hive WebHCat server. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | Restarts Hive WebHCat server. | | hive | Amazon EMR-curated settings for Apache Hive. | Sets configurations to launch Hive LLAP service. | | hive-beeline-log4j2 | Change values in Hive's beeline-log4j2.properties file. | Not available. | | hive-parquet-logging | Change values in Hive's parquet-logging.properties file. | Not available. | | hive-env | Change values in the Hive environment. | Restarts HiveServer2, HiveMetastore, and Hive HCatalog-Server. Runs Hive schemaTool CLI commands to verify hive-metastore. | | hive-exec-log4j2 | Change values in Hive's hive-exec-log4j2.properties file. | Not available. | | hive-llap-daemon-log4j2 | Change values in Hive's llap-daemon-log4j2.properties file. | Not available. | | hive-log4j2 | Change values in Hive's hive-log4j2.properties file. | Not available. | | hive-site | Change values in Hive's hive-site.xml file | Restarts HiveServer2, HiveMetastore, and Hive HCatalog-Server. Runs Hive schemaTool CLI commands to verify hive-metastore. Also restarts Oozie and Zeppelin. | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | Not available. | | hue-ini | Change values in Hue's ini file | Restarts Hue. Also activates Hue config override CLI commands to pick up new configurations. | | httpfs-env | Change values in the HTTPFS environment. | Restarts Hadoop Httpfs service. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | Restarts Hadoop Httpfs service. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | Not available. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | Restarts Hadoop-KMS service. | | hadoop-kms-java-home | Change Hadoop's KMS java home | Not available. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | Not available. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | Restarts Hadoop-KMS and Ranger-KMS service. | | hudi-env | Change values in the Hudi environment. | Not available. | | hudi-defaults | Change values in Hudi's hudi-defaults.conf file. | Not available. | | iceberg-defaults | Change values in Iceberg's iceberg-defaults.conf file. | Not available. | | delta-defaults | Change values in Delta's delta-defaults.conf file. | Not available. | | jupyter-notebook-conf | Change values in Jupyter Notebook's jupyter\$1notebook\$1config.py file. | Not available. | | jupyter-hub-conf | Change values in JupyterHubs's jupyterhub\$1config.py file. | Not available. | | jupyter-s3-conf | Configure Jupyter Notebook S3 persistence. | Not available. | | jupyter-sparkmagic-conf | Change values in Sparkmagic's config.json file. | Not available. | | livy-conf | Change values in Livy's livy.conf file. | Restarts Livy Server. | | livy-env | Change values in the Livy environment. | Restarts Livy Server. | | livy-log4j2 | Change Livy log4j2.properties settings. | Restarts Livy Server. | | mapred-env | Change values in the MapReduce application's environment. | Restarts Hadoop MapReduce-HistoryServer. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | Restarts Hadoop MapReduce-HistoryServer. | | oozie-env | Change values in Oozie's environment. | Restarts Oozie. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | Restarts Oozie. | | oozie-site | Change values in Oozie's oozie-site.xml file. | Restarts Oozie. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | Not available. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | Not available. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | Restarts Phoenix-QueryServer. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | Not available. | | pig-env | Change values in the Pig environment. | Not available. | | pig-properties | Change values in Pig's pig.properties file. | Restarts Oozie. | | pig-log4j | Change values in Pig's log4j.properties file. | Not available. | | presto-log | Change values in Presto's log.properties file. | Restarts Presto-Server (for PrestoDB) | | presto-config | Change values in Presto's config.properties file. | Restarts Presto-Server (for PrestoDB) | | presto-password-authenticator | Change values in Presto's password-authenticator.properties file. | Not available. | | presto-env | Change values in Presto's presto-env.sh file. | Restarts Presto-Server (for PrestoDB) | | presto-node | Change values in Presto's node.properties file. | Not available. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | Not available. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | Not available. | | presto-connector-hive | Change values in Presto's hive.properties file. | Restarts Presto-Server (for PrestoDB) | | presto-connector-jmx | Change values in Presto's jmx.properties file. | Not available. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | Not available. | | presto-connector-lakeformation | Change values in Presto's lakeformation.properties file. | Restarts Presto-Server (for PrestoDB) | | presto-connector-localfile | Change values in Presto's localfile.properties file. | Not available. | | presto-connector-memory | Change values in Presto's memory.properties file. | Not available. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | Not available. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | Not available. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | Not available. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | Not available. | | presto-connector-redis | Change values in Presto's redis.properties file. | Not available. | | presto-connector-redshift | Change values in Presto's redshift.properties file. | Not available. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | Not available. | | presto-connector-tpcds | Change values in Presto's tpcds.properties file. | Not available. | | trino-log | Change values in Trino's log.properties file. | Restarts Trino-Server (for Trino) | | trino-config | Change values in Trino's config.properties file. | Restarts Trino-Server (for Trino) | | trino-password-authenticator | Change values in Trino's password-authenticator.properties file. | Restarts Trino-Server (for Trino) | | trino-env | Change values in Trino's trino-env.sh file. | Restarts Trino-Server (for Trino) | | trino-node | Change values in Trino's node.properties file. | Not available. | | trino-connector-blackhole | Change values in Trino's blackhole.properties file. | Not available. | | trino-connector-cassandra | Change values in Trino's cassandra.properties file. | Not available. | | trino-connector-delta | Change values in Trino's delta.properties file. | Restarts Trino-Server (for Trino) | | trino-connector-hive | Change values in Trino's hive.properties file. | Restarts Trino-Server (for Trino) | | trino-exchange-manager | Change values in Trino's exchange-manager.properties file. | Restarts Trino-Server (for Trino) | | trino-connector-iceberg | Change values in Trino's iceberg.properties file. | Restarts Trino-Server (for Trino) | | trino-connector-hudi | Change values in Trino's hudi.properties file. | Restarts Trino-Server (for Trino) | | trino-connector-jmx | Change values in Trino's jmx.properties file. | Not available. | | trino-connector-kafka | Change values in Trino's kafka.properties file. | Not available. | | trino-connector-localfile | Change values in Trino's localfile.properties file. | Not available. | | trino-connector-memory | Change values in Trino's memory.properties file. | Not available. | | trino-connector-mongodb | Change values in Trino's mongodb.properties file. | Not available. | | trino-connector-mysql | Change values in Trino's mysql.properties file. | Not available. | | trino-connector-postgresql | Change values in Trino's postgresql.properties file. | Not available. | | trino-connector-raptor | Change values in Trino's raptor.properties file. | Not available. | | trino-connector-redis | Change values in Trino's redis.properties file. | Not available. | | trino-connector-redshift | Change values in Trino's redshift.properties file. | Not available. | | trino-connector-tpch | Change values in Trino's tpch.properties file. | Not available. | | trino-connector-tpcds | Change values in Trino's tpcds.properties file. | Not available. | | ranger-kms-dbks-site | Change values in dbks-site.xml file of Ranger KMS. | Restarts Ranger KMS Server. | | ranger-kms-site | Change values in ranger-kms-site.xml file of Ranger KMS. | Restarts Ranger KMS Server. | | ranger-kms-env | Change values in the Ranger KMS environment. | Restarts Ranger KMS Server. | | ranger-kms-logback | Change values in kms-logback.xml file of Ranger KMS. | Not available. | | ranger-kms-db-ca | Change values for CA file on S3 for MySQL SSL connection with Ranger KMS. | Not available. | | spark | Amazon EMR-curated settings for Apache Spark. | This property modifies spark-defaults. See actions there. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | Restarts Spark history server and Spark thrift server. | | spark-env | Change values in the Spark environment. | Restarts Spark history server and Spark thrift server. | | spark-hive-site | Change values in Spark's hive-site.xml file | Not available. | | spark-log4j2 | Change values in Spark's log4j2.properties file. | Restarts Spark history server and Spark thrift server. | | spark-metrics | Change values in Spark's metrics.properties file. | Restarts Spark history server and Spark thrift server. | | sqoop-env | Change values in Sqoop's environment. | Not available. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | Not available. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | Not available. | | tez-site | Change values in Tez's tez-site.xml file. | Restart Oozie and HiveServer2. | | yarn-env | Change values in the YARN environment. | Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts MapReduce-HistoryServer. | | yarn-site | Change values in YARN's yarn-site.xml file. | Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts Livy Server and MapReduce-HistoryServer. | | zeppelin-env | Change values in the Zeppelin environment. | Restarts Zeppelin. | | zeppelin-site | Change configuration settings in zeppelin-site.xml. | Restarts Zeppelin. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | Restarts Zookeeper server. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | Restarts Zookeeper server. | ## 6.13.0 change log **Change log for 6.13.0 release and release notes** | Date | Event | Description | | --- | --- | --- | | 2025-09-03 | Docs revision | Amazon EMR 6.13.0 release notes added known issue | | *2023-09-23* | Deployment complete | Amazon EMR 6.13.0 fully deployed to all [supported Regions](https://aws.amazon.com/about-aws/global-infrastructure/regional-product-services/) | | 2023-09-12 | Docs publication | Amazon EMR 6.13.0 release notes first published | | 2023-09-01 | Initial release | Amazon EMR 6.13.0 first deployed to initial commercial Regions | # Amazon EMR release 6.12.0 ## 6.12.0 application versions This release includes the following applications: [https://delta.io/](https://delta.io/), [https://flink.apache.org/](https://flink.apache.org/), [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [https://hudi.apache.org](https://hudi.apache.org), [http://gethue.com/](http://gethue.com/), [https://iceberg.apache.org/](https://iceberg.apache.org/), [https://jupyter-enterprise-gateway.readthedocs.io/en/latest/](https://jupyter-enterprise-gateway.readthedocs.io/en/latest/), [https://jupyterhub.readthedocs.io/en/latest/#](https://jupyterhub.readthedocs.io/en/latest/#), [https://livy.incubator.apache.org/](https://livy.incubator.apache.org/), [https://mxnet.incubator.apache.org/](https://mxnet.incubator.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://www.tensorflow.org/](https://www.tensorflow.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://trino.io/](https://trino.io/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-6.12.0 | emr-6.11.1 | emr-6.11.0 | emr-6.10.1 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.12.490 | 1.12.446 | 1.12.446 | 1.12.397 | | Python | 2.7, 3.7 | 2.7, 3.7 | 2.7, 3.7 | 2.7, 3.7 | | Scala | 2.12.15 | 2.12.15 | 2.12.15 | 2.12.15 | | AmazonCloudWatchAgent | - | - | - | - | | Delta | 2.4.0 | 2.2.0 | 2.2.0 | 2.2.0 | | Flink | 1.17.0 | 1.16.0 | 1.16.0 | 1.16.0 | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 2.4.17-amzn-0 | 2.4.15-amzn-1.1 | 2.4.15-amzn-1 | 2.4.15-amzn-0.1 | | HCatalog | 3.1.3-amzn-5 | 3.1.3-amzn-4.1 | 3.1.3-amzn-4 | 3.1.3-amzn-3.1 | | Hadoop | 3.3.3-amzn-4 | 3.3.3-amzn-3.1 | 3.3.3-amzn-3 | 3.3.3-amzn-2.1 | | Hive | 3.1.3-amzn-5 | 3.1.3-amzn-4.1 | 3.1.3-amzn-4 | 3.1.3-amzn-3.1 | | Hudi | 0.13.1-amzn-0 | 0.13.0-amzn-0 | 0.13.0-amzn-0 | 0.12.2-amzn-0 | | Hue | 4.11.0 | 4.11.0 | 4.11.0 | 4.10.0 | | Iceberg | 1.3.0-amzn-0 | 1.2.0-amzn-0 | 1.2.0-amzn-0 | 1.1.0-amzn-0 | | JupyterEnterpriseGateway | 2.6.0 | 2.6.0 | 2.6.0 | 2.6.0 | | JupyterHub | 1.4.1 | 1.4.1 | 1.4.1 | 1.5.0 | | Livy | 0.7.1-incubating | 0.7.1-incubating | 0.7.1-incubating | 0.7.1-incubating | | MXNet | 1.9.1 | 1.9.1 | 1.9.1 | 1.9.1 | | Mahout | - | - | - | - | | Oozie | 5.2.1 | 5.2.1 | 5.2.1 | 5.2.1 | | Phoenix | 5.1.3 | 5.1.2 | 5.1.2 | 5.1.2 | | Pig | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | | Presto | 0.281-amzn-0 | 0.279-amzn-0 | 0.279-amzn-0 | 0.278.1-amzn-0 | | Spark | 3.4.0-amzn-0 | 3.3.2-amzn-0.1 | 3.3.2-amzn-0 | 3.3.1-amzn-0.1 | | Sqoop | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | | TensorFlow | 2.11.0 | 2.11.0 | 2.11.0 | 2.11.0 | | Tez | 0.10.2-amzn-3 | 0.10.2-amzn-2.1 | 0.10.2-amzn-2 | 0.10.2-amzn-1.1 | | Trino (PrestoSQL) | 414-amzn-0 | 410-amzn-0 | 410-amzn-0 | 403-amzn-0 | | Zeppelin | 0.10.1 | 0.10.1 | 0.10.1 | 0.10.1 | | ZooKeeper | 3.5.10 | 3.5.10 | 3.5.10 | 3.5.10 | ## 6.12.0 release notes The following release notes include information for Amazon EMR release 6.12.0. Changes are relative to 6.11.0. For information on the release timeline, see the [6.12.0 change log](#6120-changelog). **New features** + Amazon EMR 6.12.0 supports Apache Spark 3.4.0, Apache Spark RAPIDS 23.06.0-amzn-0, CUDA 11.8.0, Apache Hudi 0.13.1-amzn-0, Apache Iceberg 1.3.0-amzn-0, Trino 414, and PrestoDB 0.281. + Amazon EMR releases 6.12.0 and higher support LDAP integration with Apache Livy, Apache Hive through HiveServer2 (HS2), Trino, Presto, and Hue. You can also install Apache Spark and Apache Hadoop on an EMR cluster that uses 6.12.0 or higher and configure them to use LDAP. For more information, see [Use Active Directory or LDAP servers for authentication with Amazon EMR](https://docs.aws.amazon.com/emr/latest/ManagementGuide/ldap.html). **Known issues** + An on-cluster instance-state script that monitors health of the instance can consume excessive CPU and memory resources when there are a large number of threads and/or open file handles on the node. **Changes, enhancements, and resolved issues** + Starting with Spark 3.3.1 (supported in EMR versions 6.10 and above), all executors in a decommissioning host are set to a new `ExecutorState`, called *DECOMMISSIONING* state. The executors being decommissioned cannot be used by Yarn to allocate tasks and thus it will request for new executors, if needed, for the tasks being executed. Thus, if you disable Spark DRA while using EMR Managed Scaling, EMR Auto Scaling, or any custom scaling mechanism on EMR-EC2 clusters, then Yarn may request maximum permissible executors for each job. In order to avoid this issue, leave the `spark.dynamicAllocation.enabled` property set to `TRUE` (which is the default) when you are using the above combination of features. In addition, you can also set minimum and maximum executor constraints by setting values for `spark.dynamicAllocation.maxExecutors` and `spark.dynamicAllocation.minExecutors` properties for your Spark jobs, to restrict the number of executors allocated during the job’s execution. + Amazon EMR releases 6.12.0 and higher provide Java 11 runtime support for Flink. For more information, see [Configure Flink to run with Java 11](flink-configure.md#flink-configure-java11). + The 6.12.0 release adds a new retry mechanism to the cluster scaling workflow for EMR clusters that run Presto or Trino. This improvement reduces the risk that cluster resizing will indefinitely stall due to a single failed resize operation. It also improves cluster utilization, because your cluster scales up and down faster. + The 6.12.0 release fixes an issue where cluster scale-down operations might stall when a core node that is undergoing graceful decommissioning turns unhealthy for any reason before it fully decommissions. + The 6.12.0 release improves cluster scale-down logic so that your cluster doesn't attempt a scale-down of core nodes below the HDFS replication factor setting for the cluster. This aligns with your data redundancy requirements, and reduces the chance that a scaling operation might stall. + The 6.12.0 release enhances the performance and efficiency of the health monitoring service for Amazon EMR by increasing the speed at which it logs state changes for instances. This improvement reduces the chance of degraded performance for cluster nodes that are running multiple custom client tools or third-party applications. + The 6.12.0 release improves the performance of the on-cluster log management daemon for Amazon EMR. As a result, there is less chance for degraded performance with EMR clusters that run steps with high concurrency. + With Amazon EMR release 6.12.0, the log management daemon has been upgraded to identify all logs that are in active use with open file handles on the local instance storage, and the associated processes. This upgrade ensures that Amazon EMR properly deletes the files and reclaims storage space after the logs are archived to Amazon S3. + The 6.12.0 release includes a log-management daemon enhancement that deletes empty, unused steps directories in the local cluster file system. An excessively large number of empty directories can degrade the performance of Amazon EMR daemons and result in disk over-utilization. + The 6.12.0 release enables log rotation for YARN Timeline Server logs. This minimizes disk over-utilization scenarios, especially for long-running clusters. + The default root volume size has increased to 15 GB in Amazon EMR 6.10.0 and higher. Earlier releases have default root volume size of 10 GB. + When you launch a cluster with *the latest patch release* of Amazon EMR 5.36 or higher, 6.6 or higher, or 7.0 or higher, Amazon EMR uses the latest Amazon Linux 2023 or Amazon Linux 2 release for the default Amazon EMR AMI. For more information, see [Using the default Amazon Linux AMI for Amazon EMR](https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-default-ami.html). [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-6120-release.html) ## 6.12.0 default Java versions Amazon EMR releases 6.12.0 and higher support all applications with Amazon Corretto 8 by default, except for Trino. For Trino, Amazon EMR supports Amazon Corretto 17 by default starting with Amazon EMR release 6.9.0. Amazon EMR also supports some applications with Amazon Corretto 11 and 17. Those applications are listed in the following table. If you want to change the default JVM on your cluster, follow the instructions in [Configure applications to use a specific Java Virtual Machine](configuring-java8.md) for each application that runs on the cluster. You can only use one Java runtime version for a cluster. Amazon EMR doesn't support running different nodes or applications on different runtime versions on the same cluster. While Amazon EMR supports both Amazon Corretto 11 and 17 on Apache Spark, Apache Hadoop, and Apache Hive, performance might regress for some workloads when you use these versions of Corretto. We recommend that you test your workloads before you change defaults. The following table shows the default Java versions for applications in Amazon EMR 6.12.0: | Application | Java / Amazon Corretto version (default is bold) | | --- | --- | | Delta | 17, 11, 8 | | Flink | 11, 8 | | Ganglia | 8 | | HBase | 11, 8 | | HCatalog | 17, 11, 8 | | Hadoop | 17, 11, 8 | | Hive | 17, 11, 8 | | Hudi | 17, 11, 8 | | Iceberg | 17, 11, 8 | | Livy | 17, 11, 8 | | Oozie | 17, 11, 8 | | Phoenix | 8 | | PrestoDB | 8 | | Spark | 17, 11, 8 | | Spark RAPIDS | 17, 11, 8 | | Sqoop | 8 | | Tez | 17, 11, 8 | | Trino | 17 | | Zeppelin | 8 | | Pig | 8 | | Zookeeper | 8 | ## 6.12.0 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | aws-sagemaker-spark-sdk | 1.4.2 | Amazon SageMaker Spark SDK | | delta | 2.4.0 | Delta lake is an open table format for huge analytic datasets | | delta-standalone-connectors | 0.6.0 | Delta Connectors provide different runtimes to integrate Delta Lake with engines like Flink, Hive and Presto. | | emr-ddb | 5.1.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 3.5.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.9.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-notebook-env | 1.7.0 | Conda env for emr notebook which includes jupyter enterprise gateway | | emr-s3-dist-cp | 2.26.0 | Distributed copy application optimized for Amazon S3. | | emr-s3-select | 2.5.0 | EMR S3Select Connector | | emr-wal-cli | 1.1.0 | Cli used for emrwal list/deletion. | | emrfs | 2.57.0 | Amazon S3 connector for Hadoop ecosystem applications. | | flink-client | 1.17.0 | Apache Flink command line client scripts and applications. | | flink-jobmanager-config | 1.17.0 | Managing resources on EMR nodes for Apache Flink JobManager. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 3.3.3-amzn-4 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 3.3.3-amzn-4 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 3.3.3-amzn-4 | HDFS command-line client and library | | hadoop-hdfs-namenode | 3.3.3-amzn-4 | HDFS service for tracking file names and block locations. | | hadoop-hdfs-journalnode | 3.3.3-amzn-4 | HDFS service for managing the Hadoop filesystem journal on HA clusters. | | hadoop-httpfs-server | 3.3.3-amzn-4 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 3.3.3-amzn-4 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 3.3.3-amzn-4 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 3.3.3-amzn-4 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 3.3.3-amzn-4 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 3.3.3-amzn-4 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 2.4.17-amzn-0 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 2.4.17-amzn-0 | Service for serving one or more HBase regions. | | hbase-client | 2.4.17-amzn-0 | HBase command-line client. | | hbase-rest-server | 2.4.17-amzn-0 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 2.4.17-amzn-0 | Service providing a Thrift endpoint to HBase. | | hbase-operator-tools | 2.4.17-amzn-0 | Repair tool for Apache HBase clusters. | | hcatalog-client | 3.1.3-amzn-5 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 3.1.3-amzn-5 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 3.1.3-amzn-5 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 3.1.3-amzn-5 | Hive command line client. | | hive-hbase | 3.1.3-amzn-5 | Hive-hbase client. | | hive-metastore-server | 3.1.3-amzn-5 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server2 | 3.1.3-amzn-5 | Service for accepting Hive queries as web requests. | | hudi | 0.13.1-amzn-0 | Incremental processing framework to power data pipeline at low latency and high efficiency. | | hudi-presto | 0.13.1-amzn-0 | Bundle library for running Presto with Hudi. | | hudi-trino | 0.13.1-amzn-0 | Bundle library for running Trino with Hudi. | | hudi-spark | 0.13.1-amzn-0 | Bundle library for running Spark with Hudi. | | hue-server | 4.11.0 | Web application for analyzing data using Hadoop ecosystem applications | | iceberg | 1.3.0-amzn-0 | Apache Iceberg is an open table format for huge analytic datasets | | jupyterhub | 1.4.1 | Multi-user server for Jupyter notebooks | | livy-server | 0.7.1-incubating | REST interface for interacting with Apache Spark | | nginx | 1.12.1 | nginx [engine x] is an HTTP and reverse proxy server | | mxnet | 1.9.1 | A flexible, scalable, and efficient library for deep learning. | | mariadb-server | 5.5.68\$1 | MariaDB database server. | | nvidia-cuda | 11.8.0 | Nvidia drivers and Cuda toolkit | | oozie-client | 5.2.1 | Oozie command-line client. | | oozie-server | 5.2.1 | Service for accepting Oozie workflow requests. | | opencv | 4.7.0 | Open Source Computer Vision Library. | | phoenix-library | 5.1.3 | The phoenix libraries for server and client | | phoenix-connectors | 5.1.3 | Apache Phoenix-Connectors for Spark-3 | | phoenix-query-server | 6.0.0 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.281-amzn-0 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.281-amzn-0 | Service for executing pieces of a query. | | presto-client | 0.281-amzn-0 | Presto command-line client which is installed on an HA cluster's stand-by masters where Presto server is not started. | | trino-coordinator | 414-amzn-0 | Service for accepting queries and managing query execution among trino-workers. | | trino-worker | 414-amzn-0 | Service for executing pieces of a query. | | trino-client | 414-amzn-0 | Trino command-line client which is installed on an HA cluster's stand-by masters where Trino server is not started. | | pig-client | 0.17.0 | Pig command-line client. | | r | 4.0.2 | The R Project for Statistical Computing | | ranger-kms-server | 2.0.0 | Apache Ranger Key Management System | | spark-client | 3.4.0-amzn-0 | Spark command-line clients. | | spark-history-server | 3.4.0-amzn-0 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 3.4.0-amzn-0 | In-memory execution engine for YARN. | | spark-yarn-slave | 3.4.0-amzn-0 | Apache Spark libraries needed by YARN slaves. | | spark-rapids | 23.06.0-amzn-0 | Nvidia Spark RAPIDS plugin that accelerates Apache Spark with GPUs. | | sqoop-client | 1.4.7 | Apache Sqoop command-line client. | | tensorflow | 2.11.0 | TensorFlow open source software library for high performance numerical computation. | | tez-on-yarn | 0.10.2-amzn-3 | The tez YARN application and libraries. | | tez-on-worker | 0.10.2-amzn-3 | The tez YARN application and libraries for worker nodes. | | webserver | 2.4.41\$1 | Apache HTTP server. | | zeppelin-server | 0.10.1 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.5.10 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.5.10 | ZooKeeper command line client. | ## 6.12.0 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). Reconfiguration actions occur when you specify a configuration for instance groups in a running cluster. Amazon EMR only initiates reconfiguration actions for the classifications that you modify. For more information, see [Reconfigure an instance group in a running cluster](emr-configure-apps-running-cluster.md). **emr-6.12.0 classifications** | Classifications | Description | Reconfiguration Actions | | --- | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | Restarts the ResourceManager service. | | container-executor | Change values in Hadoop YARN's container-executor.cfg file. | Not available. | | container-log4j | Change values in Hadoop YARN's container-log4j.properties file. | Not available. | | core-site | Change values in Hadoop's core-site.xml file. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts Hadoop KMS, Ranger KMS, HiveServer2, Hive MetaStore, Hadoop Httpfs, and MapReduce-HistoryServer. | | docker-conf | Change docker related settings. | Not available. | | emrfs-site | Change EMRFS settings. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts HBaseRegionserver, HBaseMaster, HBaseThrift, HBaseRest, HiveServer2, Hive MetaStore, Hadoop Httpfs, and MapReduce-HistoryServer. | | flink-conf | Change flink-conf.yaml settings. | Restarts Flink history server. | | flink-log4j | Change Flink log4j.properties settings. | Restarts Flink history server. | | flink-log4j-session | Change Flink log4j-session.properties settings for Kubernetes/Yarn session. | Restarts Flink history server. | | flink-log4j-cli | Change Flink log4j-cli.properties settings. | Restarts Flink history server. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts PhoenixQueryserver, HiveServer2, Hive MetaStore, and MapReduce-HistoryServer. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | Restarts the Hadoop HDFS services SecondaryNamenode, Datanode, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts Hadoop KMS, Hadoop Httpfs, and MapReduce-HistoryServer. | | hadoop-ssl-server | Change hadoop ssl server configuration | Not available. | | hadoop-ssl-client | Change hadoop ssl client configuration | Not available. | | hbase | Amazon EMR-curated settings for Apache HBase. | Custom EMR specific property. Sets emrfs-site and hbase-site configs. See those for their associated restarts. | | hbase-env | Change values in HBase's environment. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbase.properties file. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | Not available. | | hbase-site | Change values in HBase's hbase-site.xml file. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. Additionally restarts Phoenix QueryServer. | | hdfs-encryption-zones | Configure HDFS encryption zones. | This classification should not be reconfigured. | | hdfs-env | Change values in the HDFS environment. | Restarts Hadoop HDFS services Namenode, Datanode, and ZKFC. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Additionally restarts Hadoop Httpfs. | | hcatalog-env | Change values in HCatalog's environment. | Restarts Hive HCatalog Server. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | Restarts Hive HCatalog Server. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | Restarts Hive HCatalog Server. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | Restarts Hive WebHCat server. | | hcatalog-webhcat-log4j2 | Change values in HCatalog WebHCat's log4j2.properties. | Restarts Hive WebHCat server. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | Restarts Hive WebHCat server. | | hive | Amazon EMR-curated settings for Apache Hive. | Sets configurations to launch Hive LLAP service. | | hive-beeline-log4j2 | Change values in Hive's beeline-log4j2.properties file. | Not available. | | hive-parquet-logging | Change values in Hive's parquet-logging.properties file. | Not available. | | hive-env | Change values in the Hive environment. | Restarts HiveServer2, HiveMetastore, and Hive HCatalog-Server. Runs Hive schemaTool CLI commands to verify hive-metastore. | | hive-exec-log4j2 | Change values in Hive's hive-exec-log4j2.properties file. | Not available. | | hive-llap-daemon-log4j2 | Change values in Hive's llap-daemon-log4j2.properties file. | Not available. | | hive-log4j2 | Change values in Hive's hive-log4j2.properties file. | Not available. | | hive-site | Change values in Hive's hive-site.xml file | Restarts HiveServer2, HiveMetastore, and Hive HCatalog-Server. Runs Hive schemaTool CLI commands to verify hive-metastore. Also restarts Oozie and Zeppelin. | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | Not available. | | hue-ini | Change values in Hue's ini file | Restarts Hue. Also activates Hue config override CLI commands to pick up new configurations. | | httpfs-env | Change values in the HTTPFS environment. | Restarts Hadoop Httpfs service. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | Restarts Hadoop Httpfs service. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | Not available. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | Restarts Hadoop-KMS service. | | hadoop-kms-java-home | Change Hadoop's KMS java home | Not available. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | Not available. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | Restarts Hadoop-KMS and Ranger-KMS service. | | hudi-env | Change values in the Hudi environment. | Not available. | | hudi-defaults | Change values in Hudi's hudi-defaults.conf file. | Not available. | | iceberg-defaults | Change values in Iceberg's iceberg-defaults.conf file. | Not available. | | delta-defaults | Change values in Delta's delta-defaults.conf file. | Not available. | | jupyter-notebook-conf | Change values in Jupyter Notebook's jupyter\$1notebook\$1config.py file. | Not available. | | jupyter-hub-conf | Change values in JupyterHubs's jupyterhub\$1config.py file. | Not available. | | jupyter-s3-conf | Configure Jupyter Notebook S3 persistence. | Not available. | | jupyter-sparkmagic-conf | Change values in Sparkmagic's config.json file. | Not available. | | livy-conf | Change values in Livy's livy.conf file. | Restarts Livy Server. | | livy-env | Change values in the Livy environment. | Restarts Livy Server. | | livy-log4j2 | Change Livy log4j2.properties settings. | Restarts Livy Server. | | mapred-env | Change values in the MapReduce application's environment. | Restarts Hadoop MapReduce-HistoryServer. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | Restarts Hadoop MapReduce-HistoryServer. | | oozie-env | Change values in Oozie's environment. | Restarts Oozie. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | Restarts Oozie. | | oozie-site | Change values in Oozie's oozie-site.xml file. | Restarts Oozie. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | Not available. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | Not available. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | Restarts Phoenix-QueryServer. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | Not available. | | pig-env | Change values in the Pig environment. | Not available. | | pig-properties | Change values in Pig's pig.properties file. | Restarts Oozie. | | pig-log4j | Change values in Pig's log4j.properties file. | Not available. | | presto-log | Change values in Presto's log.properties file. | Restarts Presto-Server (for PrestoDB) | | presto-config | Change values in Presto's config.properties file. | Restarts Presto-Server (for PrestoDB) | | presto-password-authenticator | Change values in Presto's password-authenticator.properties file. | Not available. | | presto-env | Change values in Presto's presto-env.sh file. | Restarts Presto-Server (for PrestoDB) | | presto-node | Change values in Presto's node.properties file. | Not available. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | Not available. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | Not available. | | presto-connector-hive | Change values in Presto's hive.properties file. | Restarts Presto-Server (for PrestoDB) | | presto-connector-jmx | Change values in Presto's jmx.properties file. | Not available. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | Not available. | | presto-connector-lakeformation | Change values in Presto's lakeformation.properties file. | Restarts Presto-Server (for PrestoDB) | | presto-connector-localfile | Change values in Presto's localfile.properties file. | Not available. | | presto-connector-memory | Change values in Presto's memory.properties file. | Not available. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | Not available. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | Not available. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | Not available. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | Not available. | | presto-connector-redis | Change values in Presto's redis.properties file. | Not available. | | presto-connector-redshift | Change values in Presto's redshift.properties file. | Not available. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | Not available. | | presto-connector-tpcds | Change values in Presto's tpcds.properties file. | Not available. | | trino-log | Change values in Trino's log.properties file. | Restarts Trino-Server (for Trino) | | trino-config | Change values in Trino's config.properties file. | Restarts Trino-Server (for Trino) | | trino-password-authenticator | Change values in Trino's password-authenticator.properties file. | Restarts Trino-Server (for Trino) | | trino-env | Change values in Trino's trino-env.sh file. | Restarts Trino-Server (for Trino) | | trino-node | Change values in Trino's node.properties file. | Not available. | | trino-connector-blackhole | Change values in Trino's blackhole.properties file. | Not available. | | trino-connector-cassandra | Change values in Trino's cassandra.properties file. | Not available. | | trino-connector-delta | Change values in Trino's delta.properties file. | Restarts Trino-Server (for Trino) | | trino-connector-hive | Change values in Trino's hive.properties file. | Restarts Trino-Server (for Trino) | | trino-exchange-manager | Change values in Trino's exchange-manager.properties file. | Restarts Trino-Server (for Trino) | | trino-connector-iceberg | Change values in Trino's iceberg.properties file. | Restarts Trino-Server (for Trino) | | trino-connector-hudi | Change values in Trino's hudi.properties file. | Restarts Trino-Server (for Trino) | | trino-connector-jmx | Change values in Trino's jmx.properties file. | Not available. | | trino-connector-kafka | Change values in Trino's kafka.properties file. | Not available. | | trino-connector-localfile | Change values in Trino's localfile.properties file. | Not available. | | trino-connector-memory | Change values in Trino's memory.properties file. | Not available. | | trino-connector-mongodb | Change values in Trino's mongodb.properties file. | Not available. | | trino-connector-mysql | Change values in Trino's mysql.properties file. | Not available. | | trino-connector-postgresql | Change values in Trino's postgresql.properties file. | Not available. | | trino-connector-raptor | Change values in Trino's raptor.properties file. | Not available. | | trino-connector-redis | Change values in Trino's redis.properties file. | Not available. | | trino-connector-redshift | Change values in Trino's redshift.properties file. | Not available. | | trino-connector-tpch | Change values in Trino's tpch.properties file. | Not available. | | trino-connector-tpcds | Change values in Trino's tpcds.properties file. | Not available. | | ranger-kms-dbks-site | Change values in dbks-site.xml file of Ranger KMS. | Restarts Ranger KMS Server. | | ranger-kms-site | Change values in ranger-kms-site.xml file of Ranger KMS. | Restarts Ranger KMS Server. | | ranger-kms-env | Change values in the Ranger KMS environment. | Restarts Ranger KMS Server. | | ranger-kms-logback | Change values in kms-logback.xml file of Ranger KMS. | Not available. | | ranger-kms-db-ca | Change values for CA file on S3 for MySQL SSL connection with Ranger KMS. | Not available. | | spark | Amazon EMR-curated settings for Apache Spark. | This property modifies spark-defaults. See actions there. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | Restarts Spark history server and Spark thrift server. | | spark-env | Change values in the Spark environment. | Restarts Spark history server and Spark thrift server. | | spark-hive-site | Change values in Spark's hive-site.xml file | Not available. | | spark-log4j2 | Change values in Spark's log4j2.properties file. | Restarts Spark history server and Spark thrift server. | | spark-metrics | Change values in Spark's metrics.properties file. | Restarts Spark history server and Spark thrift server. | | sqoop-env | Change values in Sqoop's environment. | Not available. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | Not available. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | Not available. | | tez-site | Change values in Tez's tez-site.xml file. | Restart Oozie and HiveServer2. | | yarn-env | Change values in the YARN environment. | Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts MapReduce-HistoryServer. | | yarn-site | Change values in YARN's yarn-site.xml file. | Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts Livy Server and MapReduce-HistoryServer. | | zeppelin-env | Change values in the Zeppelin environment. | Restarts Zeppelin. | | zeppelin-site | Change configuration settings in zeppelin-site.xml. | Restarts Zeppelin. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | Restarts Zookeeper server. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | Restarts Zookeeper server. | ## 6.12.0 change log **Change log for 6.12.0 release and release notes** | Date | Event | Description | | --- | --- | --- | | 2025-09-03 | Docs revision | Amazon EMR 6.12.0 release notes added known issue | | 2023-07-27 | Update documentation | Update the Java options for 6.12 and add Oozie tutorial to update JVM | | 2023-07-21 | Deployment complete | Amazon EMR 6.12.0 fully deployed to all [supported Regions](https://aws.amazon.com/about-aws/global-infrastructure/regional-product-services/) | | 2023-07-21 | Docs publication | Amazon EMR 6.12.0 release notes first published | | 2023-07-12 | Initial release | Amazon EMR 6.12.0 first deployed to initial commercial Regions | # Amazon EMR release 6.11.1 ## 6.11.1 application versions This release includes the following applications: [https://delta.io/](https://delta.io/), [https://flink.apache.org/](https://flink.apache.org/), [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [https://hudi.apache.org](https://hudi.apache.org), [http://gethue.com/](http://gethue.com/), [https://iceberg.apache.org/](https://iceberg.apache.org/), [https://jupyter-enterprise-gateway.readthedocs.io/en/latest/](https://jupyter-enterprise-gateway.readthedocs.io/en/latest/), [https://jupyterhub.readthedocs.io/en/latest/#](https://jupyterhub.readthedocs.io/en/latest/#), [https://livy.incubator.apache.org/](https://livy.incubator.apache.org/), [https://mxnet.incubator.apache.org/](https://mxnet.incubator.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://www.tensorflow.org/](https://www.tensorflow.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://trino.io/](https://trino.io/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-6.11.1 | emr-6.11.0 | emr-6.10.1 | emr-6.10.0 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.12.446 | 1.12.446 | 1.12.397 | 1.12.397 | | Python | 2.7, 3.7 | 2.7, 3.7 | 2.7, 3.7 | 2.7, 3.7 | | Scala | 2.12.15 | 2.12.15 | 2.12.15 | 2.12.15 | | AmazonCloudWatchAgent | - | - | - | - | | Delta | 2.2.0 | 2.2.0 | 2.2.0 | 2.2.0 | | Flink | 1.16.0 | 1.16.0 | 1.16.0 | 1.16.0 | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 2.4.15-amzn-1.1 | 2.4.15-amzn-1 | 2.4.15-amzn-0.1 | 2.4.15-amzn-0 | | HCatalog | 3.1.3-amzn-4.1 | 3.1.3-amzn-4 | 3.1.3-amzn-3.1 | 3.1.3-amzn-3 | | Hadoop | 3.3.3-amzn-3.1 | 3.3.3-amzn-3 | 3.3.3-amzn-2.1 | 3.3.3-amzn-2 | | Hive | 3.1.3-amzn-4.1 | 3.1.3-amzn-4 | 3.1.3-amzn-3.1 | 3.1.3-amzn-3 | | Hudi | 0.13.0-amzn-0 | 0.13.0-amzn-0 | 0.12.2-amzn-0 | 0.12.2-amzn-0 | | Hue | 4.11.0 | 4.11.0 | 4.10.0 | 4.10.0 | | Iceberg | 1.2.0-amzn-0 | 1.2.0-amzn-0 | 1.1.0-amzn-0 | 1.1.0-amzn-0 | | JupyterEnterpriseGateway | 2.6.0 | 2.6.0 | 2.6.0 | 2.6.0 | | JupyterHub | 1.4.1 | 1.4.1 | 1.5.0 | 1.5.0 | | Livy | 0.7.1-incubating | 0.7.1-incubating | 0.7.1-incubating | 0.7.1-incubating | | MXNet | 1.9.1 | 1.9.1 | 1.9.1 | 1.9.1 | | Mahout | - | - | - | - | | Oozie | 5.2.1 | 5.2.1 | 5.2.1 | 5.2.1 | | Phoenix | 5.1.2 | 5.1.2 | 5.1.2 | 5.1.2 | | Pig | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | | Presto | 0.279-amzn-0 | 0.279-amzn-0 | 0.278.1-amzn-0 | 0.278.1-amzn-0 | | Spark | 3.3.2-amzn-0.1 | 3.3.2-amzn-0 | 3.3.1-amzn-0.1 | 3.3.1-amzn-0 | | Sqoop | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | | TensorFlow | 2.11.0 | 2.11.0 | 2.11.0 | 2.11.0 | | Tez | 0.10.2-amzn-2.1 | 0.10.2-amzn-2 | 0.10.2-amzn-1.1 | 0.10.2-amzn-1 | | Trino (PrestoSQL) | 410-amzn-0 | 410-amzn-0 | 403-amzn-0 | 403-amzn-0 | | Zeppelin | 0.10.1 | 0.10.1 | 0.10.1 | 0.10.1 | | ZooKeeper | 3.5.10 | 3.5.10 | 3.5.10 | 3.5.10 | ## 6.11.1 release notes The following release notes include information for Amazon EMR release 6.11.1. Changes are relative to 6.11.0. For information on the release timeline, see the [6.11.1 change log](#6111-changelog). **Changes, enhancements, and resolved issues** + Starting with Spark 3.3.1 (supported in EMR versions 6.10 and above), all executors in a decommissioning host are set to a new `ExecutorState`, called *DECOMMISSIONING* state. The executors being decommissioned cannot be used by Yarn to allocate tasks and thus it will request for new executors, if needed, for the tasks being executed. Thus, if you disable Spark DRA while using EMR Managed Scaling, EMR Auto Scaling, or any custom scaling mechanism on EMR-EC2 clusters, then Yarn may request maximum permissible executors for each job. In order to avoid this issue, leave the `spark.dynamicAllocation.enabled` property set to `TRUE` (which is the default) when you are using the above combination of features. In addition, you can also set minimum and maximum executor constraints by setting values for `spark.dynamicAllocation.maxExecutors` and `spark.dynamicAllocation.minExecutors` properties for your Spark jobs, to restrict the number of executors allocated during the job’s execution. + Due to lock contention, a node can enter into a deadlock if it's added or removed at the same time that it attempts to decommission. As a result, the Hadoop Resource Manager (YARN) becomes unresponsive, and affects all the incoming and currently-running containers. + This release includes a change that allows high-availability clusters to recover from failed state after restart. + This release includes security fixes for Hue and HBase. + This release fixes an issue where clusters that are running workloads on Spark with Amazon EMR might silently receive incorrect results with `contains`, `startsWith`, `endsWith`, and `like`. This issue occurs when you use the expressions on partitioned fields that have metadata in the Amazon EMR Hive3 Metastore Server (HMS). + This release fixes an issue with throttling on the Glue side when there are no user-defined functions (UDF). + This release fixes an issue that deletes container logs by the node log aggregation service before log pusher can push them to S3 in case of YARN decommissioning. + This release fixes an issue with FairShare Scheduler metrics when Node Label is enabled for Hadoop. + This release fixes an issue that impacted Spark performance when you set a default `true` value for the `spark.yarn.heterogeneousExecutors.enabled` config in `spark-defaults.conf`. + This release fixes an issue with Reduce Task failing to read shuffle data. The issue caused Hive query failures with a corrupted memory error. + This release adds a new retry mechanism to the cluster scaling workflow for EMR clusters that run Presto or Trino. This improvement reduces the risk that cluster resizing will indefinitely stall due to a single failed resize operation. It also improves cluster utilization, because your cluster scales up and down faster. + This release improves cluster scale-down logic so that your cluster doesn't attempt a scale-down of core nodes below the HDFS replication factor setting for the cluster. This aligns with your data redundancy requirements, and reduces the chance that a scaling operation might stall. + The log management daemon has been upgraded to identify all logs that are in active use with open file handles on the local instance storage, and the associated processes. This upgrade ensures that Amazon EMR properly deletes the files and reclaims storage space after the logs are archived to Amazon S3. + This release includes a log-management daemon enhancement that deletes empty, unused steps directories in the local cluster file system. An excessively large number of empty directories can degrade the performance of Amazon EMR daemons and result in disk over-utilization. + When you launch a cluster with *the latest patch release* of Amazon EMR 5.36 or higher, 6.6 or higher, or 7.0 or higher, Amazon EMR uses the latest Amazon Linux 2023 or Amazon Linux 2 release for the default Amazon EMR AMI. For more information, see [Using the default Amazon Linux AMI for Amazon EMR](https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-default-ami.html). [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-6111-release.html) ## 6.11.1 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | aws-sagemaker-spark-sdk | 1.4.2 | Amazon SageMaker Spark SDK | | delta | 2.2.0 | Delta lake is an open table format for huge analytic datasets | | delta-standalone-connectors | 0.6.0 | Delta Connectors provide different runtimes to integrate Delta Lake with engines like Flink, Hive and Presto. | | emr-ddb | 5.1.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 3.4.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.8.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-notebook-env | 1.7.0 | Conda env for emr notebook which includes jupyter enterprise gateway | | emr-s3-dist-cp | 2.25.0 | Distributed copy application optimized for Amazon S3. | | emr-s3-select | 2.4.0 | EMR S3Select Connector | | emr-wal-cli | 1.1.0 | Cli used for emrwal list/deletion. | | emrfs | 2.56.0 | Amazon S3 connector for Hadoop ecosystem applications. | | flink-client | 1.16.0 | Apache Flink command line client scripts and applications. | | flink-jobmanager-config | 1.16.0 | Managing resources on EMR nodes for Apache Flink JobManager. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 3.3.3-amzn-3.1 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 3.3.3-amzn-3.1 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 3.3.3-amzn-3.1 | HDFS command-line client and library | | hadoop-hdfs-namenode | 3.3.3-amzn-3.1 | HDFS service for tracking file names and block locations. | | hadoop-hdfs-journalnode | 3.3.3-amzn-3.1 | HDFS service for managing the Hadoop filesystem journal on HA clusters. | | hadoop-httpfs-server | 3.3.3-amzn-3.1 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 3.3.3-amzn-3.1 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 3.3.3-amzn-3.1 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 3.3.3-amzn-3.1 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 3.3.3-amzn-3.1 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 3.3.3-amzn-3.1 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 2.4.15-amzn-1.1 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 2.4.15-amzn-1.1 | Service for serving one or more HBase regions. | | hbase-client | 2.4.15-amzn-1.1 | HBase command-line client. | | hbase-rest-server | 2.4.15-amzn-1.1 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 2.4.15-amzn-1.1 | Service providing a Thrift endpoint to HBase. | | hbase-operator-tools | 2.4.15-amzn-1.1 | Repair tool for Apache HBase clusters. | | hcatalog-client | 3.1.3-amzn-4.1 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 3.1.3-amzn-4.1 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 3.1.3-amzn-4.1 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 3.1.3-amzn-4.1 | Hive command line client. | | hive-hbase | 3.1.3-amzn-4.1 | Hive-hbase client. | | hive-metastore-server | 3.1.3-amzn-4.1 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server2 | 3.1.3-amzn-4.1 | Service for accepting Hive queries as web requests. | | hudi | 0.13.0-amzn-0 | Incremental processing framework to power data pipeline at low latency and high efficiency. | | hudi-presto | 0.13.0-amzn-0 | Bundle library for running Presto with Hudi. | | hudi-trino | 0.13.0-amzn-0 | Bundle library for running Trino with Hudi. | | hudi-spark | 0.13.0-amzn-0 | Bundle library for running Spark with Hudi. | | hue-server | 4.11.0 | Web application for analyzing data using Hadoop ecosystem applications | | iceberg | 1.2.0-amzn-0 | Apache Iceberg is an open table format for huge analytic datasets | | jupyterhub | 1.4.1 | Multi-user server for Jupyter notebooks | | livy-server | 0.7.1-incubating | REST interface for interacting with Apache Spark | | nginx | 1.12.1 | nginx [engine x] is an HTTP and reverse proxy server | | mxnet | 1.9.1 | A flexible, scalable, and efficient library for deep learning. | | mariadb-server | 5.5.68\$1 | MariaDB database server. | | nvidia-cuda | 11.8.0 | Nvidia drivers and Cuda toolkit | | oozie-client | 5.2.1 | Oozie command-line client. | | oozie-server | 5.2.1 | Service for accepting Oozie workflow requests. | | opencv | 4.5.0 | Open Source Computer Vision Library. | | phoenix-library | 5.1.2 | The phoenix libraries for server and client | | phoenix-connectors | 5.1.2 | Apache Phoenix-Connectors for Spark-3 | | phoenix-query-server | 6.0.0 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.279-amzn-0 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.279-amzn-0 | Service for executing pieces of a query. | | presto-client | 0.279-amzn-0 | Presto command-line client which is installed on an HA cluster's stand-by masters where Presto server is not started. | | trino-coordinator | 410-amzn-0 | Service for accepting queries and managing query execution among trino-workers. | | trino-worker | 410-amzn-0 | Service for executing pieces of a query. | | trino-client | 410-amzn-0 | Trino command-line client which is installed on an HA cluster's stand-by masters where Trino server is not started. | | pig-client | 0.17.0 | Pig command-line client. | | r | 4.0.2 | The R Project for Statistical Computing | | ranger-kms-server | 2.0.0 | Apache Ranger Key Management System | | spark-client | 3.3.2-amzn-0.1 | Spark command-line clients. | | spark-history-server | 3.3.2-amzn-0.1 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 3.3.2-amzn-0.1 | In-memory execution engine for YARN. | | spark-yarn-slave | 3.3.2-amzn-0.1 | Apache Spark libraries needed by YARN slaves. | | spark-rapids | 23.02.0-amzn-0 | Nvidia Spark RAPIDS plugin that accelerates Apache Spark with GPUs. | | sqoop-client | 1.4.7 | Apache Sqoop command-line client. | | tensorflow | 2.11.0 | TensorFlow open source software library for high performance numerical computation. | | tez-on-yarn | 0.10.2-amzn-2.1 | The tez YARN application and libraries. | | tez-on-worker | 0.10.2-amzn-2.1 | The tez YARN application and libraries for worker nodes. | | webserver | 2.4.41\$1 | Apache HTTP server. | | zeppelin-server | 0.10.1 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.5.10 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.5.10 | ZooKeeper command line client. | ## 6.11.1 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). Reconfiguration actions occur when you specify a configuration for instance groups in a running cluster. Amazon EMR only initiates reconfiguration actions for the classifications that you modify. For more information, see [Reconfigure an instance group in a running cluster](emr-configure-apps-running-cluster.md). **emr-6.11.1 classifications** | Classifications | Description | Reconfiguration Actions | | --- | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | Restarts the ResourceManager service. | | container-executor | Change values in Hadoop YARN's container-executor.cfg file. | Not available. | | container-log4j | Change values in Hadoop YARN's container-log4j.properties file. | Not available. | | core-site | Change values in Hadoop's core-site.xml file. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts Hadoop KMS, Ranger KMS, HiveServer2, Hive MetaStore, Hadoop Httpfs, and MapReduce-HistoryServer. | | docker-conf | Change docker related settings. | Not available. | | emrfs-site | Change EMRFS settings. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts HBaseRegionserver, HBaseMaster, HBaseThrift, HBaseRest, HiveServer2, Hive MetaStore, Hadoop Httpfs, and MapReduce-HistoryServer. | | flink-conf | Change flink-conf.yaml settings. | Restarts Flink history server. | | flink-log4j | Change Flink log4j.properties settings. | Restarts Flink history server. | | flink-log4j-session | Change Flink log4j-session.properties settings for Kubernetes/Yarn session. | Restarts Flink history server. | | flink-log4j-cli | Change Flink log4j-cli.properties settings. | Restarts Flink history server. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts PhoenixQueryserver, HiveServer2, Hive MetaStore, and MapReduce-HistoryServer. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | Restarts the Hadoop HDFS services SecondaryNamenode, Datanode, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts Hadoop KMS, Hadoop Httpfs, and MapReduce-HistoryServer. | | hadoop-ssl-server | Change hadoop ssl server configuration | Not available. | | hadoop-ssl-client | Change hadoop ssl client configuration | Not available. | | hbase | Amazon EMR-curated settings for Apache HBase. | Custom EMR specific property. Sets emrfs-site and hbase-site configs. See those for their associated restarts. | | hbase-env | Change values in HBase's environment. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbase.properties file. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | Not available. | | hbase-site | Change values in HBase's hbase-site.xml file. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. Additionally restarts Phoenix QueryServer. | | hdfs-encryption-zones | Configure HDFS encryption zones. | This classification should not be reconfigured. | | hdfs-env | Change values in the HDFS environment. | Restarts Hadoop HDFS services Namenode, Datanode, and ZKFC. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Additionally restarts Hadoop Httpfs. | | hcatalog-env | Change values in HCatalog's environment. | Restarts Hive HCatalog Server. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | Restarts Hive HCatalog Server. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | Restarts Hive HCatalog Server. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | Restarts Hive WebHCat server. | | hcatalog-webhcat-log4j2 | Change values in HCatalog WebHCat's log4j2.properties. | Restarts Hive WebHCat server. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | Restarts Hive WebHCat server. | | hive | Amazon EMR-curated settings for Apache Hive. | Sets configurations to launch Hive LLAP service. | | hive-beeline-log4j2 | Change values in Hive's beeline-log4j2.properties file. | Not available. | | hive-parquet-logging | Change values in Hive's parquet-logging.properties file. | Not available. | | hive-env | Change values in the Hive environment. | Restarts HiveServer2, HiveMetastore, and Hive HCatalog-Server. Runs Hive schemaTool CLI commands to verify hive-metastore. | | hive-exec-log4j2 | Change values in Hive's hive-exec-log4j2.properties file. | Not available. | | hive-llap-daemon-log4j2 | Change values in Hive's llap-daemon-log4j2.properties file. | Not available. | | hive-log4j2 | Change values in Hive's hive-log4j2.properties file. | Not available. | | hive-site | Change values in Hive's hive-site.xml file | Restarts HiveServer2, HiveMetastore, and Hive HCatalog-Server. Runs Hive schemaTool CLI commands to verify hive-metastore. Also restarts Oozie and Zeppelin. | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | Not available. | | hue-ini | Change values in Hue's ini file | Restarts Hue. Also activates Hue config override CLI commands to pick up new configurations. | | httpfs-env | Change values in the HTTPFS environment. | Restarts Hadoop Httpfs service. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | Restarts Hadoop Httpfs service. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | Not available. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | Restarts Hadoop-KMS service. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | Not available. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | Restarts Hadoop-KMS and Ranger-KMS service. | | hudi-env | Change values in the Hudi environment. | Not available. | | hudi-defaults | Change values in Hudi's hudi-defaults.conf file. | Not available. | | iceberg-defaults | Change values in Iceberg's iceberg-defaults.conf file. | Not available. | | delta-defaults | Change values in Delta's delta-defaults.conf file. | Not available. | | jupyter-notebook-conf | Change values in Jupyter Notebook's jupyter\$1notebook\$1config.py file. | Not available. | | jupyter-hub-conf | Change values in JupyterHubs's jupyterhub\$1config.py file. | Not available. | | jupyter-s3-conf | Configure Jupyter Notebook S3 persistence. | Not available. | | jupyter-sparkmagic-conf | Change values in Sparkmagic's config.json file. | Not available. | | livy-conf | Change values in Livy's livy.conf file. | Restarts Livy Server. | | livy-env | Change values in the Livy environment. | Restarts Livy Server. | | livy-log4j2 | Change Livy log4j2.properties settings. | Restarts Livy Server. | | mapred-env | Change values in the MapReduce application's environment. | Restarts Hadoop MapReduce-HistoryServer. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | Restarts Hadoop MapReduce-HistoryServer. | | oozie-env | Change values in Oozie's environment. | Restarts Oozie. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | Restarts Oozie. | | oozie-site | Change values in Oozie's oozie-site.xml file. | Restarts Oozie. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | Not available. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | Not available. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | Restarts Phoenix-QueryServer. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | Not available. | | pig-env | Change values in the Pig environment. | Not available. | | pig-properties | Change values in Pig's pig.properties file. | Restarts Oozie. | | pig-log4j | Change values in Pig's log4j.properties file. | Not available. | | presto-log | Change values in Presto's log.properties file. | Restarts Presto-Server (for PrestoDB) | | presto-config | Change values in Presto's config.properties file. | Restarts Presto-Server (for PrestoDB) | | presto-password-authenticator | Change values in Presto's password-authenticator.properties file. | Not available. | | presto-env | Change values in Presto's presto-env.sh file. | Restarts Presto-Server (for PrestoDB) | | presto-node | Change values in Presto's node.properties file. | Not available. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | Not available. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | Not available. | | presto-connector-hive | Change values in Presto's hive.properties file. | Restarts Presto-Server (for PrestoDB) | | presto-connector-jmx | Change values in Presto's jmx.properties file. | Not available. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | Not available. | | presto-connector-lakeformation | Change values in Presto's lakeformation.properties file. | Restarts Presto-Server (for PrestoDB) | | presto-connector-localfile | Change values in Presto's localfile.properties file. | Not available. | | presto-connector-memory | Change values in Presto's memory.properties file. | Not available. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | Not available. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | Not available. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | Not available. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | Not available. | | presto-connector-redis | Change values in Presto's redis.properties file. | Not available. | | presto-connector-redshift | Change values in Presto's redshift.properties file. | Not available. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | Not available. | | presto-connector-tpcds | Change values in Presto's tpcds.properties file. | Not available. | | trino-log | Change values in Trino's log.properties file. | Restarts Trino-Server (for Trino) | | trino-config | Change values in Trino's config.properties file. | Restarts Trino-Server (for Trino) | | trino-password-authenticator | Change values in Trino's password-authenticator.properties file. | Restarts Trino-Server (for Trino) | | trino-env | Change values in Trino's trino-env.sh file. | Restarts Trino-Server (for Trino) | | trino-node | Change values in Trino's node.properties file. | Not available. | | trino-connector-blackhole | Change values in Trino's blackhole.properties file. | Not available. | | trino-connector-cassandra | Change values in Trino's cassandra.properties file. | Not available. | | trino-connector-delta | Change values in Trino's delta.properties file. | Restarts Trino-Server (for Trino) | | trino-connector-hive | Change values in Trino's hive.properties file. | Restarts Trino-Server (for Trino) | | trino-exchange-manager | Change values in Trino's exchange-manager.properties file. | Restarts Trino-Server (for Trino) | | trino-connector-iceberg | Change values in Trino's iceberg.properties file. | Restarts Trino-Server (for Trino) | | trino-connector-hudi | Change values in Trino's hudi.properties file. | Restarts Trino-Server (for Trino) | | trino-connector-jmx | Change values in Trino's jmx.properties file. | Not available. | | trino-connector-kafka | Change values in Trino's kafka.properties file. | Not available. | | trino-connector-localfile | Change values in Trino's localfile.properties file. | Not available. | | trino-connector-memory | Change values in Trino's memory.properties file. | Not available. | | trino-connector-mongodb | Change values in Trino's mongodb.properties file. | Not available. | | trino-connector-mysql | Change values in Trino's mysql.properties file. | Not available. | | trino-connector-postgresql | Change values in Trino's postgresql.properties file. | Not available. | | trino-connector-raptor | Change values in Trino's raptor.properties file. | Not available. | | trino-connector-redis | Change values in Trino's redis.properties file. | Not available. | | trino-connector-redshift | Change values in Trino's redshift.properties file. | Not available. | | trino-connector-tpch | Change values in Trino's tpch.properties file. | Not available. | | trino-connector-tpcds | Change values in Trino's tpcds.properties file. | Not available. | | ranger-kms-dbks-site | Change values in dbks-site.xml file of Ranger KMS. | Restarts Ranger KMS Server. | | ranger-kms-site | Change values in ranger-kms-site.xml file of Ranger KMS. | Restarts Ranger KMS Server. | | ranger-kms-env | Change values in the Ranger KMS environment. | Restarts Ranger KMS Server. | | ranger-kms-logback | Change values in kms-logback.xml file of Ranger KMS. | Not available. | | ranger-kms-db-ca | Change values for CA file on S3 for MySQL SSL connection with Ranger KMS. | Not available. | | spark | Amazon EMR-curated settings for Apache Spark. | This property modifies spark-defaults. See actions there. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | Restarts Spark history server and Spark thrift server. | | spark-env | Change values in the Spark environment. | Restarts Spark history server and Spark thrift server. | | spark-hive-site | Change values in Spark's hive-site.xml file | Not available. | | spark-log4j2 | Change values in Spark's log4j2.properties file. | Restarts Spark history server and Spark thrift server. | | spark-metrics | Change values in Spark's metrics.properties file. | Restarts Spark history server and Spark thrift server. | | sqoop-env | Change values in Sqoop's environment. | Not available. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | Not available. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | Not available. | | tez-site | Change values in Tez's tez-site.xml file. | Restart Oozie and HiveServer2. | | yarn-env | Change values in the YARN environment. | Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts MapReduce-HistoryServer. | | yarn-site | Change values in YARN's yarn-site.xml file. | Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts Livy Server and MapReduce-HistoryServer. | | zeppelin-env | Change values in the Zeppelin environment. | Restarts Zeppelin. | | zeppelin-site | Change configuration settings in zeppelin-site.xml. | Restarts Zeppelin. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | Restarts Zookeeper server. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | Restarts Zookeeper server. | ## 6.11.1 change log **Change log for 6.11.1 release and release notes** | Date | Event | Description | | --- | --- | --- | | 2023-08-30 | Update release notes | Added several control-plane related fixes to the release notes | | 2023-08-21 | Docs publication | Amazon EMR 6.11.1 release notes first published | | 2023-08-16 | Deployment complete | Amazon EMR 6.11.1 fully deployed to all [supported Regions](https://aws.amazon.com/about-aws/global-infrastructure/regional-product-services/) | | 2023-08-04 | Initial release | Amazon EMR 6.11.1 first deployed to limited commercial Regions | # Amazon EMR release 6.11.0 ## 6.11.0 application versions This release includes the following applications: [https://delta.io/](https://delta.io/), [https://flink.apache.org/](https://flink.apache.org/), [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [https://hudi.apache.org](https://hudi.apache.org), [http://gethue.com/](http://gethue.com/), [https://iceberg.apache.org/](https://iceberg.apache.org/), [https://jupyter-enterprise-gateway.readthedocs.io/en/latest/](https://jupyter-enterprise-gateway.readthedocs.io/en/latest/), [https://jupyterhub.readthedocs.io/en/latest/#](https://jupyterhub.readthedocs.io/en/latest/#), [https://livy.incubator.apache.org/](https://livy.incubator.apache.org/), [https://mxnet.incubator.apache.org/](https://mxnet.incubator.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://www.tensorflow.org/](https://www.tensorflow.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://trino.io/](https://trino.io/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-6.11.0 | emr-6.10.1 | emr-6.10.0 | emr-6.9.1 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.12.446 | 1.12.397 | 1.12.397 | 1.12.170 | | Python | 2.7, 3.7 | 2.7, 3.7 | 2.7, 3.7 | 2.7, 3.7 | | Scala | 2.12.15 | 2.12.15 | 2.12.15 | 2.12.15 | | AmazonCloudWatchAgent | - | - | - | - | | Delta | 2.2.0 | 2.2.0 | 2.2.0 | 2.1.0 | | Flink | 1.16.0 | 1.16.0 | 1.16.0 | 1.15.2 | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 2.4.15-amzn-1 | 2.4.15-amzn-0.1 | 2.4.15-amzn-0 | 2.4.13-amzn-0.1 | | HCatalog | 3.1.3-amzn-4 | 3.1.3-amzn-3.1 | 3.1.3-amzn-3 | 3.1.3-amzn-2.1 | | Hadoop | 3.3.3-amzn-3 | 3.3.3-amzn-2.1 | 3.3.3-amzn-2 | 3.3.3-amzn-1.1 | | Hive | 3.1.3-amzn-4 | 3.1.3-amzn-3.1 | 3.1.3-amzn-3 | 3.1.3-amzn-2.1 | | Hudi | 0.13.0-amzn-0 | 0.12.2-amzn-0 | 0.12.2-amzn-0 | 0.12.1-amzn-0 | | Hue | 4.11.0 | 4.10.0 | 4.10.0 | 4.10.0 | | Iceberg | 1.2.0-amzn-0 | 1.1.0-amzn-0 | 1.1.0-amzn-0 | 0.14.1-amzn-0 | | JupyterEnterpriseGateway | 2.6.0 | 2.6.0 | 2.6.0 | 2.6.0 | | JupyterHub | 1.4.1 | 1.5.0 | 1.5.0 | 1.4.1 | | Livy | 0.7.1-incubating | 0.7.1-incubating | 0.7.1-incubating | 0.7.1-incubating | | MXNet | 1.9.1 | 1.9.1 | 1.9.1 | 1.9.1 | | Mahout | - | - | - | - | | Oozie | 5.2.1 | 5.2.1 | 5.2.1 | 5.2.1 | | Phoenix | 5.1.2 | 5.1.2 | 5.1.2 | 5.1.2 | | Pig | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | | Presto | 0.279-amzn-0 | 0.278.1-amzn-0 | 0.278.1-amzn-0 | 0.276-amzn-0 | | Spark | 3.3.2-amzn-0 | 3.3.1-amzn-0.1 | 3.3.1-amzn-0 | 3.3.0-amzn-1.1 | | Sqoop | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | | TensorFlow | 2.11.0 | 2.11.0 | 2.11.0 | 2.10.0 | | Tez | 0.10.2-amzn-2 | 0.10.2-amzn-1.1 | 0.10.2-amzn-1 | 0.10.2-amzn-0.1 | | Trino (PrestoSQL) | 410-amzn-0 | 403-amzn-0 | 403-amzn-0 | 398-amzn-0 | | Zeppelin | 0.10.1 | 0.10.1 | 0.10.1 | 0.10.1 | | ZooKeeper | 3.5.10 | 3.5.10 | 3.5.10 | 3.5.10 | ## 6.11.0 release notes The following release notes include information for Amazon EMR release 6.11.0. Changes are relative to 6.10.0. For information on the release timeline, see the [change log](https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-6110-release.html#6110-changelog). **New features** + Amazon EMR 6.11.0 supports Apache Spark 3.3.2-amzn-0, Apache Spark RAPIDS 23.02.0-amzn-0, CUDA 11.8.0, Apache Hudi 0.13.0-amzn-0, Apache Iceberg 1.2.0-amzn-0, Trino 410-amzn-0, and PrestoDB 0.279-amzn-0. **Changes, enhancements, and resolved issues** + Starting with Spark 3.3.1 (supported in EMR versions 6.10 and above), all executors in a decommissioning host are set to a new `ExecutorState`, called *DECOMMISSIONING* state. The executors being decommissioned cannot be used by Yarn to allocate tasks and thus it will request for new executors, if needed, for the tasks being executed. Thus, if you disable Spark DRA while using EMR Managed Scaling, EMR Auto Scaling, or any custom scaling mechanism on EMR-EC2 clusters, then Yarn may request maximum permissible executors for each job. In order to avoid this issue, leave the `spark.dynamicAllocation.enabled` property set to `TRUE` (which is the default) when you are using the above combination of features. In addition, you can also set minimum and maximum executor constraints by setting values for `spark.dynamicAllocation.maxExecutors` and `spark.dynamicAllocation.minExecutors` properties for your Spark jobs, to restrict the number of executors allocated during the job’s execution. + With Amazon EMR 6.11.0, the DynamoDB connector has been upgraded to version 5.0.0. Version 5.0.0 uses AWS SDK for Java 2.x. Previous releases used AWS SDK for Java 1.x. As a result of this upgrade, we strongly advise you to test your code before you use the DynamoDB connector with Amazon EMR 6.11. + When the DynamoDB connector for Amazon EMR 6.11.0 calls the DynamoDB service, it uses the Region value that you provide for the `dynamodb.endpoint` property. We recommend that you also configure `dynamodb.region` when you use `dynamodb.endpoint`, and that both properties target the same AWS Region. If you use `dynamodb.endpoint` and you don't configure `dynamodb.region`, the DynamoDB connector for Amazon EMR 6.11.0 will return an invalid Region exception and attempt to reconcile your AWS Region information from the Amazon EC2 instance metadata service (IMDS). If the connector can't retrieve the Region from IMDS, it defaults to US East (N. Virginia) (`us-east-1`). The following error is an example of the invalid Region exception that you might get if you don't properly configure the `dynamodb.region` property: `error software.amazon.awssdk.services.dynamodb.model.DynamoDbException: Credential should be scoped to a valid region.` For more information on the classes that are affected by the AWS SDK for Java upgrade to 2.x, see the [Upgrade AWS SDK for Java from 1.x to 2.x (\$1175) ](https://github.com/awslabs/emr-dynamodb-connector/commit/1dec9d1972d3673c3fae6c6ea51f19f295147ccf) commit in the GitHub repo for the Amazon EMR - DynamoDB connector. + This release fixes an issue where column data becomes `NULL` when you use Delta Lake to store Delta table data in Amazon S3 after column rename operation. For more information about this experimental feature in Delta Lake, see [Column rename operation](https://docs.delta.io/latest/delta-batch.html#rename-columns) in the Delta Lake User Guide. + The 6.11.0 release fixes an issue that might occur when you create an edge node by replicating one of the primary nodes from a cluster with multiple primary nodes. The replicated edge node could cause delays with scale-down operations, or result in high memory-utilization on the primary nodes. For more information on how to create an edge node to communicate with your EMR cluster, see [Edge Node Creator](https://github.com/aws-samples/aws-emr-utilities/tree/main/utilities/emr-edge-node-creator) in the `aws-samples` repo on GitHub. + The 6.11.0 release improves the automation process that Amazon EMR uses to re-mount Amazon EBS volumes to an instance after a reboot. + The 6.11.0 release fixes an issue that resulted in intermittent gaps in the Hadoop metrics that Amazon EMR publishes to Amazon CloudWatch. + The 6.11.0 release fixes an issue with EMR clusters where an update to the YARN configuration file that contains the exclusion list of nodes for the cluster is interrupted due to disk over-utilization. The incomplete update hinders future cluster scale-down operations. This release ensures that your cluster remains healthy, and that scaling operations work as expected. + The default root volume size has increased to 15 GB in Amazon EMR 6.10.0 and higher. Earlier releases have default root volume size of 10 GB. + Hadoop 3.3.3 introduced a change in YARN ([YARN-9608](https://issues.apache.org/jira/browse/YARN-9608)) that keeps nodes where containers ran in a decommissioning state until the application completes. This change ensures that local data such as shuffle data doesn't get lost, and you don' need to re-run the job. This approach might also lead to underutilization of resources on clusters with or without managed scaling enabled. With Amazon EMR releases 6.11.0 and higher as well as 6.8.1, 6.9.1, and 6.10.1, the value of `yarn.resourcemanager.decommissioning-nodes-watcher.wait-for-applications` is set to `false` in `yarn-site.xml` to resolve this issue. While the fix addresses the issues that were introduced by YARN-9608, it might cause Hive jobs to fail due to shuffle data loss on clusters that have managed scaling enabled. We've mitigated that risk in this release by also setting `yarn.resourcemanager.decommissioning-nodes-watcher.wait-for-shuffle-data` for Hive workloads. This config is only available with Amazon EMR releases 6.11.0 and higher. + When you launch a cluster with *the latest patch release* of Amazon EMR 5.36 or higher, 6.6 or higher, or 7.0 or higher, Amazon EMR uses the latest Amazon Linux 2023 or Amazon Linux 2 release for the default Amazon EMR AMI. For more information, see [Using the default Amazon Linux AMI for Amazon EMR](https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-default-ami.html). **Note** This release no longer gets automatic AMI updates since it has been succeeded by 1 more more patch releases. The patch release is denoted by the number after the second decimal point (`6.8.1`). To see if you're using the latest patch release, check the available releases in the [https://docs.aws.amazon.com/emr/latest/ReleaseGuide](https://docs.aws.amazon.com/emr/latest/ReleaseGuide), or check the **Amazon EMR release** dropdown when you create a cluster in the console, or use the [https://docs.aws.amazon.com/emr/latest/APIReference/API_ListReleaseLabels.html](https://docs.aws.amazon.com/emr/latest/APIReference/API_ListReleaseLabels.html) API or [https://docs.aws.amazon.com/cli/latest/reference/emr/list-release-labels.html](https://docs.aws.amazon.com/cli/latest/reference/emr/list-release-labels.html) CLI action. To get updates about new releases, subscribe to the RSS feed on the [What's new?](https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-whatsnew.html) page. [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-6110-release.html) ## 6.11.0 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | aws-sagemaker-spark-sdk | 1.4.2 | Amazon SageMaker Spark SDK | | delta | 2.2.0 | Delta lake is an open table format for huge analytic datasets | | delta-standalone-connectors | 0.6.0 | Delta Connectors provide different runtimes to integrate Delta Lake with engines like Flink, Hive and Presto. | | emr-ddb | 5.1.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 3.4.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.8.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-notebook-env | 1.7.0 | Conda env for emr notebook which includes jupyter enterprise gateway | | emr-s3-dist-cp | 2.25.0 | Distributed copy application optimized for Amazon S3. | | emr-s3-select | 2.4.0 | EMR S3Select Connector | | emr-wal-cli | 1.1.0 | Cli used for emrwal list/deletion. | | emrfs | 2.56.0 | Amazon S3 connector for Hadoop ecosystem applications. | | flink-client | 1.16.0 | Apache Flink command line client scripts and applications. | | flink-jobmanager-config | 1.16.0 | Managing resources on EMR nodes for Apache Flink JobManager. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 3.3.3-amzn-3 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 3.3.3-amzn-3 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 3.3.3-amzn-3 | HDFS command-line client and library | | hadoop-hdfs-namenode | 3.3.3-amzn-3 | HDFS service for tracking file names and block locations. | | hadoop-hdfs-journalnode | 3.3.3-amzn-3 | HDFS service for managing the Hadoop filesystem journal on HA clusters. | | hadoop-httpfs-server | 3.3.3-amzn-3 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 3.3.3-amzn-3 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 3.3.3-amzn-3 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 3.3.3-amzn-3 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 3.3.3-amzn-3 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 3.3.3-amzn-3 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 2.4.15-amzn-1 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 2.4.15-amzn-1 | Service for serving one or more HBase regions. | | hbase-client | 2.4.15-amzn-1 | HBase command-line client. | | hbase-rest-server | 2.4.15-amzn-1 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 2.4.15-amzn-1 | Service providing a Thrift endpoint to HBase. | | hbase-operator-tools | 2.4.15-amzn-1 | Repair tool for Apache HBase clusters. | | hcatalog-client | 3.1.3-amzn-4 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 3.1.3-amzn-4 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 3.1.3-amzn-4 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 3.1.3-amzn-4 | Hive command line client. | | hive-hbase | 3.1.3-amzn-4 | Hive-hbase client. | | hive-metastore-server | 3.1.3-amzn-4 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server2 | 3.1.3-amzn-4 | Service for accepting Hive queries as web requests. | | hudi | 0.13.0-amzn-0 | Incremental processing framework to power data pipeline at low latency and high efficiency. | | hudi-presto | 0.13.0-amzn-0 | Bundle library for running Presto with Hudi. | | hudi-trino | 0.13.0-amzn-0 | Bundle library for running Trino with Hudi. | | hudi-spark | 0.13.0-amzn-0 | Bundle library for running Spark with Hudi. | | hue-server | 4.11.0 | Web application for analyzing data using Hadoop ecosystem applications | | iceberg | 1.2.0-amzn-0 | Apache Iceberg is an open table format for huge analytic datasets | | jupyterhub | 1.4.1 | Multi-user server for Jupyter notebooks | | livy-server | 0.7.1-incubating | REST interface for interacting with Apache Spark | | nginx | 1.12.1 | nginx [engine x] is an HTTP and reverse proxy server | | mxnet | 1.9.1 | A flexible, scalable, and efficient library for deep learning. | | mariadb-server | 5.5.68\$1 | MariaDB database server. | | nvidia-cuda | 11.8.0 | Nvidia drivers and Cuda toolkit | | oozie-client | 5.2.1 | Oozie command-line client. | | oozie-server | 5.2.1 | Service for accepting Oozie workflow requests. | | opencv | 4.5.0 | Open Source Computer Vision Library. | | phoenix-library | 5.1.2 | The phoenix libraries for server and client | | phoenix-connectors | 5.1.2 | Apache Phoenix-Connectors for Spark-3 | | phoenix-query-server | 6.0.0 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.279-amzn-0 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.279-amzn-0 | Service for executing pieces of a query. | | presto-client | 0.279-amzn-0 | Presto command-line client which is installed on an HA cluster's stand-by masters where Presto server is not started. | | trino-coordinator | 410-amzn-0 | Service for accepting queries and managing query execution among trino-workers. | | trino-worker | 410-amzn-0 | Service for executing pieces of a query. | | trino-client | 410-amzn-0 | Trino command-line client which is installed on an HA cluster's stand-by masters where Trino server is not started. | | pig-client | 0.17.0 | Pig command-line client. | | r | 4.0.2 | The R Project for Statistical Computing | | ranger-kms-server | 2.0.0 | Apache Ranger Key Management System | | spark-client | 3.3.2-amzn-0 | Spark command-line clients. | | spark-history-server | 3.3.2-amzn-0 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 3.3.2-amzn-0 | In-memory execution engine for YARN. | | spark-yarn-slave | 3.3.2-amzn-0 | Apache Spark libraries needed by YARN slaves. | | spark-rapids | 23.02.0-amzn-0 | Nvidia Spark RAPIDS plugin that accelerates Apache Spark with GPUs. | | sqoop-client | 1.4.7 | Apache Sqoop command-line client. | | tensorflow | 2.11.0 | TensorFlow open source software library for high performance numerical computation. | | tez-on-yarn | 0.10.2-amzn-2 | The tez YARN application and libraries. | | tez-on-worker | 0.10.2-amzn-2 | The tez YARN application and libraries for worker nodes. | | webserver | 2.4.41\$1 | Apache HTTP server. | | zeppelin-server | 0.10.1 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.5.10 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.5.10 | ZooKeeper command line client. | ## 6.11.0 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). Reconfiguration actions occur when you specify a configuration for instance groups in a running cluster. Amazon EMR only initiates reconfiguration actions for the classifications that you modify. For more information, see [Reconfigure an instance group in a running cluster](emr-configure-apps-running-cluster.md). **emr-6.11.0 classifications** | Classifications | Description | Reconfiguration Actions | | --- | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | Restarts the ResourceManager service. | | container-executor | Change values in Hadoop YARN's container-executor.cfg file. | Not available. | | container-log4j | Change values in Hadoop YARN's container-log4j.properties file. | Not available. | | core-site | Change values in Hadoop's core-site.xml file. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts Hadoop KMS, Ranger KMS, HiveServer2, Hive MetaStore, Hadoop Httpfs, and MapReduce-HistoryServer. | | docker-conf | Change docker related settings. | Not available. | | emrfs-site | Change EMRFS settings. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts HBaseRegionserver, HBaseMaster, HBaseThrift, HBaseRest, HiveServer2, Hive MetaStore, Hadoop Httpfs, and MapReduce-HistoryServer. | | flink-conf | Change flink-conf.yaml settings. | Restarts Flink history server. | | flink-log4j | Change Flink log4j.properties settings. | Restarts Flink history server. | | flink-log4j-session | Change Flink log4j-session.properties settings for Kubernetes/Yarn session. | Restarts Flink history server. | | flink-log4j-cli | Change Flink log4j-cli.properties settings. | Restarts Flink history server. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts PhoenixQueryserver, HiveServer2, Hive MetaStore, and MapReduce-HistoryServer. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | Restarts the Hadoop HDFS services SecondaryNamenode, Datanode, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts Hadoop KMS, Hadoop Httpfs, and MapReduce-HistoryServer. | | hadoop-ssl-server | Change hadoop ssl server configuration | Not available. | | hadoop-ssl-client | Change hadoop ssl client configuration | Not available. | | hbase | Amazon EMR-curated settings for Apache HBase. | Custom EMR specific property. Sets emrfs-site and hbase-site configs. See those for their associated restarts. | | hbase-env | Change values in HBase's environment. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbase.properties file. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | Not available. | | hbase-site | Change values in HBase's hbase-site.xml file. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. Additionally restarts Phoenix QueryServer. | | hdfs-encryption-zones | Configure HDFS encryption zones. | This classification should not be reconfigured. | | hdfs-env | Change values in the HDFS environment. | Restarts Hadoop HDFS services Namenode, Datanode, and ZKFC. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Additionally restarts Hadoop Httpfs. | | hcatalog-env | Change values in HCatalog's environment. | Restarts Hive HCatalog Server. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | Restarts Hive HCatalog Server. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | Restarts Hive HCatalog Server. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | Restarts Hive WebHCat server. | | hcatalog-webhcat-log4j2 | Change values in HCatalog WebHCat's log4j2.properties. | Restarts Hive WebHCat server. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | Restarts Hive WebHCat server. | | hive | Amazon EMR-curated settings for Apache Hive. | Sets configurations to launch Hive LLAP service. | | hive-beeline-log4j2 | Change values in Hive's beeline-log4j2.properties file. | Not available. | | hive-parquet-logging | Change values in Hive's parquet-logging.properties file. | Not available. | | hive-env | Change values in the Hive environment. | Restarts HiveServer2, HiveMetastore, and Hive HCatalog-Server. Runs Hive schemaTool CLI commands to verify hive-metastore. | | hive-exec-log4j2 | Change values in Hive's hive-exec-log4j2.properties file. | Not available. | | hive-llap-daemon-log4j2 | Change values in Hive's llap-daemon-log4j2.properties file. | Not available. | | hive-log4j2 | Change values in Hive's hive-log4j2.properties file. | Not available. | | hive-site | Change values in Hive's hive-site.xml file | Restarts HiveServer2, HiveMetastore, and Hive HCatalog-Server. Runs Hive schemaTool CLI commands to verify hive-metastore. Also restarts Oozie and Zeppelin. | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | Not available. | | hue-ini | Change values in Hue's ini file | Restarts Hue. Also activates Hue config override CLI commands to pick up new configurations. | | httpfs-env | Change values in the HTTPFS environment. | Restarts Hadoop Httpfs service. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | Restarts Hadoop Httpfs service. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | Not available. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | Restarts Hadoop-KMS service. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | Not available. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | Restarts Hadoop-KMS and Ranger-KMS service. | | hudi-env | Change values in the Hudi environment. | Not available. | | hudi-defaults | Change values in Hudi's hudi-defaults.conf file. | Not available. | | iceberg-defaults | Change values in Iceberg's iceberg-defaults.conf file. | Not available. | | delta-defaults | Change values in Delta's delta-defaults.conf file. | Not available. | | jupyter-notebook-conf | Change values in Jupyter Notebook's jupyter\$1notebook\$1config.py file. | Not available. | | jupyter-hub-conf | Change values in JupyterHubs's jupyterhub\$1config.py file. | Not available. | | jupyter-s3-conf | Configure Jupyter Notebook S3 persistence. | Not available. | | jupyter-sparkmagic-conf | Change values in Sparkmagic's config.json file. | Not available. | | livy-conf | Change values in Livy's livy.conf file. | Restarts Livy Server. | | livy-env | Change values in the Livy environment. | Restarts Livy Server. | | livy-log4j2 | Change Livy log4j2.properties settings. | Restarts Livy Server. | | mapred-env | Change values in the MapReduce application's environment. | Restarts Hadoop MapReduce-HistoryServer. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | Restarts Hadoop MapReduce-HistoryServer. | | oozie-env | Change values in Oozie's environment. | Restarts Oozie. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | Restarts Oozie. | | oozie-site | Change values in Oozie's oozie-site.xml file. | Restarts Oozie. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | Not available. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | Not available. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | Restarts Phoenix-QueryServer. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | Not available. | | pig-env | Change values in the Pig environment. | Not available. | | pig-properties | Change values in Pig's pig.properties file. | Restarts Oozie. | | pig-log4j | Change values in Pig's log4j.properties file. | Not available. | | presto-log | Change values in Presto's log.properties file. | Restarts Presto-Server (for PrestoDB) | | presto-config | Change values in Presto's config.properties file. | Restarts Presto-Server (for PrestoDB) | | presto-password-authenticator | Change values in Presto's password-authenticator.properties file. | Not available. | | presto-env | Change values in Presto's presto-env.sh file. | Restarts Presto-Server (for PrestoDB) | | presto-node | Change values in Presto's node.properties file. | Not available. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | Not available. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | Not available. | | presto-connector-hive | Change values in Presto's hive.properties file. | Restarts Presto-Server (for PrestoDB) | | presto-connector-jmx | Change values in Presto's jmx.properties file. | Not available. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | Not available. | | presto-connector-lakeformation | Change values in Presto's lakeformation.properties file. | Restarts Presto-Server (for PrestoDB) | | presto-connector-localfile | Change values in Presto's localfile.properties file. | Not available. | | presto-connector-memory | Change values in Presto's memory.properties file. | Not available. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | Not available. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | Not available. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | Not available. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | Not available. | | presto-connector-redis | Change values in Presto's redis.properties file. | Not available. | | presto-connector-redshift | Change values in Presto's redshift.properties file. | Not available. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | Not available. | | presto-connector-tpcds | Change values in Presto's tpcds.properties file. | Not available. | | trino-log | Change values in Trino's log.properties file. | Restarts Trino-Server (for Trino) | | trino-config | Change values in Trino's config.properties file. | Restarts Trino-Server (for Trino) | | trino-password-authenticator | Change values in Trino's password-authenticator.properties file. | Restarts Trino-Server (for Trino) | | trino-env | Change values in Trino's trino-env.sh file. | Restarts Trino-Server (for Trino) | | trino-node | Change values in Trino's node.properties file. | Not available. | | trino-connector-blackhole | Change values in Trino's blackhole.properties file. | Not available. | | trino-connector-cassandra | Change values in Trino's cassandra.properties file. | Not available. | | trino-connector-delta | Change values in Trino's delta.properties file. | Restarts Trino-Server (for Trino) | | trino-connector-hive | Change values in Trino's hive.properties file. | Restarts Trino-Server (for Trino) | | trino-exchange-manager | Change values in Trino's exchange-manager.properties file. | Restarts Trino-Server (for Trino) | | trino-connector-iceberg | Change values in Trino's iceberg.properties file. | Restarts Trino-Server (for Trino) | | trino-connector-hudi | Change values in Trino's hudi.properties file. | Restarts Trino-Server (for Trino) | | trino-connector-jmx | Change values in Trino's jmx.properties file. | Not available. | | trino-connector-kafka | Change values in Trino's kafka.properties file. | Not available. | | trino-connector-localfile | Change values in Trino's localfile.properties file. | Not available. | | trino-connector-memory | Change values in Trino's memory.properties file. | Not available. | | trino-connector-mongodb | Change values in Trino's mongodb.properties file. | Not available. | | trino-connector-mysql | Change values in Trino's mysql.properties file. | Not available. | | trino-connector-postgresql | Change values in Trino's postgresql.properties file. | Not available. | | trino-connector-raptor | Change values in Trino's raptor.properties file. | Not available. | | trino-connector-redis | Change values in Trino's redis.properties file. | Not available. | | trino-connector-redshift | Change values in Trino's redshift.properties file. | Not available. | | trino-connector-tpch | Change values in Trino's tpch.properties file. | Not available. | | trino-connector-tpcds | Change values in Trino's tpcds.properties file. | Not available. | | ranger-kms-dbks-site | Change values in dbks-site.xml file of Ranger KMS. | Restarts Ranger KMS Server. | | ranger-kms-site | Change values in ranger-kms-site.xml file of Ranger KMS. | Restarts Ranger KMS Server. | | ranger-kms-env | Change values in the Ranger KMS environment. | Restarts Ranger KMS Server. | | ranger-kms-logback | Change values in kms-logback.xml file of Ranger KMS. | Not available. | | ranger-kms-db-ca | Change values for CA file on S3 for MySQL SSL connection with Ranger KMS. | Not available. | | spark | Amazon EMR-curated settings for Apache Spark. | This property modifies spark-defaults. See actions there. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | Restarts Spark history server and Spark thrift server. | | spark-env | Change values in the Spark environment. | Restarts Spark history server and Spark thrift server. | | spark-hive-site | Change values in Spark's hive-site.xml file | Not available. | | spark-log4j2 | Change values in Spark's log4j2.properties file. | Restarts Spark history server and Spark thrift server. | | spark-metrics | Change values in Spark's metrics.properties file. | Restarts Spark history server and Spark thrift server. | | sqoop-env | Change values in Sqoop's environment. | Not available. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | Not available. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | Not available. | | tez-site | Change values in Tez's tez-site.xml file. | Restart Oozie and HiveServer2. | | yarn-env | Change values in the YARN environment. | Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts MapReduce-HistoryServer. | | yarn-site | Change values in YARN's yarn-site.xml file. | Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts Livy Server and MapReduce-HistoryServer. | | zeppelin-env | Change values in the Zeppelin environment. | Restarts Zeppelin. | | zeppelin-site | Change configuration settings in zeppelin-site.xml. | Restarts Zeppelin. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | Restarts Zookeeper server. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | Restarts Zookeeper server. | ## 6.11.0 change log **Change log for 6.11.0 release and release notes** | Date | Event | Description | | --- | --- | --- | | 2023-08-21 | Update | Fixed issue introduced with Hadoop 3.3.3. | | 2023-07-26 | Update | New OS release labels 2.0.20230612.0 and 2.0.20230628.0. | | 2023-06-09 | Deployment complete | Amazon EMR 6.11.0 fully deployed to all [supported Regions](https://aws.amazon.com/about-aws/global-infrastructure/regional-product-services/) | | 2023-06-09 | Docs publication | Amazon EMR 6.11.0 release notes first published | | 2023-06-08 | Initial release | Amazon EMR 6.11.0 first deployed to initial commercial Regions | # Amazon EMR release 6.10.1 ## 6.10.1 application versions This release includes the following applications: [https://delta.io/](https://delta.io/), [https://flink.apache.org/](https://flink.apache.org/), [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [https://hudi.apache.org](https://hudi.apache.org), [http://gethue.com/](http://gethue.com/), [https://iceberg.apache.org/](https://iceberg.apache.org/), [https://jupyter-enterprise-gateway.readthedocs.io/en/latest/](https://jupyter-enterprise-gateway.readthedocs.io/en/latest/), [https://jupyterhub.readthedocs.io/en/latest/#](https://jupyterhub.readthedocs.io/en/latest/#), [https://livy.incubator.apache.org/](https://livy.incubator.apache.org/), [https://mxnet.incubator.apache.org/](https://mxnet.incubator.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://www.tensorflow.org/](https://www.tensorflow.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://trino.io/](https://trino.io/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-6.10.1 | emr-6.10.0 | emr-6.9.1 | emr-6.9.0 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.12.397 | 1.12.397 | 1.12.170 | 1.12.170 | | Python | 2.7, 3.7 | 2.7, 3.7 | 2.7, 3.7 | 2.7, 3.7 | | Scala | 2.12.15 | 2.12.15 | 2.12.15 | 2.12.15 | | AmazonCloudWatchAgent | - | - | - | - | | Delta | 2.2.0 | 2.2.0 | 2.1.0 | 2.1.0 | | Flink | 1.16.0 | 1.16.0 | 1.15.2 | 1.15.2 | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 2.4.15-amzn-0.1 | 2.4.15-amzn-0 | 2.4.13-amzn-0.1 | 2.4.13-amzn-0 | | HCatalog | 3.1.3-amzn-3.1 | 3.1.3-amzn-3 | 3.1.3-amzn-2.1 | 3.1.3-amzn-2 | | Hadoop | 3.3.3-amzn-2.1 | 3.3.3-amzn-2 | 3.3.3-amzn-1.1 | 3.3.3-amzn-1 | | Hive | 3.1.3-amzn-3.1 | 3.1.3-amzn-3 | 3.1.3-amzn-2.1 | 3.1.3-amzn-2 | | Hudi | 0.12.2-amzn-0 | 0.12.2-amzn-0 | 0.12.1-amzn-0 | 0.12.1-amzn-0 | | Hue | 4.10.0 | 4.10.0 | 4.10.0 | 4.10.0 | | Iceberg | 1.1.0-amzn-0 | 1.1.0-amzn-0 | 0.14.1-amzn-0 | 0.14.1-amzn-0 | | JupyterEnterpriseGateway | 2.6.0 | 2.6.0 | 2.6.0 | 2.6.0 | | JupyterHub | 1.5.0 | 1.5.0 | 1.4.1 | 1.4.1 | | Livy | 0.7.1-incubating | 0.7.1-incubating | 0.7.1-incubating | 0.7.1-incubating | | MXNet | 1.9.1 | 1.9.1 | 1.9.1 | 1.9.1 | | Mahout | - | - | - | - | | Oozie | 5.2.1 | 5.2.1 | 5.2.1 | 5.2.1 | | Phoenix | 5.1.2 | 5.1.2 | 5.1.2 | 5.1.2 | | Pig | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | | Presto | 0.278.1-amzn-0 | 0.278.1-amzn-0 | 0.276-amzn-0 | 0.276-amzn-0 | | Spark | 3.3.1-amzn-0.1 | 3.3.1-amzn-0 | 3.3.0-amzn-1.1 | 3.3.0-amzn-1 | | Sqoop | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | | TensorFlow | 2.11.0 | 2.11.0 | 2.10.0 | 2.10.0 | | Tez | 0.10.2-amzn-1.1 | 0.10.2-amzn-1 | 0.10.2-amzn-0.1 | 0.10.2-amzn-0 | | Trino (PrestoSQL) | 403-amzn-0 | 403-amzn-0 | 398-amzn-0 | 398-amzn-0 | | Zeppelin | 0.10.1 | 0.10.1 | 0.10.1 | 0.10.1 | | ZooKeeper | 3.5.10 | 3.5.10 | 3.5.10 | 3.5.10 | ## 6.10.1 release notes The following release notes include information for Amazon EMR release 6.10.1. Changes are relative to 6.10.0. For information on the release timeline, see the [6.10.1 change log](#6101-changelog). **Known Issues** + Starting with Spark 3.3.1 (supported in EMR versions 6.10 and above), all executors in a decommissioning host are set to a new `ExecutorState`, called *DECOMMISSIONING* state. The executors being decommissioned cannot be used by Yarn to allocate tasks and thus it will request for new executors, if needed, for the tasks being executed. Thus, if you disable Spark DRA while using EMR Managed Scaling, EMR Auto Scaling, or any custom scaling mechanism on EMR-EC2 clusters, then Yarn may request maximum permissible executors for each job. In order to avoid this issue, leave the `spark.dynamicAllocation.enabled` property set to `TRUE` (which is the default) when you are using the above combination of features. In addition, you can also set minimum and maximum executor constraints by setting values for `spark.dynamicAllocation.maxExecutors` and `spark.dynamicAllocation.minExecutors` properties for your Spark jobs, to restrict the number of executors allocated during the job’s execution. **Changes, enhancements, and resolved issues** + Due to lock contention, a node can enter into a deadlock if it's added or removed at the same time that it attempts to decommission. As a result, the Hadoop Resource Manager (YARN) becomes unresponsive, and affects all the incoming and currently-running containers. + Hadoop 3.3.3 introduced a change in YARN ([YARN-9608](https://issues.apache.org/jira/browse/YARN-9608)) that keeps nodes where containers ran in a decommissioning state until the application completes. This change ensures that local data such as shuffle data doesn't get lost, and you don' need to re-run the job. This approach might also lead to underutilization of resources on clusters with or without managed scaling enabled. With Amazon EMR releases 6.11.0 and higher as well as 6.8.1, 6.9.1, and 6.10.1, the value of `yarn.resourcemanager.decommissioning-nodes-watcher.wait-for-applications` is set to `false` in `yarn-site.xml` to resolve this issue. While the fix addresses the issues that were introduced by YARN-9608, it might cause Hive jobs to fail due to shuffle data loss on clusters that have managed scaling enabled. We've mitigated that risk in this release by also setting `yarn.resourcemanager.decommissioning-nodes-watcher.wait-for-shuffle-data` for Hive workloads. This config is only available with Amazon EMR releases 6.11.0 and higher. + Metrics collector will not send any metrics to the control plane after failover of primary node in clusters with the instance groups configuration. + This release includes a change that allows high-availability clusters to recover from failed state after restart. + This release includes security fixes for Hue and HBase. + This release fixes an issue where clusters that are running workloads on Spark with Amazon EMR might silently receive incorrect results with `contains`, `startsWith`, `endsWith`, and `like`. This issue occurs when you use the expressions on partitioned fields that have metadata in the Amazon EMR Hive3 Metastore Server (HMS). + This release fixes an issue with throttling on the Glue side when there are no user-defined functions (UDF). + This release fixes an issue that deletes container logs by the node log aggregation service before log pusher can push them to S3 in case of YARN decommissioning. + This release fixes an issue with FairShare Scheduler metrics when Node Label is enabled for Hadoop. + This release fixes an issue that impacted Spark performance when you set a default `true` value for the `spark.yarn.heterogeneousExecutors.enabled` config in `spark-defaults.conf`. + This release fixes an issue with Reduce Task failing to read shuffle data. The issue caused Hive query failures with a corrupted memory error. + This release adds a new retry mechanism to the cluster scaling workflow for EMR clusters that run Presto or Trino. This improvement reduces the risk that cluster resizing will indefinitely stall due to a single failed resize operation. It also improves cluster utilization, because your cluster scales up and down faster. + This release improves cluster scale-down logic so that your cluster doesn't attempt a scale-down of core nodes below the HDFS replication factor setting for the cluster. This aligns with your data redundancy requirements, and reduces the chance that a scaling operation might stall. + The log management daemon has been upgraded to identify all logs that are in active use with open file handles on the local instance storage, and the associated processes. This upgrade ensures that Amazon EMR properly deletes the files and reclaims storage space after the logs are archived to Amazon S3. + This release includes a log-management daemon enhancement that deletes empty, unused steps directories in the local cluster file system. An excessively large number of empty directories can degrade the performance of Amazon EMR daemons and result in disk over-utilization. + This release fixes an issue that might occur when you create an edge node by replicating one of the primary nodes from a cluster with multiple primary nodes. The replicated edge node could cause delays with scale-down operations, or result in high memory-utilization on the primary nodes. For more information on how to create an edge node to communicate with your EMR cluster, see [Edge Node Creator](https://github.com/aws-samples/aws-emr-utilities/tree/main/utilities/emr-edge-node-creator) in the `aws-samples` repo on GitHub. + This release improves the automation process that Amazon EMR uses to re-mount Amazon EBS volumes to an instance after a reboot. + This release fixes an issue that resulted in intermittent gaps in the Hadoop metrics that Amazon EMR publishes to Amazon CloudWatch. + This release fixes an issue with EMR clusters where an update to the YARN configuration file that contains the exclusion list of nodes for the cluster is interrupted due to disk over-utilization. The incomplete update hinders future cluster scale-down operations. This release ensures that your cluster remains healthy, and that scaling operations work as expected. + When you launch a cluster with *the latest patch release* of Amazon EMR 5.36 or higher, 6.6 or higher, or 7.0 or higher, Amazon EMR uses the latest Amazon Linux 2023 or Amazon Linux 2 release for the default Amazon EMR AMI. For more information, see [Using the default Amazon Linux AMI for Amazon EMR](https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-default-ami.html). [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-6101-release.html) ## 6.10.1 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | aws-sagemaker-spark-sdk | 1.4.2 | Amazon SageMaker Spark SDK | | delta | 2.2.0 | Delta lake is an open table format for huge analytic datasets | | emr-ddb | 4.16.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 3.3.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.7.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-notebook-env | 1.7.0 | Conda env for emr notebook which includes jupyter enterprise gateway | | emr-s3-dist-cp | 2.24.0 | Distributed copy application optimized for Amazon S3. | | emr-s3-select | 2.3.0 | EMR S3Select Connector | | emr-wal-cli | 1.0.0 | Cli used for emrwal list/deletion. | | emrfs | 2.55.0 | Amazon S3 connector for Hadoop ecosystem applications. | | flink-client | 1.16.0 | Apache Flink command line client scripts and applications. | | flink-jobmanager-config | 1.16.0 | Managing resources on EMR nodes for Apache Flink JobManager. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 3.3.3-amzn-2.1 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 3.3.3-amzn-2.1 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 3.3.3-amzn-2.1 | HDFS command-line client and library | | hadoop-hdfs-namenode | 3.3.3-amzn-2.1 | HDFS service for tracking file names and block locations. | | hadoop-hdfs-journalnode | 3.3.3-amzn-2.1 | HDFS service for managing the Hadoop filesystem journal on HA clusters. | | hadoop-httpfs-server | 3.3.3-amzn-2.1 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 3.3.3-amzn-2.1 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 3.3.3-amzn-2.1 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 3.3.3-amzn-2.1 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 3.3.3-amzn-2.1 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 3.3.3-amzn-2.1 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 2.4.15-amzn-0.1 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 2.4.15-amzn-0.1 | Service for serving one or more HBase regions. | | hbase-client | 2.4.15-amzn-0.1 | HBase command-line client. | | hbase-rest-server | 2.4.15-amzn-0.1 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 2.4.15-amzn-0.1 | Service providing a Thrift endpoint to HBase. | | hbase-operator-tools | 2.4.15-amzn-0.1 | Repair tool for Apache HBase clusters. | | hcatalog-client | 3.1.3-amzn-3.1 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 3.1.3-amzn-3.1 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 3.1.3-amzn-3.1 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 3.1.3-amzn-3.1 | Hive command line client. | | hive-hbase | 3.1.3-amzn-3.1 | Hive-hbase client. | | hive-metastore-server | 3.1.3-amzn-3.1 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server2 | 3.1.3-amzn-3.1 | Service for accepting Hive queries as web requests. | | hudi | 0.12.2-amzn-0 | Incremental processing framework to power data pipeline at low latency and high efficiency. | | hudi-presto | 0.12.2-amzn-0 | Bundle library for running Presto with Hudi. | | hudi-trino | 0.12.2-amzn-0 | Bundle library for running Trino with Hudi. | | hudi-spark | 0.12.2-amzn-0 | Bundle library for running Spark with Hudi. | | hue-server | 4.10.0 | Web application for analyzing data using Hadoop ecosystem applications | | iceberg | 1.1.0-amzn-0 | Apache Iceberg is an open table format for huge analytic datasets | | jupyterhub | 1.5.0 | Multi-user server for Jupyter notebooks | | livy-server | 0.7.1-incubating | REST interface for interacting with Apache Spark | | nginx | 1.12.1 | nginx [engine x] is an HTTP and reverse proxy server | | mxnet | 1.9.1 | A flexible, scalable, and efficient library for deep learning. | | mariadb-server | 5.5.68\$1 | MariaDB database server. | | nvidia-cuda | 11.8.0 | Nvidia drivers and Cuda toolkit | | oozie-client | 5.2.1 | Oozie command-line client. | | oozie-server | 5.2.1 | Service for accepting Oozie workflow requests. | | opencv | 4.5.0 | Open Source Computer Vision Library. | | phoenix-library | 5.1.2 | The phoenix libraries for server and client | | phoenix-connectors | 6.0.0-SNAPSHOT | Apache Phoenix-Connectors for Spark-3 | | phoenix-query-server | 6.0.0 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.278.1-amzn-0 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.278.1-amzn-0 | Service for executing pieces of a query. | | presto-client | 0.278.1-amzn-0 | Presto command-line client which is installed on an HA cluster's stand-by masters where Presto server is not started. | | trino-coordinator | 403-amzn-0 | Service for accepting queries and managing query execution among trino-workers. | | trino-worker | 403-amzn-0 | Service for executing pieces of a query. | | trino-client | 403-amzn-0 | Trino command-line client which is installed on an HA cluster's stand-by masters where Trino server is not started. | | pig-client | 0.17.0 | Pig command-line client. | | r | 4.0.2 | The R Project for Statistical Computing | | ranger-kms-server | 2.0.0 | Apache Ranger Key Management System | | spark-client | 3.3.1-amzn-0.1 | Spark command-line clients. | | spark-history-server | 3.3.1-amzn-0.1 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 3.3.1-amzn-0.1 | In-memory execution engine for YARN. | | spark-yarn-slave | 3.3.1-amzn-0.1 | Apache Spark libraries needed by YARN slaves. | | spark-rapids | 22.12.0-amzn-0 | Nvidia Spark RAPIDS plugin that accelerates Apache Spark with GPUs. | | sqoop-client | 1.4.7 | Apache Sqoop command-line client. | | tensorflow | 2.11.0 | TensorFlow open source software library for high performance numerical computation. | | tez-on-yarn | 0.10.2-amzn-1.1 | The tez YARN application and libraries. | | tez-on-worker | 0.10.2-amzn-1.1 | The tez YARN application and libraries for worker nodes. | | webserver | 2.4.41\$1 | Apache HTTP server. | | zeppelin-server | 0.10.1 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.5.10 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.5.10 | ZooKeeper command line client. | ## 6.10.1 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). Reconfiguration actions occur when you specify a configuration for instance groups in a running cluster. Amazon EMR only initiates reconfiguration actions for the classifications that you modify. For more information, see [Reconfigure an instance group in a running cluster](emr-configure-apps-running-cluster.md). **emr-6.10.1 classifications** | Classifications | Description | Reconfiguration Actions | | --- | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | Restarts the ResourceManager service. | | container-executor | Change values in Hadoop YARN's container-executor.cfg file. | Not available. | | container-log4j | Change values in Hadoop YARN's container-log4j.properties file. | Not available. | | core-site | Change values in Hadoop's core-site.xml file. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts Hadoop KMS, Ranger KMS, HiveServer2, Hive MetaStore, Hadoop Httpfs, and MapReduce-HistoryServer. | | docker-conf | Change docker related settings. | Not available. | | emrfs-site | Change EMRFS settings. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts HBaseRegionserver, HBaseMaster, HBaseThrift, HBaseRest, HiveServer2, Hive MetaStore, Hadoop Httpfs, and MapReduce-HistoryServer. | | flink-conf | Change flink-conf.yaml settings. | Restarts Flink history server. | | flink-log4j | Change Flink log4j.properties settings. | Restarts Flink history server. | | flink-log4j-session | Change Flink log4j-session.properties settings for Kubernetes/Yarn session. | Restarts Flink history server. | | flink-log4j-cli | Change Flink log4j-cli.properties settings. | Restarts Flink history server. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts PhoenixQueryserver, HiveServer2, Hive MetaStore, and MapReduce-HistoryServer. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | Restarts the Hadoop HDFS services SecondaryNamenode, Datanode, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts Hadoop KMS, Hadoop Httpfs, and MapReduce-HistoryServer. | | hadoop-ssl-server | Change hadoop ssl server configuration | Not available. | | hadoop-ssl-client | Change hadoop ssl client configuration | Not available. | | hbase | Amazon EMR-curated settings for Apache HBase. | Custom EMR specific property. Sets emrfs-site and hbase-site configs. See those for their associated restarts. | | hbase-env | Change values in HBase's environment. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbase.properties file. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | Not available. | | hbase-site | Change values in HBase's hbase-site.xml file. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. Additionally restarts Phoenix QueryServer. | | hdfs-encryption-zones | Configure HDFS encryption zones. | This classification should not be reconfigured. | | hdfs-env | Change values in the HDFS environment. | Restarts Hadoop HDFS services Namenode, Datanode, and ZKFC. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Additionally restarts Hadoop Httpfs. | | hcatalog-env | Change values in HCatalog's environment. | Restarts Hive HCatalog Server. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | Restarts Hive HCatalog Server. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | Restarts Hive HCatalog Server. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | Restarts Hive WebHCat server. | | hcatalog-webhcat-log4j2 | Change values in HCatalog WebHCat's log4j2.properties. | Restarts Hive WebHCat server. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | Restarts Hive WebHCat server. | | hive | Amazon EMR-curated settings for Apache Hive. | Sets configurations to launch Hive LLAP service. | | hive-beeline-log4j2 | Change values in Hive's beeline-log4j2.properties file. | Not available. | | hive-parquet-logging | Change values in Hive's parquet-logging.properties file. | Not available. | | hive-env | Change values in the Hive environment. | Restarts HiveServer2, HiveMetastore, and Hive HCatalog-Server. Runs Hive schemaTool CLI commands to verify hive-metastore. | | hive-exec-log4j2 | Change values in Hive's hive-exec-log4j2.properties file. | Not available. | | hive-llap-daemon-log4j2 | Change values in Hive's llap-daemon-log4j2.properties file. | Not available. | | hive-log4j2 | Change values in Hive's hive-log4j2.properties file. | Not available. | | hive-site | Change values in Hive's hive-site.xml file | Restarts HiveServer2, HiveMetastore, and Hive HCatalog-Server. Runs Hive schemaTool CLI commands to verify hive-metastore. Also restarts Oozie and Zeppelin. | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | Not available. | | hue-ini | Change values in Hue's ini file | Restarts Hue. Also activates Hue config override CLI commands to pick up new configurations. | | httpfs-env | Change values in the HTTPFS environment. | Restarts Hadoop Httpfs service. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | Restarts Hadoop Httpfs service. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | Not available. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | Restarts Hadoop-KMS service. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | Not available. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | Restarts Hadoop-KMS and Ranger-KMS service. | | hudi-env | Change values in the Hudi environment. | Not available. | | hudi-defaults | Change values in Hudi's hudi-defaults.conf file. | Not available. | | iceberg-defaults | Change values in Iceberg's iceberg-defaults.conf file. | Not available. | | delta-defaults | Change values in Delta's delta-defaults.conf file. | Not available. | | jupyter-notebook-conf | Change values in Jupyter Notebook's jupyter\$1notebook\$1config.py file. | Not available. | | jupyter-hub-conf | Change values in JupyterHubs's jupyterhub\$1config.py file. | Not available. | | jupyter-s3-conf | Configure Jupyter Notebook S3 persistence. | Not available. | | jupyter-sparkmagic-conf | Change values in Sparkmagic's config.json file. | Not available. | | livy-conf | Change values in Livy's livy.conf file. | Restarts Livy Server. | | livy-env | Change values in the Livy environment. | Restarts Livy Server. | | livy-log4j2 | Change Livy log4j2.properties settings. | Restarts Livy Server. | | mapred-env | Change values in the MapReduce application's environment. | Restarts Hadoop MapReduce-HistoryServer. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | Restarts Hadoop MapReduce-HistoryServer. | | oozie-env | Change values in Oozie's environment. | Restarts Oozie. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | Restarts Oozie. | | oozie-site | Change values in Oozie's oozie-site.xml file. | Restarts Oozie. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | Not available. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | Not available. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | Restarts Phoenix-QueryServer. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | Not available. | | pig-env | Change values in the Pig environment. | Not available. | | pig-properties | Change values in Pig's pig.properties file. | Restarts Oozie. | | pig-log4j | Change values in Pig's log4j.properties file. | Not available. | | presto-log | Change values in Presto's log.properties file. | Restarts Presto-Server (for PrestoDB) | | presto-config | Change values in Presto's config.properties file. | Restarts Presto-Server (for PrestoDB) | | presto-password-authenticator | Change values in Presto's password-authenticator.properties file. | Not available. | | presto-env | Change values in Presto's presto-env.sh file. | Restarts Presto-Server (for PrestoDB) | | presto-node | Change values in Presto's node.properties file. | Not available. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | Not available. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | Not available. | | presto-connector-hive | Change values in Presto's hive.properties file. | Restarts Presto-Server (for PrestoDB) | | presto-connector-jmx | Change values in Presto's jmx.properties file. | Not available. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | Not available. | | presto-connector-lakeformation | Change values in Presto's lakeformation.properties file. | Restarts Presto-Server (for PrestoDB) | | presto-connector-localfile | Change values in Presto's localfile.properties file. | Not available. | | presto-connector-memory | Change values in Presto's memory.properties file. | Not available. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | Not available. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | Not available. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | Not available. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | Not available. | | presto-connector-redis | Change values in Presto's redis.properties file. | Not available. | | presto-connector-redshift | Change values in Presto's redshift.properties file. | Not available. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | Not available. | | presto-connector-tpcds | Change values in Presto's tpcds.properties file. | Not available. | | trino-log | Change values in Trino's log.properties file. | Restarts Trino-Server (for Trino) | | trino-config | Change values in Trino's config.properties file. | Restarts Trino-Server (for Trino) | | trino-password-authenticator | Change values in Trino's password-authenticator.properties file. | Restarts Trino-Server (for Trino) | | trino-env | Change values in Trino's trino-env.sh file. | Restarts Trino-Server (for Trino) | | trino-node | Change values in Trino's node.properties file. | Not available. | | trino-connector-blackhole | Change values in Trino's blackhole.properties file. | Not available. | | trino-connector-cassandra | Change values in Trino's cassandra.properties file. | Not available. | | trino-connector-delta | Change values in Trino's delta.properties file. | Restarts Trino-Server (for Trino) | | trino-connector-hive | Change values in Trino's hive.properties file. | Restarts Trino-Server (for Trino) | | trino-exchange-manager | Change values in Trino's exchange-manager.properties file. | Restarts Trino-Server (for Trino) | | trino-connector-iceberg | Change values in Trino's iceberg.properties file. | Restarts Trino-Server (for Trino) | | trino-connector-hudi | Change values in Trino's hudi.properties file. | Restarts Trino-Server (for Trino) | | trino-connector-jmx | Change values in Trino's jmx.properties file. | Not available. | | trino-connector-kafka | Change values in Trino's kafka.properties file. | Not available. | | trino-connector-localfile | Change values in Trino's localfile.properties file. | Not available. | | trino-connector-memory | Change values in Trino's memory.properties file. | Not available. | | trino-connector-mongodb | Change values in Trino's mongodb.properties file. | Not available. | | trino-connector-mysql | Change values in Trino's mysql.properties file. | Not available. | | trino-connector-postgresql | Change values in Trino's postgresql.properties file. | Not available. | | trino-connector-raptor | Change values in Trino's raptor.properties file. | Not available. | | trino-connector-redis | Change values in Trino's redis.properties file. | Not available. | | trino-connector-redshift | Change values in Trino's redshift.properties file. | Not available. | | trino-connector-tpch | Change values in Trino's tpch.properties file. | Not available. | | trino-connector-tpcds | Change values in Trino's tpcds.properties file. | Not available. | | ranger-kms-dbks-site | Change values in dbks-site.xml file of Ranger KMS. | Restarts Ranger KMS Server. | | ranger-kms-site | Change values in ranger-kms-site.xml file of Ranger KMS. | Restarts Ranger KMS Server. | | ranger-kms-env | Change values in the Ranger KMS environment. | Restarts Ranger KMS Server. | | ranger-kms-logback | Change values in kms-logback.xml file of Ranger KMS. | Not available. | | ranger-kms-db-ca | Change values for CA file on S3 for MySQL SSL connection with Ranger KMS. | Not available. | | spark | Amazon EMR-curated settings for Apache Spark. | This property modifies spark-defaults. See actions there. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | Restarts Spark history server and Spark thrift server. | | spark-env | Change values in the Spark environment. | Restarts Spark history server and Spark thrift server. | | spark-hive-site | Change values in Spark's hive-site.xml file | Not available. | | spark-log4j2 | Change values in Spark's log4j2.properties file. | Restarts Spark history server and Spark thrift server. | | spark-metrics | Change values in Spark's metrics.properties file. | Restarts Spark history server and Spark thrift server. | | sqoop-env | Change values in Sqoop's environment. | Not available. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | Not available. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | Not available. | | tez-site | Change values in Tez's tez-site.xml file. | Restart Oozie and HiveServer2. | | yarn-env | Change values in the YARN environment. | Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts MapReduce-HistoryServer. | | yarn-site | Change values in YARN's yarn-site.xml file. | Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts Livy Server and MapReduce-HistoryServer. | | zeppelin-env | Change values in the Zeppelin environment. | Restarts Zeppelin. | | zeppelin-site | Change configuration settings in zeppelin-site.xml. | Restarts Zeppelin. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | Restarts Zookeeper server. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | Restarts Zookeeper server. | ## 6.10.1 change log **Change log for 6.10.1 release and release notes** | Date | Event | Description | | --- | --- | --- | | 2023-08-30 | Update release notes | Added several control-plane related fixes to the release notes | | 2023-08-21 | Docs publication | Amazon EMR 6.10.1 release notes first published | | 2023-08-16 | Deployment complete | Amazon EMR 6.10.1 fully deployed to all [supported Regions](https://aws.amazon.com/about-aws/global-infrastructure/regional-product-services/) | | 2023-08-04 | Initial release | Amazon EMR 6.10.1 first deployed to limited commercial Regions | # Amazon EMR release 6.10.0 ## 6.10.0 application versions This release includes the following applications: [https://delta.io/](https://delta.io/), [https://flink.apache.org/](https://flink.apache.org/), [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [https://hudi.apache.org](https://hudi.apache.org), [http://gethue.com/](http://gethue.com/), [https://iceberg.apache.org/](https://iceberg.apache.org/), [https://jupyter-enterprise-gateway.readthedocs.io/en/latest/](https://jupyter-enterprise-gateway.readthedocs.io/en/latest/), [https://jupyterhub.readthedocs.io/en/latest/#](https://jupyterhub.readthedocs.io/en/latest/#), [https://livy.incubator.apache.org/](https://livy.incubator.apache.org/), [https://mxnet.incubator.apache.org/](https://mxnet.incubator.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://www.tensorflow.org/](https://www.tensorflow.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://trino.io/](https://trino.io/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-6.10.0 | emr-6.9.1 | emr-6.9.0 | emr-6.8.1 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.12.397 | 1.12.170 | 1.12.170 | 1.12.170 | | Python | 2.7, 3.7 | 2.7, 3.7 | 2.7, 3.7 | 2.7, 3.7 | | Scala | 2.12.15 | 2.12.15 | 2.12.15 | 2.12.15 | | AmazonCloudWatchAgent | - | - | - | - | | Delta | 2.2.0 | 2.1.0 | 2.1.0 | - | | Flink | 1.16.0 | 1.15.2 | 1.15.2 | 1.15.1 | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 2.4.15-amzn-0 | 2.4.13-amzn-0.1 | 2.4.13-amzn-0 | 2.4.12-amzn-0.1 | | HCatalog | 3.1.3-amzn-3 | 3.1.3-amzn-2.1 | 3.1.3-amzn-2 | 3.1.3-amzn-1.1 | | Hadoop | 3.3.3-amzn-2 | 3.3.3-amzn-1.1 | 3.3.3-amzn-1 | 3.2.1-amzn-8.1 | | Hive | 3.1.3-amzn-3 | 3.1.3-amzn-2.1 | 3.1.3-amzn-2 | 3.1.3-amzn-1.1 | | Hudi | 0.12.2-amzn-0 | 0.12.1-amzn-0 | 0.12.1-amzn-0 | 0.11.1-amzn-0 | | Hue | 4.10.0 | 4.10.0 | 4.10.0 | 4.10.0 | | Iceberg | 1.1.0-amzn-0 | 0.14.1-amzn-0 | 0.14.1-amzn-0 | 0.14.0-amzn-0 | | JupyterEnterpriseGateway | 2.6.0 | 2.6.0 | 2.6.0 | 2.1.0 | | JupyterHub | 1.5.0 | 1.4.1 | 1.4.1 | 1.4.1 | | Livy | 0.7.1-incubating | 0.7.1-incubating | 0.7.1-incubating | 0.7.1-incubating | | MXNet | 1.9.1 | 1.9.1 | 1.9.1 | 1.9.1 | | Mahout | - | - | - | - | | Oozie | 5.2.1 | 5.2.1 | 5.2.1 | 5.2.1 | | Phoenix | 5.1.2 | 5.1.2 | 5.1.2 | 5.1.2 | | Pig | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | | Presto | 0.278.1-amzn-0 | 0.276-amzn-0 | 0.276-amzn-0 | 0.273.3-amzn-0 | | Spark | 3.3.1-amzn-0 | 3.3.0-amzn-1.1 | 3.3.0-amzn-1 | 3.3.0-amzn-0.1 | | Sqoop | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | | TensorFlow | 2.11.0 | 2.10.0 | 2.10.0 | 2.9.1 | | Tez | 0.10.2-amzn-1 | 0.10.2-amzn-0.1 | 0.10.2-amzn-0 | 0.9.2 | | Trino (PrestoSQL) | 403-amzn-0 | 398-amzn-0 | 398-amzn-0 | 388-amzn-0 | | Zeppelin | 0.10.1 | 0.10.1 | 0.10.1 | 0.10.1 | | ZooKeeper | 3.5.10 | 3.5.10 | 3.5.10 | 3.5.10 | ## 6.10.0 release notes The following release notes include information for Amazon EMR release 6.10.0. Changes are relative to 6.9.0. For information on the release timeline, see the [change log](https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-6100-release.html#6100-changelog). **New features** + Amazon EMR 6.10.0 supports Apache Spark 3.3.1, Apache Spark RAPIDS 22.12.0, CUDA 11.8.0, Apache Hudi 0.12.2-amzn-0, Apache Iceberg 1.1.0-amzn-0, Trino 403, and PrestoDB 0.278.1. + Amazon EMR 6.10.0 includes a native Trino-Hudi connector that provides read access to data in Hudi tables. You can activate the connector with `trino-cli --catalog hudi`, and configure the connector for your requirements with `trino-connector-hudi`. The native integration with Amazon EMR means that you no longer need to use `trino-connector-hive` to query Hudi tables. For a list of supported configurations with the new connector, see the [Hudi connector](https://trino.io/docs/current/connector/hudi.html) page of the Trino documentation. + Amazon EMR releases 6.10.0 and higher support Apache Zeppelin integration with Apache Flink. See [Working with Flink jobs from Zeppelin in Amazon EMR](flink-zeppelin.md) for more information. **Known Issues** + Hadoop 3.3.3 introduced a change in YARN ([YARN-9608](https://issues.apache.org/jira/browse/YARN-9608)) that keeps nodes where containers ran in a decommissioning state until the application completes. This change ensures that local data such as shuffle data doesn't get lost, and you don' need to re-run the job. This approach might also lead to underutilization of resources on clusters with or without managed scaling enabled. To work around this issue in Amazon EMR 6.10.0, you can set the value of `yarn.resourcemanager.decommissioning-nodes-watcher.wait-for-applications` to `false` in `yarn-site.xml`. In Amazon EMR releases 6.11.0 and higher as well as 6.8.1, 6.9.1, and 6.10.1, the config is set to `false` by default to resolve this issue. + Starting with Spark 3.3.1 (supported in EMR versions 6.10 and above), all executors in a decommissioning host are set to a new `ExecutorState`, called *DECOMMISSIONING* state. The executors being decommissioned cannot be used by Yarn to allocate tasks and thus it will request for new executors, if needed, for the tasks being executed. Thus, if you disable Spark DRA while using EMR Managed Scaling, EMR Auto Scaling, or any custom scaling mechanism on EMR-EC2 clusters, then Yarn may request maximum permissible executors for each job. In order to avoid this issue, leave the `spark.dynamicAllocation.enabled` property set to `TRUE` (which is the default) when you are using the above combination of features. In addition, you can also set minimum and maximum executor constraints by setting values for `spark.dynamicAllocation.maxExecutors` and `spark.dynamicAllocation.minExecutors` properties for your Spark jobs, to restrict the number of executors allocated during the job’s execution. **Changes, enhancements, and resolved issues** + Amazon EMR 6.10.0 removes the dependency on `minimal-json.jar` for the [Amazon Redshift integration for Apache Spark](emr-spark-redshift-launch.md), and automatically adds the required Spark-Redshift related jars to the executor class path for Spark: `spark-redshift.jar`, `spark-avro.jar`, and `RedshiftJDBC.jar`. + The 6.10.0 release improves the on-cluster log management daemon to monitor additional log folders in your EMR cluster. This improvement minimizes disk over-utilization scenarios. + The 6.10.0 release automatically restarts the on-cluster log management daemon when it stops. This improvement reduces the risk for nodes to appear unhealthy due to disk over-utilization. + Amazon EMR 6.10.0 supports regional endpoints for EMRFS user mapping. + The default root volume size has increased to 15 GB in Amazon EMR 6.10.0 and higher. Earlier releases have default root volume size of 10 GB. + The 6.10.0 release fixes an issue that caused Spark jobs to stall when all remaining Spark executors are on a decommissioning host with the YARN resource manager. + With Amazon EMR 6.6.0 through 6.9.x, INSERT queries with dynamic partition and an ORDER BY or SORT BY clause will always have two reducers. This issue is caused by OSS change [HIVE-20703](https://issues.apache.org/jira/browse/HIVE-20703), which puts dynamic sort partition optimization under cost-based decision. If your workload doesn't require sorting of dynamic partitions, we recommend that you set the `hive.optimize.sort.dynamic.partition.threshold` property to `-1` to disable the new feature and get the correctly calculated number of reducers. This issue is fixed in OSS Hive as part of [HIVE-22269](https://issues.apache.org/jira/browse/HIVE-22269) and is fixed in Amazon EMR 6.10.0. + When you launch a cluster with *the latest patch release* of Amazon EMR 5.36 or higher, 6.6 or higher, or 7.0 or higher, Amazon EMR uses the latest Amazon Linux 2023 or Amazon Linux 2 release for the default Amazon EMR AMI. For more information, see [Using the default Amazon Linux AMI for Amazon EMR](https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-default-ami.html). **Note** This release no longer gets automatic AMI updates since it has been succeeded by 1 more more patch releases. The patch release is denoted by the number after the second decimal point (`6.8.1`). To see if you're using the latest patch release, check the available releases in the [https://docs.aws.amazon.com/emr/latest/ReleaseGuide](https://docs.aws.amazon.com/emr/latest/ReleaseGuide), or check the **Amazon EMR release** dropdown when you create a cluster in the console, or use the [https://docs.aws.amazon.com/emr/latest/APIReference/API_ListReleaseLabels.html](https://docs.aws.amazon.com/emr/latest/APIReference/API_ListReleaseLabels.html) API or [https://docs.aws.amazon.com/cli/latest/reference/emr/list-release-labels.html](https://docs.aws.amazon.com/cli/latest/reference/emr/list-release-labels.html) CLI action. To get updates about new releases, subscribe to the RSS feed on the [What's new?](https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-whatsnew.html) page. [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-6100-release.html) ## 6.10.0 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | aws-sagemaker-spark-sdk | 1.4.2 | Amazon SageMaker Spark SDK | | delta | 2.2.0 | Delta lake is an open table format for huge analytic datasets | | emr-ddb | 4.16.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 3.3.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.7.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-notebook-env | 1.7.0 | Conda env for emr notebook which includes jupyter enterprise gateway | | emr-s3-dist-cp | 2.24.0 | Distributed copy application optimized for Amazon S3. | | emr-s3-select | 2.3.0 | EMR S3Select Connector | | emr-wal-cli | 1.0.0 | Cli used for emrwal list/deletion. | | emrfs | 2.55.0 | Amazon S3 connector for Hadoop ecosystem applications. | | flink-client | 1.16.0 | Apache Flink command line client scripts and applications. | | flink-jobmanager-config | 1.16.0 | Managing resources on EMR nodes for Apache Flink JobManager. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 3.3.3-amzn-2 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 3.3.3-amzn-2 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 3.3.3-amzn-2 | HDFS command-line client and library | | hadoop-hdfs-namenode | 3.3.3-amzn-2 | HDFS service for tracking file names and block locations. | | hadoop-hdfs-journalnode | 3.3.3-amzn-2 | HDFS service for managing the Hadoop filesystem journal on HA clusters. | | hadoop-httpfs-server | 3.3.3-amzn-2 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 3.3.3-amzn-2 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 3.3.3-amzn-2 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 3.3.3-amzn-2 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 3.3.3-amzn-2 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 3.3.3-amzn-2 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 2.4.15-amzn-0 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 2.4.15-amzn-0 | Service for serving one or more HBase regions. | | hbase-client | 2.4.15-amzn-0 | HBase command-line client. | | hbase-rest-server | 2.4.15-amzn-0 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 2.4.15-amzn-0 | Service providing a Thrift endpoint to HBase. | | hbase-operator-tools | 2.4.15-amzn-0 | Repair tool for Apache HBase clusters. | | hcatalog-client | 3.1.3-amzn-3 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 3.1.3-amzn-3 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 3.1.3-amzn-3 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 3.1.3-amzn-3 | Hive command line client. | | hive-hbase | 3.1.3-amzn-3 | Hive-hbase client. | | hive-metastore-server | 3.1.3-amzn-3 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server2 | 3.1.3-amzn-3 | Service for accepting Hive queries as web requests. | | hudi | 0.12.2-amzn-0 | Incremental processing framework to power data pipeline at low latency and high efficiency. | | hudi-presto | 0.12.2-amzn-0 | Bundle library for running Presto with Hudi. | | hudi-trino | 0.12.2-amzn-0 | Bundle library for running Trino with Hudi. | | hudi-spark | 0.12.2-amzn-0 | Bundle library for running Spark with Hudi. | | hue-server | 4.10.0 | Web application for analyzing data using Hadoop ecosystem applications | | iceberg | 1.1.0-amzn-0 | Apache Iceberg is an open table format for huge analytic datasets | | jupyterhub | 1.5.0 | Multi-user server for Jupyter notebooks | | livy-server | 0.7.1-incubating | REST interface for interacting with Apache Spark | | nginx | 1.12.1 | nginx [engine x] is an HTTP and reverse proxy server | | mxnet | 1.9.1 | A flexible, scalable, and efficient library for deep learning. | | mariadb-server | 5.5.68\$1 | MariaDB database server. | | nvidia-cuda | 11.8.0 | Nvidia drivers and Cuda toolkit | | oozie-client | 5.2.1 | Oozie command-line client. | | oozie-server | 5.2.1 | Service for accepting Oozie workflow requests. | | opencv | 4.5.0 | Open Source Computer Vision Library. | | phoenix-library | 5.1.2 | The phoenix libraries for server and client | | phoenix-connectors | 6.0.0-SNAPSHOT | Apache Phoenix-Connectors for Spark-3 | | phoenix-query-server | 6.0.0 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.278.1-amzn-0 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.278.1-amzn-0 | Service for executing pieces of a query. | | presto-client | 0.278.1-amzn-0 | Presto command-line client which is installed on an HA cluster's stand-by masters where Presto server is not started. | | trino-coordinator | 403-amzn-0 | Service for accepting queries and managing query execution among trino-workers. | | trino-worker | 403-amzn-0 | Service for executing pieces of a query. | | trino-client | 403-amzn-0 | Trino command-line client which is installed on an HA cluster's stand-by masters where Trino server is not started. | | pig-client | 0.17.0 | Pig command-line client. | | r | 4.0.2 | The R Project for Statistical Computing | | ranger-kms-server | 2.0.0 | Apache Ranger Key Management System | | spark-client | 3.3.1-amzn-0 | Spark command-line clients. | | spark-history-server | 3.3.1-amzn-0 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 3.3.1-amzn-0 | In-memory execution engine for YARN. | | spark-yarn-slave | 3.3.1-amzn-0 | Apache Spark libraries needed by YARN slaves. | | spark-rapids | 22.12.0-amzn-0 | Nvidia Spark RAPIDS plugin that accelerates Apache Spark with GPUs. | | sqoop-client | 1.4.7 | Apache Sqoop command-line client. | | tensorflow | 2.11.0 | TensorFlow open source software library for high performance numerical computation. | | tez-on-yarn | 0.10.2-amzn-1 | The tez YARN application and libraries. | | tez-on-worker | 0.10.2-amzn-1 | The tez YARN application and libraries for worker nodes. | | webserver | 2.4.41\$1 | Apache HTTP server. | | zeppelin-server | 0.10.1 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.5.10 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.5.10 | ZooKeeper command line client. | ## 6.10.0 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). Reconfiguration actions occur when you specify a configuration for instance groups in a running cluster. Amazon EMR only initiates reconfiguration actions for the classifications that you modify. For more information, see [Reconfigure an instance group in a running cluster](emr-configure-apps-running-cluster.md). **emr-6.10.0 classifications** | Classifications | Description | Reconfiguration Actions | | --- | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | Restarts the ResourceManager service. | | container-executor | Change values in Hadoop YARN's container-executor.cfg file. | Not available. | | container-log4j | Change values in Hadoop YARN's container-log4j.properties file. | Not available. | | core-site | Change values in Hadoop's core-site.xml file. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts Hadoop KMS, Ranger KMS, HiveServer2, Hive MetaStore, Hadoop Httpfs, and MapReduce-HistoryServer. | | docker-conf | Change docker related settings. | Not available. | | emrfs-site | Change EMRFS settings. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts HBaseRegionserver, HBaseMaster, HBaseThrift, HBaseRest, HiveServer2, Hive MetaStore, Hadoop Httpfs, and MapReduce-HistoryServer. | | flink-conf | Change flink-conf.yaml settings. | Restarts Flink history server. | | flink-log4j | Change Flink log4j.properties settings. | Restarts Flink history server. | | flink-log4j-session | Change Flink log4j-session.properties settings for Kubernetes/Yarn session. | Restarts Flink history server. | | flink-log4j-cli | Change Flink log4j-cli.properties settings. | Restarts Flink history server. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts PhoenixQueryserver, HiveServer2, Hive MetaStore, and MapReduce-HistoryServer. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | Restarts the Hadoop HDFS services SecondaryNamenode, Datanode, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts Hadoop KMS, Hadoop Httpfs, and MapReduce-HistoryServer. | | hadoop-ssl-server | Change hadoop ssl server configuration | Not available. | | hadoop-ssl-client | Change hadoop ssl client configuration | Not available. | | hbase | Amazon EMR-curated settings for Apache HBase. | Custom EMR specific property. Sets emrfs-site and hbase-site configs. See those for their associated restarts. | | hbase-env | Change values in HBase's environment. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbase.properties file. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | Not available. | | hbase-site | Change values in HBase's hbase-site.xml file. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. Additionally restarts Phoenix QueryServer. | | hdfs-encryption-zones | Configure HDFS encryption zones. | This classification should not be reconfigured. | | hdfs-env | Change values in the HDFS environment. | Restarts Hadoop HDFS services Namenode, Datanode, and ZKFC. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Additionally restarts Hadoop Httpfs. | | hcatalog-env | Change values in HCatalog's environment. | Restarts Hive HCatalog Server. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | Restarts Hive HCatalog Server. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | Restarts Hive HCatalog Server. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | Restarts Hive WebHCat server. | | hcatalog-webhcat-log4j2 | Change values in HCatalog WebHCat's log4j2.properties. | Restarts Hive WebHCat server. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | Restarts Hive WebHCat server. | | hive | Amazon EMR-curated settings for Apache Hive. | Sets configurations to launch Hive LLAP service. | | hive-beeline-log4j2 | Change values in Hive's beeline-log4j2.properties file. | Not available. | | hive-parquet-logging | Change values in Hive's parquet-logging.properties file. | Not available. | | hive-env | Change values in the Hive environment. | Restarts HiveServer2, HiveMetastore, and Hive HCatalog-Server. Runs Hive schemaTool CLI commands to verify hive-metastore. | | hive-exec-log4j2 | Change values in Hive's hive-exec-log4j2.properties file. | Not available. | | hive-llap-daemon-log4j2 | Change values in Hive's llap-daemon-log4j2.properties file. | Not available. | | hive-log4j2 | Change values in Hive's hive-log4j2.properties file. | Not available. | | hive-site | Change values in Hive's hive-site.xml file | Restarts HiveServer2, HiveMetastore, and Hive HCatalog-Server. Runs Hive schemaTool CLI commands to verify hive-metastore. Also restarts Oozie and Zeppelin. | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | Not available. | | hue-ini | Change values in Hue's ini file | Restarts Hue. Also activates Hue config override CLI commands to pick up new configurations. | | httpfs-env | Change values in the HTTPFS environment. | Restarts Hadoop Httpfs service. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | Restarts Hadoop Httpfs service. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | Not available. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | Restarts Hadoop-KMS service. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | Not available. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | Restarts Hadoop-KMS and Ranger-KMS service. | | hudi-env | Change values in the Hudi environment. | Not available. | | hudi-defaults | Change values in Hudi's hudi-defaults.conf file. | Not available. | | iceberg-defaults | Change values in Iceberg's iceberg-defaults.conf file. | Not available. | | delta-defaults | Change values in Delta's delta-defaults.conf file. | Not available. | | jupyter-notebook-conf | Change values in Jupyter Notebook's jupyter\$1notebook\$1config.py file. | Not available. | | jupyter-hub-conf | Change values in JupyterHubs's jupyterhub\$1config.py file. | Not available. | | jupyter-s3-conf | Configure Jupyter Notebook S3 persistence. | Not available. | | jupyter-sparkmagic-conf | Change values in Sparkmagic's config.json file. | Not available. | | livy-conf | Change values in Livy's livy.conf file. | Restarts Livy Server. | | livy-env | Change values in the Livy environment. | Restarts Livy Server. | | livy-log4j2 | Change Livy log4j2.properties settings. | Restarts Livy Server. | | mapred-env | Change values in the MapReduce application's environment. | Restarts Hadoop MapReduce-HistoryServer. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | Restarts Hadoop MapReduce-HistoryServer. | | oozie-env | Change values in Oozie's environment. | Restarts Oozie. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | Restarts Oozie. | | oozie-site | Change values in Oozie's oozie-site.xml file. | Restarts Oozie. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | Not available. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | Not available. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | Restarts Phoenix-QueryServer. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | Not available. | | pig-env | Change values in the Pig environment. | Not available. | | pig-properties | Change values in Pig's pig.properties file. | Restarts Oozie. | | pig-log4j | Change values in Pig's log4j.properties file. | Not available. | | presto-log | Change values in Presto's log.properties file. | Restarts Presto-Server (for PrestoDB) | | presto-config | Change values in Presto's config.properties file. | Restarts Presto-Server (for PrestoDB) | | presto-password-authenticator | Change values in Presto's password-authenticator.properties file. | Not available. | | presto-env | Change values in Presto's presto-env.sh file. | Restarts Presto-Server (for PrestoDB) | | presto-node | Change values in Presto's node.properties file. | Not available. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | Not available. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | Not available. | | presto-connector-hive | Change values in Presto's hive.properties file. | Restarts Presto-Server (for PrestoDB) | | presto-connector-jmx | Change values in Presto's jmx.properties file. | Not available. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | Not available. | | presto-connector-lakeformation | Change values in Presto's lakeformation.properties file. | Restarts Presto-Server (for PrestoDB) | | presto-connector-localfile | Change values in Presto's localfile.properties file. | Not available. | | presto-connector-memory | Change values in Presto's memory.properties file. | Not available. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | Not available. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | Not available. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | Not available. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | Not available. | | presto-connector-redis | Change values in Presto's redis.properties file. | Not available. | | presto-connector-redshift | Change values in Presto's redshift.properties file. | Not available. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | Not available. | | presto-connector-tpcds | Change values in Presto's tpcds.properties file. | Not available. | | trino-log | Change values in Trino's log.properties file. | Restarts Trino-Server (for Trino) | | trino-config | Change values in Trino's config.properties file. | Restarts Trino-Server (for Trino) | | trino-password-authenticator | Change values in Trino's password-authenticator.properties file. | Restarts Trino-Server (for Trino) | | trino-env | Change values in Trino's trino-env.sh file. | Restarts Trino-Server (for Trino) | | trino-node | Change values in Trino's node.properties file. | Not available. | | trino-connector-blackhole | Change values in Trino's blackhole.properties file. | Not available. | | trino-connector-cassandra | Change values in Trino's cassandra.properties file. | Not available. | | trino-connector-delta | Change values in Trino's delta.properties file. | Restarts Trino-Server (for Trino) | | trino-connector-hive | Change values in Trino's hive.properties file. | Restarts Trino-Server (for Trino) | | trino-exchange-manager | Change values in Trino's exchange-manager.properties file. | Restarts Trino-Server (for Trino) | | trino-connector-iceberg | Change values in Trino's iceberg.properties file. | Restarts Trino-Server (for Trino) | | trino-connector-hudi | Change values in Trino's hudi.properties file. | Restarts Trino-Server (for Trino) | | trino-connector-jmx | Change values in Trino's jmx.properties file. | Not available. | | trino-connector-kafka | Change values in Trino's kafka.properties file. | Not available. | | trino-connector-localfile | Change values in Trino's localfile.properties file. | Not available. | | trino-connector-memory | Change values in Trino's memory.properties file. | Not available. | | trino-connector-mongodb | Change values in Trino's mongodb.properties file. | Not available. | | trino-connector-mysql | Change values in Trino's mysql.properties file. | Not available. | | trino-connector-postgresql | Change values in Trino's postgresql.properties file. | Not available. | | trino-connector-raptor | Change values in Trino's raptor.properties file. | Not available. | | trino-connector-redis | Change values in Trino's redis.properties file. | Not available. | | trino-connector-redshift | Change values in Trino's redshift.properties file. | Not available. | | trino-connector-tpch | Change values in Trino's tpch.properties file. | Not available. | | trino-connector-tpcds | Change values in Trino's tpcds.properties file. | Not available. | | ranger-kms-dbks-site | Change values in dbks-site.xml file of Ranger KMS. | Restarts Ranger KMS Server. | | ranger-kms-site | Change values in ranger-kms-site.xml file of Ranger KMS. | Restarts Ranger KMS Server. | | ranger-kms-env | Change values in the Ranger KMS environment. | Restarts Ranger KMS Server. | | ranger-kms-logback | Change values in kms-logback.xml file of Ranger KMS. | Not available. | | ranger-kms-db-ca | Change values for CA file on S3 for MySQL SSL connection with Ranger KMS. | Not available. | | spark | Amazon EMR-curated settings for Apache Spark. | This property modifies spark-defaults. See actions there. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | Restarts Spark history server and Spark thrift server. | | spark-env | Change values in the Spark environment. | Restarts Spark history server and Spark thrift server. | | spark-hive-site | Change values in Spark's hive-site.xml file | Not available. | | spark-log4j2 | Change values in Spark's log4j2.properties file. | Restarts Spark history server and Spark thrift server. | | spark-metrics | Change values in Spark's metrics.properties file. | Restarts Spark history server and Spark thrift server. | | sqoop-env | Change values in Sqoop's environment. | Not available. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | Not available. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | Not available. | | tez-site | Change values in Tez's tez-site.xml file. | Restart Oozie and HiveServer2. | | yarn-env | Change values in the YARN environment. | Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts MapReduce-HistoryServer. | | yarn-site | Change values in YARN's yarn-site.xml file. | Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts Livy Server and MapReduce-HistoryServer. | | zeppelin-env | Change values in the Zeppelin environment. | Restarts Zeppelin. | | zeppelin-site | Change configuration settings in zeppelin-site.xml. | Restarts Zeppelin. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | Restarts Zookeeper server. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | Restarts Zookeeper server. | ## 6.10.0 change log **Change log for 6.10.0 release and release notes** | Date | Event | Description | | --- | --- | --- | | 2023-08-21 | Update | Added a known issue introduced with Hadoop 3.3.3. | | 2023-07-26 | Update | New OS release labels 2.0.20230612.0 and 2.0.20230628.0. | | 2023-03-02 | Deployment complete | Amazon EMR 6.10 fully deployed to all [supported Regions](https://aws.amazon.com/about-aws/global-infrastructure/regional-product-services/) | | 2023-03-02 | Docs publication | Amazon EMR 6.10 release notes first published | | 2023-02-27 | Initial release | Amazon EMR 6.10 deployed to limited commercial Regions | # Amazon EMR release 6.9.1 ## 6.9.1 application versions This release includes the following applications: [https://delta.io/](https://delta.io/), [https://flink.apache.org/](https://flink.apache.org/), [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [https://hudi.apache.org](https://hudi.apache.org), [http://gethue.com/](http://gethue.com/), [https://iceberg.apache.org/](https://iceberg.apache.org/), [https://jupyter-enterprise-gateway.readthedocs.io/en/latest/](https://jupyter-enterprise-gateway.readthedocs.io/en/latest/), [https://jupyterhub.readthedocs.io/en/latest/#](https://jupyterhub.readthedocs.io/en/latest/#), [https://livy.incubator.apache.org/](https://livy.incubator.apache.org/), [https://mxnet.incubator.apache.org/](https://mxnet.incubator.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://www.tensorflow.org/](https://www.tensorflow.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://trino.io/](https://trino.io/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-6.9.1 | emr-6.9.0 | emr-6.8.1 | emr-6.8.0 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.12.170 | 1.12.170 | 1.12.170 | 1.12.170 | | Python | 2.7, 3.7 | 2.7, 3.7 | 2.7, 3.7 | 2.7, 3.7 | | Scala | 2.12.15 | 2.12.15 | 2.12.15 | 2.12.15 | | AmazonCloudWatchAgent | - | - | - | - | | Delta | 2.1.0 | 2.1.0 | - | - | | Flink | 1.15.2 | 1.15.2 | 1.15.1 | 1.15.1 | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 2.4.13-amzn-0.1 | 2.4.13-amzn-0 | 2.4.12-amzn-0.1 | 2.4.12-amzn-0 | | HCatalog | 3.1.3-amzn-2.1 | 3.1.3-amzn-2 | 3.1.3-amzn-1.1 | 3.1.3-amzn-1 | | Hadoop | 3.3.3-amzn-1.1 | 3.3.3-amzn-1 | 3.2.1-amzn-8.1 | 3.2.1-amzn-8 | | Hive | 3.1.3-amzn-2.1 | 3.1.3-amzn-2 | 3.1.3-amzn-1.1 | 3.1.3-amzn-1 | | Hudi | 0.12.1-amzn-0 | 0.12.1-amzn-0 | 0.11.1-amzn-0 | 0.11.1-amzn-0 | | Hue | 4.10.0 | 4.10.0 | 4.10.0 | 4.10.0 | | Iceberg | 0.14.1-amzn-0 | 0.14.1-amzn-0 | 0.14.0-amzn-0 | 0.14.0-amzn-0 | | JupyterEnterpriseGateway | 2.6.0 | 2.6.0 | 2.1.0 | 2.1.0 | | JupyterHub | 1.4.1 | 1.4.1 | 1.4.1 | 1.4.1 | | Livy | 0.7.1-incubating | 0.7.1-incubating | 0.7.1-incubating | 0.7.1-incubating | | MXNet | 1.9.1 | 1.9.1 | 1.9.1 | 1.9.1 | | Mahout | - | - | - | - | | Oozie | 5.2.1 | 5.2.1 | 5.2.1 | 5.2.1 | | Phoenix | 5.1.2 | 5.1.2 | 5.1.2 | 5.1.2 | | Pig | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | | Presto | 0.276-amzn-0 | 0.276-amzn-0 | 0.273.3-amzn-0 | 0.273.3-amzn-0 | | Spark | 3.3.0-amzn-1.1 | 3.3.0-amzn-1 | 3.3.0-amzn-0.1 | 3.3.0-amzn-0 | | Sqoop | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | | TensorFlow | 2.10.0 | 2.10.0 | 2.9.1 | 2.9.1 | | Tez | 0.10.2-amzn-0.1 | 0.10.2-amzn-0 | 0.9.2 | 0.9.2 | | Trino (PrestoSQL) | 398-amzn-0 | 398-amzn-0 | 388-amzn-0 | 388-amzn-0 | | Zeppelin | 0.10.1 | 0.10.1 | 0.10.1 | 0.10.1 | | ZooKeeper | 3.5.10 | 3.5.10 | 3.5.10 | 3.5.10 | ## 6.9.1 release notes The following release notes include information for Amazon EMR release 6.9.1. Changes are relative to 6.9.0. For information on the release timeline, see the [6.9.1 change log](#691-changelog). **Changes, enhancements, and resolved issues** + Hadoop 3.3.3 introduced a change in YARN ([YARN-9608](https://issues.apache.org/jira/browse/YARN-9608)) that keeps nodes where containers ran in a decommissioning state until the application completes. This change ensures that local data such as shuffle data doesn't get lost, and you don' need to re-run the job. This approach might also lead to underutilization of resources on clusters with or without managed scaling enabled. With Amazon EMR releases 6.11.0 and higher as well as 6.8.1, 6.9.1, and 6.10.1, the value of `yarn.resourcemanager.decommissioning-nodes-watcher.wait-for-applications` is set to `false` in `yarn-site.xml` to resolve this issue. While the fix addresses the issues that were introduced by YARN-9608, it might cause Hive jobs to fail due to shuffle data loss on clusters that have managed scaling enabled. We've mitigated that risk in this release by also setting `yarn.resourcemanager.decommissioning-nodes-watcher.wait-for-shuffle-data` for Hive workloads. This config is only available with Amazon EMR releases 6.11.0 and higher. + Metrics collector won't send any metrics to the control plane after failover of primary node in clusters with the instance groups configuration. + This release eliminates retries on failed HTTP requests to metrics collector endpoints. + This release includes a change that allows high-availability clusters to recover from failed state after restart. + This release fixes an issue where large user-created UIDs caused overflow exceptions. + This release fixes timeout issues with the Amazon EMR reconfiguration process. + This release includes security fixes. + This release fixes an issue where clusters that are running workloads on Spark with Amazon EMR might silently receive incorrect results with `contains`, `startsWith`, `endsWith`, and `like`. This issue occurs when you use the expressions on partitioned fields that have metadata in the Amazon EMR Hive3 Metastore Server (HMS). + With Amazon EMR 6.6.0 through 6.9.x, INSERT queries with dynamic partition and an ORDER BY or SORT BY clause will always have two reducers. This issue is caused by OSS change [HIVE-20703](https://issues.apache.org/jira/browse/HIVE-20703), which puts dynamic sort partition optimization under cost-based decision. If your workload doesn't require sorting of dynamic partitions, we recommend that you set the `hive.optimize.sort.dynamic.partition.threshold` property to `-1` to disable the new feature and get the correctly calculated number of reducers. This issue is fixed in OSS Hive as part of [HIVE-22269](https://issues.apache.org/jira/browse/HIVE-22269) and is fixed in Amazon EMR 6.10.0. + Hive might experience data loss when you use HDFS as a scratch directory and you have enabled merge small files, and the table contains static partition paths. + This release fixes a performance issue with Hive if merge small files (disabled by default) is enabled at the end of ETL job. + This release fixes an issue with throttling on the Glue side when there are no user-defined functions (UDF). + This release fixes an issue that deletes container logs by the node log aggregation service before log pusher can push them to S3 in case of YARN decommissioning. + This release fixes handling of compacted/archived files with persistent storefile tracking for HBase. + This release fixes an issue that impacted Spark performance when you set a default `true` value for the `spark.yarn.heterogeneousExecutors.enabled` config in `spark-defaults.conf`. + This release fixes an issue with Reduce Task failing to read shuffle data. The issue caused Hive query failures with a corrupted memory error. + This release fixes an issue that caused the node provisioner to fail if the HDFS NameNode (NN) service was stuck in safemode during node replacement. + This release adds a new retry mechanism to the cluster scaling workflow for EMR clusters that run Presto or Trino. This improvement reduces the risk that cluster resizing will indefinitely stall due to a single failed resize operation. It also improves cluster utilization, because your cluster scales up and down faster. + This release improves cluster scale-down logic so that your cluster doesn't attempt a scale-down of core nodes below the HDFS replication factor setting for the cluster. This aligns with your data redundancy requirements, and reduces the chance that a scaling operation might stall. + The log management daemon has been upgraded to identify all logs that are in active use with open file handles on the local instance storage, and the associated processes. This upgrade ensures that Amazon EMR properly deletes the files and reclaims storage space after the logs are archived to Amazon S3. + This release includes a log-management daemon enhancement that deletes empty, unused steps directories in the local cluster file system. An excessively large number of empty directories can degrade the performance of Amazon EMR daemons and result in disk over-utilization. + This release fixes an issue that might occur when you create an edge node by replicating one of the primary nodes from a cluster with multiple primary nodes. The replicated edge node could cause delays with scale-down operations, or result in high memory-utilization on the primary nodes. For more information on how to create an edge node to communicate with your EMR cluster, see [Edge Node Creator](https://github.com/aws-samples/aws-emr-utilities/tree/main/utilities/emr-edge-node-creator) in the `aws-samples` repo on GitHub. + This release improves the automation process that Amazon EMR uses to re-mount Amazon EBS volumes to an instance after a reboot. + This release fixes an issue that resulted in intermittent gaps in the Hadoop metrics that Amazon EMR publishes to Amazon CloudWatch. + This release fixes an issue with EMR clusters where an update to the YARN configuration file that contains the exclusion list of nodes for the cluster is interrupted due to disk over-utilization. The incomplete update hinders future cluster scale-down operations. This release ensures that your cluster remains healthy, and that scaling operations work as expected. + This release improves the on-cluster log management daemon to monitor additional log folders in your EMR cluster. This improvement minimizes disk over-utilization scenarios. + This release automatically restarts the on-cluster log management daemon when it stops. This improvement reduces the risk for nodes to appear unhealthy due to disk over-utilization. + When you launch a cluster with *the latest patch release* of Amazon EMR 5.36 or higher, 6.6 or higher, or 7.0 or higher, Amazon EMR uses the latest Amazon Linux 2023 or Amazon Linux 2 release for the default Amazon EMR AMI. For more information, see [Using the default Amazon Linux AMI for Amazon EMR](https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-default-ami.html). [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-691-release.html) ## 6.9.1 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | aws-sagemaker-spark-sdk | 1.4.2 | Amazon SageMaker Spark SDK | | delta | 2.1.0 | Delta lake is an open table format for huge analytic datasets | | emr-ddb | 4.16.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 3.3.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.6.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-notebook-env | 1.7.0 | Conda env for emr notebook which includes jupyter enterprise gateway | | emr-s3-dist-cp | 2.23.0 | Distributed copy application optimized for Amazon S3. | | emr-s3-select | 2.2.0 | EMR S3Select Connector | | emrfs | 2.54.0 | Amazon S3 connector for Hadoop ecosystem applications. | | flink-client | 1.15.2 | Apache Flink command line client scripts and applications. | | flink-jobmanager-config | 1.15.2 | Managing resources on EMR nodes for Apache Flink JobManager. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 3.3.3-amzn-1.1 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 3.3.3-amzn-1.1 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 3.3.3-amzn-1.1 | HDFS command-line client and library | | hadoop-hdfs-namenode | 3.3.3-amzn-1.1 | HDFS service for tracking file names and block locations. | | hadoop-hdfs-journalnode | 3.3.3-amzn-1.1 | HDFS service for managing the Hadoop filesystem journal on HA clusters. | | hadoop-httpfs-server | 3.3.3-amzn-1.1 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 3.3.3-amzn-1.1 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 3.3.3-amzn-1.1 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 3.3.3-amzn-1.1 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 3.3.3-amzn-1.1 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 3.3.3-amzn-1.1 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 2.4.13-amzn-0.1 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 2.4.13-amzn-0.1 | Service for serving one or more HBase regions. | | hbase-client | 2.4.13-amzn-0.1 | HBase command-line client. | | hbase-rest-server | 2.4.13-amzn-0.1 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 2.4.13-amzn-0.1 | Service providing a Thrift endpoint to HBase. | | hbase-operator-tools | 2.4.13-amzn-0.1 | Repair tool for Apache HBase clusters. | | hcatalog-client | 3.1.3-amzn-2.1 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 3.1.3-amzn-2.1 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 3.1.3-amzn-2.1 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 3.1.3-amzn-2.1 | Hive command line client. | | hive-hbase | 3.1.3-amzn-2.1 | Hive-hbase client. | | hive-metastore-server | 3.1.3-amzn-2.1 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server2 | 3.1.3-amzn-2.1 | Service for accepting Hive queries as web requests. | | hudi | 0.12.1-amzn-0 | Incremental processing framework to power data pipeline at low latency and high efficiency. | | hudi-presto | 0.12.1-amzn-0 | Bundle library for running Presto with Hudi. | | hudi-trino | 0.12.1-amzn-0 | Bundle library for running Trino with Hudi. | | hudi-spark | 0.12.1-amzn-0 | Bundle library for running Spark with Hudi. | | hue-server | 4.10.0 | Web application for analyzing data using Hadoop ecosystem applications | | iceberg | 0.14.1-amzn-0 | Apache Iceberg is an open table format for huge analytic datasets | | jupyterhub | 1.4.1 | Multi-user server for Jupyter notebooks | | livy-server | 0.7.1-incubating | REST interface for interacting with Apache Spark | | nginx | 1.12.1 | nginx [engine x] is an HTTP and reverse proxy server | | mxnet | 1.9.1 | A flexible, scalable, and efficient library for deep learning. | | mariadb-server | 5.5.68\$1 | MariaDB database server. | | nvidia-cuda | 11.7.0 | Nvidia drivers and Cuda toolkit | | oozie-client | 5.2.1 | Oozie command-line client. | | oozie-server | 5.2.1 | Service for accepting Oozie workflow requests. | | opencv | 4.5.0 | Open Source Computer Vision Library. | | phoenix-library | 5.1.2 | The phoenix libraries for server and client | | phoenix-connectors | 6.0.0-SNAPSHOT | Apache Phoenix-Connectors for Spark-3 | | phoenix-query-server | 6.0.0 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.276-amzn-0 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.276-amzn-0 | Service for executing pieces of a query. | | presto-client | 0.276-amzn-0 | Presto command-line client which is installed on an HA cluster's stand-by masters where Presto server is not started. | | trino-coordinator | 398-amzn-0 | Service for accepting queries and managing query execution among trino-workers. | | trino-worker | 398-amzn-0 | Service for executing pieces of a query. | | trino-client | 398-amzn-0 | Trino command-line client which is installed on an HA cluster's stand-by masters where Trino server is not started. | | pig-client | 0.17.0 | Pig command-line client. | | r | 4.0.2 | The R Project for Statistical Computing | | ranger-kms-server | 2.0.0 | Apache Ranger Key Management System | | spark-client | 3.3.0-amzn-1.1 | Spark command-line clients. | | spark-history-server | 3.3.0-amzn-1.1 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 3.3.0-amzn-1.1 | In-memory execution engine for YARN. | | spark-yarn-slave | 3.3.0-amzn-1.1 | Apache Spark libraries needed by YARN slaves. | | spark-rapids | 22.08.0-amzn-0 | Nvidia Spark RAPIDS plugin that accelerates Apache Spark with GPUs. | | sqoop-client | 1.4.7 | Apache Sqoop command-line client. | | tensorflow | 2.10.0 | TensorFlow open source software library for high performance numerical computation. | | tez-on-yarn | 0.10.2-amzn-0.1 | The tez YARN application and libraries. | | webserver | 2.4.41\$1 | Apache HTTP server. | | zeppelin-server | 0.10.1 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.5.10 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.5.10 | ZooKeeper command line client. | ## 6.9.1 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). Reconfiguration actions occur when you specify a configuration for instance groups in a running cluster. Amazon EMR only initiates reconfiguration actions for the classifications that you modify. For more information, see [Reconfigure an instance group in a running cluster](emr-configure-apps-running-cluster.md). **emr-6.9.1 classifications** | Classifications | Description | Reconfiguration Actions | | --- | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | Restarts the ResourceManager service. | | container-executor | Change values in Hadoop YARN's container-executor.cfg file. | Not available. | | container-log4j | Change values in Hadoop YARN's container-log4j.properties file. | Not available. | | core-site | Change values in Hadoop's core-site.xml file. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts Hadoop KMS, Ranger KMS, HiveServer2, Hive MetaStore, Hadoop Httpfs, and MapReduce-HistoryServer. | | docker-conf | Change docker related settings. | Not available. | | emrfs-site | Change EMRFS settings. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts HBaseRegionserver, HBaseMaster, HBaseThrift, HBaseRest, HiveServer2, Hive MetaStore, Hadoop Httpfs, and MapReduce-HistoryServer. | | flink-conf | Change flink-conf.yaml settings. | Restarts Flink history server. | | flink-log4j | Change Flink log4j.properties settings. | Restarts Flink history server. | | flink-log4j-session | Change Flink log4j-session.properties settings for Kubernetes/Yarn session. | Restarts Flink history server. | | flink-log4j-cli | Change Flink log4j-cli.properties settings. | Restarts Flink history server. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts PhoenixQueryserver, HiveServer2, Hive MetaStore, and MapReduce-HistoryServer. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | Restarts the Hadoop HDFS services SecondaryNamenode, Datanode, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts Hadoop KMS, Hadoop Httpfs, and MapReduce-HistoryServer. | | hadoop-ssl-server | Change hadoop ssl server configuration | Not available. | | hadoop-ssl-client | Change hadoop ssl client configuration | Not available. | | hbase | Amazon EMR-curated settings for Apache HBase. | Custom EMR specific property. Sets emrfs-site and hbase-site configs. See those for their associated restarts. | | hbase-env | Change values in HBase's environment. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbase.properties file. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | Not available. | | hbase-site | Change values in HBase's hbase-site.xml file. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. Additionally restarts Phoenix QueryServer. | | hdfs-encryption-zones | Configure HDFS encryption zones. | This classification should not be reconfigured. | | hdfs-env | Change values in the HDFS environment. | Restarts Hadoop HDFS services Namenode, Datanode, and ZKFC. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Additionally restarts Hadoop Httpfs. | | hcatalog-env | Change values in HCatalog's environment. | Restarts Hive HCatalog Server. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | Restarts Hive HCatalog Server. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | Restarts Hive HCatalog Server. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | Restarts Hive WebHCat server. | | hcatalog-webhcat-log4j2 | Change values in HCatalog WebHCat's log4j2.properties. | Restarts Hive WebHCat server. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | Restarts Hive WebHCat server. | | hive | Amazon EMR-curated settings for Apache Hive. | Sets configurations to launch Hive LLAP service. | | hive-beeline-log4j2 | Change values in Hive's beeline-log4j2.properties file. | Not available. | | hive-parquet-logging | Change values in Hive's parquet-logging.properties file. | Not available. | | hive-env | Change values in the Hive environment. | Restarts HiveServer2, HiveMetastore, and Hive HCatalog-Server. Runs Hive schemaTool CLI commands to verify hive-metastore. | | hive-exec-log4j2 | Change values in Hive's hive-exec-log4j2.properties file. | Not available. | | hive-llap-daemon-log4j2 | Change values in Hive's llap-daemon-log4j2.properties file. | Not available. | | hive-log4j2 | Change values in Hive's hive-log4j2.properties file. | Not available. | | hive-site | Change values in Hive's hive-site.xml file | Restarts HiveServer2, HiveMetastore, and Hive HCatalog-Server. Runs Hive schemaTool CLI commands to verify hive-metastore. Also restarts Oozie and Zeppelin. | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | Not available. | | hue-ini | Change values in Hue's ini file | Restarts Hue. Also activates Hue config override CLI commands to pick up new configurations. | | httpfs-env | Change values in the HTTPFS environment. | Restarts Hadoop Httpfs service. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | Restarts Hadoop Httpfs service. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | Not available. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | Restarts Hadoop-KMS service. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | Not available. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | Restarts Hadoop-KMS and Ranger-KMS service. | | hudi-env | Change values in the Hudi environment. | Not available. | | hudi-defaults | Change values in Hudi's hudi-defaults.conf file. | Not available. | | iceberg-defaults | Change values in Iceberg's iceberg-defaults.conf file. | Not available. | | delta-defaults | Change values in Delta's delta-defaults.conf file. | Not available. | | jupyter-notebook-conf | Change values in Jupyter Notebook's jupyter\$1notebook\$1config.py file. | Not available. | | jupyter-hub-conf | Change values in JupyterHubs's jupyterhub\$1config.py file. | Not available. | | jupyter-s3-conf | Configure Jupyter Notebook S3 persistence. | Not available. | | jupyter-sparkmagic-conf | Change values in Sparkmagic's config.json file. | Not available. | | livy-conf | Change values in Livy's livy.conf file. | Restarts Livy Server. | | livy-env | Change values in the Livy environment. | Restarts Livy Server. | | livy-log4j2 | Change Livy log4j2.properties settings. | Restarts Livy Server. | | mapred-env | Change values in the MapReduce application's environment. | Restarts Hadoop MapReduce-HistoryServer. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | Restarts Hadoop MapReduce-HistoryServer. | | oozie-env | Change values in Oozie's environment. | Restarts Oozie. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | Restarts Oozie. | | oozie-site | Change values in Oozie's oozie-site.xml file. | Restarts Oozie. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | Not available. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | Not available. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | Restarts Phoenix-QueryServer. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | Not available. | | pig-env | Change values in the Pig environment. | Not available. | | pig-properties | Change values in Pig's pig.properties file. | Restarts Oozie. | | pig-log4j | Change values in Pig's log4j.properties file. | Not available. | | presto-log | Change values in Presto's log.properties file. | Restarts Presto-Server (for PrestoDB) | | presto-config | Change values in Presto's config.properties file. | Restarts Presto-Server (for PrestoDB) | | presto-password-authenticator | Change values in Presto's password-authenticator.properties file. | Not available. | | presto-env | Change values in Presto's presto-env.sh file. | Restarts Presto-Server (for PrestoDB) | | presto-node | Change values in Presto's node.properties file. | Not available. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | Not available. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | Not available. | | presto-connector-hive | Change values in Presto's hive.properties file. | Restarts Presto-Server (for PrestoDB) | | presto-connector-jmx | Change values in Presto's jmx.properties file. | Not available. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | Not available. | | presto-connector-lakeformation | Change values in Presto's lakeformation.properties file. | Restarts Presto-Server (for PrestoDB) | | presto-connector-localfile | Change values in Presto's localfile.properties file. | Not available. | | presto-connector-memory | Change values in Presto's memory.properties file. | Not available. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | Not available. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | Not available. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | Not available. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | Not available. | | presto-connector-redis | Change values in Presto's redis.properties file. | Not available. | | presto-connector-redshift | Change values in Presto's redshift.properties file. | Not available. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | Not available. | | presto-connector-tpcds | Change values in Presto's tpcds.properties file. | Not available. | | trino-log | Change values in Trino's log.properties file. | Restarts Trino-Server (for Trino) | | trino-config | Change values in Trino's config.properties file. | Restarts Trino-Server (for Trino) | | trino-password-authenticator | Change values in Trino's password-authenticator.properties file. | Restarts Trino-Server (for Trino) | | trino-env | Change values in Trino's trino-env.sh file. | Restarts Trino-Server (for Trino) | | trino-node | Change values in Trino's node.properties file. | Not available. | | trino-connector-blackhole | Change values in Trino's blackhole.properties file. | Not available. | | trino-connector-cassandra | Change values in Trino's cassandra.properties file. | Not available. | | trino-connector-delta | Change values in Trino's delta.properties file. | Restarts Trino-Server (for Trino) | | trino-connector-hive | Change values in Trino's hive.properties file. | Restarts Trino-Server (for Trino) | | trino-exchange-manager | Change values in Trino's exchange-manager.properties file. | Restarts Trino-Server (for Trino) | | trino-connector-iceberg | Change values in Trino's iceberg.properties file. | Restarts Trino-Server (for Trino) | | trino-connector-jmx | Change values in Trino's jmx.properties file. | Not available. | | trino-connector-kafka | Change values in Trino's kafka.properties file. | Not available. | | trino-connector-localfile | Change values in Trino's localfile.properties file. | Not available. | | trino-connector-memory | Change values in Trino's memory.properties file. | Not available. | | trino-connector-mongodb | Change values in Trino's mongodb.properties file. | Not available. | | trino-connector-mysql | Change values in Trino's mysql.properties file. | Not available. | | trino-connector-postgresql | Change values in Trino's postgresql.properties file. | Not available. | | trino-connector-raptor | Change values in Trino's raptor.properties file. | Not available. | | trino-connector-redis | Change values in Trino's redis.properties file. | Not available. | | trino-connector-redshift | Change values in Trino's redshift.properties file. | Not available. | | trino-connector-tpch | Change values in Trino's tpch.properties file. | Not available. | | trino-connector-tpcds | Change values in Trino's tpcds.properties file. | Not available. | | ranger-kms-dbks-site | Change values in dbks-site.xml file of Ranger KMS. | Restarts Ranger KMS Server. | | ranger-kms-site | Change values in ranger-kms-site.xml file of Ranger KMS. | Restarts Ranger KMS Server. | | ranger-kms-env | Change values in the Ranger KMS environment. | Restarts Ranger KMS Server. | | ranger-kms-log4j | Change values in kms-log4j.properties file of Ranger KMS. | Not available. | | ranger-kms-db-ca | Change values for CA file on S3 for MySQL SSL connection with Ranger KMS. | Not available. | | spark | Amazon EMR-curated settings for Apache Spark. | This property modifies spark-defaults. See actions there. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | Restarts Spark history server and Spark thrift server. | | spark-env | Change values in the Spark environment. | Restarts Spark history server and Spark thrift server. | | spark-hive-site | Change values in Spark's hive-site.xml file | Not available. | | spark-log4j2 | Change values in Spark's log4j2.properties file. | Restarts Spark history server and Spark thrift server. | | spark-metrics | Change values in Spark's metrics.properties file. | Restarts Spark history server and Spark thrift server. | | sqoop-env | Change values in Sqoop's environment. | Not available. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | Not available. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | Not available. | | tez-site | Change values in Tez's tez-site.xml file. | Restart Oozie and HiveServer2. | | yarn-env | Change values in the YARN environment. | Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts MapReduce-HistoryServer. | | yarn-site | Change values in YARN's yarn-site.xml file. | Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts Livy Server and MapReduce-HistoryServer. | | zeppelin-env | Change values in the Zeppelin environment. | Restarts Zeppelin. | | zeppelin-site | Change configuration settings in zeppelin-site.xml. | Restarts Zeppelin. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | Restarts Zookeeper server. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | Restarts Zookeeper server. | ## 6.9.1 change log **Change log for 6.9.1 release and release notes** | Date | Event | Description | | --- | --- | --- | | 2023-08-30 | Update release notes | Added several control-plane related fixes to the release notes | | 2023-08-21 | Docs publication | Amazon EMR 6.9.1 release notes first published | | 2023-08-16 | Deployment complete | Amazon EMR 6.9.1 fully deployed to all [supported Regions](https://aws.amazon.com/about-aws/global-infrastructure/regional-product-services/) | | 2023-08-04 | Initial release | Amazon EMR 6.9.1 first deployed to limited commercial Regions | # Amazon EMR release 6.9.0 ## 6.9.0 application versions This release includes the following applications: [https://delta.io/](https://delta.io/), [https://flink.apache.org/](https://flink.apache.org/), [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [https://hudi.apache.org](https://hudi.apache.org), [http://gethue.com/](http://gethue.com/), [https://iceberg.apache.org/](https://iceberg.apache.org/), [https://jupyter-enterprise-gateway.readthedocs.io/en/latest/](https://jupyter-enterprise-gateway.readthedocs.io/en/latest/), [https://jupyterhub.readthedocs.io/en/latest/#](https://jupyterhub.readthedocs.io/en/latest/#), [https://livy.incubator.apache.org/](https://livy.incubator.apache.org/), [https://mxnet.incubator.apache.org/](https://mxnet.incubator.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://www.tensorflow.org/](https://www.tensorflow.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://trino.io/](https://trino.io/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-6.9.0 | emr-6.8.1 | emr-6.8.0 | emr-6.7.0 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.12.170 | 1.12.170 | 1.12.170 | 1.12.170 | | Python | 2.7, 3.7 | 2.7, 3.7 | 2.7, 3.7 | 2.7, 3.7 | | Scala | 2.12.15 | 2.12.15 | 2.12.15 | 2.12.15 | | AmazonCloudWatchAgent | - | - | - | - | | Delta | 2.1.0 | - | - | - | | Flink | 1.15.2 | 1.15.1 | 1.15.1 | 1.14.2 | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 2.4.13-amzn-0 | 2.4.12-amzn-0.1 | 2.4.12-amzn-0 | 2.4.4-amzn-3 | | HCatalog | 3.1.3-amzn-2 | 3.1.3-amzn-1.1 | 3.1.3-amzn-1 | 3.1.3-amzn-0 | | Hadoop | 3.3.3-amzn-1 | 3.2.1-amzn-8.1 | 3.2.1-amzn-8 | 3.2.1-amzn-7 | | Hive | 3.1.3-amzn-2 | 3.1.3-amzn-1.1 | 3.1.3-amzn-1 | 3.1.3-amzn-0 | | Hudi | 0.12.1-amzn-0 | 0.11.1-amzn-0 | 0.11.1-amzn-0 | 0.11.0-amzn-0 | | Hue | 4.10.0 | 4.10.0 | 4.10.0 | 4.10.0 | | Iceberg | 0.14.1-amzn-0 | 0.14.0-amzn-0 | 0.14.0-amzn-0 | 0.13.1-amzn-0 | | JupyterEnterpriseGateway | 2.6.0 | 2.1.0 | 2.1.0 | 2.1.0 | | JupyterHub | 1.4.1 | 1.4.1 | 1.4.1 | 1.4.1 | | Livy | 0.7.1-incubating | 0.7.1-incubating | 0.7.1-incubating | 0.7.1-incubating | | MXNet | 1.9.1 | 1.9.1 | 1.9.1 | 1.8.0 | | Mahout | - | - | - | - | | Oozie | 5.2.1 | 5.2.1 | 5.2.1 | 5.2.1 | | Phoenix | 5.1.2 | 5.1.2 | 5.1.2 | 5.1.2 | | Pig | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | | Presto | 0.276-amzn-0 | 0.273.3-amzn-0 | 0.273.3-amzn-0 | 0.272-amzn-0 | | Spark | 3.3.0-amzn-1 | 3.3.0-amzn-0.1 | 3.3.0-amzn-0 | 3.2.1-amzn-0 | | Sqoop | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | | TensorFlow | 2.10.0 | 2.9.1 | 2.9.1 | 2.4.1 | | Tez | 0.10.2-amzn-0 | 0.9.2 | 0.9.2 | 0.9.2 | | Trino (PrestoSQL) | 398-amzn-0 | 388-amzn-0 | 388-amzn-0 | 378-amzn-0 | | Zeppelin | 0.10.1 | 0.10.1 | 0.10.1 | 0.10.0 | | ZooKeeper | 3.5.10 | 3.5.10 | 3.5.10 | 3.5.7 | ## 6.9.0 release notes The following release notes include information for Amazon EMR release 6.9.0. Changes are relative to Amazon EMR release 6.8.0. For information on the release timeline, see the [change log](https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-690-release.html#690-changelog). **New Features** + Amazon EMR release 6.9.0 supports Apache Spark RAPIDS 22.08.0, Apache Hudi 0.12.1, Apache Iceberg 0.14.1, Trino 398, and Tez 0.10.2. + Amazon EMR release 6.9.0 includes a new open-source application, [Delta Lake](emr-delta.md) 2.1.0. + The Amazon Redshift integration for Apache Spark is included in Amazon EMR releases 6.9.0 and later. Previously an open-source tool, the native integration is a Spark connector that you can use to build Apache Spark applications that read from and write to data in Amazon Redshift and Amazon Redshift Serverless. For more information, see [Using Amazon Redshift integration for Apache Spark with Amazon EMR](emr-spark-redshift.md). + Amazon EMR release 6.9.0 adds support for archiving logs to Amazon S3 during cluster scale-down. Previously, you could only archive log files to Amazon S3 during cluster termination. The new capability ensures that log files generated on the cluster persist on Amazon S3 even after the node is terminated. For more information, see [Configure cluster logging and debugging](https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-plan-debugging.html). + To support long running queries, Trino now includes a fault-tolerant execution mechanism. Fault-tolerant execution mitigates query failures by retrying failed queries or their component tasks. + You can use Apache Flink on Amazon EMR for unified `BATCH` and `STREAM` processing of Apache Hive Tables or metadata of any Flink tablesource such as Iceberg, Kinesis or Kafka. You can specify the AWS Glue Data Catalog as the metastore for Flink using the AWS Management Console, AWS CLI, or Amazon EMR API. For more information, see [Configuring Flink in Amazon EMR](flink-configure.md). + You can now specify AWS Identity and Access Management (IAM) runtime roles and AWS Lake Formation-based access control for Apache Spark, Apache Hive, and Presto queries on Amazon EMR on EC2 clusters with Amazon SageMaker AI Studio. For more information, see [Configure runtime roles for Amazon EMR steps](https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-steps-runtime-roles.html). **Known Issues** + For Amazon EMR release 6.9.0, Trino does not work on clusters enabled for Apache Ranger. If you need to use Trino with Ranger, contact [Support](https://console.aws.amazon.com/support/home#/). + If you use the Amazon Redshift integration for Apache Spark and have a time, timetz, timestamp, or timestamptz with microsecond precision in Parquet format, the connector rounds the time values to the nearest millisecond value. As a workaround, use the text unload format `unload_s3_format` parameter. + When you use Spark with Hive partition location formatting to read data in Amazon S3, and you run Spark on Amazon EMR releases 5.30.0 to 5.36.0, and 6.2.0 to 6.9.0, you might encounter an issue that prevents your cluster from reading data correctly. This can happen if your partitions have all of the following characteristics: + Two or more partitions are scanned from the same table. + At least one partition directory path is a prefix of at least one other partition directory path, for example, `s3://bucket/table/p=a` is a prefix of `s3://bucket/table/p=a b`. + The first character that follows the prefix in the other partition directory has a UTF-8 value that’s less than than the `/` character (U\$1002F). For example, the space character (U\$10020) that occurs between a and b in `s3://bucket/table/p=a b` falls into this category. Note that there are 14 other non-control characters: `!"#$%&‘()*+,-`. For more information, see [UTF-8 encoding table and Unicode characters](https://www.utf8-chartable.de/). As a workaround to this issue, set the `spark.sql.sources.fastS3PartitionDiscovery.enabled` configuration to `false` in the `spark-defaults` classification. + Connections to Amazon EMR clusters from Amazon SageMaker AI Studio may intermittently fail with a **403 Forbidden** response code. This error happens when setup of the IAM role on the cluster takes longer than 60 seconds. As a workaround, you can install an Amazon EMR patch to enable retries and increase the timeout to a minimum of 300 seconds. Use the following steps to apply the bootstrap action when you launch your cluster. 1. Download the bootstrap script and RPM files from the following Amazon S3 URIs. ``` s3://emr-data-access-control-us-east-1/customer-bootstrap-actions/gcsc/replace-rpms.sh s3://emr-data-access-control-us-east-1/customer-bootstrap-actions/gcsc/emr-secret-agent-1.18.0-SNAPSHOT20221121212949.noarch.rpm ``` 1. Upload the files from the previous step to an Amazon S3 bucket that you own. The bucket must be in the same AWS Region where you plan to launch the cluster. 1. Include the following bootstrap action when you launch your EMR cluster. Replace *bootstrap\$1URI* and *RPM\$1URI* with the corresponding URIs from Amazon S3. ``` --bootstrap-actions "Path=bootstrap_URI,Args=[RPM_URI]" ``` + With Amazon EMR releases 5.36.0 and 6.6.0 through 6.9.0, `SecretAgent` and `RecordServer` service components may experience log data loss due to an incorrect file name pattern configuration in Log4j2 properties. The incorrect configuration causes the components to generate only one log file per day. When the rotation strategy occurs, it overwrites the existing file instead of generating a new log file as expected. As a workaround, use a bootstrap action to generate log files each hour and append an auto-increment integer in the file name to handle the rotation. For Amazon EMR 6.6.0 through 6.9.0 releases, use the following bootstrap action when you launch a cluster. ``` ‑‑bootstrap‑actions "Path=s3://emr-data-access-control-us-east-1/customer-bootstrap-actions/log-rotation-emr-6x/replace-puppet.sh,Args=[]" ``` For Amazon EMR 5.36.0, use the following bootstrap action when you launch a cluster. ``` ‑‑bootstrap‑actions "Path=s3://emr-data-access-control-us-east-1/customer-bootstrap-actions/log-rotation-emr-5x/replace-puppet.sh,Args=[]" ``` + Apache Flink provides Native S3 FileSystem and Hadoop FileSystem Connectors, which let applications create a FileSink and write the data into Amazon S3. This FileSink fails with one of the following two exceptions. ``` java.lang.UnsupportedOperationException: Recoverable writers on Hadoop are only supported for HDFS ``` ``` Caused by: java.lang.NoSuchMethodError: org.apache.hadoop.io.retry.RetryPolicies.retryOtherThanRemoteAndSaslException(Lorg/apache/hadoop/io/retry/RetryPolicy;Ljava/util/Map;)Lorg/apache/hadoop/io/retry/RetryPolicy; at org.apache.hadoop.yarn.client.RMProxy.createRetryPolicy(RMProxy.java:302) ~[hadoop-yarn-common-3.3.3-amzn-0.jar:?] ``` As a workaround, you can install an Amazon EMR patch, which fixes the above issue in Flink. To apply the bootstrap action when you launch your cluster, complete the following steps. 1. Download the flink-rpm to your Amazon S3 bucket. Your RPM path is `s3://DOC-EXAMPLE-BUCKET/rpms/flink/`. 1. Download the bootstrap script and RPM files from Amazon S3 using the following URI. Replace `regionName` with the AWS Region where you plan to launch the cluster. ``` s3://emr-data-access-control-regionName/customer-bootstrap-actions/gcsc/replace-rpms.sh ``` 1. Hadoop 3.3.3 introduced a change in YARN ([YARN-9608](https://issues.apache.org/jira/browse/YARN-9608)) that keeps nodes where containers ran in a decommissioning state until the application completes. This change ensures that local data such as shuffle data doesn't get lost, and you don' need to re-run the job. In Amazon EMR 6.8.0 and 6.9.0, this approach might also lead to underutilization of resources on clusters with or without managed scaling enabled. With [Amazon EMR 6.10.0](emr-6100-release.md#emr-6100-relnotes), there's a workaround for this issue to set the value of `yarn.resourcemanager.decommissioning-nodes-watcher.wait-for-applications` to `false` in `yarn-site.xml`. In Amazon EMR releases 6.11.0 and higher as well as 6.8.1, 6.9.1, and 6.10.1, the config is set to `false` by default to resolve this issue. **Changes, Enhancements, and Resolved Issues** + For Amazon EMR release 6.9.0 and later, all components installed by Amazon EMR that use Log4j libraries use Log4j version 2.17.1 or later. + When you use the DynamoDB connector with Spark on Amazon EMR versions 6.6.0, 6.7.0, and 6.8.0, all reads from your table return an empty result, even though the input split references non-empty data. Amazon EMR release 6.9.0 fixes this issue. + Amazon EMR 6.9.0 adds limited support for Lake Formation-based access control with Apache Hudi when reading data using Spark SQL. The support is for SELECT queries using Spark SQL and is limited to column-level access control. For more information, see [Hudi and Lake Formation](https://docs.aws.amazon.com/emr/latest/ManagementGuide/hudi-with-lake-formation.html). + When you use Amazon EMR 6.9.0 to create a Hadoop cluster with [Node Labels](https://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/NodeLabel.html) enabled, the [YARN metrics API](https://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/ResourceManagerRest.html#Cluster_Metrics_API) returns aggregated information across all partitions, instead of the default partition. For more information, see [YARN-11414](https://issues.apache.org/jira/browse/YARN-11414). + With Amazon EMR release 6.9.0, we've updated Trino to version 398, which uses Java 17. The previous supported version of Trino for Amazon EMR 6.8.0 was Trino 388 running on Java 11. For more information about this change, see [Trino updates to Java 17](https://trino.io/blog/2022/07/14/trino-updates-to-java-17.html) on the Trino blog. + This releases fixes a timing sequence mismatch issue between Apache BigTop and the Amazon EMR on EC2 cluster startup sequence. This timing sequence mismatch occurs when a system attempts to perform two or more operations at the same time instead of doing them in the proper sequence. As a result, certain cluster configurations experienced instance startup timeouts and slower cluster startup times. + When you launch a cluster with *the latest patch release* of Amazon EMR 5.36 or higher, 6.6 or higher, or 7.0 or higher, Amazon EMR uses the latest Amazon Linux 2023 or Amazon Linux 2 release for the default Amazon EMR AMI. For more information, see [Using the default Amazon Linux AMI for Amazon EMR](https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-default-ami.html). **Note** This release no longer gets automatic AMI updates since it has been succeeded by 1 more more patch releases. The patch release is denoted by the number after the second decimal point (`6.8.1`). To see if you're using the latest patch release, check the available releases in the [https://docs.aws.amazon.com/emr/latest/ReleaseGuide](https://docs.aws.amazon.com/emr/latest/ReleaseGuide), or check the **Amazon EMR release** dropdown when you create a cluster in the console, or use the [https://docs.aws.amazon.com/emr/latest/APIReference/API_ListReleaseLabels.html](https://docs.aws.amazon.com/emr/latest/APIReference/API_ListReleaseLabels.html) API or [https://docs.aws.amazon.com/cli/latest/reference/emr/list-release-labels.html](https://docs.aws.amazon.com/cli/latest/reference/emr/list-release-labels.html) CLI action. To get updates about new releases, subscribe to the RSS feed on the [What's new?](https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-whatsnew.html) page. [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-690-release.html) ## 6.9.0 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | aws-sagemaker-spark-sdk | 1.4.2 | Amazon SageMaker Spark SDK | | delta | 2.1.0 | Delta lake is an open table format for huge analytic datasets | | emr-ddb | 4.16.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 3.3.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.6.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-notebook-env | 1.7.0 | Conda env for emr notebook which includes jupyter enterprise gateway | | emr-s3-dist-cp | 2.23.0 | Distributed copy application optimized for Amazon S3. | | emr-s3-select | 2.2.0 | EMR S3Select Connector | | emrfs | 2.54.0 | Amazon S3 connector for Hadoop ecosystem applications. | | flink-client | 1.15.2 | Apache Flink command line client scripts and applications. | | flink-jobmanager-config | 1.15.2 | Managing resources on EMR nodes for Apache Flink JobManager. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 3.3.3-amzn-1 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 3.3.3-amzn-1 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 3.3.3-amzn-1 | HDFS command-line client and library | | hadoop-hdfs-namenode | 3.3.3-amzn-1 | HDFS service for tracking file names and block locations. | | hadoop-hdfs-journalnode | 3.3.3-amzn-1 | HDFS service for managing the Hadoop filesystem journal on HA clusters. | | hadoop-httpfs-server | 3.3.3-amzn-1 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 3.3.3-amzn-1 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 3.3.3-amzn-1 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 3.3.3-amzn-1 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 3.3.3-amzn-1 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 3.3.3-amzn-1 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 2.4.13-amzn-0 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 2.4.13-amzn-0 | Service for serving one or more HBase regions. | | hbase-client | 2.4.13-amzn-0 | HBase command-line client. | | hbase-rest-server | 2.4.13-amzn-0 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 2.4.13-amzn-0 | Service providing a Thrift endpoint to HBase. | | hbase-operator-tools | 2.4.13-amzn-0 | Repair tool for Apache HBase clusters. | | hcatalog-client | 3.1.3-amzn-2 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 3.1.3-amzn-2 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 3.1.3-amzn-2 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 3.1.3-amzn-2 | Hive command line client. | | hive-hbase | 3.1.3-amzn-2 | Hive-hbase client. | | hive-metastore-server | 3.1.3-amzn-2 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server2 | 3.1.3-amzn-2 | Service for accepting Hive queries as web requests. | | hudi | 0.12.1-amzn-0 | Incremental processing framework to power data pipeline at low latency and high efficiency. | | hudi-presto | 0.12.1-amzn-0 | Bundle library for running Presto with Hudi. | | hudi-trino | 0.12.1-amzn-0 | Bundle library for running Trino with Hudi. | | hudi-spark | 0.12.1-amzn-0 | Bundle library for running Spark with Hudi. | | hue-server | 4.10.0 | Web application for analyzing data using Hadoop ecosystem applications | | iceberg | 0.14.1-amzn-0 | Apache Iceberg is an open table format for huge analytic datasets | | jupyterhub | 1.4.1 | Multi-user server for Jupyter notebooks | | livy-server | 0.7.1-incubating | REST interface for interacting with Apache Spark | | nginx | 1.12.1 | nginx [engine x] is an HTTP and reverse proxy server | | mxnet | 1.9.1 | A flexible, scalable, and efficient library for deep learning. | | mariadb-server | 5.5.68\$1 | MariaDB database server. | | nvidia-cuda | 11.7.0 | Nvidia drivers and Cuda toolkit | | oozie-client | 5.2.1 | Oozie command-line client. | | oozie-server | 5.2.1 | Service for accepting Oozie workflow requests. | | opencv | 4.5.0 | Open Source Computer Vision Library. | | phoenix-library | 5.1.2 | The phoenix libraries for server and client | | phoenix-connectors | 6.0.0-SNAPSHOT | Apache Phoenix-Connectors for Spark-3 | | phoenix-query-server | 6.0.0 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.276-amzn-0 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.276-amzn-0 | Service for executing pieces of a query. | | presto-client | 0.276-amzn-0 | Presto command-line client which is installed on an HA cluster's stand-by masters where Presto server is not started. | | trino-coordinator | 398-amzn-0 | Service for accepting queries and managing query execution among trino-workers. | | trino-worker | 398-amzn-0 | Service for executing pieces of a query. | | trino-client | 398-amzn-0 | Trino command-line client which is installed on an HA cluster's stand-by masters where Trino server is not started. | | pig-client | 0.17.0 | Pig command-line client. | | r | 4.0.2 | The R Project for Statistical Computing | | ranger-kms-server | 2.0.0 | Apache Ranger Key Management System | | spark-client | 3.3.0-amzn-1 | Spark command-line clients. | | spark-history-server | 3.3.0-amzn-1 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 3.3.0-amzn-1 | In-memory execution engine for YARN. | | spark-yarn-slave | 3.3.0-amzn-1 | Apache Spark libraries needed by YARN slaves. | | spark-rapids | 22.08.0-amzn-0 | Nvidia Spark RAPIDS plugin that accelerates Apache Spark with GPUs. | | sqoop-client | 1.4.7 | Apache Sqoop command-line client. | | tensorflow | 2.10.0 | TensorFlow open source software library for high performance numerical computation. | | tez-on-yarn | 0.10.2-amzn-0 | The tez YARN application and libraries. | | webserver | 2.4.41\$1 | Apache HTTP server. | | zeppelin-server | 0.10.1 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.5.10 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.5.10 | ZooKeeper command line client. | ## 6.9.0 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). Reconfiguration actions occur when you specify a configuration for instance groups in a running cluster. Amazon EMR only initiates reconfiguration actions for the classifications that you modify. For more information, see [Reconfigure an instance group in a running cluster](emr-configure-apps-running-cluster.md). **emr-6.9.0 classifications** | Classifications | Description | Reconfiguration Actions | | --- | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | Restarts the ResourceManager service. | | container-executor | Change values in Hadoop YARN's container-executor.cfg file. | Not available. | | container-log4j | Change values in Hadoop YARN's container-log4j.properties file. | Not available. | | core-site | Change values in Hadoop's core-site.xml file. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts Hadoop KMS, Ranger KMS, HiveServer2, Hive MetaStore, Hadoop Httpfs, and MapReduce-HistoryServer. | | docker-conf | Change docker related settings. | Not available. | | emrfs-site | Change EMRFS settings. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts HBaseRegionserver, HBaseMaster, HBaseThrift, HBaseRest, HiveServer2, Hive MetaStore, Hadoop Httpfs, and MapReduce-HistoryServer. | | flink-conf | Change flink-conf.yaml settings. | Restarts Flink history server. | | flink-log4j | Change Flink log4j.properties settings. | Restarts Flink history server. | | flink-log4j-session | Change Flink log4j-session.properties settings for Kubernetes/Yarn session. | Restarts Flink history server. | | flink-log4j-cli | Change Flink log4j-cli.properties settings. | Restarts Flink history server. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts PhoenixQueryserver, HiveServer2, Hive MetaStore, and MapReduce-HistoryServer. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | Restarts the Hadoop HDFS services SecondaryNamenode, Datanode, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts Hadoop KMS, Hadoop Httpfs, and MapReduce-HistoryServer. | | hadoop-ssl-server | Change hadoop ssl server configuration | Not available. | | hadoop-ssl-client | Change hadoop ssl client configuration | Not available. | | hbase | Amazon EMR-curated settings for Apache HBase. | Custom EMR specific property. Sets emrfs-site and hbase-site configs. See those for their associated restarts. | | hbase-env | Change values in HBase's environment. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbase.properties file. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | Not available. | | hbase-site | Change values in HBase's hbase-site.xml file. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. Additionally restarts Phoenix QueryServer. | | hdfs-encryption-zones | Configure HDFS encryption zones. | This classification should not be reconfigured. | | hdfs-env | Change values in the HDFS environment. | Restarts Hadoop HDFS services Namenode, Datanode, and ZKFC. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Additionally restarts Hadoop Httpfs. | | hcatalog-env | Change values in HCatalog's environment. | Restarts Hive HCatalog Server. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | Restarts Hive HCatalog Server. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | Restarts Hive HCatalog Server. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | Restarts Hive WebHCat server. | | hcatalog-webhcat-log4j2 | Change values in HCatalog WebHCat's log4j2.properties. | Restarts Hive WebHCat server. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | Restarts Hive WebHCat server. | | hive | Amazon EMR-curated settings for Apache Hive. | Sets configurations to launch Hive LLAP service. | | hive-beeline-log4j2 | Change values in Hive's beeline-log4j2.properties file. | Not available. | | hive-parquet-logging | Change values in Hive's parquet-logging.properties file. | Not available. | | hive-env | Change values in the Hive environment. | Restarts HiveServer2, HiveMetastore, and Hive HCatalog-Server. Runs Hive schemaTool CLI commands to verify hive-metastore. | | hive-exec-log4j2 | Change values in Hive's hive-exec-log4j2.properties file. | Not available. | | hive-llap-daemon-log4j2 | Change values in Hive's llap-daemon-log4j2.properties file. | Not available. | | hive-log4j2 | Change values in Hive's hive-log4j2.properties file. | Not available. | | hive-site | Change values in Hive's hive-site.xml file | Restarts HiveServer2, HiveMetastore, and Hive HCatalog-Server. Runs Hive schemaTool CLI commands to verify hive-metastore. Also restarts Oozie and Zeppelin. | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | Not available. | | hue-ini | Change values in Hue's ini file | Restarts Hue. Also activates Hue config override CLI commands to pick up new configurations. | | httpfs-env | Change values in the HTTPFS environment. | Restarts Hadoop Httpfs service. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | Restarts Hadoop Httpfs service. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | Not available. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | Restarts Hadoop-KMS service. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | Not available. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | Restarts Hadoop-KMS and Ranger-KMS service. | | hudi-env | Change values in the Hudi environment. | Not available. | | hudi-defaults | Change values in Hudi's hudi-defaults.conf file. | Not available. | | iceberg-defaults | Change values in Iceberg's iceberg-defaults.conf file. | Not available. | | delta-defaults | Change values in Delta's delta-defaults.conf file. | Not available. | | jupyter-notebook-conf | Change values in Jupyter Notebook's jupyter\$1notebook\$1config.py file. | Not available. | | jupyter-hub-conf | Change values in JupyterHubs's jupyterhub\$1config.py file. | Not available. | | jupyter-s3-conf | Configure Jupyter Notebook S3 persistence. | Not available. | | jupyter-sparkmagic-conf | Change values in Sparkmagic's config.json file. | Not available. | | livy-conf | Change values in Livy's livy.conf file. | Restarts Livy Server. | | livy-env | Change values in the Livy environment. | Restarts Livy Server. | | livy-log4j2 | Change Livy log4j2.properties settings. | Restarts Livy Server. | | mapred-env | Change values in the MapReduce application's environment. | Restarts Hadoop MapReduce-HistoryServer. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | Restarts Hadoop MapReduce-HistoryServer. | | oozie-env | Change values in Oozie's environment. | Restarts Oozie. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | Restarts Oozie. | | oozie-site | Change values in Oozie's oozie-site.xml file. | Restarts Oozie. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | Not available. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | Not available. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | Restarts Phoenix-QueryServer. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | Not available. | | pig-env | Change values in the Pig environment. | Not available. | | pig-properties | Change values in Pig's pig.properties file. | Restarts Oozie. | | pig-log4j | Change values in Pig's log4j.properties file. | Not available. | | presto-log | Change values in Presto's log.properties file. | Restarts Presto-Server (for PrestoDB) | | presto-config | Change values in Presto's config.properties file. | Restarts Presto-Server (for PrestoDB) | | presto-password-authenticator | Change values in Presto's password-authenticator.properties file. | Not available. | | presto-env | Change values in Presto's presto-env.sh file. | Restarts Presto-Server (for PrestoDB) | | presto-node | Change values in Presto's node.properties file. | Not available. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | Not available. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | Not available. | | presto-connector-hive | Change values in Presto's hive.properties file. | Restarts Presto-Server (for PrestoDB) | | presto-connector-jmx | Change values in Presto's jmx.properties file. | Not available. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | Not available. | | presto-connector-lakeformation | Change values in Presto's lakeformation.properties file. | Restarts Presto-Server (for PrestoDB) | | presto-connector-localfile | Change values in Presto's localfile.properties file. | Not available. | | presto-connector-memory | Change values in Presto's memory.properties file. | Not available. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | Not available. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | Not available. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | Not available. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | Not available. | | presto-connector-redis | Change values in Presto's redis.properties file. | Not available. | | presto-connector-redshift | Change values in Presto's redshift.properties file. | Not available. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | Not available. | | presto-connector-tpcds | Change values in Presto's tpcds.properties file. | Not available. | | trino-log | Change values in Trino's log.properties file. | Restarts Trino-Server (for Trino) | | trino-config | Change values in Trino's config.properties file. | Restarts Trino-Server (for Trino) | | trino-password-authenticator | Change values in Trino's password-authenticator.properties file. | Restarts Trino-Server (for Trino) | | trino-env | Change values in Trino's trino-env.sh file. | Restarts Trino-Server (for Trino) | | trino-node | Change values in Trino's node.properties file. | Not available. | | trino-connector-blackhole | Change values in Trino's blackhole.properties file. | Not available. | | trino-connector-cassandra | Change values in Trino's cassandra.properties file. | Not available. | | trino-connector-delta | Change values in Trino's delta.properties file. | Restarts Trino-Server (for Trino) | | trino-connector-hive | Change values in Trino's hive.properties file. | Restarts Trino-Server (for Trino) | | trino-exchange-manager | Change values in Trino's exchange-manager.properties file. | Restarts Trino-Server (for Trino) | | trino-connector-iceberg | Change values in Trino's iceberg.properties file. | Restarts Trino-Server (for Trino) | | trino-connector-jmx | Change values in Trino's jmx.properties file. | Not available. | | trino-connector-kafka | Change values in Trino's kafka.properties file. | Not available. | | trino-connector-localfile | Change values in Trino's localfile.properties file. | Not available. | | trino-connector-memory | Change values in Trino's memory.properties file. | Not available. | | trino-connector-mongodb | Change values in Trino's mongodb.properties file. | Not available. | | trino-connector-mysql | Change values in Trino's mysql.properties file. | Not available. | | trino-connector-postgresql | Change values in Trino's postgresql.properties file. | Not available. | | trino-connector-raptor | Change values in Trino's raptor.properties file. | Not available. | | trino-connector-redis | Change values in Trino's redis.properties file. | Not available. | | trino-connector-redshift | Change values in Trino's redshift.properties file. | Not available. | | trino-connector-tpch | Change values in Trino's tpch.properties file. | Not available. | | trino-connector-tpcds | Change values in Trino's tpcds.properties file. | Not available. | | ranger-kms-dbks-site | Change values in dbks-site.xml file of Ranger KMS. | Restarts Ranger KMS Server. | | ranger-kms-site | Change values in ranger-kms-site.xml file of Ranger KMS. | Restarts Ranger KMS Server. | | ranger-kms-env | Change values in the Ranger KMS environment. | Restarts Ranger KMS Server. | | ranger-kms-log4j | Change values in kms-log4j.properties file of Ranger KMS. | Not available. | | ranger-kms-db-ca | Change values for CA file on S3 for MySQL SSL connection with Ranger KMS. | Not available. | | spark | Amazon EMR-curated settings for Apache Spark. | This property modifies spark-defaults. See actions there. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | Restarts Spark history server and Spark thrift server. | | spark-env | Change values in the Spark environment. | Restarts Spark history server and Spark thrift server. | | spark-hive-site | Change values in Spark's hive-site.xml file | Not available. | | spark-log4j2 | Change values in Spark's log4j2.properties file. | Restarts Spark history server and Spark thrift server. | | spark-metrics | Change values in Spark's metrics.properties file. | Restarts Spark history server and Spark thrift server. | | sqoop-env | Change values in Sqoop's environment. | Not available. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | Not available. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | Not available. | | tez-site | Change values in Tez's tez-site.xml file. | Restart Oozie and HiveServer2. | | yarn-env | Change values in the YARN environment. | Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts MapReduce-HistoryServer. | | yarn-site | Change values in YARN's yarn-site.xml file. | Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts Livy Server and MapReduce-HistoryServer. | | zeppelin-env | Change values in the Zeppelin environment. | Restarts Zeppelin. | | zeppelin-site | Change configuration settings in zeppelin-site.xml. | Restarts Zeppelin. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | Restarts Zookeeper server. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | Restarts Zookeeper server. | ## 6.9.0 change log **Change log for 6.9.0 release and release notes** | Date | Event | Description | | --- | --- | --- | | 2023-08-30 | Update release notes | Added fix for timing sequence mismatch issue | | 2023-08-21 | Update release notes | Added a known issue with Hadoop 3.3.3. | | 2023-07-26 | Update | New OS release labels 2.0.20230612.0 and 2.0.20230628.0. | | 2022-12-13 | Release notes updated | Added feature and known issue for runtime with SageMaker AI | | 2022-11-29 | Release notes and documentation updated | Added feature for Amazon Redshift integration for Apache Spark | | 2022-11-23 | Release notes updated | Removed Log4j entry | | 2022-11-18 | Deployment complete | Amazon EMR 6.9 fully deployed to all [supported Regions](https://aws.amazon.com/about-aws/global-infrastructure/regional-product-services/) | | 2022-11-18 | Docs publication | Amazon EMR 6.9 release notes first published | | 2022-11-14 | Initial release | Amazon EMR 6.9 deployed to limited commercial Regions | # Amazon EMR release 6.8.1 ## 6.8.1 application versions This release includes the following applications: [https://flink.apache.org/](https://flink.apache.org/), [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [https://hudi.apache.org](https://hudi.apache.org), [http://gethue.com/](http://gethue.com/), [https://iceberg.apache.org/](https://iceberg.apache.org/), [https://jupyter-enterprise-gateway.readthedocs.io/en/latest/](https://jupyter-enterprise-gateway.readthedocs.io/en/latest/), [https://jupyterhub.readthedocs.io/en/latest/#](https://jupyterhub.readthedocs.io/en/latest/#), [https://livy.incubator.apache.org/](https://livy.incubator.apache.org/), [https://mxnet.incubator.apache.org/](https://mxnet.incubator.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://www.tensorflow.org/](https://www.tensorflow.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://trino.io/](https://trino.io/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-6.8.1 | emr-6.8.0 | emr-6.7.0 | emr-6.6.0 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.12.170 | 1.12.170 | 1.12.170 | 1.12.170 | | Python | 2.7, 3.7 | 2.7, 3.7 | 2.7, 3.7 | 2.7, 3.7 | | Scala | 2.12.15 | 2.12.15 | 2.12.15 | 2.12.10 | | AmazonCloudWatchAgent | - | - | - | - | | Delta | - | - | - | - | | Flink | 1.15.1 | 1.15.1 | 1.14.2 | 1.14.2 | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 2.4.12-amzn-0.1 | 2.4.12-amzn-0 | 2.4.4-amzn-3 | 2.4.4-amzn-2 | | HCatalog | 3.1.3-amzn-1.1 | 3.1.3-amzn-1 | 3.1.3-amzn-0 | 3.1.2-amzn-7 | | Hadoop | 3.2.1-amzn-8.1 | 3.2.1-amzn-8 | 3.2.1-amzn-7 | 3.2.1-amzn-6 | | Hive | 3.1.3-amzn-1.1 | 3.1.3-amzn-1 | 3.1.3-amzn-0 | 3.1.2-amzn-7 | | Hudi | 0.11.1-amzn-0 | 0.11.1-amzn-0 | 0.11.0-amzn-0 | 0.10.1-amzn-0 | | Hue | 4.10.0 | 4.10.0 | 4.10.0 | 4.10.0 | | Iceberg | 0.14.0-amzn-0 | 0.14.0-amzn-0 | 0.13.1-amzn-0 | 0.13.1 | | JupyterEnterpriseGateway | 2.1.0 | 2.1.0 | 2.1.0 | 2.1.0 | | JupyterHub | 1.4.1 | 1.4.1 | 1.4.1 | 1.4.1 | | Livy | 0.7.1-incubating | 0.7.1-incubating | 0.7.1-incubating | 0.7.1-incubating | | MXNet | 1.9.1 | 1.9.1 | 1.8.0 | 1.8.0 | | Mahout | - | - | - | - | | Oozie | 5.2.1 | 5.2.1 | 5.2.1 | 5.2.1 | | Phoenix | 5.1.2 | 5.1.2 | 5.1.2 | 5.1.2 | | Pig | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | | Presto | 0.273.3-amzn-0 | 0.273.3-amzn-0 | 0.272-amzn-0 | 0.267-amzn-0 | | Spark | 3.3.0-amzn-0.1 | 3.3.0-amzn-0 | 3.2.1-amzn-0 | 3.2.0-amzn-0 | | Sqoop | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | | TensorFlow | 2.9.1 | 2.9.1 | 2.4.1 | 2.4.1 | | Tez | 0.9.2 | 0.9.2 | 0.9.2 | 0.9.2 | | Trino (PrestoSQL) | 388-amzn-0 | 388-amzn-0 | 378-amzn-0 | 367-amzn-0 | | Zeppelin | 0.10.1 | 0.10.1 | 0.10.0 | 0.10.0 | | ZooKeeper | 3.5.10 | 3.5.10 | 3.5.7 | 3.5.7 | ## 6.8.1 release notes The following release notes include information for Amazon EMR release 6.8.1. Changes are relative to 6.8.0. For information on the release timeline, see the [6.8.1 change log](#681-changelog). **Changes, enhancements, and resolved issues** + Hadoop 3.3.3 introduced a change in YARN ([YARN-9608](https://issues.apache.org/jira/browse/YARN-9608)) that keeps nodes where containers ran in a decommissioning state until the application completes. This change ensures that local data such as shuffle data doesn't get lost, and you don' need to re-run the job. This approach might also lead to underutilization of resources on clusters with or without managed scaling enabled. With Amazon EMR releases 6.11.0 and higher as well as 6.8.1, 6.9.1, and 6.10.1, the value of `yarn.resourcemanager.decommissioning-nodes-watcher.wait-for-applications` is set to `false` in `yarn-site.xml` to resolve this issue. While the fix addresses the issues that were introduced by YARN-9608, it might cause Hive jobs to fail due to shuffle data loss on clusters that have managed scaling enabled. We've mitigated that risk in this release by also setting `yarn.resourcemanager.decommissioning-nodes-watcher.wait-for-shuffle-data` for Hive workloads. This config is only available with Amazon EMR releases 6.11.0 and higher. + Metrics collector won't send any metrics to the control plane after failover of primary node in clusters with the instance groups configuration. + This release eliminates retries on failed HTTP requests to metrics collector endpoints. + This release includes a change that allows high-availability clusters to recover from failed state after restart. + This release fixes an issue where large user-created UIDs caused overflow exceptions. + This release fixes timeout issues with the Amazon EMR reconfiguration process. + This release prevents an issue where failed reconfiguration might break other, unrelated processes. + This release includes security fixes. + This release fixes an issue where clusters that are running workloads on Spark with Amazon EMR might silently receive incorrect results with `contains`, `startsWith`, `endsWith`, and `like`. This issue occurs when you use the expressions on partitioned fields that have metadata in the Amazon EMR Hive3 Metastore Server (HMS). + With Amazon EMR 6.6.0 through 6.9.x, INSERT queries with dynamic partition and an ORDER BY or SORT BY clause will always have two reducers. This issue is caused by OSS change [HIVE-20703](https://issues.apache.org/jira/browse/HIVE-20703), which puts dynamic sort partition optimization under cost-based decision. If your workload doesn't require sorting of dynamic partitions, we recommend that you set the `hive.optimize.sort.dynamic.partition.threshold` property to `-1` to disable the new feature and get the correctly calculated number of reducers. This issue is fixed in OSS Hive as part of [HIVE-22269](https://issues.apache.org/jira/browse/HIVE-22269) and is fixed in Amazon EMR 6.10.0. + Hive might experience data loss when you use HDFS as a scratch directory and you have enabled merge small files, and the table contains static partition paths. + This release fixes a performance issue with Hive if merge small files (disabled by default) is enabled at the end of ETL job. + This release fixes an issue with throttling on the Glue side when there are no user-defined functions (UDF). + This release fixes an issue that deletes container logs by the node log aggregation service before log pusher can push them to S3 in case of YARN decommissioning. + This release fixes handling of compacted/archived files with persistent storefile tracking for HBase. + This release fixes an issue that impacted Spark performance when you set a default `true` value for the `spark.yarn.heterogeneousExecutors.enabled` config in `spark-defaults.conf`. + This release fixes an issue with Reduce Task failing to read shuffle data. The issue caused Hive query failures with a corrupted memory error. + This release fixes an issue that caused the node provisioner to fail if the HDFS NameNode (NN) service was stuck in safemode during node replacement. + This release adds a new retry mechanism to the cluster scaling workflow for EMR clusters that run Presto or Trino. This improvement reduces the risk that cluster resizing will indefinitely stall due to a single failed resize operation. It also improves cluster utilization, because your cluster scales up and down faster. + This release improves cluster scale-down logic so that your cluster doesn't attempt a scale-down of core nodes below the HDFS replication factor setting for the cluster. This aligns with your data redundancy requirements, and reduces the chance that a scaling operation might stall. + The log management daemon has been upgraded to identify all logs that are in active use with open file handles on the local instance storage, and the associated processes. This upgrade ensures that Amazon EMR properly deletes the files and reclaims storage space after the logs are archived to Amazon S3. + This release includes a log-management daemon enhancement that deletes empty, unused steps directories in the local cluster file system. An excessively large number of empty directories can degrade the performance of Amazon EMR daemons and result in disk over-utilization. + This release fixes an issue that might occur when you create an edge node by replicating one of the primary nodes from a cluster with multiple primary nodes. The replicated edge node could cause delays with scale-down operations, or result in high memory-utilization on the primary nodes. For more information on how to create an edge node to communicate with your EMR cluster, see [Edge Node Creator](https://github.com/aws-samples/aws-emr-utilities/tree/main/utilities/emr-edge-node-creator) in the `aws-samples` repo on GitHub. + This release improves the automation process that Amazon EMR uses to re-mount Amazon EBS volumes to an instance after a reboot. + This release fixes an issue that resulted in intermittent gaps in the Hadoop metrics that Amazon EMR publishes to Amazon CloudWatch. + This release fixes an issue with EMR clusters where an update to the YARN configuration file that contains the exclusion list of nodes for the cluster is interrupted due to disk over-utilization. The incomplete update hinders future cluster scale-down operations. This release ensures that your cluster remains healthy, and that scaling operations work as expected. + This release improves the on-cluster log management daemon to monitor additional log folders in your EMR cluster. This improvement minimizes disk over-utilization scenarios. + This release automatically restarts the on-cluster log management daemon when it stops. This improvement reduces the risk for nodes to appear unhealthy due to disk over-utilization. + This release adds support for archiving logs to Amazon S3 during cluster scale-down. Previously, you could only archive log files to Amazon S3 during cluster termination. The new capability ensures that log files generated on the cluster persist on Amazon S3 even after the node is terminated. For more information, see [Configure cluster logging and debugging](https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-plan-debugging.html). + This release fixes an issue that occurred when the Amazon S3 URI for a bootstrap action ended with a port number, for example: `a.b.c.d:4345`. Amazon EMR was incorrectly parsing these URIs, so any associated bootstrap actions would fail. + This releases fixes a timing sequence mismatch issue between Apache BigTop and the Amazon EMR on EC2 cluster startup sequence. This timing sequence mismatch occurs when a system attempts to perform two or more operations at the same time instead of doing them in the proper sequence. As a result, certain cluster configurations experienced instance startup timeouts and slower cluster startup times. + When you launch a cluster with *the latest patch release* of Amazon EMR 5.36 or higher, 6.6 or higher, or 7.0 or higher, Amazon EMR uses the latest Amazon Linux 2023 or Amazon Linux 2 release for the default Amazon EMR AMI. For more information, see [Using the default Amazon Linux AMI for Amazon EMR](https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-default-ami.html). [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-681-release.html) ## 6.8.1 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | aws-sagemaker-spark-sdk | 1.4.2 | Amazon SageMaker Spark SDK | | emr-ddb | 4.16.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 3.2.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.5.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-notebook-env | 1.7.0 | Conda env for emr notebook which includes jupyter enterprise gateway | | emr-s3-dist-cp | 2.22.0 | Distributed copy application optimized for Amazon S3. | | emr-s3-select | 2.1.0 | EMR S3Select Connector | | emrfs | 2.53.0 | Amazon S3 connector for Hadoop ecosystem applications. | | flink-client | 1.15.1 | Apache Flink command line client scripts and applications. | | flink-jobmanager-config | 1.15.1 | Managing resources on EMR nodes for Apache Flink JobManager. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 3.2.1-amzn-8.1 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 3.2.1-amzn-8.1 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 3.2.1-amzn-8.1 | HDFS command-line client and library | | hadoop-hdfs-namenode | 3.2.1-amzn-8.1 | HDFS service for tracking file names and block locations. | | hadoop-hdfs-journalnode | 3.2.1-amzn-8.1 | HDFS service for managing the Hadoop filesystem journal on HA clusters. | | hadoop-httpfs-server | 3.2.1-amzn-8.1 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 3.2.1-amzn-8.1 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 3.2.1-amzn-8.1 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 3.2.1-amzn-8.1 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 3.2.1-amzn-8.1 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 3.2.1-amzn-8.1 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 2.4.12-amzn-0.1 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 2.4.12-amzn-0.1 | Service for serving one or more HBase regions. | | hbase-client | 2.4.12-amzn-0.1 | HBase command-line client. | | hbase-rest-server | 2.4.12-amzn-0.1 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 2.4.12-amzn-0.1 | Service providing a Thrift endpoint to HBase. | | hbase-operator-tools | 2.4.12-amzn-0.1 | Repair tool for Apache HBase clusters. | | hcatalog-client | 3.1.3-amzn-1.1 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 3.1.3-amzn-1.1 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 3.1.3-amzn-1.1 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 3.1.3-amzn-1.1 | Hive command line client. | | hive-hbase | 3.1.3-amzn-1.1 | Hive-hbase client. | | hive-metastore-server | 3.1.3-amzn-1.1 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server2 | 3.1.3-amzn-1.1 | Service for accepting Hive queries as web requests. | | hudi | 0.11.1-amzn-0 | Incremental processing framework to power data pipeline at low latency and high efficiency. | | hudi-presto | 0.11.1-amzn-0 | Bundle library for running Presto with Hudi. | | hudi-trino | 0.11.1-amzn-0 | Bundle library for running Trino with Hudi. | | hudi-spark | 0.11.1-amzn-0 | Bundle library for running Spark with Hudi. | | hue-server | 4.10.0 | Web application for analyzing data using Hadoop ecosystem applications | | iceberg | 0.14.0-amzn-0 | Apache Iceberg is an open table format for huge analytic datasets | | jupyterhub | 1.4.1 | Multi-user server for Jupyter notebooks | | livy-server | 0.7.1-incubating | REST interface for interacting with Apache Spark | | nginx | 1.12.1 | nginx [engine x] is an HTTP and reverse proxy server | | mxnet | 1.9.1 | A flexible, scalable, and efficient library for deep learning. | | mariadb-server | 5.5.68\$1 | MariaDB database server. | | nvidia-cuda | 11.7.0 | Nvidia drivers and Cuda toolkit | | oozie-client | 5.2.1 | Oozie command-line client. | | oozie-server | 5.2.1 | Service for accepting Oozie workflow requests. | | opencv | 4.5.0 | Open Source Computer Vision Library. | | phoenix-library | 5.1.2 | The phoenix libraries for server and client | | phoenix-connectors | 5.1.2 | Apache Phoenix-Connectors for Spark-3 | | phoenix-query-server | 5.1.2 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.273.3-amzn-0 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.273.3-amzn-0 | Service for executing pieces of a query. | | presto-client | 0.273.3-amzn-0 | Presto command-line client which is installed on an HA cluster's stand-by masters where Presto server is not started. | | trino-coordinator | 388-amzn-0 | Service for accepting queries and managing query execution among trino-workers. | | trino-worker | 388-amzn-0 | Service for executing pieces of a query. | | trino-client | 388-amzn-0 | Trino command-line client which is installed on an HA cluster's stand-by masters where Trino server is not started. | | pig-client | 0.17.0 | Pig command-line client. | | r | 4.0.2 | The R Project for Statistical Computing | | ranger-kms-server | 2.0.0 | Apache Ranger Key Management System | | spark-client | 3.3.0-amzn-0.1 | Spark command-line clients. | | spark-history-server | 3.3.0-amzn-0.1 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 3.3.0-amzn-0.1 | In-memory execution engine for YARN. | | spark-yarn-slave | 3.3.0-amzn-0.1 | Apache Spark libraries needed by YARN slaves. | | spark-rapids | 22.06.0-amzn-0 | Nvidia Spark RAPIDS plugin that accelerates Apache Spark with GPUs. | | sqoop-client | 1.4.7 | Apache Sqoop command-line client. | | tensorflow | 2.9.1 | TensorFlow open source software library for high performance numerical computation. | | tez-on-yarn | 0.9.2 | The tez YARN application and libraries. | | webserver | 2.4.41\$1 | Apache HTTP server. | | zeppelin-server | 0.10.1 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.5.10 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.5.10 | ZooKeeper command line client. | ## 6.8.1 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). Reconfiguration actions occur when you specify a configuration for instance groups in a running cluster. Amazon EMR only initiates reconfiguration actions for the classifications that you modify. For more information, see [Reconfigure an instance group in a running cluster](emr-configure-apps-running-cluster.md). **emr-6.8.1 classifications** | Classifications | Description | Reconfiguration Actions | | --- | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | Restarts the ResourceManager service. | | container-executor | Change values in Hadoop YARN's container-executor.cfg file. | Not available. | | container-log4j | Change values in Hadoop YARN's container-log4j.properties file. | Not available. | | core-site | Change values in Hadoop's core-site.xml file. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts Hadoop KMS, Ranger KMS, HiveServer2, Hive MetaStore, Hadoop Httpfs, and MapReduce-HistoryServer. | | docker-conf | Change docker related settings. | Not available. | | emrfs-site | Change EMRFS settings. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts HBaseRegionserver, HBaseMaster, HBaseThrift, HBaseRest, HiveServer2, Hive MetaStore, Hadoop Httpfs, and MapReduce-HistoryServer. | | flink-conf | Change flink-conf.yaml settings. | Restarts Flink history server. | | flink-log4j | Change Flink log4j.properties settings. | Restarts Flink history server. | | flink-log4j-session | Change Flink log4j-session.properties settings for Kubernetes/Yarn session. | Restarts Flink history server. | | flink-log4j-cli | Change Flink log4j-cli.properties settings. | Restarts Flink history server. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts PhoenixQueryserver, HiveServer2, Hive MetaStore, and MapReduce-HistoryServer. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | Restarts the Hadoop HDFS services SecondaryNamenode, Datanode, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts Hadoop KMS, Hadoop Httpfs, and MapReduce-HistoryServer. | | hadoop-ssl-server | Change hadoop ssl server configuration | Not available. | | hadoop-ssl-client | Change hadoop ssl client configuration | Not available. | | hbase | Amazon EMR-curated settings for Apache HBase. | Custom EMR specific property. Sets emrfs-site and hbase-site configs. See those for their associated restarts. | | hbase-env | Change values in HBase's environment. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbase.properties file. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | Not available. | | hbase-site | Change values in HBase's hbase-site.xml file. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. Additionally restarts Phoenix QueryServer. | | hdfs-encryption-zones | Configure HDFS encryption zones. | This classification should not be reconfigured. | | hdfs-env | Change values in the HDFS environment. | Restarts Hadoop HDFS services Namenode, Datanode, and ZKFC. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Additionally restarts Hadoop Httpfs. | | hcatalog-env | Change values in HCatalog's environment. | Restarts Hive HCatalog Server. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | Restarts Hive HCatalog Server. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | Restarts Hive HCatalog Server. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | Restarts Hive WebHCat server. | | hcatalog-webhcat-log4j2 | Change values in HCatalog WebHCat's log4j2.properties. | Restarts Hive WebHCat server. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | Restarts Hive WebHCat server. | | hive | Amazon EMR-curated settings for Apache Hive. | Sets configurations to launch Hive LLAP service. | | hive-beeline-log4j2 | Change values in Hive's beeline-log4j2.properties file. | Not available. | | hive-parquet-logging | Change values in Hive's parquet-logging.properties file. | Not available. | | hive-env | Change values in the Hive environment. | Restarts HiveServer2, HiveMetastore, and Hive HCatalog-Server. Runs Hive schemaTool CLI commands to verify hive-metastore. | | hive-exec-log4j2 | Change values in Hive's hive-exec-log4j2.properties file. | Not available. | | hive-llap-daemon-log4j2 | Change values in Hive's llap-daemon-log4j2.properties file. | Not available. | | hive-log4j2 | Change values in Hive's hive-log4j2.properties file. | Not available. | | hive-site | Change values in Hive's hive-site.xml file | Restarts HiveServer2, HiveMetastore, and Hive HCatalog-Server. Runs Hive schemaTool CLI commands to verify hive-metastore. Also restarts Oozie and Zeppelin. | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | Not available. | | hue-ini | Change values in Hue's ini file | Restarts Hue. Also activates Hue config override CLI commands to pick up new configurations. | | httpfs-env | Change values in the HTTPFS environment. | Restarts Hadoop Httpfs service. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | Restarts Hadoop Httpfs service. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | Not available. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | Restarts Hadoop-KMS service. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | Not available. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | Restarts Hadoop-KMS and Ranger-KMS service. | | hudi-env | Change values in the Hudi environment. | Not available. | | hudi-defaults | Change values in Hudi's hudi-defaults.conf file. | Not available. | | iceberg-defaults | Change values in Iceberg's iceberg-defaults.conf file. | Not available. | | jupyter-notebook-conf | Change values in Jupyter Notebook's jupyter\$1notebook\$1config.py file. | Not available. | | jupyter-hub-conf | Change values in JupyterHubs's jupyterhub\$1config.py file. | Not available. | | jupyter-s3-conf | Configure Jupyter Notebook S3 persistence. | Not available. | | jupyter-sparkmagic-conf | Change values in Sparkmagic's config.json file. | Not available. | | livy-conf | Change values in Livy's livy.conf file. | Restarts Livy Server. | | livy-env | Change values in the Livy environment. | Restarts Livy Server. | | livy-log4j | Change Livy log4j.properties settings. | Restarts Livy Server. | | mapred-env | Change values in the MapReduce application's environment. | Restarts Hadoop MapReduce-HistoryServer. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | Restarts Hadoop MapReduce-HistoryServer. | | oozie-env | Change values in Oozie's environment. | Restarts Oozie. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | Restarts Oozie. | | oozie-site | Change values in Oozie's oozie-site.xml file. | Restarts Oozie. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | Not available. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | Not available. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | Restarts Phoenix-QueryServer. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | Not available. | | pig-env | Change values in the Pig environment. | Not available. | | pig-properties | Change values in Pig's pig.properties file. | Restarts Oozie. | | pig-log4j | Change values in Pig's log4j.properties file. | Not available. | | presto-log | Change values in Presto's log.properties file. | Restarts Presto-Server (for PrestoDB) | | presto-config | Change values in Presto's config.properties file. | Restarts Presto-Server (for PrestoDB) | | presto-password-authenticator | Change values in Presto's password-authenticator.properties file. | Not available. | | presto-env | Change values in Presto's presto-env.sh file. | Restarts Presto-Server (for PrestoDB) | | presto-node | Change values in Presto's node.properties file. | Not available. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | Not available. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | Not available. | | presto-connector-hive | Change values in Presto's hive.properties file. | Restarts Presto-Server (for PrestoDB) | | presto-connector-jmx | Change values in Presto's jmx.properties file. | Not available. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | Not available. | | presto-connector-localfile | Change values in Presto's localfile.properties file. | Not available. | | presto-connector-memory | Change values in Presto's memory.properties file. | Not available. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | Not available. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | Not available. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | Not available. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | Not available. | | presto-connector-redis | Change values in Presto's redis.properties file. | Not available. | | presto-connector-redshift | Change values in Presto's redshift.properties file. | Not available. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | Not available. | | presto-connector-tpcds | Change values in Presto's tpcds.properties file. | Not available. | | trino-log | Change values in Trino's log.properties file. | Restarts Trino-Server (for Trino) | | trino-config | Change values in Trino's config.properties file. | Restarts Trino-Server (for Trino) | | trino-password-authenticator | Change values in Trino's password-authenticator.properties file. | Restarts Trino-Server (for Trino) | | trino-env | Change values in Trino's trino-env.sh file. | Restarts Trino-Server (for Trino) | | trino-node | Change values in Trino's node.properties file. | Not available. | | trino-connector-blackhole | Change values in Trino's blackhole.properties file. | Not available. | | trino-connector-cassandra | Change values in Trino's cassandra.properties file. | Not available. | | trino-connector-hive | Change values in Trino's hive.properties file. | Restarts Trino-Server (for Trino) | | trino-connector-iceberg | Change values in Trino's iceberg.properties file. | Restarts Trino-Server (for Trino) | | trino-connector-jmx | Change values in Trino's jmx.properties file. | Not available. | | trino-connector-kafka | Change values in Trino's kafka.properties file. | Not available. | | trino-connector-localfile | Change values in Trino's localfile.properties file. | Not available. | | trino-connector-memory | Change values in Trino's memory.properties file. | Not available. | | trino-connector-mongodb | Change values in Trino's mongodb.properties file. | Not available. | | trino-connector-mysql | Change values in Trino's mysql.properties file. | Not available. | | trino-connector-postgresql | Change values in Trino's postgresql.properties file. | Not available. | | trino-connector-raptor | Change values in Trino's raptor.properties file. | Not available. | | trino-connector-redis | Change values in Trino's redis.properties file. | Not available. | | trino-connector-redshift | Change values in Trino's redshift.properties file. | Not available. | | trino-connector-tpch | Change values in Trino's tpch.properties file. | Not available. | | trino-connector-tpcds | Change values in Trino's tpcds.properties file. | Not available. | | ranger-kms-dbks-site | Change values in dbks-site.xml file of Ranger KMS. | Restarts Ranger KMS Server. | | ranger-kms-site | Change values in ranger-kms-site.xml file of Ranger KMS. | Restarts Ranger KMS Server. | | ranger-kms-env | Change values in the Ranger KMS environment. | Restarts Ranger KMS Server. | | ranger-kms-log4j | Change values in kms-log4j.properties file of Ranger KMS. | Not available. | | ranger-kms-db-ca | Change values for CA file on S3 for MySQL SSL connection with Ranger KMS. | Not available. | | spark | Amazon EMR-curated settings for Apache Spark. | This property modifies spark-defaults. See actions there. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | Restarts Spark history server and Spark thrift server. | | spark-env | Change values in the Spark environment. | Restarts Spark history server and Spark thrift server. | | spark-hive-site | Change values in Spark's hive-site.xml file | Not available. | | spark-log4j2 | Change values in Spark's log4j2.properties file. | Restarts Spark history server and Spark thrift server. | | spark-metrics | Change values in Spark's metrics.properties file. | Restarts Spark history server and Spark thrift server. | | sqoop-env | Change values in Sqoop's environment. | Not available. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | Not available. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | Not available. | | tez-site | Change values in Tez's tez-site.xml file. | Restart Oozie and HiveServer2. | | yarn-env | Change values in the YARN environment. | Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts MapReduce-HistoryServer. | | yarn-site | Change values in YARN's yarn-site.xml file. | Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts Livy Server and MapReduce-HistoryServer. | | zeppelin-env | Change values in the Zeppelin environment. | Restarts Zeppelin. | | zeppelin-site | Change configuration settings in zeppelin-site.xml. | Restarts Zeppelin. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | Restarts Zookeeper server. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | Restarts Zookeeper server. | ## 6.8.1 change log **Change log for 6.8.1 release and release notes** | Date | Event | Description | | --- | --- | --- | | 2023-08-30 | Update release notes | Added several control-plane related fixes to the release notes | | 2023-08-21 | Docs publication | Amazon EMR 6.8.1 release notes first published | | 2023-08-16 | Deployment complete | Amazon EMR 6.8.1 fully deployed to all [supported Regions](https://aws.amazon.com/about-aws/global-infrastructure/regional-product-services/) | | 2023-08-04 | Initial release | Amazon EMR 6.8.1 first deployed to limited commercial Regions | # Amazon EMR release 6.8.0 ## 6.8.0 application versions This release includes the following applications: [https://flink.apache.org/](https://flink.apache.org/), [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [https://hudi.apache.org](https://hudi.apache.org), [http://gethue.com/](http://gethue.com/), [https://iceberg.apache.org/](https://iceberg.apache.org/), [https://jupyter-enterprise-gateway.readthedocs.io/en/latest/](https://jupyter-enterprise-gateway.readthedocs.io/en/latest/), [https://jupyterhub.readthedocs.io/en/latest/#](https://jupyterhub.readthedocs.io/en/latest/#), [https://livy.incubator.apache.org/](https://livy.incubator.apache.org/), [https://mxnet.incubator.apache.org/](https://mxnet.incubator.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://www.tensorflow.org/](https://www.tensorflow.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://trino.io/](https://trino.io/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-6.8.0 | emr-6.7.0 | emr-6.6.0 | emr-6.5.0 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.12.170 | 1.12.170 | 1.12.170 | 1.12.31 | | Python | 2.7, 3.7 | 2.7, 3.7 | 2.7, 3.7 | 2.7, 3.7 | | Scala | 2.12.15 | 2.12.15 | 2.12.10 | 2.12.10 | | AmazonCloudWatchAgent | - | - | - | - | | Delta | - | - | - | - | | Flink | 1.15.1 | 1.14.2 | 1.14.2 | 1.14.0 | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 2.4.12-amzn-0 | 2.4.4-amzn-3 | 2.4.4-amzn-2 | 2.4.4-amzn-1 | | HCatalog | 3.1.3-amzn-1 | 3.1.3-amzn-0 | 3.1.2-amzn-7 | 3.1.2-amzn-6 | | Hadoop | 3.2.1-amzn-8 | 3.2.1-amzn-7 | 3.2.1-amzn-6 | 3.2.1-amzn-5 | | Hive | 3.1.3-amzn-1 | 3.1.3-amzn-0 | 3.1.2-amzn-7 | 3.1.2-amzn-6 | | Hudi | 0.11.1-amzn-0 | 0.11.0-amzn-0 | 0.10.1-amzn-0 | 0.9.0-amzn-1 | | Hue | 4.10.0 | 4.10.0 | 4.10.0 | 4.9.0 | | Iceberg | 0.14.0-amzn-0 | 0.13.1-amzn-0 | 0.13.1 | 0.12.0 | | JupyterEnterpriseGateway | 2.1.0 | 2.1.0 | 2.1.0 | 2.1.0 | | JupyterHub | 1.4.1 | 1.4.1 | 1.4.1 | 1.4.1 | | Livy | 0.7.1-incubating | 0.7.1-incubating | 0.7.1-incubating | 0.7.1-incubating | | MXNet | 1.9.1 | 1.8.0 | 1.8.0 | 1.8.0 | | Mahout | - | - | - | - | | Oozie | 5.2.1 | 5.2.1 | 5.2.1 | 5.2.1 | | Phoenix | 5.1.2 | 5.1.2 | 5.1.2 | 5.1.2 | | Pig | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | | Presto | 0.273.3-amzn-0 | 0.272-amzn-0 | 0.267-amzn-0 | 0.261-amzn-0 | | Spark | 3.3.0-amzn-0 | 3.2.1-amzn-0 | 3.2.0-amzn-0 | 3.1.2-amzn-1 | | Sqoop | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | | TensorFlow | 2.9.1 | 2.4.1 | 2.4.1 | 2.4.1 | | Tez | 0.9.2 | 0.9.2 | 0.9.2 | 0.9.2 | | Trino (PrestoSQL) | 388-amzn-0 | 378-amzn-0 | 367-amzn-0 | 360 | | Zeppelin | 0.10.1 | 0.10.0 | 0.10.0 | 0.10.0 | | ZooKeeper | 3.5.10 | 3.5.7 | 3.5.7 | 3.5.7 | ## 6.8.0 release notes The following release notes include information for Amazon EMR release 6.8.0. Changes are relative to 6.7.0. **New Features** + Amazon EMR steps feature now supports Apache Livy endpoint and JDBC/ODBC clients. For more information, see [Configure runtime roles for Amazon EMR steps](https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-steps-runtime-roles.html). + Amazon EMR release 6.8.0 comes with Apache HBase release 2.4.12. With this HBase release, you can both archive and delete your HBase tables. The Amazon S3 archive process renames all table files to the archive directory. This can be a costly and lengthy process. Now, you can skip the archive process and quickly drop and delete large tables. For more information, see [Using the HBase shell](emr-hbase-connect.md). **Known Issues** + Hadoop 3.3.3 introduced a change in YARN ([YARN-9608](https://issues.apache.org/jira/browse/YARN-9608)) that keeps nodes where containers ran in a decommissioning state until the application completes. This change ensures that local data such as shuffle data doesn't get lost, and you don' need to re-run the job. In Amazon EMR 6.8.0 and 6.9.0, this approach might also lead to underutilization of resources on clusters with or without managed scaling enabled. With [Amazon EMR 6.10.0](emr-6100-release.md#emr-6100-relnotes), there's a workaround for this issue to set the value of `yarn.resourcemanager.decommissioning-nodes-watcher.wait-for-applications` to `false` in `yarn-site.xml`. In Amazon EMR releases 6.11.0 and higher as well as 6.8.1, 6.9.1, and 6.10.1, the config is set to `false` by default to resolve this issue. **Changes, Enhancements, and Resolved Issues** + When Amazon EMR release 6.5.0, 6.6.0, or 6.7.0 read Apache Phoenix tables through the Apache Spark shell, Amazon EMR produced a `NoSuchMethodError`. Amazon EMR release 6.8.0 fixes this issue. + Amazon EMR release 6.8.0 comes with [Apache Hudi](https://hudi.apache.org/) 0.11.1; however, Amazon EMR 6.8.0 clusters are also compatible with the open-source `hudi-spark3.3-bundle_2.12` from Hudi 0.12.0. + Amazon EMR release 6.8.0 comes with Apache Spark 3.3.0. This Spark release uses Apache Log4j 2 and the `log4j2.properties` file to configure Log4j in Spark processes. If you use Spark in the cluster or create EMR clusters with custom configuration parameters, and you want to upgrade to Amazon EMR release 6.8.0, you must migrate to the new `spark-log4j2` configuration classification and key format for Apache Log4j 2. For more information, see [Migrating from Apache Log4j 1.x to Log4j 2.x](emr-spark-configure.md#spark-migrate-logj42). + When you launch a cluster with *the latest patch release* of Amazon EMR 5.36 or higher, 6.6 or higher, or 7.0 or higher, Amazon EMR uses the latest Amazon Linux 2023 or Amazon Linux 2 release for the default Amazon EMR AMI. For more information, see [Using the default Amazon Linux AMI for Amazon EMR](https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-default-ami.html). **Note** This release no longer gets automatic AMI updates since it has been succeeded by 1 more more patch releases. The patch release is denoted by the number after the second decimal point (`6.8.1`). To see if you're using the latest patch release, check the available releases in the [https://docs.aws.amazon.com/emr/latest/ReleaseGuide](https://docs.aws.amazon.com/emr/latest/ReleaseGuide), or check the **Amazon EMR release** dropdown when you create a cluster in the console, or use the [https://docs.aws.amazon.com/emr/latest/APIReference/API_ListReleaseLabels.html](https://docs.aws.amazon.com/emr/latest/APIReference/API_ListReleaseLabels.html) API or [https://docs.aws.amazon.com/cli/latest/reference/emr/list-release-labels.html](https://docs.aws.amazon.com/cli/latest/reference/emr/list-release-labels.html) CLI action. To get updates about new releases, subscribe to the RSS feed on the [What's new?](https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-whatsnew.html) page. [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-680-release.html) **Known Issues** + When you use the DynamoDB connector with Spark on Amazon EMR versions 6.6.0, 6.7.0, and 6.8.0, all reads from your table return an empty result, even though the input split references non-empty data. This is because Spark 3.2.0 sets `spark.hadoopRDD.ignoreEmptySplits` to `true` by default. As a workaround, explicitly set `spark.hadoopRDD.ignoreEmptySplits` to `false`. Amazon EMR release 6.9.0 fixes this issue. + When you use Spark with Hive partition location formatting to read data in Amazon S3, and you run Spark on Amazon EMR releases 5.30.0 to 5.36.0, and 6.2.0 to 6.9.0, you might encounter an issue that prevents your cluster from reading data correctly. This can happen if your partitions have all of the following characteristics: + Two or more partitions are scanned from the same table. + At least one partition directory path is a prefix of at least one other partition directory path, for example, `s3://bucket/table/p=a` is a prefix of `s3://bucket/table/p=a b`. + The first character that follows the prefix in the other partition directory has a UTF-8 value that’s less than than the `/` character (U\$1002F). For example, the space character (U\$10020) that occurs between a and b in `s3://bucket/table/p=a b` falls into this category. Note that there are 14 other non-control characters: `!"#$%&‘()*+,-`. For more information, see [UTF-8 encoding table and Unicode characters](https://www.utf8-chartable.de/). As a workaround to this issue, set the `spark.sql.sources.fastS3PartitionDiscovery.enabled` configuration to `false` in the `spark-defaults` classification. + With Amazon EMR releases 5.36.0 and 6.6.0 through 6.9.0, `SecretAgent` and `RecordServer` service components may experience log data loss due to an incorrect file name pattern configuration in Log4j2 properties. The incorrect configuration causes the components to generate only one log file per day. When the rotation strategy occurs, it overwrites the existing file instead of generating a new log file as expected. As a workaround, use a bootstrap action to generate log files each hour and append an auto-increment integer in the file name to handle the rotation. For Amazon EMR 6.6.0 through 6.9.0 releases, use the following bootstrap action when you launch a cluster. ``` ‑‑bootstrap‑actions "Path=s3://emr-data-access-control-us-east-1/customer-bootstrap-actions/log-rotation-emr-6x/replace-puppet.sh,Args=[]" ``` For Amazon EMR 5.36.0, use the following bootstrap action when you launch a cluster. ``` ‑‑bootstrap‑actions "Path=s3://emr-data-access-control-us-east-1/customer-bootstrap-actions/log-rotation-emr-5x/replace-puppet.sh,Args=[]" ``` For more information on the release timeline, see the [change log](https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-680-release.html#680-changelog). ## 6.8.0 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | aws-sagemaker-spark-sdk | 1.4.2 | Amazon SageMaker Spark SDK | | emr-ddb | 4.16.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 3.2.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.5.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-notebook-env | 1.7.0 | Conda env for emr notebook which includes jupyter enterprise gateway | | emr-s3-dist-cp | 2.22.0 | Distributed copy application optimized for Amazon S3. | | emr-s3-select | 2.1.0 | EMR S3Select Connector | | emrfs | 2.53.0 | Amazon S3 connector for Hadoop ecosystem applications. | | flink-client | 1.15.1 | Apache Flink command line client scripts and applications. | | flink-jobmanager-config | 1.15.1 | Managing resources on EMR nodes for Apache Flink JobManager. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 3.2.1-amzn-8 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 3.2.1-amzn-8 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 3.2.1-amzn-8 | HDFS command-line client and library | | hadoop-hdfs-namenode | 3.2.1-amzn-8 | HDFS service for tracking file names and block locations. | | hadoop-hdfs-journalnode | 3.2.1-amzn-8 | HDFS service for managing the Hadoop filesystem journal on HA clusters. | | hadoop-httpfs-server | 3.2.1-amzn-8 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 3.2.1-amzn-8 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 3.2.1-amzn-8 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 3.2.1-amzn-8 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 3.2.1-amzn-8 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 3.2.1-amzn-8 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 2.4.12-amzn-0 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 2.4.12-amzn-0 | Service for serving one or more HBase regions. | | hbase-client | 2.4.12-amzn-0 | HBase command-line client. | | hbase-rest-server | 2.4.12-amzn-0 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 2.4.12-amzn-0 | Service providing a Thrift endpoint to HBase. | | hbase-operator-tools | 2.4.12-amzn-0 | Repair tool for Apache HBase clusters. | | hcatalog-client | 3.1.3-amzn-1 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 3.1.3-amzn-1 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 3.1.3-amzn-1 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 3.1.3-amzn-1 | Hive command line client. | | hive-hbase | 3.1.3-amzn-1 | Hive-hbase client. | | hive-metastore-server | 3.1.3-amzn-1 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server2 | 3.1.3-amzn-1 | Service for accepting Hive queries as web requests. | | hudi | 0.11.1-amzn-0 | Incremental processing framework to power data pipeline at low latency and high efficiency. | | hudi-presto | 0.11.1-amzn-0 | Bundle library for running Presto with Hudi. | | hudi-trino | 0.11.1-amzn-0 | Bundle library for running Trino with Hudi. | | hudi-spark | 0.11.1-amzn-0 | Bundle library for running Spark with Hudi. | | hue-server | 4.10.0 | Web application for analyzing data using Hadoop ecosystem applications | | iceberg | 0.14.0-amzn-0 | Apache Iceberg is an open table format for huge analytic datasets | | jupyterhub | 1.4.1 | Multi-user server for Jupyter notebooks | | livy-server | 0.7.1-incubating | REST interface for interacting with Apache Spark | | nginx | 1.12.1 | nginx [engine x] is an HTTP and reverse proxy server | | mxnet | 1.9.1 | A flexible, scalable, and efficient library for deep learning. | | mariadb-server | 5.5.68\$1 | MariaDB database server. | | nvidia-cuda | 11.7.0 | Nvidia drivers and Cuda toolkit | | oozie-client | 5.2.1 | Oozie command-line client. | | oozie-server | 5.2.1 | Service for accepting Oozie workflow requests. | | opencv | 4.5.0 | Open Source Computer Vision Library. | | phoenix-library | 5.1.2 | The phoenix libraries for server and client | | phoenix-connectors | 5.1.2 | Apache Phoenix-Connectors for Spark-3 | | phoenix-query-server | 5.1.2 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.273.3-amzn-0 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.273.3-amzn-0 | Service for executing pieces of a query. | | presto-client | 0.273.3-amzn-0 | Presto command-line client which is installed on an HA cluster's stand-by masters where Presto server is not started. | | trino-coordinator | 388-amzn-0 | Service for accepting queries and managing query execution among trino-workers. | | trino-worker | 388-amzn-0 | Service for executing pieces of a query. | | trino-client | 388-amzn-0 | Trino command-line client which is installed on an HA cluster's stand-by masters where Trino server is not started. | | pig-client | 0.17.0 | Pig command-line client. | | r | 4.0.2 | The R Project for Statistical Computing | | ranger-kms-server | 2.0.0 | Apache Ranger Key Management System | | spark-client | 3.3.0-amzn-0 | Spark command-line clients. | | spark-history-server | 3.3.0-amzn-0 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 3.3.0-amzn-0 | In-memory execution engine for YARN. | | spark-yarn-slave | 3.3.0-amzn-0 | Apache Spark libraries needed by YARN slaves. | | spark-rapids | 22.06.0-amzn-0 | Nvidia Spark RAPIDS plugin that accelerates Apache Spark with GPUs. | | sqoop-client | 1.4.7 | Apache Sqoop command-line client. | | tensorflow | 2.9.1 | TensorFlow open source software library for high performance numerical computation. | | tez-on-yarn | 0.9.2 | The tez YARN application and libraries. | | webserver | 2.4.41\$1 | Apache HTTP server. | | zeppelin-server | 0.10.1 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.5.10 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.5.10 | ZooKeeper command line client. | ## 6.8.0 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). Reconfiguration actions occur when you specify a configuration for instance groups in a running cluster. Amazon EMR only initiates reconfiguration actions for the classifications that you modify. For more information, see [Reconfigure an instance group in a running cluster](emr-configure-apps-running-cluster.md). **emr-6.8.0 classifications** | Classifications | Description | Reconfiguration Actions | | --- | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | Restarts the ResourceManager service. | | container-executor | Change values in Hadoop YARN's container-executor.cfg file. | Not available. | | container-log4j | Change values in Hadoop YARN's container-log4j.properties file. | Not available. | | core-site | Change values in Hadoop's core-site.xml file. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts Hadoop KMS, Ranger KMS, HiveServer2, Hive MetaStore, Hadoop Httpfs, and MapReduce-HistoryServer. | | docker-conf | Change docker related settings. | Not available. | | emrfs-site | Change EMRFS settings. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts HBaseRegionserver, HBaseMaster, HBaseThrift, HBaseRest, HiveServer2, Hive MetaStore, Hadoop Httpfs, and MapReduce-HistoryServer. | | flink-conf | Change flink-conf.yaml settings. | Restarts Flink history server. | | flink-log4j | Change Flink log4j.properties settings. | Restarts Flink history server. | | flink-log4j-session | Change Flink log4j-session.properties settings for Kubernetes/Yarn session. | Restarts Flink history server. | | flink-log4j-cli | Change Flink log4j-cli.properties settings. | Restarts Flink history server. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts PhoenixQueryserver, HiveServer2, Hive MetaStore, and MapReduce-HistoryServer. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | Restarts the Hadoop HDFS services SecondaryNamenode, Datanode, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts Hadoop KMS, Hadoop Httpfs, and MapReduce-HistoryServer. | | hadoop-ssl-server | Change hadoop ssl server configuration | Not available. | | hadoop-ssl-client | Change hadoop ssl client configuration | Not available. | | hbase | Amazon EMR-curated settings for Apache HBase. | Custom EMR specific property. Sets emrfs-site and hbase-site configs. See those for their associated restarts. | | hbase-env | Change values in HBase's environment. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbase.properties file. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | Not available. | | hbase-site | Change values in HBase's hbase-site.xml file. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. Additionally restarts Phoenix QueryServer. | | hdfs-encryption-zones | Configure HDFS encryption zones. | This classification should not be reconfigured. | | hdfs-env | Change values in the HDFS environment. | Restarts Hadoop HDFS services Namenode, Datanode, and ZKFC. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Additionally restarts Hadoop Httpfs. | | hcatalog-env | Change values in HCatalog's environment. | Restarts Hive HCatalog Server. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | Restarts Hive HCatalog Server. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | Restarts Hive HCatalog Server. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | Restarts Hive WebHCat server. | | hcatalog-webhcat-log4j2 | Change values in HCatalog WebHCat's log4j2.properties. | Restarts Hive WebHCat server. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | Restarts Hive WebHCat server. | | hive | Amazon EMR-curated settings for Apache Hive. | Sets configurations to launch Hive LLAP service. | | hive-beeline-log4j2 | Change values in Hive's beeline-log4j2.properties file. | Not available. | | hive-parquet-logging | Change values in Hive's parquet-logging.properties file. | Not available. | | hive-env | Change values in the Hive environment. | Restarts HiveServer2, HiveMetastore, and Hive HCatalog-Server. Runs Hive schemaTool CLI commands to verify hive-metastore. | | hive-exec-log4j2 | Change values in Hive's hive-exec-log4j2.properties file. | Not available. | | hive-llap-daemon-log4j2 | Change values in Hive's llap-daemon-log4j2.properties file. | Not available. | | hive-log4j2 | Change values in Hive's hive-log4j2.properties file. | Not available. | | hive-site | Change values in Hive's hive-site.xml file | Restarts HiveServer2, HiveMetastore, and Hive HCatalog-Server. Runs Hive schemaTool CLI commands to verify hive-metastore. Also restarts Oozie and Zeppelin. | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | Not available. | | hue-ini | Change values in Hue's ini file | Restarts Hue. Also activates Hue config override CLI commands to pick up new configurations. | | httpfs-env | Change values in the HTTPFS environment. | Restarts Hadoop Httpfs service. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | Restarts Hadoop Httpfs service. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | Not available. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | Restarts Hadoop-KMS service. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | Not available. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | Restarts Hadoop-KMS and Ranger-KMS service. | | hudi-env | Change values in the Hudi environment. | Not available. | | hudi-defaults | Change values in Hudi's hudi-defaults.conf file. | Not available. | | iceberg-defaults | Change values in Iceberg's iceberg-defaults.conf file. | Not available. | | jupyter-notebook-conf | Change values in Jupyter Notebook's jupyter\$1notebook\$1config.py file. | Not available. | | jupyter-hub-conf | Change values in JupyterHubs's jupyterhub\$1config.py file. | Not available. | | jupyter-s3-conf | Configure Jupyter Notebook S3 persistence. | Not available. | | jupyter-sparkmagic-conf | Change values in Sparkmagic's config.json file. | Not available. | | livy-conf | Change values in Livy's livy.conf file. | Restarts Livy Server. | | livy-env | Change values in the Livy environment. | Restarts Livy Server. | | livy-log4j | Change Livy log4j.properties settings. | Restarts Livy Server. | | mapred-env | Change values in the MapReduce application's environment. | Restarts Hadoop MapReduce-HistoryServer. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | Restarts Hadoop MapReduce-HistoryServer. | | oozie-env | Change values in Oozie's environment. | Restarts Oozie. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | Restarts Oozie. | | oozie-site | Change values in Oozie's oozie-site.xml file. | Restarts Oozie. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | Not available. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | Not available. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | Restarts Phoenix-QueryServer. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | Not available. | | pig-env | Change values in the Pig environment. | Not available. | | pig-properties | Change values in Pig's pig.properties file. | Restarts Oozie. | | pig-log4j | Change values in Pig's log4j.properties file. | Not available. | | presto-log | Change values in Presto's log.properties file. | Restarts Presto-Server (for PrestoDB) | | presto-config | Change values in Presto's config.properties file. | Restarts Presto-Server (for PrestoDB) | | presto-password-authenticator | Change values in Presto's password-authenticator.properties file. | Not available. | | presto-env | Change values in Presto's presto-env.sh file. | Restarts Presto-Server (for PrestoDB) | | presto-node | Change values in Presto's node.properties file. | Not available. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | Not available. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | Not available. | | presto-connector-hive | Change values in Presto's hive.properties file. | Restarts Presto-Server (for PrestoDB) | | presto-connector-jmx | Change values in Presto's jmx.properties file. | Not available. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | Not available. | | presto-connector-localfile | Change values in Presto's localfile.properties file. | Not available. | | presto-connector-memory | Change values in Presto's memory.properties file. | Not available. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | Not available. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | Not available. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | Not available. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | Not available. | | presto-connector-redis | Change values in Presto's redis.properties file. | Not available. | | presto-connector-redshift | Change values in Presto's redshift.properties file. | Not available. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | Not available. | | presto-connector-tpcds | Change values in Presto's tpcds.properties file. | Not available. | | trino-log | Change values in Trino's log.properties file. | Restarts Trino-Server (for Trino) | | trino-config | Change values in Trino's config.properties file. | Restarts Trino-Server (for Trino) | | trino-password-authenticator | Change values in Trino's password-authenticator.properties file. | Restarts Trino-Server (for Trino) | | trino-env | Change values in Trino's trino-env.sh file. | Restarts Trino-Server (for Trino) | | trino-node | Change values in Trino's node.properties file. | Not available. | | trino-connector-blackhole | Change values in Trino's blackhole.properties file. | Not available. | | trino-connector-cassandra | Change values in Trino's cassandra.properties file. | Not available. | | trino-connector-hive | Change values in Trino's hive.properties file. | Restarts Trino-Server (for Trino) | | trino-connector-iceberg | Change values in Trino's iceberg.properties file. | Restarts Trino-Server (for Trino) | | trino-connector-jmx | Change values in Trino's jmx.properties file. | Not available. | | trino-connector-kafka | Change values in Trino's kafka.properties file. | Not available. | | trino-connector-localfile | Change values in Trino's localfile.properties file. | Not available. | | trino-connector-memory | Change values in Trino's memory.properties file. | Not available. | | trino-connector-mongodb | Change values in Trino's mongodb.properties file. | Not available. | | trino-connector-mysql | Change values in Trino's mysql.properties file. | Not available. | | trino-connector-postgresql | Change values in Trino's postgresql.properties file. | Not available. | | trino-connector-raptor | Change values in Trino's raptor.properties file. | Not available. | | trino-connector-redis | Change values in Trino's redis.properties file. | Not available. | | trino-connector-redshift | Change values in Trino's redshift.properties file. | Not available. | | trino-connector-tpch | Change values in Trino's tpch.properties file. | Not available. | | trino-connector-tpcds | Change values in Trino's tpcds.properties file. | Not available. | | ranger-kms-dbks-site | Change values in dbks-site.xml file of Ranger KMS. | Restarts Ranger KMS Server. | | ranger-kms-site | Change values in ranger-kms-site.xml file of Ranger KMS. | Restarts Ranger KMS Server. | | ranger-kms-env | Change values in the Ranger KMS environment. | Restarts Ranger KMS Server. | | ranger-kms-log4j | Change values in kms-log4j.properties file of Ranger KMS. | Not available. | | ranger-kms-db-ca | Change values for CA file on S3 for MySQL SSL connection with Ranger KMS. | Not available. | | spark | Amazon EMR-curated settings for Apache Spark. | This property modifies spark-defaults. See actions there. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | Restarts Spark history server and Spark thrift server. | | spark-env | Change values in the Spark environment. | Restarts Spark history server and Spark thrift server. | | spark-hive-site | Change values in Spark's hive-site.xml file | Not available. | | spark-log4j2 | Change values in Spark's log4j2.properties file. | Restarts Spark history server and Spark thrift server. | | spark-metrics | Change values in Spark's metrics.properties file. | Restarts Spark history server and Spark thrift server. | | sqoop-env | Change values in Sqoop's environment. | Not available. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | Not available. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | Not available. | | tez-site | Change values in Tez's tez-site.xml file. | Restart Oozie and HiveServer2. | | yarn-env | Change values in the YARN environment. | Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts MapReduce-HistoryServer. | | yarn-site | Change values in YARN's yarn-site.xml file. | Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts Livy Server and MapReduce-HistoryServer. | | zeppelin-env | Change values in the Zeppelin environment. | Restarts Zeppelin. | | zeppelin-site | Change configuration settings in zeppelin-site.xml. | Restarts Zeppelin. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | Restarts Zookeeper server. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | Restarts Zookeeper server. | ## 6.8.0 change log **Change log for 6.8.0 release and release notes** | Date | Event | Description | | --- | --- | --- | | 2023-08-21 | Update | Added a known issue with Hadoop 3.3.3. | | 2023-07-26 | Update | New OS release labels 2.0.20230612.0 and 2.0.20230628.0. | | 2022-09-06 | Deployment complete | Amazon EMR 6.8 fully deployed to all [supported Regions](https://aws.amazon.com/about-aws/global-infrastructure/regional-product-services/) | | 2022-09-06 | Initial publication | Amazon EMR 6.8 release notes first published | | 2022-08-31 | Initial release | Amazon EMR 6.8 released to limited commercial Regions | # Amazon EMR release 6.7.0 ## 6.7.0 application versions This release includes the following applications: [https://flink.apache.org/](https://flink.apache.org/), [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [https://hudi.apache.org](https://hudi.apache.org), [http://gethue.com/](http://gethue.com/), [https://iceberg.apache.org/](https://iceberg.apache.org/), [https://jupyter-enterprise-gateway.readthedocs.io/en/latest/](https://jupyter-enterprise-gateway.readthedocs.io/en/latest/), [https://jupyterhub.readthedocs.io/en/latest/#](https://jupyterhub.readthedocs.io/en/latest/#), [https://livy.incubator.apache.org/](https://livy.incubator.apache.org/), [https://mxnet.incubator.apache.org/](https://mxnet.incubator.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://www.tensorflow.org/](https://www.tensorflow.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://trino.io/](https://trino.io/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-6.7.0 | emr-6.6.0 | emr-6.5.0 | emr-6.4.0 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.12.170 | 1.12.170 | 1.12.31 | 1.12.31 | | Python | 2.7, 3.7 | 2.7, 3.7 | 2.7, 3.7 | 2.7, 3.7 | | Scala | 2.12.15 | 2.12.10 | 2.12.10 | 2.12.10 | | AmazonCloudWatchAgent | - | - | - | - | | Delta | - | - | - | - | | Flink | 1.14.2 | 1.14.2 | 1.14.0 | 1.13.1 | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 2.4.4-amzn-3 | 2.4.4-amzn-2 | 2.4.4-amzn-1 | 2.4.4-amzn-0 | | HCatalog | 3.1.3-amzn-0 | 3.1.2-amzn-7 | 3.1.2-amzn-6 | 3.1.2-amzn-5 | | Hadoop | 3.2.1-amzn-7 | 3.2.1-amzn-6 | 3.2.1-amzn-5 | 3.2.1-amzn-4 | | Hive | 3.1.3-amzn-0 | 3.1.2-amzn-7 | 3.1.2-amzn-6 | 3.1.2-amzn-5 | | Hudi | 0.11.0-amzn-0 | 0.10.1-amzn-0 | 0.9.0-amzn-1 | 0.8.0-amzn-0 | | Hue | 4.10.0 | 4.10.0 | 4.9.0 | 4.9.0 | | Iceberg | 0.13.1-amzn-0 | 0.13.1 | 0.12.0 | - | | JupyterEnterpriseGateway | 2.1.0 | 2.1.0 | 2.1.0 | 2.1.0 | | JupyterHub | 1.4.1 | 1.4.1 | 1.4.1 | 1.4.1 | | Livy | 0.7.1-incubating | 0.7.1-incubating | 0.7.1-incubating | 0.7.1-incubating | | MXNet | 1.8.0 | 1.8.0 | 1.8.0 | 1.8.0 | | Mahout | - | - | - | - | | Oozie | 5.2.1 | 5.2.1 | 5.2.1 | 5.2.1 | | Phoenix | 5.1.2 | 5.1.2 | 5.1.2 | 5.1.2 | | Pig | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | | Presto | 0.272-amzn-0 | 0.267-amzn-0 | 0.261-amzn-0 | 0.254.1-amzn-0 | | Spark | 3.2.1-amzn-0 | 3.2.0-amzn-0 | 3.1.2-amzn-1 | 3.1.2-amzn-0 | | Sqoop | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | | TensorFlow | 2.4.1 | 2.4.1 | 2.4.1 | 2.4.1 | | Tez | 0.9.2 | 0.9.2 | 0.9.2 | 0.9.2 | | Trino (PrestoSQL) | 378-amzn-0 | 367-amzn-0 | 360 | 359 | | Zeppelin | 0.10.0 | 0.10.0 | 0.10.0 | 0.9.0 | | ZooKeeper | 3.5.7 | 3.5.7 | 3.5.7 | 3.5.7 | ## 6.7.0 release notes The following release notes include information for Amazon EMR release 6.7.0. Changes are relative to 6.6.0. Initial release date: July 15, 2022 **New Features** + Amazon EMR now supports Apache Spark 3.2.1, Apache Hive 3.1.3, HUDI 0.11, PrestoDB 0.272, and Trino 0.378. + Supports IAM Role and Lake Formation-based access controls with EMR steps (Spark, Hive) for Amazon EMR on EC2 clusters. + Supports Apache Spark data definition statements on Apache Ranger enabled clusters. This now includes support for Trino applications reading and writing Apache Hive metadata on Apache Ranger enabled clusters. For more information, see [Enable federated governance using Trino and Apache Ranger on Amazon EMR](https://aws.amazon.com/blogs/big-data/enable-federated-governance-using-trino-and-apache-ranger-on-amazon-emr/). + When you launch a cluster with *the latest patch release* of Amazon EMR 5.36 or higher, 6.6 or higher, or 7.0 or higher, Amazon EMR uses the latest Amazon Linux 2023 or Amazon Linux 2 release for the default Amazon EMR AMI. For more information, see [Using the default Amazon Linux AMI for Amazon EMR](https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-default-ami.html). [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-670-release.html) **Known Issues** + When Amazon EMR release 6.5.0, 6.6.0, or 6.7.0 read Apache Phoenix tables through the Apache Spark shell, a `NoSuchMethodError` occurs because Amazon EMR uses an incorrect `Hbase.compat.version`. Amazon EMR release 6.8.0 fixes this issue. + When you use the DynamoDB connector with Spark on Amazon EMR versions 6.6.0, 6.7.0, and 6.8.0, all reads from your table return an empty result, even though the input split references non-empty data. This is because Spark 3.2.0 sets `spark.hadoopRDD.ignoreEmptySplits` to `true` by default. As a workaround, explicitly set `spark.hadoopRDD.ignoreEmptySplits` to `false`. Amazon EMR release 6.9.0 fixes this issue. + When you use Spark with Hive partition location formatting to read data in Amazon S3, and you run Spark on Amazon EMR releases 5.30.0 to 5.36.0, and 6.2.0 to 6.9.0, you might encounter an issue that prevents your cluster from reading data correctly. This can happen if your partitions have all of the following characteristics: + Two or more partitions are scanned from the same table. + At least one partition directory path is a prefix of at least one other partition directory path, for example, `s3://bucket/table/p=a` is a prefix of `s3://bucket/table/p=a b`. + The first character that follows the prefix in the other partition directory has a UTF-8 value that’s less than than the `/` character (U\$1002F). For example, the space character (U\$10020) that occurs between a and b in `s3://bucket/table/p=a b` falls into this category. Note that there are 14 other non-control characters: `!"#$%&‘()*+,-`. For more information, see [UTF-8 encoding table and Unicode characters](https://www.utf8-chartable.de/). As a workaround to this issue, set the `spark.sql.sources.fastS3PartitionDiscovery.enabled` configuration to `false` in the `spark-defaults` classification. + With Amazon EMR releases 5.36.0 and 6.6.0 through 6.9.0, `SecretAgent` and `RecordServer` service components may experience log data loss due to an incorrect file name pattern configuration in Log4j2 properties. The incorrect configuration causes the components to generate only one log file per day. When the rotation strategy occurs, it overwrites the existing file instead of generating a new log file as expected. As a workaround, use a bootstrap action to generate log files each hour and append an auto-increment integer in the file name to handle the rotation. For Amazon EMR 6.6.0 through 6.9.0 releases, use the following bootstrap action when you launch a cluster. ``` ‑‑bootstrap‑actions "Path=s3://emr-data-access-control-us-east-1/customer-bootstrap-actions/log-rotation-emr-6x/replace-puppet.sh,Args=[]" ``` For Amazon EMR 5.36.0, use the following bootstrap action when you launch a cluster. ``` ‑‑bootstrap‑actions "Path=s3://emr-data-access-control-us-east-1/customer-bootstrap-actions/log-rotation-emr-5x/replace-puppet.sh,Args=[]" ``` + The `GetClusterSessionCredentials` API isn't supported with clusters that run on Amazon EMR 6.7 or lower. + The following Hadoop commits were backported. - [[HADOOP-16080]](https://issues.apache.org/jira/browse/HADOOP-16080) Fix issue where `hadoop-aws` not working with `hadoop-client-api`. - [[HADOOP-18237]](https://issues.apache.org/jira/browse/HADOOP-18237) Upgrade Apache Xerces Java to 2.12.2. - [[YARN-11092]](https://issues.apache.org/jira/browse/YARN-11092) Upgrade jquery to ui to 1.13.1. - [[YARN-10720]](https://issues.apache.org/jira/browse/YARN-10720) YARN WebAppProxyServlet should support connection timeout to prevent proxy server from hanging. ## 6.7.0 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | aws-sagemaker-spark-sdk | 1.4.1 | Amazon SageMaker Spark SDK | | emr-ddb | 4.16.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 3.2.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.5.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-notebook-env | 1.6.0 | Conda env for emr notebook which includes jupyter enterprise gateway | | emr-s3-dist-cp | 2.22.0 | Distributed copy application optimized for Amazon S3. | | emr-s3-select | 2.1.0 | EMR S3Select Connector | | emrfs | 2.52.0 | Amazon S3 connector for Hadoop ecosystem applications. | | flink-client | 1.14.2 | Apache Flink command line client scripts and applications. | | flink-jobmanager-config | 1.14.2 | Managing resources on EMR nodes for Apache Flink JobManager. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 3.2.1-amzn-7 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 3.2.1-amzn-7 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 3.2.1-amzn-7 | HDFS command-line client and library | | hadoop-hdfs-namenode | 3.2.1-amzn-7 | HDFS service for tracking file names and block locations. | | hadoop-hdfs-journalnode | 3.2.1-amzn-7 | HDFS service for managing the Hadoop filesystem journal on HA clusters. | | hadoop-httpfs-server | 3.2.1-amzn-7 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 3.2.1-amzn-7 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 3.2.1-amzn-7 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 3.2.1-amzn-7 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 3.2.1-amzn-7 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 3.2.1-amzn-7 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 2.4.4-amzn-3 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 2.4.4-amzn-3 | Service for serving one or more HBase regions. | | hbase-client | 2.4.4-amzn-3 | HBase command-line client. | | hbase-rest-server | 2.4.4-amzn-3 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 2.4.4-amzn-3 | Service providing a Thrift endpoint to HBase. | | hbase-operator-tools | 2.4.4-amzn-3 | Repair tool for Apache HBase clusters. | | hcatalog-client | 3.1.3-amzn-0 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 3.1.3-amzn-0 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 3.1.3-amzn-0 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 3.1.3-amzn-0 | Hive command line client. | | hive-hbase | 3.1.3-amzn-0 | Hive-hbase client. | | hive-metastore-server | 3.1.3-amzn-0 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server2 | 3.1.3-amzn-0 | Service for accepting Hive queries as web requests. | | hudi | 0.11.0-amzn-0 | Incremental processing framework to power data pipeline at low latency and high efficiency. | | hudi-presto | 0.11.0-amzn-0 | Bundle library for running Presto with Hudi. | | hudi-trino | 0.11.0-amzn-0 | Bundle library for running Trino with Hudi. | | hudi-spark | 0.11.0-amzn-0 | Bundle library for running Spark with Hudi. | | hue-server | 4.10.0 | Web application for analyzing data using Hadoop ecosystem applications | | iceberg | 0.13.1-amzn-0 | Apache Iceberg is an open table format for huge analytic datasets | | jupyterhub | 1.4.1 | Multi-user server for Jupyter notebooks | | livy-server | 0.7.1-incubating | REST interface for interacting with Apache Spark | | nginx | 1.12.1 | nginx [engine x] is an HTTP and reverse proxy server | | mxnet | 1.8.0 | A flexible, scalable, and efficient library for deep learning. | | mariadb-server | 5.5.68\$1 | MariaDB database server. | | nvidia-cuda | 11.0.194 | Nvidia drivers and Cuda toolkit | | oozie-client | 5.2.1 | Oozie command-line client. | | oozie-server | 5.2.1 | Service for accepting Oozie workflow requests. | | opencv | 4.5.0 | Open Source Computer Vision Library. | | phoenix-library | 5.1.2 | The phoenix libraries for server and client | | phoenix-connectors | 5.1.2 | Apache Phoenix-Connectors for Spark-3 | | phoenix-query-server | 5.1.2 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.272-amzn-0 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.272-amzn-0 | Service for executing pieces of a query. | | presto-client | 0.272-amzn-0 | Presto command-line client which is installed on an HA cluster's stand-by masters where Presto server is not started. | | trino-coordinator | 378-amzn-0 | Service for accepting queries and managing query execution among trino-workers. | | trino-worker | 378-amzn-0 | Service for executing pieces of a query. | | trino-client | 378-amzn-0 | Trino command-line client which is installed on an HA cluster's stand-by masters where Trino server is not started. | | pig-client | 0.17.0 | Pig command-line client. | | r | 4.0.2 | The R Project for Statistical Computing | | ranger-kms-server | 2.0.0 | Apache Ranger Key Management System | | spark-client | 3.2.1-amzn-0 | Spark command-line clients. | | spark-history-server | 3.2.1-amzn-0 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 3.2.1-amzn-0 | In-memory execution engine for YARN. | | spark-yarn-slave | 3.2.1-amzn-0 | Apache Spark libraries needed by YARN slaves. | | spark-rapids | 22.02.0-amzn-1 | Nvidia Spark RAPIDS plugin that accelerates Apache Spark with GPUs. | | sqoop-client | 1.4.7 | Apache Sqoop command-line client. | | tensorflow | 2.4.1 | TensorFlow open source software library for high performance numerical computation. | | tez-on-yarn | 0.9.2 | The tez YARN application and libraries. | | webserver | 2.4.41\$1 | Apache HTTP server. | | zeppelin-server | 0.10.0 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.5.7 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.5.7 | ZooKeeper command line client. | ## 6.7.0 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). Reconfiguration actions occur when you specify a configuration for instance groups in a running cluster. Amazon EMR only initiates reconfiguration actions for the classifications that you modify. For more information, see [Reconfigure an instance group in a running cluster](emr-configure-apps-running-cluster.md). **emr-6.7.0 classifications** | Classifications | Description | Reconfiguration Actions | | --- | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | Restarts the ResourceManager service. | | container-executor | Change values in Hadoop YARN's container-executor.cfg file. | Not available. | | container-log4j | Change values in Hadoop YARN's container-log4j.properties file. | Not available. | | core-site | Change values in Hadoop's core-site.xml file. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts Hadoop KMS, Ranger KMS, HiveServer2, Hive MetaStore, Hadoop Httpfs, and MapReduce-HistoryServer. | | docker-conf | Change docker related settings. | Not available. | | emrfs-site | Change EMRFS settings. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts HBaseRegionserver, HBaseMaster, HBaseThrift, HBaseRest, HiveServer2, Hive MetaStore, Hadoop Httpfs, and MapReduce-HistoryServer. | | flink-conf | Change flink-conf.yaml settings. | Restarts Flink history server. | | flink-log4j | Change Flink log4j.properties settings. | Restarts Flink history server. | | flink-log4j-session | Change Flink log4j-session.properties settings for Kubernetes/Yarn session. | Restarts Flink history server. | | flink-log4j-cli | Change Flink log4j-cli.properties settings. | Restarts Flink history server. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts PhoenixQueryserver, HiveServer2, Hive MetaStore, and MapReduce-HistoryServer. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | Restarts the Hadoop HDFS services SecondaryNamenode, Datanode, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts Hadoop KMS, Hadoop Httpfs, and MapReduce-HistoryServer. | | hadoop-ssl-server | Change hadoop ssl server configuration | Not available. | | hadoop-ssl-client | Change hadoop ssl client configuration | Not available. | | hbase | Amazon EMR-curated settings for Apache HBase. | Custom EMR specific property. Sets emrfs-site and hbase-site configs. See those for their associated restarts. | | hbase-env | Change values in HBase's environment. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbase.properties file. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | Not available. | | hbase-site | Change values in HBase's hbase-site.xml file. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. Additionally restarts Phoenix QueryServer. | | hdfs-encryption-zones | Configure HDFS encryption zones. | This classification should not be reconfigured. | | hdfs-env | Change values in the HDFS environment. | Restarts Hadoop HDFS services Namenode, Datanode, and ZKFC. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Additionally restarts Hadoop Httpfs. | | hcatalog-env | Change values in HCatalog's environment. | Restarts Hive HCatalog Server. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | Restarts Hive HCatalog Server. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | Restarts Hive HCatalog Server. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | Restarts Hive WebHCat server. | | hcatalog-webhcat-log4j2 | Change values in HCatalog WebHCat's log4j2.properties. | Restarts Hive WebHCat server. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | Restarts Hive WebHCat server. | | hive | Amazon EMR-curated settings for Apache Hive. | Sets configurations to launch Hive LLAP service. | | hive-beeline-log4j2 | Change values in Hive's beeline-log4j2.properties file. | Not available. | | hive-parquet-logging | Change values in Hive's parquet-logging.properties file. | Not available. | | hive-env | Change values in the Hive environment. | Restarts HiveServer2, HiveMetastore, and Hive HCatalog-Server. Runs Hive schemaTool CLI commands to verify hive-metastore. | | hive-exec-log4j2 | Change values in Hive's hive-exec-log4j2.properties file. | Not available. | | hive-llap-daemon-log4j2 | Change values in Hive's llap-daemon-log4j2.properties file. | Not available. | | hive-log4j2 | Change values in Hive's hive-log4j2.properties file. | Not available. | | hive-site | Change values in Hive's hive-site.xml file | Restarts HiveServer2, HiveMetastore, and Hive HCatalog-Server. Runs Hive schemaTool CLI commands to verify hive-metastore. Also restarts Oozie and Zeppelin. | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | Not available. | | hue-ini | Change values in Hue's ini file | Restarts Hue. Also activates Hue config override CLI commands to pick up new configurations. | | httpfs-env | Change values in the HTTPFS environment. | Restarts Hadoop Httpfs service. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | Restarts Hadoop Httpfs service. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | Not available. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | Restarts Hadoop-KMS service. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | Not available. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | Restarts Hadoop-KMS and Ranger-KMS service. | | hudi-env | Change values in the Hudi environment. | Not available. | | hudi-defaults | Change values in Hudi's hudi-defaults.conf file. | Not available. | | iceberg-defaults | Change values in Iceberg's iceberg-defaults.conf file. | Not available. | | jupyter-notebook-conf | Change values in Jupyter Notebook's jupyter\$1notebook\$1config.py file. | Not available. | | jupyter-hub-conf | Change values in JupyterHubs's jupyterhub\$1config.py file. | Not available. | | jupyter-s3-conf | Configure Jupyter Notebook S3 persistence. | Not available. | | jupyter-sparkmagic-conf | Change values in Sparkmagic's config.json file. | Not available. | | livy-conf | Change values in Livy's livy.conf file. | Restarts Livy Server. | | livy-env | Change values in the Livy environment. | Restarts Livy Server. | | livy-log4j | Change Livy log4j.properties settings. | Restarts Livy Server. | | mapred-env | Change values in the MapReduce application's environment. | Restarts Hadoop MapReduce-HistoryServer. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | Restarts Hadoop MapReduce-HistoryServer. | | oozie-env | Change values in Oozie's environment. | Restarts Oozie. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | Restarts Oozie. | | oozie-site | Change values in Oozie's oozie-site.xml file. | Restarts Oozie. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | Not available. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | Not available. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | Restarts Phoenix-QueryServer. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | Not available. | | pig-env | Change values in the Pig environment. | Not available. | | pig-properties | Change values in Pig's pig.properties file. | Restarts Oozie. | | pig-log4j | Change values in Pig's log4j.properties file. | Not available. | | presto-log | Change values in Presto's log.properties file. | Restarts Presto-Server (for PrestoDB) | | presto-config | Change values in Presto's config.properties file. | Restarts Presto-Server (for PrestoDB) | | presto-password-authenticator | Change values in Presto's password-authenticator.properties file. | Not available. | | presto-env | Change values in Presto's presto-env.sh file. | Restarts Presto-Server (for PrestoDB) | | presto-node | Change values in Presto's node.properties file. | Not available. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | Not available. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | Not available. | | presto-connector-hive | Change values in Presto's hive.properties file. | Restarts Presto-Server (for PrestoDB) | | presto-connector-jmx | Change values in Presto's jmx.properties file. | Not available. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | Not available. | | presto-connector-localfile | Change values in Presto's localfile.properties file. | Not available. | | presto-connector-memory | Change values in Presto's memory.properties file. | Not available. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | Not available. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | Not available. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | Not available. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | Not available. | | presto-connector-redis | Change values in Presto's redis.properties file. | Not available. | | presto-connector-redshift | Change values in Presto's redshift.properties file. | Not available. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | Not available. | | presto-connector-tpcds | Change values in Presto's tpcds.properties file. | Not available. | | trino-log | Change values in Trino's log.properties file. | Restarts Trino-Server (for Trino) | | trino-config | Change values in Trino's config.properties file. | Restarts Trino-Server (for Trino) | | trino-password-authenticator | Change values in Trino's password-authenticator.properties file. | Restarts Trino-Server (for Trino) | | trino-env | Change values in Trino's trino-env.sh file. | Restarts Trino-Server (for Trino) | | trino-node | Change values in Trino's node.properties file. | Not available. | | trino-connector-blackhole | Change values in Trino's blackhole.properties file. | Not available. | | trino-connector-cassandra | Change values in Trino's cassandra.properties file. | Not available. | | trino-connector-hive | Change values in Trino's hive.properties file. | Restarts Trino-Server (for Trino) | | trino-connector-iceberg | Change values in Trino's iceberg.properties file. | Restarts Trino-Server (for Trino) | | trino-connector-jmx | Change values in Trino's jmx.properties file. | Not available. | | trino-connector-kafka | Change values in Trino's kafka.properties file. | Not available. | | trino-connector-localfile | Change values in Trino's localfile.properties file. | Not available. | | trino-connector-memory | Change values in Trino's memory.properties file. | Not available. | | trino-connector-mongodb | Change values in Trino's mongodb.properties file. | Not available. | | trino-connector-mysql | Change values in Trino's mysql.properties file. | Not available. | | trino-connector-postgresql | Change values in Trino's postgresql.properties file. | Not available. | | trino-connector-raptor | Change values in Trino's raptor.properties file. | Not available. | | trino-connector-redis | Change values in Trino's redis.properties file. | Not available. | | trino-connector-redshift | Change values in Trino's redshift.properties file. | Not available. | | trino-connector-tpch | Change values in Trino's tpch.properties file. | Not available. | | trino-connector-tpcds | Change values in Trino's tpcds.properties file. | Not available. | | ranger-kms-dbks-site | Change values in dbks-site.xml file of Ranger KMS. | Restarts Ranger KMS Server. | | ranger-kms-site | Change values in ranger-kms-site.xml file of Ranger KMS. | Restarts Ranger KMS Server. | | ranger-kms-env | Change values in the Ranger KMS environment. | Restarts Ranger KMS Server. | | ranger-kms-log4j | Change values in kms-log4j.properties file of Ranger KMS. | Not available. | | ranger-kms-db-ca | Change values for CA file on S3 for MySQL SSL connection with Ranger KMS. | Not available. | | spark | Amazon EMR-curated settings for Apache Spark. | This property modifies spark-defaults. See actions there. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | Restarts Spark history server and Spark thrift server. | | spark-env | Change values in the Spark environment. | Restarts Spark history server and Spark thrift server. | | spark-hive-site | Change values in Spark's hive-site.xml file | Not available. | | spark-log4j | Change values in Spark's log4j.properties file. | Restarts Spark history server and Spark thrift server. | | spark-metrics | Change values in Spark's metrics.properties file. | Restarts Spark history server and Spark thrift server. | | sqoop-env | Change values in Sqoop's environment. | Not available. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | Not available. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | Not available. | | tez-site | Change values in Tez's tez-site.xml file. | Restart Oozie and HiveServer2. | | yarn-env | Change values in the YARN environment. | Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts MapReduce-HistoryServer. | | yarn-site | Change values in YARN's yarn-site.xml file. | Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts Livy Server and MapReduce-HistoryServer. | | zeppelin-env | Change values in the Zeppelin environment. | Restarts Zeppelin. | | zeppelin-site | Change configuration settings in zeppelin-site.xml. | Restarts Zeppelin. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | Restarts Zookeeper server. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | Restarts Zookeeper server. | # Amazon EMR release 6.6.0 ## 6.6.0 application versions This release includes the following applications: [https://flink.apache.org/](https://flink.apache.org/), [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [https://hudi.apache.org](https://hudi.apache.org), [http://gethue.com/](http://gethue.com/), [https://iceberg.apache.org/](https://iceberg.apache.org/), [https://jupyter-enterprise-gateway.readthedocs.io/en/latest/](https://jupyter-enterprise-gateway.readthedocs.io/en/latest/), [https://jupyterhub.readthedocs.io/en/latest/#](https://jupyterhub.readthedocs.io/en/latest/#), [https://livy.incubator.apache.org/](https://livy.incubator.apache.org/), [https://mxnet.incubator.apache.org/](https://mxnet.incubator.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://www.tensorflow.org/](https://www.tensorflow.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://trino.io/](https://trino.io/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-6.6.0 | emr-6.5.0 | emr-6.4.0 | emr-6.3.1 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.12.170 | 1.12.31 | 1.12.31 | 1.11.977 | | Python | 2.7, 3.7 | 2.7, 3.7 | 2.7, 3.7 | 2.7, 3.7 | | Scala | 2.12.10 | 2.12.10 | 2.12.10 | 2.12.10 | | AmazonCloudWatchAgent | - | - | - | - | | Delta | - | - | - | - | | Flink | 1.14.2 | 1.14.0 | 1.13.1 | 1.12.1 | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 2.4.4-amzn-2 | 2.4.4-amzn-1 | 2.4.4-amzn-0 | 2.2.6-amzn-1 | | HCatalog | 3.1.2-amzn-7 | 3.1.2-amzn-6 | 3.1.2-amzn-5 | 3.1.2-amzn-4 | | Hadoop | 3.2.1-amzn-6 | 3.2.1-amzn-5 | 3.2.1-amzn-4 | 3.2.1-amzn-3.1 | | Hive | 3.1.2-amzn-7 | 3.1.2-amzn-6 | 3.1.2-amzn-5 | 3.1.2-amzn-4 | | Hudi | 0.10.1-amzn-0 | 0.9.0-amzn-1 | 0.8.0-amzn-0 | 0.7.0-amzn-0 | | Hue | 4.10.0 | 4.9.0 | 4.9.0 | 4.9.0 | | Iceberg | 0.13.1 | 0.12.0 | - | - | | JupyterEnterpriseGateway | 2.1.0 | 2.1.0 | 2.1.0 | 2.1.0 | | JupyterHub | 1.4.1 | 1.4.1 | 1.4.1 | 1.2.2 | | Livy | 0.7.1-incubating | 0.7.1-incubating | 0.7.1-incubating | 0.7.0-incubating | | MXNet | 1.8.0 | 1.8.0 | 1.8.0 | 1.7.0 | | Mahout | - | - | - | - | | Oozie | 5.2.1 | 5.2.1 | 5.2.1 | 5.2.1 | | Phoenix | 5.1.2 | 5.1.2 | 5.1.2 | 5.0.0-HBase-2.0 | | Pig | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | | Presto | 0.267-amzn-0 | 0.261-amzn-0 | 0.254.1-amzn-0 | 0.245.1-amzn-0 | | Spark | 3.2.0-amzn-0 | 3.1.2-amzn-1 | 3.1.2-amzn-0 | 3.1.1-amzn-0.1 | | Sqoop | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | | TensorFlow | 2.4.1 | 2.4.1 | 2.4.1 | 2.4.1 | | Tez | 0.9.2 | 0.9.2 | 0.9.2 | 0.9.2 | | Trino (PrestoSQL) | 367-amzn-0 | 360 | 359 | 350 | | Zeppelin | 0.10.0 | 0.10.0 | 0.9.0 | 0.9.0 | | ZooKeeper | 3.5.7 | 3.5.7 | 3.5.7 | 3.4.14 | ## 6.6.0 release notes The following release notes include information for Amazon EMR release 6.6.0. Changes are relative to 6.5.0. Initial release date: May 9, 2022 Updated documentation date: June 15, 2022 **New Features** + Amazon EMR 6.6 now supports Apache Spark 3.2, Apache Spark RAPIDS 22.02, CUDA 11, Apache Hudi 0.10.1, Apache Iceberg 0.13, Trino 0.367 and PrestoDB 0.267. + When you launch a cluster with *the latest patch release* of Amazon EMR 5.36 or higher, 6.6 or higher, or 7.0 or higher, Amazon EMR uses the latest Amazon Linux 2023 or Amazon Linux 2 release for the default Amazon EMR AMI. For more information, see [Using the default Amazon Linux AMI for Amazon EMR](https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-default-ami.html). [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-660-release.html) + With Amazon EMR 6.6 and later, applications that use Log4j 1.x and Log4j 2.x are upgraded to use Log4j 1.2.17 (or higher) and Log4j 2.17.1 (or higher) respectively, and do not require using the [bootstrap actions](https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-log4j-vulnerability.html) provided to mitigate the CVE issues. + **[Managed scaling] Spark shuffle data managed scaling optimization** - For Amazon EMR versions 5.34.0 and later, and EMR versions 6.4.0 and later, managed scaling is now Spark shuffle data aware (data that Spark redistributes across partitions to perform specific operations). For more information on shuffle operations, see [Using EMR managed scaling in Amazon EMR](https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-managed-scaling.html) in the *Amazon EMR Management Guide* and [Spark Programming Guide](https://spark.apache.org/docs/latest/rdd-programming-guide.html#shuffle-operations). + Starting with Amazon EMR 5.32.0 and 6.5.0, dynamic executor sizing for Apache Spark is enabled by default. To turn this feature on or off, you can use the `spark.yarn.heterogeneousExecutors.enabled` configuration parameter. **Changes, Enhancements, and Resolved Issues** + Amazon EMR reduces cluster startup time by up to 80 seconds on average for clusters that use the EMR default AMI option and only install common applications, such as Apache Hadoop, Apache Spark and Apache Hive. **Known Issues** + When Amazon EMR release 6.5.0, 6.6.0, or 6.7.0 read Apache Phoenix tables through the Apache Spark shell, a `NoSuchMethodError` occurs because Amazon EMR uses an incorrect `Hbase.compat.version`. Amazon EMR release 6.8.0 fixes this issue. + When you use the DynamoDB connector with Spark on Amazon EMR versions 6.6.0, 6.7.0, and 6.8.0, all reads from your table return an empty result, even though the input split references non-empty data. This is because Spark 3.2.0 sets `spark.hadoopRDD.ignoreEmptySplits` to `true` by default. As a workaround, explicitly set `spark.hadoopRDD.ignoreEmptySplits` to `false`. Amazon EMR release 6.9.0 fixes this issue. + On Trino long-running clusters, Amazon EMR 6.6.0 enables Garbage Collection logging parameters in the Trino jvm.config to get better insights from the Garbage Collection logs. This change appends many Garbage Collection logs to the launcher.log (/var/log/trino/launcher.log) file. If you are running Trino clusters in Amazon EMR 6.6.0, you may encounter nodes running out of disk space after the cluster has been running for a couple of days due to the appended logs. The workaround for this issue is to run the script below as a Bootstrap Action to disable the Garbage Collection logging parameters in jvm.config while creating or cloning the cluster for Amazon EMR 6.6.0. ``` #!/bin/bash set -ex PRESTO_PUPPET_DIR='/var/aws/emr/bigtop-deploy/puppet/modules/trino' sudo bash -c "sed -i '/-Xlog/d' ${PRESTO_PUPPET_DIR}/templates/jvm.config" ``` + When you use Spark with Hive partition location formatting to read data in Amazon S3, and you run Spark on Amazon EMR releases 5.30.0 to 5.36.0, and 6.2.0 to 6.9.0, you might encounter an issue that prevents your cluster from reading data correctly. This can happen if your partitions have all of the following characteristics: + Two or more partitions are scanned from the same table. + At least one partition directory path is a prefix of at least one other partition directory path, for example, `s3://bucket/table/p=a` is a prefix of `s3://bucket/table/p=a b`. + The first character that follows the prefix in the other partition directory has a UTF-8 value that’s less than than the `/` character (U\$1002F). For example, the space character (U\$10020) that occurs between a and b in `s3://bucket/table/p=a b` falls into this category. Note that there are 14 other non-control characters: `!"#$%&‘()*+,-`. For more information, see [UTF-8 encoding table and Unicode characters](https://www.utf8-chartable.de/). As a workaround to this issue, set the `spark.sql.sources.fastS3PartitionDiscovery.enabled` configuration to `false` in the `spark-defaults` classification. + With Amazon EMR releases 5.36.0 and 6.6.0 through 6.9.0, `SecretAgent` and `RecordServer` service components may experience log data loss due to an incorrect file name pattern configuration in Log4j2 properties. The incorrect configuration causes the components to generate only one log file per day. When the rotation strategy occurs, it overwrites the existing file instead of generating a new log file as expected. As a workaround, use a bootstrap action to generate log files each hour and append an auto-increment integer in the file name to handle the rotation. For Amazon EMR 6.6.0 through 6.9.0 releases, use the following bootstrap action when you launch a cluster. ``` ‑‑bootstrap‑actions "Path=s3://emr-data-access-control-us-east-1/customer-bootstrap-actions/log-rotation-emr-6x/replace-puppet.sh,Args=[]" ``` For Amazon EMR 5.36.0, use the following bootstrap action when you launch a cluster. ``` ‑‑bootstrap‑actions "Path=s3://emr-data-access-control-us-east-1/customer-bootstrap-actions/log-rotation-emr-5x/replace-puppet.sh,Args=[]" ``` ## 6.6.0 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | aws-sagemaker-spark-sdk | 1.4.1 | Amazon SageMaker Spark SDK | | emr-ddb | 4.16.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 3.2.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.5.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-notebook-env | 1.5.0 | Conda env for emr notebook which includes jupyter enterprise gateway | | emr-s3-dist-cp | 2.20.0 | Distributed copy application optimized for Amazon S3. | | emr-s3-select | 2.1.0 | EMR S3Select Connector | | emrfs | 2.50.0 | Amazon S3 connector for Hadoop ecosystem applications. | | flink-client | 1.14.2 | Apache Flink command line client scripts and applications. | | flink-jobmanager-config | 1.14.2 | Managing resources on EMR nodes for Apache Flink JobManager. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 3.2.1-amzn-6 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 3.2.1-amzn-6 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 3.2.1-amzn-6 | HDFS command-line client and library | | hadoop-hdfs-namenode | 3.2.1-amzn-6 | HDFS service for tracking file names and block locations. | | hadoop-hdfs-journalnode | 3.2.1-amzn-6 | HDFS service for managing the Hadoop filesystem journal on HA clusters. | | hadoop-httpfs-server | 3.2.1-amzn-6 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 3.2.1-amzn-6 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 3.2.1-amzn-6 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 3.2.1-amzn-6 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 3.2.1-amzn-6 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 3.2.1-amzn-6 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 2.4.4-amzn-2 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 2.4.4-amzn-2 | Service for serving one or more HBase regions. | | hbase-client | 2.4.4-amzn-2 | HBase command-line client. | | hbase-rest-server | 2.4.4-amzn-2 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 2.4.4-amzn-2 | Service providing a Thrift endpoint to HBase. | | hbase-operator-tools | 2.4.4-amzn-2 | Repair tool for Apache HBase clusters. | | hcatalog-client | 3.1.2-amzn-7 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 3.1.2-amzn-7 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 3.1.2-amzn-7 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 3.1.2-amzn-7 | Hive command line client. | | hive-hbase | 3.1.2-amzn-7 | Hive-hbase client. | | hive-metastore-server | 3.1.2-amzn-7 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server2 | 3.1.2-amzn-7 | Service for accepting Hive queries as web requests. | | hudi | 0.10.1-amzn-0 | Incremental processing framework to power data pipeline at low latency and high efficiency. | | hudi-presto | 0.10.1-amzn-0 | Bundle library for running Presto with Hudi. | | hudi-trino | 0.10.1-amzn-0 | Bundle library for running Trino with Hudi. | | hudi-spark | 0.10.1-amzn-0 | Bundle library for running Spark with Hudi. | | hue-server | 4.10.0 | Web application for analyzing data using Hadoop ecosystem applications | | iceberg | 0.13.1 | Apache Iceberg is an open table format for huge analytic datasets | | jupyterhub | 1.4.1 | Multi-user server for Jupyter notebooks | | livy-server | 0.7.1-incubating | REST interface for interacting with Apache Spark | | nginx | 1.12.1 | nginx [engine x] is an HTTP and reverse proxy server | | mxnet | 1.8.0 | A flexible, scalable, and efficient library for deep learning. | | mariadb-server | 5.5.68\$1 | MariaDB database server. | | nvidia-cuda | 11.0.194 | Nvidia drivers and Cuda toolkit | | oozie-client | 5.2.1 | Oozie command-line client. | | oozie-server | 5.2.1 | Service for accepting Oozie workflow requests. | | opencv | 4.5.0 | Open Source Computer Vision Library. | | phoenix-library | 5.1.2 | The phoenix libraries for server and client | | phoenix-connectors | 5.1.2 | Apache Phoenix-Connectors for Spark-3 | | phoenix-query-server | 5.1.2 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.267-amzn-0 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.267-amzn-0 | Service for executing pieces of a query. | | presto-client | 0.267-amzn-0 | Presto command-line client which is installed on an HA cluster's stand-by masters where Presto server is not started. | | trino-coordinator | 367-amzn-0 | Service for accepting queries and managing query execution among trino-workers. | | trino-worker | 367-amzn-0 | Service for executing pieces of a query. | | trino-client | 367-amzn-0 | Trino command-line client which is installed on an HA cluster's stand-by masters where Trino server is not started. | | pig-client | 0.17.0 | Pig command-line client. | | r | 4.0.2 | The R Project for Statistical Computing | | ranger-kms-server | 2.0.0 | Apache Ranger Key Management System | | spark-client | 3.2.0-amzn-0 | Spark command-line clients. | | spark-history-server | 3.2.0-amzn-0 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 3.2.0-amzn-0 | In-memory execution engine for YARN. | | spark-yarn-slave | 3.2.0-amzn-0 | Apache Spark libraries needed by YARN slaves. | | spark-rapids | 22.02.0-amzn-0 | Nvidia Spark RAPIDS plugin that accelerates Apache Spark with GPUs. | | sqoop-client | 1.4.7 | Apache Sqoop command-line client. | | tensorflow | 2.4.1 | TensorFlow open source software library for high performance numerical computation. | | tez-on-yarn | 0.9.2 | The tez YARN application and libraries. | | webserver | 2.4.41\$1 | Apache HTTP server. | | zeppelin-server | 0.10.0 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.5.7 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.5.7 | ZooKeeper command line client. | ## 6.6.0 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). Reconfiguration actions occur when you specify a configuration for instance groups in a running cluster. Amazon EMR only initiates reconfiguration actions for the classifications that you modify. For more information, see [Reconfigure an instance group in a running cluster](emr-configure-apps-running-cluster.md). **emr-6.6.0 classifications** | Classifications | Description | Reconfiguration Actions | | --- | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | Restarts the ResourceManager service. | | container-executor | Change values in Hadoop YARN's container-executor.cfg file. | Not available. | | container-log4j | Change values in Hadoop YARN's container-log4j.properties file. | Not available. | | core-site | Change values in Hadoop's core-site.xml file. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts Hadoop KMS, Ranger KMS, HiveServer2, Hive MetaStore, Hadoop Httpfs, and MapReduce-HistoryServer. | | docker-conf | Change docker related settings. | Not available. | | emrfs-site | Change EMRFS settings. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts HBaseRegionserver, HBaseMaster, HBaseThrift, HBaseRest, HiveServer2, Hive MetaStore, Hadoop Httpfs, and MapReduce-HistoryServer. | | flink-conf | Change flink-conf.yaml settings. | Restarts Flink history server. | | flink-log4j | Change Flink log4j.properties settings. | Restarts Flink history server. | | flink-log4j-session | Change Flink log4j-session.properties settings for Kubernetes/Yarn session. | Restarts Flink history server. | | flink-log4j-cli | Change Flink log4j-cli.properties settings. | Restarts Flink history server. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts PhoenixQueryserver, HiveServer2, Hive MetaStore, and MapReduce-HistoryServer. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | Restarts the Hadoop HDFS services SecondaryNamenode, Datanode, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts Hadoop KMS, Hadoop Httpfs, and MapReduce-HistoryServer. | | hadoop-ssl-server | Change hadoop ssl server configuration | Not available. | | hadoop-ssl-client | Change hadoop ssl client configuration | Not available. | | hbase | Amazon EMR-curated settings for Apache HBase. | Custom EMR specific property. Sets emrfs-site and hbase-site configs. See those for their associated restarts. | | hbase-env | Change values in HBase's environment. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbase.properties file. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | Not available. | | hbase-site | Change values in HBase's hbase-site.xml file. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. Additionally restarts Phoenix QueryServer. | | hdfs-encryption-zones | Configure HDFS encryption zones. | This classification should not be reconfigured. | | hdfs-env | Change values in the HDFS environment. | Restarts Hadoop HDFS services Namenode, Datanode, and ZKFC. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Additionally restarts Hadoop Httpfs. | | hcatalog-env | Change values in HCatalog's environment. | Restarts Hive HCatalog Server. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | Restarts Hive HCatalog Server. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | Restarts Hive HCatalog Server. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | Restarts Hive WebHCat server. | | hcatalog-webhcat-log4j2 | Change values in HCatalog WebHCat's log4j2.properties. | Restarts Hive WebHCat server. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | Restarts Hive WebHCat server. | | hive | Amazon EMR-curated settings for Apache Hive. | Sets configurations to launch Hive LLAP service. | | hive-beeline-log4j2 | Change values in Hive's beeline-log4j2.properties file. | Not available. | | hive-parquet-logging | Change values in Hive's parquet-logging.properties file. | Not available. | | hive-env | Change values in the Hive environment. | Restarts HiveServer2, HiveMetastore, and Hive HCatalog-Server. Runs Hive schemaTool CLI commands to verify hive-metastore. | | hive-exec-log4j2 | Change values in Hive's hive-exec-log4j2.properties file. | Not available. | | hive-llap-daemon-log4j2 | Change values in Hive's llap-daemon-log4j2.properties file. | Not available. | | hive-log4j2 | Change values in Hive's hive-log4j2.properties file. | Not available. | | hive-site | Change values in Hive's hive-site.xml file | Restarts HiveServer2, HiveMetastore, and Hive HCatalog-Server. Runs Hive schemaTool CLI commands to verify hive-metastore. Also restarts Oozie and Zeppelin. | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | Not available. | | hue-ini | Change values in Hue's ini file | Restarts Hue. Also activates Hue config override CLI commands to pick up new configurations. | | httpfs-env | Change values in the HTTPFS environment. | Restarts Hadoop Httpfs service. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | Restarts Hadoop Httpfs service. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | Not available. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | Restarts Hadoop-KMS service. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | Not available. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | Restarts Hadoop-KMS and Ranger-KMS service. | | hudi-env | Change values in the Hudi environment. | Not available. | | hudi-defaults | Change values in Hudi's hudi-defaults.conf file. | Not available. | | iceberg-defaults | Change values in Iceberg's iceberg-defaults.conf file. | Not available. | | jupyter-notebook-conf | Change values in Jupyter Notebook's jupyter\$1notebook\$1config.py file. | Not available. | | jupyter-hub-conf | Change values in JupyterHubs's jupyterhub\$1config.py file. | Not available. | | jupyter-s3-conf | Configure Jupyter Notebook S3 persistence. | Not available. | | jupyter-sparkmagic-conf | Change values in Sparkmagic's config.json file. | Not available. | | livy-conf | Change values in Livy's livy.conf file. | Restarts Livy Server. | | livy-env | Change values in the Livy environment. | Restarts Livy Server. | | livy-log4j | Change Livy log4j.properties settings. | Restarts Livy Server. | | mapred-env | Change values in the MapReduce application's environment. | Restarts Hadoop MapReduce-HistoryServer. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | Restarts Hadoop MapReduce-HistoryServer. | | oozie-env | Change values in Oozie's environment. | Restarts Oozie. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | Restarts Oozie. | | oozie-site | Change values in Oozie's oozie-site.xml file. | Restarts Oozie. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | Not available. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | Not available. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | Restarts Phoenix-QueryServer. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | Not available. | | pig-env | Change values in the Pig environment. | Not available. | | pig-properties | Change values in Pig's pig.properties file. | Restarts Oozie. | | pig-log4j | Change values in Pig's log4j.properties file. | Not available. | | presto-log | Change values in Presto's log.properties file. | Restarts Presto-Server (for PrestoDB) | | presto-config | Change values in Presto's config.properties file. | Restarts Presto-Server (for PrestoDB) | | presto-password-authenticator | Change values in Presto's password-authenticator.properties file. | Not available. | | presto-env | Change values in Presto's presto-env.sh file. | Restarts Presto-Server (for PrestoDB) | | presto-node | Change values in Presto's node.properties file. | Not available. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | Not available. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | Not available. | | presto-connector-hive | Change values in Presto's hive.properties file. | Restarts Presto-Server (for PrestoDB) | | presto-connector-jmx | Change values in Presto's jmx.properties file. | Not available. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | Not available. | | presto-connector-localfile | Change values in Presto's localfile.properties file. | Not available. | | presto-connector-memory | Change values in Presto's memory.properties file. | Not available. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | Not available. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | Not available. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | Not available. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | Not available. | | presto-connector-redis | Change values in Presto's redis.properties file. | Not available. | | presto-connector-redshift | Change values in Presto's redshift.properties file. | Not available. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | Not available. | | presto-connector-tpcds | Change values in Presto's tpcds.properties file. | Not available. | | trino-log | Change values in Trino's log.properties file. | Restarts Trino-Server (for Trino) | | trino-config | Change values in Trino's config.properties file. | Restarts Trino-Server (for Trino) | | trino-password-authenticator | Change values in Trino's password-authenticator.properties file. | Restarts Trino-Server (for Trino) | | trino-env | Change values in Trino's trino-env.sh file. | Restarts Trino-Server (for Trino) | | trino-node | Change values in Trino's node.properties file. | Not available. | | trino-connector-blackhole | Change values in Trino's blackhole.properties file. | Not available. | | trino-connector-cassandra | Change values in Trino's cassandra.properties file. | Not available. | | trino-connector-hive | Change values in Trino's hive.properties file. | Restarts Trino-Server (for Trino) | | trino-connector-iceberg | Change values in Trino's iceberg.properties file. | Restarts Trino-Server (for Trino) | | trino-connector-jmx | Change values in Trino's jmx.properties file. | Not available. | | trino-connector-kafka | Change values in Trino's kafka.properties file. | Not available. | | trino-connector-localfile | Change values in Trino's localfile.properties file. | Not available. | | trino-connector-memory | Change values in Trino's memory.properties file. | Not available. | | trino-connector-mongodb | Change values in Trino's mongodb.properties file. | Not available. | | trino-connector-mysql | Change values in Trino's mysql.properties file. | Not available. | | trino-connector-postgresql | Change values in Trino's postgresql.properties file. | Not available. | | trino-connector-raptor | Change values in Trino's raptor.properties file. | Not available. | | trino-connector-redis | Change values in Trino's redis.properties file. | Not available. | | trino-connector-redshift | Change values in Trino's redshift.properties file. | Not available. | | trino-connector-tpch | Change values in Trino's tpch.properties file. | Not available. | | trino-connector-tpcds | Change values in Trino's tpcds.properties file. | Not available. | | ranger-kms-dbks-site | Change values in dbks-site.xml file of Ranger KMS. | Restarts Ranger KMS Server. | | ranger-kms-site | Change values in ranger-kms-site.xml file of Ranger KMS. | Restarts Ranger KMS Server. | | ranger-kms-env | Change values in the Ranger KMS environment. | Restarts Ranger KMS Server. | | ranger-kms-log4j | Change values in kms-log4j.properties file of Ranger KMS. | Not available. | | ranger-kms-db-ca | Change values for CA file on S3 for MySQL SSL connection with Ranger KMS. | Not available. | | spark | Amazon EMR-curated settings for Apache Spark. | This property modifies spark-defaults. See actions there. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | Restarts Spark history server and Spark thrift server. | | spark-env | Change values in the Spark environment. | Restarts Spark history server and Spark thrift server. | | spark-hive-site | Change values in Spark's hive-site.xml file | Not available. | | spark-log4j | Change values in Spark's log4j.properties file. | Restarts Spark history server and Spark thrift server. | | spark-metrics | Change values in Spark's metrics.properties file. | Restarts Spark history server and Spark thrift server. | | sqoop-env | Change values in Sqoop's environment. | Not available. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | Not available. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | Not available. | | tez-site | Change values in Tez's tez-site.xml file. | Restart Oozie and HiveServer2. | | yarn-env | Change values in the YARN environment. | Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts MapReduce-HistoryServer. | | yarn-site | Change values in YARN's yarn-site.xml file. | Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts Livy Server and MapReduce-HistoryServer. | | zeppelin-env | Change values in the Zeppelin environment. | Restarts Zeppelin. | | zeppelin-site | Change configuration settings in zeppelin-site.xml. | Restarts Zeppelin. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | Restarts Zookeeper server. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | Restarts Zookeeper server. | # Amazon EMR release 6.5.0 ## 6.5.0 application versions This release includes the following applications: [https://flink.apache.org/](https://flink.apache.org/), [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [https://hudi.apache.org](https://hudi.apache.org), [http://gethue.com/](http://gethue.com/), [https://iceberg.apache.org/](https://iceberg.apache.org/), [https://jupyter-enterprise-gateway.readthedocs.io/en/latest/](https://jupyter-enterprise-gateway.readthedocs.io/en/latest/), [https://jupyterhub.readthedocs.io/en/latest/#](https://jupyterhub.readthedocs.io/en/latest/#), [https://livy.incubator.apache.org/](https://livy.incubator.apache.org/), [https://mxnet.incubator.apache.org/](https://mxnet.incubator.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://www.tensorflow.org/](https://www.tensorflow.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://trino.io/](https://trino.io/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-6.5.0 | emr-6.4.0 | emr-6.3.1 | emr-6.3.0 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.12.31 | 1.12.31 | 1.11.977 | 1.11.977 | | Python | 2.7, 3.7 | 2.7, 3.7 | 2.7, 3.7 | 2.7, 3.7 | | Scala | 2.12.10 | 2.12.10 | 2.12.10 | 2.12.10 | | AmazonCloudWatchAgent | - | - | - | - | | Delta | - | - | - | - | | Flink | 1.14.0 | 1.13.1 | 1.12.1 | 1.12.1 | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 2.4.4-amzn-1 | 2.4.4-amzn-0 | 2.2.6-amzn-1 | 2.2.6-amzn-1 | | HCatalog | 3.1.2-amzn-6 | 3.1.2-amzn-5 | 3.1.2-amzn-4 | 3.1.2-amzn-4 | | Hadoop | 3.2.1-amzn-5 | 3.2.1-amzn-4 | 3.2.1-amzn-3.1 | 3.2.1-amzn-3 | | Hive | 3.1.2-amzn-6 | 3.1.2-amzn-5 | 3.1.2-amzn-4 | 3.1.2-amzn-4 | | Hudi | 0.9.0-amzn-1 | 0.8.0-amzn-0 | 0.7.0-amzn-0 | 0.7.0-amzn-0 | | Hue | 4.9.0 | 4.9.0 | 4.9.0 | 4.9.0 | | Iceberg | 0.12.0 | - | - | - | | JupyterEnterpriseGateway | 2.1.0 | 2.1.0 | 2.1.0 | 2.1.0 | | JupyterHub | 1.4.1 | 1.4.1 | 1.2.2 | 1.2.2 | | Livy | 0.7.1-incubating | 0.7.1-incubating | 0.7.0-incubating | 0.7.0-incubating | | MXNet | 1.8.0 | 1.8.0 | 1.7.0 | 1.7.0 | | Mahout | - | - | - | - | | Oozie | 5.2.1 | 5.2.1 | 5.2.1 | 5.2.1 | | Phoenix | 5.1.2 | 5.1.2 | 5.0.0-HBase-2.0 | 5.0.0-HBase-2.0 | | Pig | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | | Presto | 0.261-amzn-0 | 0.254.1-amzn-0 | 0.245.1-amzn-0 | 0.245.1-amzn-0 | | Spark | 3.1.2-amzn-1 | 3.1.2-amzn-0 | 3.1.1-amzn-0.1 | 3.1.1-amzn-0 | | Sqoop | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | | TensorFlow | 2.4.1 | 2.4.1 | 2.4.1 | 2.4.1 | | Tez | 0.9.2 | 0.9.2 | 0.9.2 | 0.9.2 | | Trino (PrestoSQL) | 360 | 359 | 350 | 350 | | Zeppelin | 0.10.0 | 0.9.0 | 0.9.0 | 0.9.0 | | ZooKeeper | 3.5.7 | 3.5.7 | 3.4.14 | 3.4.14 | ## 6.5.0 release notes The following release notes include information for Amazon EMR release 6.5.0. Changes are relative to 6.4.0. Initial release date: January 20, 2022 Updated release date: March 21, 2022 **New Features** + **[Managed scaling] Spark shuffle data managed scaling optimization** - For Amazon EMR versions 5.34.0 and later, and EMR versions 6.4.0 and later, managed scaling is now Spark shuffle data aware (data that Spark redistributes across partitions to perform specific operations). For more information on shuffle operations, see [Using EMR managed scaling in Amazon EMR](https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-managed-scaling.html) in the *Amazon EMR Management Guide* and [Spark Programming Guide](https://spark.apache.org/docs/latest/rdd-programming-guide.html#shuffle-operations). + Starting with Amazon EMR 5.32.0 and 6.5.0, dynamic executor sizing for Apache Spark is enabled by default. To turn this feature on or off, you can use the `spark.yarn.heterogeneousExecutors.enabled` configuration parameter. + Support for Apache Iceberg open table format for huge analytic datasets. + Support for ranger-trino-plugin 2.0.1-amzn-1 + Support for toree 0.5.0 **Changes, Enhancements, and Resolved Issues** + Amazon EMR 6.5 release version now supports Apache Iceberg 0.12.0, and provides runtime improvements with Amazon EMR Runtime for Apache Spark, Amazon EMR Runtime for Presto, and Amazon EMR Runtime for Apache Hive. + [Apache Iceberg](https://iceberg.apache.org/) is an open table format for large data sets in Amazon S3 and provides fast query performance over large tables, atomic commits, concurrent writes, and SQL-compatible table evolution. With EMR 6.5, you can use Apache Spark 3.1.2 with the Iceberg table format. + Apache Hudi 0.9 adds Spark SQL DDL and DML support. This allows you to create, upsert Hudi tables using just SQL statements. Apache Hudi 0.9 also includes query side and writer side performance improvements. + Amazon EMR Runtime for Apache Hive improves Apache Hive performance on Amazon S3 by removing rename operations during staging operations, and improves performance for metastore check (MSCK) commands used for repairing tables. **Known Issues** + When Amazon EMR release 6.5.0, 6.6.0, or 6.7.0 read Apache Phoenix tables through the Apache Spark shell, a `NoSuchMethodError` occurs because Amazon EMR uses an incorrect `Hbase.compat.version`. Amazon EMR release 6.8.0 fixes this issue. + Hbase bundle clusters in high availability (HA) fail to provision with the default volume size and instance type. The workaround for this issue is to increase the root volume size. + To use Spark actions with Apache Oozie, you must add the following configuration to your Oozie `workflow.xml` file. Otherwise, several critical libraries such as Hadoop and EMRFS will be missing from the classpath of the Spark executors that Oozie launches. ``` --conf spark.yarn.populateHadoopClasspath=true ``` + When you use Spark with Hive partition location formatting to read data in Amazon S3, and you run Spark on Amazon EMR releases 5.30.0 to 5.36.0, and 6.2.0 to 6.9.0, you might encounter an issue that prevents your cluster from reading data correctly. This can happen if your partitions have all of the following characteristics: + Two or more partitions are scanned from the same table. + At least one partition directory path is a prefix of at least one other partition directory path, for example, `s3://bucket/table/p=a` is a prefix of `s3://bucket/table/p=a b`. + The first character that follows the prefix in the other partition directory has a UTF-8 value that’s less than than the `/` character (U\$1002F). For example, the space character (U\$10020) that occurs between a and b in `s3://bucket/table/p=a b` falls into this category. Note that there are 14 other non-control characters: `!"#$%&‘()*+,-`. For more information, see [UTF-8 encoding table and Unicode characters](https://www.utf8-chartable.de/). As a workaround to this issue, set the `spark.sql.sources.fastS3PartitionDiscovery.enabled` configuration to `false` in the `spark-defaults` classification. ## 6.5.0 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | aws-sagemaker-spark-sdk | 1.4.1 | Amazon SageMaker Spark SDK | | emr-ddb | 4.16.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 3.2.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.5.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-notebook-env | 1.4.0 | Conda env for emr notebook which includes jupyter enterprise gateway | | emr-s3-dist-cp | 2.19.0 | Distributed copy application optimized for Amazon S3. | | emr-s3-select | 2.1.0 | EMR S3Select Connector | | emrfs | 2.48.0 | Amazon S3 connector for Hadoop ecosystem applications. | | flink-client | 1.14.0 | Apache Flink command line client scripts and applications. | | flink-jobmanager-config | 1.14.0 | Managing resources on EMR nodes for Apache Flink JobManager. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 3.2.1-amzn-5 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 3.2.1-amzn-5 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 3.2.1-amzn-5 | HDFS command-line client and library | | hadoop-hdfs-namenode | 3.2.1-amzn-5 | HDFS service for tracking file names and block locations. | | hadoop-hdfs-journalnode | 3.2.1-amzn-5 | HDFS service for managing the Hadoop filesystem journal on HA clusters. | | hadoop-httpfs-server | 3.2.1-amzn-5 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 3.2.1-amzn-5 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 3.2.1-amzn-5 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 3.2.1-amzn-5 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 3.2.1-amzn-5 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 3.2.1-amzn-5 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 2.4.4-amzn-1 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 2.4.4-amzn-1 | Service for serving one or more HBase regions. | | hbase-client | 2.4.4-amzn-1 | HBase command-line client. | | hbase-rest-server | 2.4.4-amzn-1 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 2.4.4-amzn-1 | Service providing a Thrift endpoint to HBase. | | hcatalog-client | 3.1.2-amzn-6 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 3.1.2-amzn-6 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 3.1.2-amzn-6 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 3.1.2-amzn-6 | Hive command line client. | | hive-hbase | 3.1.2-amzn-6 | Hive-hbase client. | | hive-metastore-server | 3.1.2-amzn-6 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server2 | 3.1.2-amzn-6 | Service for accepting Hive queries as web requests. | | hudi | 0.9.0-amzn-1 | Incremental processing framework to power data pipeline at low latency and high efficiency. | | hudi-presto | 0.9.0-amzn-1 | Bundle library for running Presto with Hudi. | | hudi-trino | 0.9.0-amzn-1 | Bundle library for running Trino with Hudi. | | hudi-spark | 0.9.0-amzn-1 | Bundle library for running Spark with Hudi. | | hue-server | 4.9.0 | Web application for analyzing data using Hadoop ecosystem applications | | iceberg | 0.12.0 | Apache Iceberg is an open table format for huge analytic datasets | | jupyterhub | 1.4.1 | Multi-user server for Jupyter notebooks | | livy-server | 0.7.1-incubating | REST interface for interacting with Apache Spark | | nginx | 1.12.1 | nginx [engine x] is an HTTP and reverse proxy server | | mxnet | 1.8.0 | A flexible, scalable, and efficient library for deep learning. | | mariadb-server | 5.5.68\$1 | MariaDB database server. | | nvidia-cuda | 10.1.243 | Nvidia drivers and Cuda toolkit | | oozie-client | 5.2.1 | Oozie command-line client. | | oozie-server | 5.2.1 | Service for accepting Oozie workflow requests. | | opencv | 4.5.0 | Open Source Computer Vision Library. | | phoenix-library | 5.1.2 | The phoenix libraries for server and client | | phoenix-query-server | 5.1.2 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.261-amzn-0 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.261-amzn-0 | Service for executing pieces of a query. | | presto-client | 0.261-amzn-0 | Presto command-line client which is installed on an HA cluster's stand-by masters where Presto server is not started. | | trino-coordinator | 360 | Service for accepting queries and managing query execution among trino-workers. | | trino-worker | 360 | Service for executing pieces of a query. | | trino-client | 360 | Trino command-line client which is installed on an HA cluster's stand-by masters where Trino server is not started. | | pig-client | 0.17.0 | Pig command-line client. | | r | 4.0.2 | The R Project for Statistical Computing | | ranger-kms-server | 2.0.0 | Apache Ranger Key Management System | | spark-client | 3.1.2-amzn-1 | Spark command-line clients. | | spark-history-server | 3.1.2-amzn-1 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 3.1.2-amzn-1 | In-memory execution engine for YARN. | | spark-yarn-slave | 3.1.2-amzn-1 | Apache Spark libraries needed by YARN slaves. | | spark-rapids | 0.4.1 | Nvidia Spark RAPIDS plugin that accelerates Apache Spark with GPUs. | | sqoop-client | 1.4.7 | Apache Sqoop command-line client. | | tensorflow | 2.4.1 | TensorFlow open source software library for high performance numerical computation. | | tez-on-yarn | 0.9.2 | The tez YARN application and libraries. | | webserver | 2.4.41\$1 | Apache HTTP server. | | zeppelin-server | 0.10.0 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.5.7 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.5.7 | ZooKeeper command line client. | ## 6.5.0 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). Reconfiguration actions occur when you specify a configuration for instance groups in a running cluster. Amazon EMR only initiates reconfiguration actions for the classifications that you modify. For more information, see [Reconfigure an instance group in a running cluster](emr-configure-apps-running-cluster.md). **emr-6.5.0 classifications** | Classifications | Description | Reconfiguration Actions | | --- | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | Restarts the ResourceManager service. | | container-executor | Change values in Hadoop YARN's container-executor.cfg file. | Not available. | | container-log4j | Change values in Hadoop YARN's container-log4j.properties file. | Not available. | | core-site | Change values in Hadoop's core-site.xml file. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts Hadoop KMS, Ranger KMS, HiveServer2, Hive MetaStore, Hadoop Httpfs, and MapReduce-HistoryServer. | | docker-conf | Change docker related settings. | Not available. | | emrfs-site | Change EMRFS settings. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts HBaseRegionserver, HBaseMaster, HBaseThrift, HBaseRest, HiveServer2, Hive MetaStore, Hadoop Httpfs, and MapReduce-HistoryServer. | | flink-conf | Change flink-conf.yaml settings. | Restarts Flink history server. | | flink-log4j | Change Flink log4j.properties settings. | Restarts Flink history server. | | flink-log4j-session | Change Flink log4j-session.properties settings for Kubernetes/Yarn session. | Restarts Flink history server. | | flink-log4j-cli | Change Flink log4j-cli.properties settings. | Restarts Flink history server. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts PhoenixQueryserver, HiveServer2, Hive MetaStore, and MapReduce-HistoryServer. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | Restarts the Hadoop HDFS services SecondaryNamenode, Datanode, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts Hadoop KMS, Hadoop Httpfs, and MapReduce-HistoryServer. | | hadoop-ssl-server | Change hadoop ssl server configuration | Not available. | | hadoop-ssl-client | Change hadoop ssl client configuration | Not available. | | hbase | Amazon EMR-curated settings for Apache HBase. | Custom EMR specific property. Sets emrfs-site and hbase-site configs. See those for their associated restarts. | | hbase-env | Change values in HBase's environment. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbase.properties file. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | Not available. | | hbase-site | Change values in HBase's hbase-site.xml file. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. Additionally restarts Phoenix QueryServer. | | hdfs-encryption-zones | Configure HDFS encryption zones. | This classification should not be reconfigured. | | hdfs-env | Change values in the HDFS environment. | Restarts Hadoop HDFS services Namenode, Datanode, and ZKFC. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Additionally restarts Hadoop Httpfs. | | hcatalog-env | Change values in HCatalog's environment. | Restarts Hive HCatalog Server. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | Restarts Hive HCatalog Server. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | Restarts Hive HCatalog Server. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | Restarts Hive WebHCat server. | | hcatalog-webhcat-log4j2 | Change values in HCatalog WebHCat's log4j2.properties. | Restarts Hive WebHCat server. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | Restarts Hive WebHCat server. | | hive | Amazon EMR-curated settings for Apache Hive. | Sets configurations to launch Hive LLAP service. | | hive-beeline-log4j2 | Change values in Hive's beeline-log4j2.properties file. | Not available. | | hive-parquet-logging | Change values in Hive's parquet-logging.properties file. | Not available. | | hive-env | Change values in the Hive environment. | Restarts HiveServer2, HiveMetastore, and Hive HCatalog-Server. Runs Hive schemaTool CLI commands to verify hive-metastore. | | hive-exec-log4j2 | Change values in Hive's hive-exec-log4j2.properties file. | Not available. | | hive-llap-daemon-log4j2 | Change values in Hive's llap-daemon-log4j2.properties file. | Not available. | | hive-log4j2 | Change values in Hive's hive-log4j2.properties file. | Not available. | | hive-site | Change values in Hive's hive-site.xml file | Restarts HiveServer2, HiveMetastore, and Hive HCatalog-Server. Runs Hive schemaTool CLI commands to verify hive-metastore. Also restarts Oozie and Zeppelin. | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | Not available. | | hue-ini | Change values in Hue's ini file | Restarts Hue. Also activates Hue config override CLI commands to pick up new configurations. | | httpfs-env | Change values in the HTTPFS environment. | Restarts Hadoop Httpfs service. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | Restarts Hadoop Httpfs service. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | Not available. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | Restarts Hadoop-KMS service. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | Not available. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | Restarts Hadoop-KMS and Ranger-KMS service. | | hudi-env | Change values in the Hudi environment. | Not available. | | hudi-defaults | Change values in Hudi's hudi-defaults.conf file. | Not available. | | iceberg-defaults | Change values in Iceberg's iceberg-defaults.conf file. | Not available. | | jupyter-notebook-conf | Change values in Jupyter Notebook's jupyter\$1notebook\$1config.py file. | Not available. | | jupyter-hub-conf | Change values in JupyterHubs's jupyterhub\$1config.py file. | Not available. | | jupyter-s3-conf | Configure Jupyter Notebook S3 persistence. | Not available. | | jupyter-sparkmagic-conf | Change values in Sparkmagic's config.json file. | Not available. | | livy-conf | Change values in Livy's livy.conf file. | Restarts Livy Server. | | livy-env | Change values in the Livy environment. | Restarts Livy Server. | | livy-log4j | Change Livy log4j.properties settings. | Restarts Livy Server. | | mapred-env | Change values in the MapReduce application's environment. | Restarts Hadoop MapReduce-HistoryServer. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | Restarts Hadoop MapReduce-HistoryServer. | | oozie-env | Change values in Oozie's environment. | Restarts Oozie. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | Restarts Oozie. | | oozie-site | Change values in Oozie's oozie-site.xml file. | Restarts Oozie. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | Not available. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | Not available. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | Restarts Phoenix-QueryServer. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | Not available. | | pig-env | Change values in the Pig environment. | Not available. | | pig-properties | Change values in Pig's pig.properties file. | Restarts Oozie. | | pig-log4j | Change values in Pig's log4j.properties file. | Not available. | | presto-log | Change values in Presto's log.properties file. | Restarts Presto-Server (for PrestoDB) | | presto-config | Change values in Presto's config.properties file. | Restarts Presto-Server (for PrestoDB) | | presto-password-authenticator | Change values in Presto's password-authenticator.properties file. | Not available. | | presto-env | Change values in Presto's presto-env.sh file. | Restarts Presto-Server (for PrestoDB) | | presto-node | Change values in Presto's node.properties file. | Not available. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | Not available. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | Not available. | | presto-connector-hive | Change values in Presto's hive.properties file. | Restarts Presto-Server (for PrestoDB) | | presto-connector-jmx | Change values in Presto's jmx.properties file. | Not available. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | Not available. | | presto-connector-localfile | Change values in Presto's localfile.properties file. | Not available. | | presto-connector-memory | Change values in Presto's memory.properties file. | Not available. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | Not available. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | Not available. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | Not available. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | Not available. | | presto-connector-redis | Change values in Presto's redis.properties file. | Not available. | | presto-connector-redshift | Change values in Presto's redshift.properties file. | Not available. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | Not available. | | presto-connector-tpcds | Change values in Presto's tpcds.properties file. | Not available. | | trino-log | Change values in Trino's log.properties file. | Restarts Trino-Server (for Trino) | | trino-config | Change values in Trino's config.properties file. | Restarts Trino-Server (for Trino) | | trino-password-authenticator | Change values in Trino's password-authenticator.properties file. | Restarts Trino-Server (for Trino) | | trino-env | Change values in Trino's trino-env.sh file. | Restarts Trino-Server (for Trino) | | trino-node | Change values in Trino's node.properties file. | Not available. | | trino-connector-blackhole | Change values in Trino's blackhole.properties file. | Not available. | | trino-connector-cassandra | Change values in Trino's cassandra.properties file. | Not available. | | trino-connector-hive | Change values in Trino's hive.properties file. | Restarts Trino-Server (for Trino) | | trino-connector-jmx | Change values in Trino's jmx.properties file. | Not available. | | trino-connector-kafka | Change values in Trino's kafka.properties file. | Not available. | | trino-connector-localfile | Change values in Trino's localfile.properties file. | Not available. | | trino-connector-memory | Change values in Trino's memory.properties file. | Not available. | | trino-connector-mongodb | Change values in Trino's mongodb.properties file. | Not available. | | trino-connector-mysql | Change values in Trino's mysql.properties file. | Not available. | | trino-connector-postgresql | Change values in Trino's postgresql.properties file. | Not available. | | trino-connector-raptor | Change values in Trino's raptor.properties file. | Not available. | | trino-connector-redis | Change values in Trino's redis.properties file. | Not available. | | trino-connector-redshift | Change values in Trino's redshift.properties file. | Not available. | | trino-connector-tpch | Change values in Trino's tpch.properties file. | Not available. | | trino-connector-tpcds | Change values in Trino's tpcds.properties file. | Not available. | | ranger-kms-dbks-site | Change values in dbks-site.xml file of Ranger KMS. | Restarts Ranger KMS Server. | | ranger-kms-site | Change values in ranger-kms-site.xml file of Ranger KMS. | Restarts Ranger KMS Server. | | ranger-kms-env | Change values in the Ranger KMS environment. | Restarts Ranger KMS Server. | | ranger-kms-log4j | Change values in kms-log4j.properties file of Ranger KMS. | Not available. | | ranger-kms-db-ca | Change values for CA file on S3 for MySQL SSL connection with Ranger KMS. | Not available. | | spark | Amazon EMR-curated settings for Apache Spark. | This property modifies spark-defaults. See actions there. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | Restarts Spark history server and Spark thrift server. | | spark-env | Change values in the Spark environment. | Restarts Spark history server and Spark thrift server. | | spark-hive-site | Change values in Spark's hive-site.xml file | Not available. | | spark-log4j | Change values in Spark's log4j.properties file. | Restarts Spark history server and Spark thrift server. | | spark-metrics | Change values in Spark's metrics.properties file. | Restarts Spark history server and Spark thrift server. | | sqoop-env | Change values in Sqoop's environment. | Not available. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | Not available. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | Not available. | | tez-site | Change values in Tez's tez-site.xml file. | Restart Oozie and HiveServer2. | | yarn-env | Change values in the YARN environment. | Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts MapReduce-HistoryServer. | | yarn-site | Change values in YARN's yarn-site.xml file. | Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts Livy Server and MapReduce-HistoryServer. | | zeppelin-env | Change values in the Zeppelin environment. | Restarts Zeppelin. | | zeppelin-site | Change configuration settings in zeppelin-site.xml. | Restarts Zeppelin. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | Restarts Zookeeper server. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | Restarts Zookeeper server. | # Amazon EMR release 6.4.0 ## 6.4.0 application versions This release includes the following applications: [https://flink.apache.org/](https://flink.apache.org/), [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [https://hudi.apache.org](https://hudi.apache.org), [http://gethue.com/](http://gethue.com/), [https://jupyter-enterprise-gateway.readthedocs.io/en/latest/](https://jupyter-enterprise-gateway.readthedocs.io/en/latest/), [https://jupyterhub.readthedocs.io/en/latest/#](https://jupyterhub.readthedocs.io/en/latest/#), [https://livy.incubator.apache.org/](https://livy.incubator.apache.org/), [https://mxnet.incubator.apache.org/](https://mxnet.incubator.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://www.tensorflow.org/](https://www.tensorflow.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://trino.io/](https://trino.io/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-6.4.0 | emr-6.3.1 | emr-6.3.0 | emr-6.2.1 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.12.31 | 1.11.977 | 1.11.977 | 1.11.880 | | Python | 2.7, 3.7 | 2.7, 3.7 | 2.7, 3.7 | 2.7, 3.7 | | Scala | 2.12.10 | 2.12.10 | 2.12.10 | 2.12.10 | | AmazonCloudWatchAgent | - | - | - | - | | Delta | - | - | - | - | | Flink | 1.13.1 | 1.12.1 | 1.12.1 | 1.11.2 | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 2.4.4-amzn-0 | 2.2.6-amzn-1 | 2.2.6-amzn-1 | 2.2.6-amzn-0 | | HCatalog | 3.1.2-amzn-5 | 3.1.2-amzn-4 | 3.1.2-amzn-4 | 3.1.2-amzn-3 | | Hadoop | 3.2.1-amzn-4 | 3.2.1-amzn-3.1 | 3.2.1-amzn-3 | 3.2.1-amzn-2.1 | | Hive | 3.1.2-amzn-5 | 3.1.2-amzn-4 | 3.1.2-amzn-4 | 3.1.2-amzn-3 | | Hudi | 0.8.0-amzn-0 | 0.7.0-amzn-0 | 0.7.0-amzn-0 | 0.6.0-amzn-1 | | Hue | 4.9.0 | 4.9.0 | 4.9.0 | 4.8.0 | | Iceberg | - | - | - | - | | JupyterEnterpriseGateway | 2.1.0 | 2.1.0 | 2.1.0 | 2.1.0 | | JupyterHub | 1.4.1 | 1.2.2 | 1.2.2 | 1.1.0 | | Livy | 0.7.1-incubating | 0.7.0-incubating | 0.7.0-incubating | 0.7.0-incubating | | MXNet | 1.8.0 | 1.7.0 | 1.7.0 | 1.7.0 | | Mahout | - | - | - | - | | Oozie | 5.2.1 | 5.2.1 | 5.2.1 | 5.2.0 | | Phoenix | 5.1.2 | 5.0.0-HBase-2.0 | 5.0.0-HBase-2.0 | 5.0.0-HBase-2.0 | | Pig | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | | Presto | 0.254.1-amzn-0 | 0.245.1-amzn-0 | 0.245.1-amzn-0 | 0.238.3-amzn-1 | | Spark | 3.1.2-amzn-0 | 3.1.1-amzn-0.1 | 3.1.1-amzn-0 | 3.0.1-amzn-0.1 | | Sqoop | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | | TensorFlow | 2.4.1 | 2.4.1 | 2.4.1 | 2.3.1 | | Tez | 0.9.2 | 0.9.2 | 0.9.2 | 0.9.2 | | Trino (PrestoSQL) | 359 | 350 | 350 | 343 | | Zeppelin | 0.9.0 | 0.9.0 | 0.9.0 | 0.9.0-preview1 | | ZooKeeper | 3.5.7 | 3.4.14 | 3.4.14 | 3.4.14 | ## 6.4.0 release notes The following release notes include information for Amazon EMR release 6.4.0. Changes are relative to 6.3.0. Initial release date: Sept 20, 2021 Updated release date: March 21, 2022 **Supported applications** + AWS SDK for Java version 1.12.31 + CloudWatch Sink version 2.2.0 + DynamoDB Connector version 4.16.0 + EMRFS version 2.47.0 + Amazon EMR Goodies version 3.2.0 + Amazon EMR Kinesis Connector version 3.5.0 + Amazon EMR Record Server version 2.1.0 + Amazon EMR Scripts version 2.5.0 + Flink version 1.13.1 + Ganglia version 3.7.2 + AWS Glue Hive Metastore Client version 3.3.0 + Hadoop version 3.2.1-amzn-4 + HBase version 2.4.4-amzn-0 + HBase-operator-tools 1.1.0 + HCatalog version 3.1.2-amzn-5 + Hive version 3.1.2-amzn-5 + Hudi version 0.8.0-amzn-0 + Hue version 4.9.0 + Java JDK version Corretto-8.302.08.1 (build 1.8.0\$1302-b08) + JupyterHub version 1.4.1 + Livy version 0.7.1-incubating + MXNet version 1.8.0 + Oozie version 5.2.1 + Phoenix version 5.1.2 + Pig version 0.17.0 + Presto version 0.254.1-amzn-0 + Trino version 359 + Apache Ranger KMS (multi-master transparent encryption) version 2.0.0 + ranger-plugins 2.0.1-amzn-0 + ranger-s3-plugin 1.2.0 + SageMaker Spark SDK version 1.4.1 + Scala version 2.12.10 (OpenJDK 64-Bit Server VM, Java 1.8.0\$1282) + Spark version 3.1.2-amzn-0 + spark-rapids 0.4.1 + Sqoop version 1.4.7 + TensorFlow version 2.4.1 + tez version 0.9.2 + Zeppelin version 0.9.0 + Zookeeper version 3.5.7 + Connectors and drivers: DynamoDB Connector 4.16.0 **New features** + **[Managed scaling] Spark shuffle data managed scaling optimization** - For Amazon EMR versions 5.34.0 and later, and EMR versions 6.4.0 and later, managed scaling is now Spark shuffle data aware (data that Spark redistributes across partitions to perform specific operations). For more information on shuffle operations, see [Using EMR managed scaling in Amazon EMR](https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-managed-scaling.html) in the *Amazon EMR Management Guide* and [Spark Programming Guide](https://spark.apache.org/docs/latest/rdd-programming-guide.html#shuffle-operations). + On Apache Ranger-enabled Amazon EMR clusters, you can use Apache Spark SQL to insert data into or update the Apache Hive metastore tables using `INSERT INTO`, `INSERT OVERWRITE`, and `ALTER TABLE`. When using ALTER TABLE with Spark SQL, a partition location must be the child directory of a table location. Amazon EMR does not currently support inserting data into a partition where the partition location is different from the table location. + PrestoSQL has been [renamed to Trino.](https://trino.io/blog/2020/12/27/announcing-trino.html) + Hive: Execution of simple SELECT queries with LIMIT clause are accelerated by stopping the query execution as soon as the number of records mentioned in LIMIT clause is fetched. Simple SELECT queries are queries that do not have GROUP BY / ORDER by clause or queries that do not have a reducer stage. For example, `SELECT * from WHERE LIMIT `. **Hudi Concurrency Control** + Hudi now supports Optimistic Concurrency Control (OCC), which can be leveraged with write operations like UPSERT and INSERT to allow changes from multiple writers to the same Hudi table. This is file-level OCC, so any two commits (or writers) can write to the same table, if their changes do not conflict. For more information, see the [Hudi concurrency control](https://hudi.apache.org/docs/concurrency_control/). + Amazon EMR clusters have Zookeeper installed, which can be leveraged as the lock provider for OCC. To make it easier to use this feature, Amazon EMR clusters have the following properties pre-configured: ``` hoodie.write.lock.provider=org.apache.hudi.client.transaction.lock.ZookeeperBasedLockProvider hoodie.write.lock.zookeeper.url= hoodie.write.lock.zookeeper.port= hoodie.write.lock.zookeeper.base_path=/hudi ``` To enable OCC, you need to configure the following properties either with their Hudi job options or at the cluster-level using the Amazon EMR configurations API: ``` hoodie.write.concurrency.mode=optimistic_concurrency_control hoodie.cleaner.policy.failed.writes=LAZY (Performs cleaning of failed writes lazily instead of inline with every write) hoodie.write.lock.zookeeper.lock_key= (Table Name is a good option) ``` **Hudi Monitoring: Amazon CloudWatch integration to report Hudi Metrics** + Amazon EMR supports publishing Hudi Metrics to Amazon CloudWatch. It is enabled by setting the following required configurations: ``` hoodie.metrics.on=true hoodie.metrics.reporter.type=CLOUDWATCH ``` + The following are optional Hudi configurations that you can change: [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-640-release.html) **Amazon EMR Hudi configurations support and improvements** + Customers can now leverage EMR Configurations API and Reconfiguration feature to configure Hudi configurations at cluster level. A new file based configuration support has been introduced via /etc/hudi/conf/hudi-defaults.conf along the lines of other applications like Spark, Hive etc. EMR configures few defaults to improve user experience: — `hoodie.datasource.hive_sync.jdbcurl ` is configured to the cluster Hive server URL and no longer needs to be specified. This is particularly useful when running a job in Spark cluster mode, where you previously had to specify the Amazon EMR master IP. — HBase specific configurations, which are useful for using HBase index with Hudi. — Zookeeper lock provider specific configuration, as discussed under concurrency control, which makes it easier to use Optimistic Concurrency Control (OCC). + Additional changes have been introduced to reduce the number of configurations that you need to pass, and to infer automatically where possible: — The `partitionBy ` keyword can be used to specify the partition column. — When enabling Hive Sync, it is no longer mandatory to pass `HIVE_TABLE_OPT_KEY, HIVE_PARTITION_FIELDS_OPT_KEY, HIVE_PARTITION_EXTRACTOR_CLASS_OPT_KEY`. Those values can be inferred from the Hudi table name and partition field. — `KEYGENERATOR_CLASS_OPT_KEY` is not mandatory to pass, and can be inferred from simpler cases of `SimpleKeyGenerator` and `ComplexKeyGenerator`. **Hudi Caveats** + Hudi does not support vectorized execution in Hive for Merge on Read (MoR) and Bootstrap tables. For example, `count(*)` fails with Hudi realtime table when `hive.vectorized.execution.enabled` is set to true. As a workaround, you can disable vectorized reading by setting `hive.vectorized.execution.enabled` to `false`. + Multi-writer support is not compatible with the Hudi bootstrap feature. + Flink Streamer and Flink SQL are experimental features in this release. These features are not recommended for use in production deployments. **Changes, enhancements, and resolved issues** This is a release to fix issues with Amazon EMR Scaling when it fails to scale up/scale down a cluster successfully or causes application failures. + Previously, manual restart of the resource manager on a multi-master cluster caused Amazon EMR on-cluster daemons, like Zookeeper, to reload all previously decommissioned or lost nodes in the Zookeeper znode file. This caused default limits to be exceeded in certain situations. Amazon EMR now removes the decommissioned or lost node records older than one hour from the Zookeeper file and the internal limits have been increased. + Fixed an issue where scaling requests failed for a large, highly utilized cluster when Amazon EMR on-cluster daemons were running health checking activities, such as gathering YARN node state and HDFS node state. This was happening because on-cluster daemons were not able to communicate the health status data of a node to internal Amazon EMR components. + Improved EMR on-cluster daemons to correctly track the node states when IP addresses are reused to improve reliability during scaling operations. + [SPARK-29683](https://issues.apache.org/jira/browse/SPARK-29683). Fixed an issue where job failures occurred during cluster scale-down as Spark was assuming all available nodes were deny-listed. + [YARN-9011](https://issues.apache.org/jira/browse/YARN-9011). Fixed an issue where job failures occurred due to a race condition in YARN decommissioning when cluster tried to scale up or down. + Fixed issue with step or job failures during cluster scaling by ensuring that the node states are always consistent between the Amazon EMR on-cluster daemons and YARN/HDFS. + Fixed an issue where cluster operations such as scale down and step submission failed for Amazon EMR clusters enabled with Kerberos authentication. This was because the Amazon EMR on-cluster daemon did not renew the Kerberos ticket, which is required to securely communicate with HDFS/YARN running on the primary node. + **Configuring a cluster to fix Apache YARN Timeline Server version 1 and 1.5 performance issues** Apache YARN Timeline Server version 1 and 1.5 can cause performance issues with very active, large EMR clusters, particularly with `yarn.resourcemanager.system-metrics-publisher.enabled=true`, which is the default setting in Amazon EMR. An open source YARN Timeline Server v2 solves the performance issue related to YARN Timeline Server scalability. Other workarounds for this issue include: + Configuring yarn.resourcemanager.system-metrics-publisher.enabled=false in yarn-site.xml. + Enabling the fix for this issue when creating a cluster, as described below. The following Amazon EMR releases contain a fix for this YARN Timeline Server performance issue. EMR 5.30.2, 5.31.1, 5.32.1, 5.33.1, 5.34.x, 6.0.1, 6.1.1, 6.2.1, 6.3.1, 6.4.x To enable the fix on any of the above specified Amazon EMR releases, set these properties to `true` in a configurations JSON file that is passed in using the [`aws emr create-cluster` command parameter](https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-configure-apps-create-cluster.html): `--configurations file://./configurations.json`. Or enable the fix using the [reconfiguration console UI](https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-configure-apps-running-cluster.html). Example of the configurations.json file contents: ``` [ { "Classification": "yarn-site", "Properties": { "yarn.resourcemanager.system-metrics-publisher.timeline-server-v1.enable-batch": "true", "yarn.resourcemanager.system-metrics-publisher.enabled": "true" }, "Configurations": [] } ] ``` + WebHDFS and HttpFS server are disabled by default. You can re-enable WebHDFS using the Hadoop configuration, `dfs.webhdfs.enabled`. HttpFS server can be started by using `sudo systemctl start hadoop-httpfs`. + HTTPS is now enabled by default for Amazon Linux repositories. If you are using an Amazon S3 VPCE policy to restrict access to specific buckets, you must add the new Amazon Linux bucket ARN `arn:aws:s3:::amazonlinux-2-repos-$region/*` to your policy (replace `$region` with the region where the endpoint is). For more information, see this topic in the AWS discussion forums. [Announcement: Amazon Linux 2 now supports the ability to use HTTPS while connecting to package repositories ](https://forums.aws.amazon.com/ann.jspa?annID=8528). + Hive: Write query performance is improved by enabling the use of a scratch directory on HDFS for the last job. The temporary data for final job is written to HDFS instead of Amazon S3 and performance is improved because the data is moved from HDFS to the final table location (Amazon S3) instead of between Amazon S3 devices. + Hive: Query compilation time improvement up to 2.5x with Glue metastore Partition Pruning. + By default, when built-in UDFs are passed by Hive to the Hive Metastore Server, only a subset of those built-in UDFs are passed to the Glue Metastore since Glue supports only limited expression operators. If you set `hive.glue.partition.pruning.client=true`, then all partition pruning happens on the client side. If the you set `hive.glue.partition.pruning.server=true`, then all partition pruning happens on the server side. **Known issues** + Hue queries do not work in Amazon EMR 6.4.0 because Apache Hadoop HttpFS server is disabled by default. To use Hue on Amazon EMR 6.4.0, either manually start HttpFS server on the Amazon EMR primary node using `sudo systemctl start hadoop-httpfs`, or [use an Amazon EMR step](https://docs.aws.amazon.com/emr/latest/ManagementGuide/add-step-cli.html). + The Amazon EMR Notebooks feature used with Livy user impersonation does not work because HttpFS is disabled by default. In this case, the EMR notebook cannot connect to the cluster that has Livy impersonation enabled. The workaround is to start HttpFS server before connecting the EMR notebook to the cluster using `sudo systemctl start hadoop-httpfs`. + In Amazon EMR version 6.4.0, Phoenix does not support the Phoenix connectors component. + To use Spark actions with Apache Oozie, you must add the following configuration to your Oozie `workflow.xml` file. Otherwise, several critical libraries such as Hadoop and EMRFS will be missing from the classpath of the Spark executors that Oozie launches. ``` --conf spark.yarn.populateHadoopClasspath=true ``` + When you use Spark with Hive partition location formatting to read data in Amazon S3, and you run Spark on Amazon EMR releases 5.30.0 to 5.36.0, and 6.2.0 to 6.9.0, you might encounter an issue that prevents your cluster from reading data correctly. This can happen if your partitions have all of the following characteristics: + Two or more partitions are scanned from the same table. + At least one partition directory path is a prefix of at least one other partition directory path, for example, `s3://bucket/table/p=a` is a prefix of `s3://bucket/table/p=a b`. + The first character that follows the prefix in the other partition directory has a UTF-8 value that’s less than than the `/` character (U\$1002F). For example, the space character (U\$10020) that occurs between a and b in `s3://bucket/table/p=a b` falls into this category. Note that there are 14 other non-control characters: `!"#$%&‘()*+,-`. For more information, see [UTF-8 encoding table and Unicode characters](https://www.utf8-chartable.de/). As a workaround to this issue, set the `spark.sql.sources.fastS3PartitionDiscovery.enabled` configuration to `false` in the `spark-defaults` classification. ## 6.4.0 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | aws-sagemaker-spark-sdk | 1.4.1 | Amazon SageMaker Spark SDK | | emr-ddb | 4.16.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 3.2.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.5.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-notebook-env | 1.3.0 | Conda env for emr notebook which includes jupyter enterprise gateway | | emr-s3-dist-cp | 2.18.0 | Distributed copy application optimized for Amazon S3. | | emr-s3-select | 2.1.0 | EMR S3Select Connector | | emrfs | 2.47.0 | Amazon S3 connector for Hadoop ecosystem applications. | | flink-client | 1.13.1 | Apache Flink command line client scripts and applications. | | flink-jobmanager-config | 1.13.1 | Managing resources on EMR nodes for Apache Flink JobManager. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 3.2.1-amzn-4 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 3.2.1-amzn-4 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 3.2.1-amzn-4 | HDFS command-line client and library | | hadoop-hdfs-namenode | 3.2.1-amzn-4 | HDFS service for tracking file names and block locations. | | hadoop-hdfs-journalnode | 3.2.1-amzn-4 | HDFS service for managing the Hadoop filesystem journal on HA clusters. | | hadoop-httpfs-server | 3.2.1-amzn-4 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 3.2.1-amzn-4 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 3.2.1-amzn-4 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 3.2.1-amzn-4 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 3.2.1-amzn-4 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 3.2.1-amzn-4 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 2.4.4-amzn-0 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 2.4.4-amzn-0 | Service for serving one or more HBase regions. | | hbase-client | 2.4.4-amzn-0 | HBase command-line client. | | hbase-rest-server | 2.4.4-amzn-0 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 2.4.4-amzn-0 | Service providing a Thrift endpoint to HBase. | | hcatalog-client | 3.1.2-amzn-5 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 3.1.2-amzn-5 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 3.1.2-amzn-5 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 3.1.2-amzn-5 | Hive command line client. | | hive-hbase | 3.1.2-amzn-5 | Hive-hbase client. | | hive-metastore-server | 3.1.2-amzn-5 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server2 | 3.1.2-amzn-5 | Service for accepting Hive queries as web requests. | | hudi | 0.8.0-amzn-0 | Incremental processing framework to power data pipline at low latency and high efficiency. | | hudi-presto | 0.8.0-amzn-0 | Bundle library for running Presto with Hudi. | | hudi-trino | 0.8.0-amzn-0 | Bundle library for running Trino with Hudi. | | hudi-spark | 0.8.0-amzn-0 | Bundle library for running Spark with Hudi. | | hue-server | 4.9.0 | Web application for analyzing data using Hadoop ecosystem applications | | jupyterhub | 1.4.1 | Multi-user server for Jupyter notebooks | | livy-server | 0.7.1-incubating | REST interface for interacting with Apache Spark | | nginx | 1.12.1 | nginx [engine x] is an HTTP and reverse proxy server | | mxnet | 1.8.0 | A flexible, scalable, and efficient library for deep learning. | | mariadb-server | 5.5.68\$1 | MariaDB database server. | | nvidia-cuda | 10.1.243 | Nvidia drivers and Cuda toolkit | | oozie-client | 5.2.1 | Oozie command-line client. | | oozie-server | 5.2.1 | Service for accepting Oozie workflow requests. | | opencv | 4.5.0 | Open Source Computer Vision Library. | | phoenix-library | 5.1.2 | The phoenix libraries for server and client | | phoenix-query-server | 5.1.2 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.254.1-amzn-0 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.254.1-amzn-0 | Service for executing pieces of a query. | | presto-client | 0.254.1-amzn-0 | Presto command-line client which is installed on an HA cluster's stand-by masters where Presto server is not started. | | trino-coordinator | 359 | Service for accepting queries and managing query execution among trino-workers. | | trino-worker | 359 | Service for executing pieces of a query. | | trino-client | 359 | Trino command-line client which is installed on an HA cluster's stand-by masters where Trino server is not started. | | pig-client | 0.17.0 | Pig command-line client. | | r | 4.0.2 | The R Project for Statistical Computing | | ranger-kms-server | 2.0.0 | Apache Ranger Key Management System | | spark-client | 3.1.2-amzn-0 | Spark command-line clients. | | spark-history-server | 3.1.2-amzn-0 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 3.1.2-amzn-0 | In-memory execution engine for YARN. | | spark-yarn-slave | 3.1.2-amzn-0 | Apache Spark libraries needed by YARN slaves. | | spark-rapids | 0.4.1 | Nvidia Spark RAPIDS plugin that accelerates Apache Spark with GPUs. | | sqoop-client | 1.4.7 | Apache Sqoop command-line client. | | tensorflow | 2.4.1 | TensorFlow open source software library for high performance numerical computation. | | tez-on-yarn | 0.9.2 | The tez YARN application and libraries. | | webserver | 2.4.41\$1 | Apache HTTP server. | | zeppelin-server | 0.9.0 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.5.7 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.5.7 | ZooKeeper command line client. | ## 6.4.0 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). Reconfiguration actions occur when you specify a configuration for instance groups in a running cluster. Amazon EMR only initiates reconfiguration actions for the classifications that you modify. For more information, see [Reconfigure an instance group in a running cluster](emr-configure-apps-running-cluster.md). **emr-6.4.0 classifications** | Classifications | Description | Reconfiguration Actions | | --- | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | Restarts the ResourceManager service. | | container-executor | Change values in Hadoop YARN's container-executor.cfg file. | Not available. | | container-log4j | Change values in Hadoop YARN's container-log4j.properties file. | Not available. | | core-site | Change values in Hadoop's core-site.xml file. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts Hadoop KMS, Ranger KMS, HiveServer2, Hive MetaStore, Hadoop Httpfs, and MapReduce-HistoryServer. | | docker-conf | Change docker related settings. | Not available. | | emrfs-site | Change EMRFS settings. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts HBaseRegionserver, HBaseMaster, HBaseThrift, HBaseRest, HiveServer2, Hive MetaStore, Hadoop Httpfs, and MapReduce-HistoryServer. | | flink-conf | Change flink-conf.yaml settings. | Restarts Flink history server. | | flink-log4j | Change Flink log4j.properties settings. | Restarts Flink history server. | | flink-log4j-session | Change Flink log4j-session.properties settings for Kubernetes/Yarn session. | Restarts Flink history server. | | flink-log4j-cli | Change Flink log4j-cli.properties settings. | Restarts Flink history server. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts PhoenixQueryserver, HiveServer2, Hive MetaStore, and MapReduce-HistoryServer. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | Restarts the Hadoop HDFS services SecondaryNamenode, Datanode, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts Hadoop KMS, Hadoop Httpfs, and MapReduce-HistoryServer. | | hadoop-ssl-server | Change hadoop ssl server configuration | Not available. | | hadoop-ssl-client | Change hadoop ssl client configuration | Not available. | | hbase | Amazon EMR-curated settings for Apache HBase. | Custom EMR specific property. Sets emrfs-site and hbase-site configs. See those for their associated restarts. | | hbase-env | Change values in HBase's environment. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbase.properties file. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | Not available. | | hbase-site | Change values in HBase's hbase-site.xml file. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. Additionally restarts Phoenix QueryServer. | | hdfs-encryption-zones | Configure HDFS encryption zones. | This classification should not be reconfigured. | | hdfs-env | Change values in the HDFS environment. | Restarts Hadoop HDFS services Namenode, Datanode, and ZKFC. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Additionally restarts Hadoop Httpfs. | | hcatalog-env | Change values in HCatalog's environment. | Restarts Hive HCatalog Server. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | Restarts Hive HCatalog Server. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | Restarts Hive HCatalog Server. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | Restarts Hive WebHCat server. | | hcatalog-webhcat-log4j2 | Change values in HCatalog WebHCat's log4j2.properties. | Restarts Hive WebHCat server. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | Restarts Hive WebHCat server. | | hive | Amazon EMR-curated settings for Apache Hive. | Sets configurations to launch Hive LLAP service. | | hive-beeline-log4j2 | Change values in Hive's beeline-log4j2.properties file. | Not available. | | hive-parquet-logging | Change values in Hive's parquet-logging.properties file. | Not available. | | hive-env | Change values in the Hive environment. | Restarts HiveServer2, HiveMetastore, and Hive HCatalog-Server. Runs Hive schemaTool CLI commands to verify hive-metastore. | | hive-exec-log4j2 | Change values in Hive's hive-exec-log4j2.properties file. | Not available. | | hive-llap-daemon-log4j2 | Change values in Hive's llap-daemon-log4j2.properties file. | Not available. | | hive-log4j2 | Change values in Hive's hive-log4j2.properties file. | Not available. | | hive-site | Change values in Hive's hive-site.xml file | Restarts HiveServer2, HiveMetastore, and Hive HCatalog-Server. Runs Hive schemaTool CLI commands to verify hive-metastore. Also restarts Oozie and Zeppelin. | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | Not available. | | hue-ini | Change values in Hue's ini file | Restarts Hue. Also activates Hue config override CLI commands to pick up new configurations. | | httpfs-env | Change values in the HTTPFS environment. | Restarts Hadoop Httpfs service. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | Restarts Hadoop Httpfs service. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | Not available. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | Restarts Hadoop-KMS service. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | Not available. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | Restarts Hadoop-KMS and Ranger-KMS service. | | hudi-env | Change values in the Hudi environment. | Not available. | | hudi-defaults | Change values in Hudi's hudi-defaults.conf file. | Not available. | | jupyter-notebook-conf | Change values in Jupyter Notebook's jupyter\$1notebook\$1config.py file. | Not available. | | jupyter-hub-conf | Change values in JupyterHubs's jupyterhub\$1config.py file. | Not available. | | jupyter-s3-conf | Configure Jupyter Notebook S3 persistence. | Not available. | | jupyter-sparkmagic-conf | Change values in Sparkmagic's config.json file. | Not available. | | livy-conf | Change values in Livy's livy.conf file. | Restarts Livy Server. | | livy-env | Change values in the Livy environment. | Restarts Livy Server. | | livy-log4j | Change Livy log4j.properties settings. | Restarts Livy Server. | | mapred-env | Change values in the MapReduce application's environment. | Restarts Hadoop MapReduce-HistoryServer. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | Restarts Hadoop MapReduce-HistoryServer. | | oozie-env | Change values in Oozie's environment. | Restarts Oozie. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | Restarts Oozie. | | oozie-site | Change values in Oozie's oozie-site.xml file. | Restarts Oozie. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | Not available. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | Not available. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | Restarts Phoenix-QueryServer. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | Not available. | | pig-env | Change values in the Pig environment. | Not available. | | pig-properties | Change values in Pig's pig.properties file. | Restarts Oozie. | | pig-log4j | Change values in Pig's log4j.properties file. | Not available. | | presto-log | Change values in Presto's log.properties file. | Restarts Presto-Server (for PrestoDB) | | presto-config | Change values in Presto's config.properties file. | Restarts Presto-Server (for PrestoDB) | | presto-password-authenticator | Change values in Presto's password-authenticator.properties file. | Not available. | | presto-env | Change values in Presto's presto-env.sh file. | Restarts Presto-Server (for PrestoDB) | | presto-node | Change values in Presto's node.properties file. | Not available. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | Not available. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | Not available. | | presto-connector-hive | Change values in Presto's hive.properties file. | Restarts Presto-Server (for PrestoDB) | | presto-connector-jmx | Change values in Presto's jmx.properties file. | Not available. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | Not available. | | presto-connector-localfile | Change values in Presto's localfile.properties file. | Not available. | | presto-connector-memory | Change values in Presto's memory.properties file. | Not available. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | Not available. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | Not available. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | Not available. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | Not available. | | presto-connector-redis | Change values in Presto's redis.properties file. | Not available. | | presto-connector-redshift | Change values in Presto's redshift.properties file. | Not available. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | Not available. | | presto-connector-tpcds | Change values in Presto's tpcds.properties file. | Not available. | | trino-log | Change values in Trino's log.properties file. | Restarts Trino-Server (for Trino) | | trino-config | Change values in Trino's config.properties file. | Restarts Trino-Server (for Trino) | | trino-password-authenticator | Change values in Trino's password-authenticator.properties file. | Restarts Trino-Server (for Trino) | | trino-env | Change values in Trino's trino-env.sh file. | Restarts Trino-Server (for Trino) | | trino-node | Change values in Trino's node.properties file. | Not available. | | trino-connector-blackhole | Change values in Trino's blackhole.properties file. | Not available. | | trino-connector-cassandra | Change values in Trino's cassandra.properties file. | Not available. | | trino-connector-hive | Change values in Trino's hive.properties file. | Restarts Trino-Server (for Trino) | | trino-connector-jmx | Change values in Trino's jmx.properties file. | Not available. | | trino-connector-kafka | Change values in Trino's kafka.properties file. | Not available. | | trino-connector-localfile | Change values in Trino's localfile.properties file. | Not available. | | trino-connector-memory | Change values in Trino's memory.properties file. | Not available. | | trino-connector-mongodb | Change values in Trino's mongodb.properties file. | Not available. | | trino-connector-mysql | Change values in Trino's mysql.properties file. | Not available. | | trino-connector-postgresql | Change values in Trino's postgresql.properties file. | Not available. | | trino-connector-raptor | Change values in Trino's raptor.properties file. | Not available. | | trino-connector-redis | Change values in Trino's redis.properties file. | Not available. | | trino-connector-redshift | Change values in Trino's redshift.properties file. | Not available. | | trino-connector-tpch | Change values in Trino's tpch.properties file. | Not available. | | trino-connector-tpcds | Change values in Trino's tpcds.properties file. | Not available. | | ranger-kms-dbks-site | Change values in dbks-site.xml file of Ranger KMS. | Restarts Ranger KMS Server. | | ranger-kms-site | Change values in ranger-kms-site.xml file of Ranger KMS. | Restarts Ranger KMS Server. | | ranger-kms-env | Change values in the Ranger KMS environment. | Restarts Ranger KMS Server. | | ranger-kms-log4j | Change values in kms-log4j.properties file of Ranger KMS. | Not available. | | ranger-kms-db-ca | Change values for CA file on S3 for MySQL SSL connection with Ranger KMS. | Not available. | | spark | Amazon EMR-curated settings for Apache Spark. | This property modifies spark-defaults. See actions there. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | Restarts Spark history server and Spark thrift server. | | spark-env | Change values in the Spark environment. | Restarts Spark history server and Spark thrift server. | | spark-hive-site | Change values in Spark's hive-site.xml file | Not available. | | spark-log4j | Change values in Spark's log4j.properties file. | Restarts Spark history server and Spark thrift server. | | spark-metrics | Change values in Spark's metrics.properties file. | Restarts Spark history server and Spark thrift server. | | sqoop-env | Change values in Sqoop's environment. | Not available. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | Not available. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | Not available. | | tez-site | Change values in Tez's tez-site.xml file. | Restart Oozie and HiveServer2. | | yarn-env | Change values in the YARN environment. | Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts MapReduce-HistoryServer. | | yarn-site | Change values in YARN's yarn-site.xml file. | Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts Livy Server and MapReduce-HistoryServer. | | zeppelin-env | Change values in the Zeppelin environment. | Restarts Zeppelin. | | zeppelin-site | Change configuration settings in zeppelin-site.xml. | Restarts Zeppelin. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | Restarts Zookeeper server. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | Restarts Zookeeper server. | # Amazon EMR release 6.3.1 ## 6.3.1 application versions This release includes the following applications: [https://flink.apache.org/](https://flink.apache.org/), [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [https://hudi.apache.org](https://hudi.apache.org), [http://gethue.com/](http://gethue.com/), [https://jupyter-enterprise-gateway.readthedocs.io/en/latest/](https://jupyter-enterprise-gateway.readthedocs.io/en/latest/), [https://jupyterhub.readthedocs.io/en/latest/#](https://jupyterhub.readthedocs.io/en/latest/#), [https://livy.incubator.apache.org/](https://livy.incubator.apache.org/), [https://mxnet.incubator.apache.org/](https://mxnet.incubator.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://prestosql.io/](https://prestosql.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://www.tensorflow.org/](https://www.tensorflow.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-6.3.1 | emr-6.3.0 | emr-6.2.1 | emr-6.2.0 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.11.977 | 1.11.977 | 1.11.880 | 1.11.880 | | Python | 2.7, 3.7 | 2.7, 3.7 | 2.7, 3.7 | 2.7, 3.7 | | Scala | 2.12.10 | 2.12.10 | 2.12.10 | 2.12.10 | | AmazonCloudWatchAgent | - | - | - | - | | Delta | - | - | - | - | | Flink | 1.12.1 | 1.12.1 | 1.11.2 | 1.11.2 | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 2.2.6-amzn-1 | 2.2.6-amzn-1 | 2.2.6-amzn-0 | 2.2.6-amzn-0 | | HCatalog | 3.1.2-amzn-4 | 3.1.2-amzn-4 | 3.1.2-amzn-3 | 3.1.2-amzn-3 | | Hadoop | 3.2.1-amzn-3.1 | 3.2.1-amzn-3 | 3.2.1-amzn-2.1 | 3.2.1-amzn-2 | | Hive | 3.1.2-amzn-4 | 3.1.2-amzn-4 | 3.1.2-amzn-3 | 3.1.2-amzn-3 | | Hudi | 0.7.0-amzn-0 | 0.7.0-amzn-0 | 0.6.0-amzn-1 | 0.6.0-amzn-1 | | Hue | 4.9.0 | 4.9.0 | 4.8.0 | 4.8.0 | | Iceberg | - | - | - | - | | JupyterEnterpriseGateway | 2.1.0 | 2.1.0 | 2.1.0 | 2.1.0 | | JupyterHub | 1.2.2 | 1.2.2 | 1.1.0 | 1.1.0 | | Livy | 0.7.0-incubating | 0.7.0-incubating | 0.7.0-incubating | 0.7.0-incubating | | MXNet | 1.7.0 | 1.7.0 | 1.7.0 | 1.7.0 | | Mahout | - | - | - | - | | Oozie | 5.2.1 | 5.2.1 | 5.2.0 | 5.2.0 | | Phoenix | 5.0.0-HBase-2.0 | 5.0.0-HBase-2.0 | 5.0.0-HBase-2.0 | 5.0.0-HBase-2.0 | | Pig | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | | Presto | 0.245.1-amzn-0 | 0.245.1-amzn-0 | 0.238.3-amzn-1 | 0.238.3-amzn-1 | | Spark | 3.1.1-amzn-0.1 | 3.1.1-amzn-0 | 3.0.1-amzn-0.1 | 3.0.1-amzn-0 | | Sqoop | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | | TensorFlow | 2.4.1 | 2.4.1 | 2.3.1 | 2.3.1 | | Tez | 0.9.2 | 0.9.2 | 0.9.2 | 0.9.2 | | Trino (PrestoSQL) | 350 | 350 | 343 | 343 | | Zeppelin | 0.9.0 | 0.9.0 | 0.9.0-preview1 | 0.9.0-preview1 | | ZooKeeper | 3.4.14 | 3.4.14 | 3.4.14 | 3.4.14 | ## 6.3.1 release notes This is a release to fix issues with Amazon EMR Scaling when it fails to scale up/scale down a cluster successfully or causes application failures. **Changes, Enhancements, and Resolved Issues** + Fixed an issue where scaling requests failed for a large, highly utilized cluster when Amazon EMR on-cluster daemons were running health checking activities, such as gathering YARN node state and HDFS node state. This was happening because on-cluster daemons were not able to communicate the health status data of a node to internal Amazon EMR components. + Improved EMR on-cluster daemons to correctly track the node states when IP addresses are reused to improve reliability during scaling operations. + [SPARK-29683](https://issues.apache.org/jira/browse/SPARK-29683). Fixed an issue where job failures occurred during cluster scale-down as Spark was assuming all available nodes were deny-listed. + [YARN-9011](https://issues.apache.org/jira/browse/YARN-9011). Fixed an issue where job failures occurred due to a race condition in YARN decommissioning when cluster tried to scale up or down. + Fixed issue with step or job failures during cluster scaling by ensuring that the node states are always consistent between the Amazon EMR on-cluster daemons and YARN/HDFS. + Fixed an issue where cluster operations such as scale down and step submission failed for Amazon EMR clusters enabled with Kerberos authentication. This was because the Amazon EMR on-cluster daemon did not renew the Kerberos ticket, which is required to securely communicate with HDFS/YARN running on the primary node. + HTTPS is now enabled by default for Amazon Linux repositories. If you are using an Amazon S3 VPCE policy to restrict access to specific buckets, you must add the new Amazon Linux bucket ARN `arn:aws:s3:::amazonlinux-2-repos-$region/*` to your policy (replace `$region` with the region where the endpoint is). For more information, see this topic in the AWS discussion forums. [Announcement: Amazon Linux 2 now supports the ability to use HTTPS while connecting to package repositories ](https://forums.aws.amazon.com/ann.jspa?annID=8528). **Known issues** + When you use Spark with Hive partition location formatting to read data in Amazon S3, and you run Spark on Amazon EMR releases 5.30.0 to 5.36.0, and 6.2.0 to 6.9.0, you might encounter an issue that prevents your cluster from reading data correctly. This can happen if your partitions have all of the following characteristics: + Two or more partitions are scanned from the same table. + At least one partition directory path is a prefix of at least one other partition directory path, for example, `s3://bucket/table/p=a` is a prefix of `s3://bucket/table/p=a b`. + The first character that follows the prefix in the other partition directory has a UTF-8 value that’s less than than the `/` character (U\$1002F). For example, the space character (U\$10020) that occurs between a and b in `s3://bucket/table/p=a b` falls into this category. Note that there are 14 other non-control characters: `!"#$%&‘()*+,-`. For more information, see [UTF-8 encoding table and Unicode characters](https://www.utf8-chartable.de/). As a workaround to this issue, set the `spark.sql.sources.fastS3PartitionDiscovery.enabled` configuration to `false` in the `spark-defaults` classification. ## 6.3.1 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | aws-sagemaker-spark-sdk | 1.4.1 | Amazon SageMaker Spark SDK | | emr-ddb | 4.16.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 3.2.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.5.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-notebook-env | 1.2.0 | Conda env for emr notebook which includes jupyter enterprise gateway | | emr-s3-dist-cp | 2.18.0 | Distributed copy application optimized for Amazon S3. | | emr-s3-select | 2.1.0 | EMR S3Select Connector | | emrfs | 2.46.0 | Amazon S3 connector for Hadoop ecosystem applications. | | flink-client | 1.12.1 | Apache Flink command line client scripts and applications. | | flink-jobmanager-config | 1.12.1 | Managing resources on EMR nodes for Apache Flink JobManager. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 3.2.1-amzn-3.1 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 3.2.1-amzn-3.1 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 3.2.1-amzn-3.1 | HDFS command-line client and library | | hadoop-hdfs-namenode | 3.2.1-amzn-3.1 | HDFS service for tracking file names and block locations. | | hadoop-hdfs-journalnode | 3.2.1-amzn-3.1 | HDFS service for managing the Hadoop filesystem journal on HA clusters. | | hadoop-httpfs-server | 3.2.1-amzn-3.1 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 3.2.1-amzn-3.1 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 3.2.1-amzn-3.1 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 3.2.1-amzn-3.1 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 3.2.1-amzn-3.1 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 3.2.1-amzn-3.1 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 2.2.6-amzn-1 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 2.2.6-amzn-1 | Service for serving one or more HBase regions. | | hbase-client | 2.2.6-amzn-1 | HBase command-line client. | | hbase-rest-server | 2.2.6-amzn-1 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 2.2.6-amzn-1 | Service providing a Thrift endpoint to HBase. | | hcatalog-client | 3.1.2-amzn-4 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 3.1.2-amzn-4 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 3.1.2-amzn-4 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 3.1.2-amzn-4 | Hive command line client. | | hive-hbase | 3.1.2-amzn-4 | Hive-hbase client. | | hive-metastore-server | 3.1.2-amzn-4 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server2 | 3.1.2-amzn-4 | Service for accepting Hive queries as web requests. | | hudi | 0.7.0-amzn-0 | Incremental processing framework to power data pipline at low latency and high efficiency. | | hudi-presto | 0.7.0-amzn-0 | Bundle library for running Presto with Hudi. | | hudi-prestosql | 0.7.0-amzn-0 | Bundle library for running PrestoSQL with Hudi. | | hudi-spark | 0.7.0-amzn-0 | Bundle library for running Spark with Hudi. | | hue-server | 4.9.0 | Web application for analyzing data using Hadoop ecosystem applications | | jupyterhub | 1.2.2 | Multi-user server for Jupyter notebooks | | livy-server | 0.7.0-incubating | REST interface for interacting with Apache Spark | | nginx | 1.12.1 | nginx [engine x] is an HTTP and reverse proxy server | | mxnet | 1.7.0 | A flexible, scalable, and efficient library for deep learning. | | mariadb-server | 5.5.68\$1 | MariaDB database server. | | nvidia-cuda | 10.1.243 | Nvidia drivers and Cuda toolkit | | oozie-client | 5.2.1 | Oozie command-line client. | | oozie-server | 5.2.1 | Service for accepting Oozie workflow requests. | | opencv | 4.5.0 | Open Source Computer Vision Library. | | phoenix-library | 5.0.0-HBase-2.0 | The phoenix libraries for server and client | | phoenix-query-server | 5.0.0-HBase-2.0 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.245.1-amzn-0 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.245.1-amzn-0 | Service for executing pieces of a query. | | presto-client | 0.245.1-amzn-0 | Presto command-line client which is installed on an HA cluster's stand-by masters where Presto server is not started. | | prestosql-coordinator | 350 | Service for accepting queries and managing query execution among prestosql-workers. | | prestosql-worker | 350 | Service for executing pieces of a query. | | prestosql-client | 350 | Presto command-line client which is installed on an HA cluster's stand-by masters where Presto server is not started. | | pig-client | 0.17.0 | Pig command-line client. | | r | 4.0.2 | The R Project for Statistical Computing | | ranger-kms-server | 2.0.0 | Apache Ranger Key Management System | | spark-client | 3.1.1-amzn-0.1 | Spark command-line clients. | | spark-history-server | 3.1.1-amzn-0.1 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 3.1.1-amzn-0.1 | In-memory execution engine for YARN. | | spark-yarn-slave | 3.1.1-amzn-0.1 | Apache Spark libraries needed by YARN slaves. | | spark-rapids | 0.4.1 | Nvidia Spark RAPIDS plugin that accelerates Apache Spark with GPUs. | | sqoop-client | 1.4.7 | Apache Sqoop command-line client. | | tensorflow | 2.4.1 | TensorFlow open source software library for high performance numerical computation. | | tez-on-yarn | 0.9.2 | The tez YARN application and libraries. | | webserver | 2.4.41\$1 | Apache HTTP server. | | zeppelin-server | 0.9.0 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.4.14 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.4.14 | ZooKeeper command line client. | ## 6.3.1 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). Reconfiguration actions occur when you specify a configuration for instance groups in a running cluster. Amazon EMR only initiates reconfiguration actions for the classifications that you modify. For more information, see [Reconfigure an instance group in a running cluster](emr-configure-apps-running-cluster.md). **emr-6.3.1 classifications** | Classifications | Description | Reconfiguration Actions | | --- | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | Restarts the ResourceManager service. | | container-executor | Change values in Hadoop YARN's container-executor.cfg file. | Not available. | | container-log4j | Change values in Hadoop YARN's container-log4j.properties file. | Not available. | | core-site | Change values in Hadoop's core-site.xml file. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts Hadoop KMS, Ranger KMS, HiveServer2, Hive MetaStore, Hadoop Httpfs, and MapReduce-HistoryServer. | | docker-conf | Change docker related settings. | Not available. | | emrfs-site | Change EMRFS settings. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts HBaseRegionserver, HBaseMaster, HBaseThrift, HBaseRest, HiveServer2, Hive MetaStore, Hadoop Httpfs, and MapReduce-HistoryServer. | | flink-conf | Change flink-conf.yaml settings. | Restarts Flink history server. | | flink-log4j | Change Flink log4j.properties settings. | Restarts Flink history server. | | flink-log4j-session | Change Flink log4j-session.properties settings for Kubernetes/Yarn session. | Restarts Flink history server. | | flink-log4j-cli | Change Flink log4j-cli.properties settings. | Restarts Flink history server. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts PhoenixQueryserver, HiveServer2, Hive MetaStore, and MapReduce-HistoryServer. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | Restarts the Hadoop HDFS services SecondaryNamenode, Datanode, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts Hadoop KMS, Hadoop Httpfs, and MapReduce-HistoryServer. | | hadoop-ssl-server | Change hadoop ssl server configuration | Not available. | | hadoop-ssl-client | Change hadoop ssl client configuration | Not available. | | hbase | Amazon EMR-curated settings for Apache HBase. | Custom EMR specific property. Sets emrfs-site and hbase-site configs. See those for their associated restarts. | | hbase-env | Change values in HBase's environment. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbase.properties file. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | Not available. | | hbase-site | Change values in HBase's hbase-site.xml file. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. Additionally restarts Phoenix QueryServer. | | hdfs-encryption-zones | Configure HDFS encryption zones. | This classification should not be reconfigured. | | hdfs-env | Change values in the HDFS environment. | Restarts Hadoop HDFS services Namenode, Datanode, and ZKFC. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Additionally restarts Hadoop Httpfs. | | hcatalog-env | Change values in HCatalog's environment. | Restarts Hive HCatalog Server. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | Restarts Hive HCatalog Server. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | Restarts Hive HCatalog Server. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | Restarts Hive WebHCat server. | | hcatalog-webhcat-log4j2 | Change values in HCatalog WebHCat's log4j2.properties. | Restarts Hive WebHCat server. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | Restarts Hive WebHCat server. | | hive | Amazon EMR-curated settings for Apache Hive. | Sets configurations to launch Hive LLAP service. | | hive-beeline-log4j2 | Change values in Hive's beeline-log4j2.properties file. | Not available. | | hive-parquet-logging | Change values in Hive's parquet-logging.properties file. | Not available. | | hive-env | Change values in the Hive environment. | Restarts HiveServer2, HiveMetastore, and Hive HCatalog-Server. Runs Hive schemaTool CLI commands to verify hive-metastore. | | hive-exec-log4j2 | Change values in Hive's hive-exec-log4j2.properties file. | Not available. | | hive-llap-daemon-log4j2 | Change values in Hive's llap-daemon-log4j2.properties file. | Not available. | | hive-log4j2 | Change values in Hive's hive-log4j2.properties file. | Not available. | | hive-site | Change values in Hive's hive-site.xml file | Restarts HiveServer2, HiveMetastore, and Hive HCatalog-Server. Runs Hive schemaTool CLI commands to verify hive-metastore. Also restarts Oozie and Zeppelin. | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | Not available. | | hue-ini | Change values in Hue's ini file | Restarts Hue. Also activates Hue config override CLI commands to pick up new configurations. | | httpfs-env | Change values in the HTTPFS environment. | Restarts Hadoop Httpfs service. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | Restarts Hadoop Httpfs service. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | Not available. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | Restarts Hadoop-KMS service. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | Not available. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | Restarts Hadoop-KMS and Ranger-KMS service. | | hudi-env | Change values in the Hudi environment. | Not available. | | jupyter-notebook-conf | Change values in Jupyter Notebook's jupyter\$1notebook\$1config.py file. | Not available. | | jupyter-hub-conf | Change values in JupyterHubs's jupyterhub\$1config.py file. | Not available. | | jupyter-s3-conf | Configure Jupyter Notebook S3 persistence. | Not available. | | jupyter-sparkmagic-conf | Change values in Sparkmagic's config.json file. | Not available. | | livy-conf | Change values in Livy's livy.conf file. | Restarts Livy Server. | | livy-env | Change values in the Livy environment. | Restarts Livy Server. | | livy-log4j | Change Livy log4j.properties settings. | Restarts Livy Server. | | mapred-env | Change values in the MapReduce application's environment. | Restarts Hadoop MapReduce-HistoryServer. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | Restarts Hadoop MapReduce-HistoryServer. | | oozie-env | Change values in Oozie's environment. | Restarts Oozie. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | Restarts Oozie. | | oozie-site | Change values in Oozie's oozie-site.xml file. | Restarts Oozie. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | Not available. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | Not available. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | Restarts Phoenix-QueryServer. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | Not available. | | pig-env | Change values in the Pig environment. | Not available. | | pig-properties | Change values in Pig's pig.properties file. | Restarts Oozie. | | pig-log4j | Change values in Pig's log4j.properties file. | Not available. | | presto-log | Change values in Presto's log.properties file. | Restarts Presto-Server (for PrestoDB) | | presto-config | Change values in Presto's config.properties file. | Restarts Presto-Server (for PrestoDB) | | presto-password-authenticator | Change values in Presto's password-authenticator.properties file. | Not available. | | presto-env | Change values in Presto's presto-env.sh file. | Restarts Presto-Server (for PrestoDB) | | presto-node | Change values in Presto's node.properties file. | Not available. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | Not available. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | Not available. | | presto-connector-hive | Change values in Presto's hive.properties file. | Restarts Presto-Server (for PrestoDB) | | presto-connector-jmx | Change values in Presto's jmx.properties file. | Not available. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | Not available. | | presto-connector-localfile | Change values in Presto's localfile.properties file. | Not available. | | presto-connector-memory | Change values in Presto's memory.properties file. | Not available. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | Not available. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | Not available. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | Not available. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | Not available. | | presto-connector-redis | Change values in Presto's redis.properties file. | Not available. | | presto-connector-redshift | Change values in Presto's redshift.properties file. | Not available. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | Not available. | | presto-connector-tpcds | Change values in Presto's tpcds.properties file. | Not available. | | prestosql-log | Change values in Presto's log.properties file. | Restarts Presto-Server (for PrestoSQL) | | prestosql-config | Change values in Presto's config.properties file. | Restarts Presto-Server (for PrestoSQL) | | prestosql-password-authenticator | Change values in Presto's password-authenticator.properties file. | Restarts Presto-Server (for PrestoSQL) | | prestosql-env | Change values in Presto's presto-env.sh file. | Restarts Presto-Server (for PrestoSQL) | | prestosql-node | Change values in PrestoSQL's node.properties file. | Not available. | | prestosql-connector-blackhole | Change values in PrestoSQL's blackhole.properties file. | Not available. | | prestosql-connector-cassandra | Change values in PrestoSQL's cassandra.properties file. | Not available. | | prestosql-connector-hive | Change values in PrestoSQL's hive.properties file. | Restarts Presto-Server (for PrestoSQL) | | prestosql-connector-jmx | Change values in PrestoSQL's jmx.properties file. | Not available. | | prestosql-connector-kafka | Change values in PrestoSQL's kafka.properties file. | Not available. | | prestosql-connector-localfile | Change values in PrestoSQL's localfile.properties file. | Not available. | | prestosql-connector-memory | Change values in PrestoSQL's memory.properties file. | Not available. | | prestosql-connector-mongodb | Change values in PrestoSQL's mongodb.properties file. | Not available. | | prestosql-connector-mysql | Change values in PrestoSQL's mysql.properties file. | Not available. | | prestosql-connector-postgresql | Change values in PrestoSQL's postgresql.properties file. | Not available. | | prestosql-connector-raptor | Change values in PrestoSQL's raptor.properties file. | Not available. | | prestosql-connector-redis | Change values in PrestoSQL's redis.properties file. | Not available. | | prestosql-connector-redshift | Change values in PrestoSQL's redshift.properties file. | Not available. | | prestosql-connector-tpch | Change values in PrestoSQL's tpch.properties file. | Not available. | | prestosql-connector-tpcds | Change values in PrestoSQL's tpcds.properties file. | Not available. | | ranger-kms-dbks-site | Change values in dbks-site.xml file of Ranger KMS. | Restarts Ranger KMS Server. | | ranger-kms-site | Change values in ranger-kms-site.xml file of Ranger KMS. | Restarts Ranger KMS Server. | | ranger-kms-env | Change values in the Ranger KMS environment. | Restarts Ranger KMS Server. | | ranger-kms-log4j | Change values in kms-log4j.properties file of Ranger KMS. | Not available. | | ranger-kms-db-ca | Change values for CA file on S3 for MySQL SSL connection with Ranger KMS. | Not available. | | spark | Amazon EMR-curated settings for Apache Spark. | This property modifies spark-defaults. See actions there. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | Restarts Spark history server and Spark thrift server. | | spark-env | Change values in the Spark environment. | Restarts Spark history server and Spark thrift server. | | spark-hive-site | Change values in Spark's hive-site.xml file | Not available. | | spark-log4j | Change values in Spark's log4j.properties file. | Restarts Spark history server and Spark thrift server. | | spark-metrics | Change values in Spark's metrics.properties file. | Restarts Spark history server and Spark thrift server. | | sqoop-env | Change values in Sqoop's environment. | Not available. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | Not available. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | Not available. | | tez-site | Change values in Tez's tez-site.xml file. | Restart Oozie and HiveServer2. | | yarn-env | Change values in the YARN environment. | Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts MapReduce-HistoryServer. | | yarn-site | Change values in YARN's yarn-site.xml file. | Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts Livy Server and MapReduce-HistoryServer. | | zeppelin-env | Change values in the Zeppelin environment. | Restarts Zeppelin. | | zeppelin-site | Change configuration settings in zeppelin-site.xml. | Restarts Zeppelin. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | Restarts Zookeeper server. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | Restarts Zookeeper server. | # Amazon EMR release 6.3.0 ## 6.3.0 application versions This release includes the following applications: [https://flink.apache.org/](https://flink.apache.org/), [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [https://hudi.apache.org](https://hudi.apache.org), [http://gethue.com/](http://gethue.com/), [https://jupyter-enterprise-gateway.readthedocs.io/en/latest/](https://jupyter-enterprise-gateway.readthedocs.io/en/latest/), [https://jupyterhub.readthedocs.io/en/latest/#](https://jupyterhub.readthedocs.io/en/latest/#), [https://livy.incubator.apache.org/](https://livy.incubator.apache.org/), [https://mxnet.incubator.apache.org/](https://mxnet.incubator.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://prestosql.io/](https://prestosql.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://www.tensorflow.org/](https://www.tensorflow.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-6.3.0 | emr-6.2.1 | emr-6.2.0 | emr-6.1.1 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.11.977 | 1.11.880 | 1.11.880 | 1.11.828 | | Python | 2.7, 3.7 | 2.7, 3.7 | 2.7, 3.7 | 2.7, 3.7 | | Scala | 2.12.10 | 2.12.10 | 2.12.10 | 2.12.10 | | AmazonCloudWatchAgent | - | - | - | - | | Delta | - | - | - | - | | Flink | 1.12.1 | 1.11.2 | 1.11.2 | 1.11.0 | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 2.2.6-amzn-1 | 2.2.6-amzn-0 | 2.2.6-amzn-0 | 2.2.5 | | HCatalog | 3.1.2-amzn-4 | 3.1.2-amzn-3 | 3.1.2-amzn-3 | 3.1.2-amzn-2 | | Hadoop | 3.2.1-amzn-3 | 3.2.1-amzn-2.1 | 3.2.1-amzn-2 | 3.2.1-amzn-1.1 | | Hive | 3.1.2-amzn-4 | 3.1.2-amzn-3 | 3.1.2-amzn-3 | 3.1.2-amzn-2 | | Hudi | 0.7.0-amzn-0 | 0.6.0-amzn-1 | 0.6.0-amzn-1 | 0.5.2-incubating-amzn-2 | | Hue | 4.9.0 | 4.8.0 | 4.8.0 | 4.7.1 | | Iceberg | - | - | - | - | | JupyterEnterpriseGateway | 2.1.0 | 2.1.0 | 2.1.0 | - | | JupyterHub | 1.2.2 | 1.1.0 | 1.1.0 | 1.1.0 | | Livy | 0.7.0-incubating | 0.7.0-incubating | 0.7.0-incubating | 0.7.0-incubating | | MXNet | 1.7.0 | 1.7.0 | 1.7.0 | 1.6.0 | | Mahout | - | - | - | - | | Oozie | 5.2.1 | 5.2.0 | 5.2.0 | 5.2.0 | | Phoenix | 5.0.0-HBase-2.0 | 5.0.0-HBase-2.0 | 5.0.0-HBase-2.0 | 5.0.0-HBase-2.0 | | Pig | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | | Presto | 0.245.1-amzn-0 | 0.238.3-amzn-1 | 0.238.3-amzn-1 | 0.232 | | Spark | 3.1.1-amzn-0 | 3.0.1-amzn-0.1 | 3.0.1-amzn-0 | 3.0.0-amzn-0.1 | | Sqoop | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | | TensorFlow | 2.4.1 | 2.3.1 | 2.3.1 | 2.1.0 | | Tez | 0.9.2 | 0.9.2 | 0.9.2 | 0.9.2 | | Trino (PrestoSQL) | 350 | 343 | 343 | 338 | | Zeppelin | 0.9.0 | 0.9.0-preview1 | 0.9.0-preview1 | 0.9.0-preview1 | | ZooKeeper | 3.4.14 | 3.4.14 | 3.4.14 | 3.4.14 | ## 6.3.0 release notes The following release notes include information for Amazon EMR release 6.3.0. Changes are relative to 6.2.0. Initial release date: May 12, 2021 Last updated date: August 9, 2021 **Supported applications** + AWS SDK for Java version 1.11.977 + CloudWatch Sink version 2.1.0 + DynamoDB Connector version 4.16.0 + EMRFS version 2.46.0 + Amazon EMR Goodies version 3.2.0 + Amazon EMR Kinesis Connector version 3.5.0 + Amazon EMR Record Server version 2.0.0 + Amazon EMR Scripts version 2.5.0 + Flink version 1.12.1 + Ganglia version 3.7.2 + AWS Glue Hive Metastore Client version 3.2.0 + Hadoop version 3.2.1-amzn-3 + HBase version 2.2.6-amzn-1 + HBase-operator-tools 1.0.0 + HCatalog version 3.1.2-amzn-0 + Hive version 3.1.2-amzn-4 + Hudi version 0.7.0-amzn-0 + Hue version 4.9.0 + Java JDK version Corretto-8.282.08.1 (build 1.8.0\$1282-b08) + JupyterHub version 1.2.0 + Livy version 0.7.0-incubating + MXNet version 1.7.0 + Oozie version 5.2.1 + Phoenix version 5.0.0 + Pig version 0.17.0 + Presto version 0.245.1-amzn-0 + PrestoSQL version 350 + Apache Ranger KMS (multi-master transparent encryption) version 2.0.0 + ranger-plugins 2.0.1-amzn-0 + ranger-s3-plugin 1.1.0 + SageMaker Spark SDK version 1.4.1 + Scala version 2.12.10 (OpenJDK 64-Bit Server VM, Java 1.8.0\$1282) + Spark version 3.1.1-amzn-0 + spark-rapids 0.4.1 + Sqoop version 1.4.7 + TensorFlow version 2.4.1 + tez version 0.9.2 + Zeppelin version 0.9.0 + Zookeeper version 3.4.14 + Connectors and drivers: DynamoDB Connector 4.16.0 **New features** + Amazon EMR supports Amazon S3 Access Points, a feature of Amazon S3 that allows you to easily manage access for shared data lakes. Using your Amazon S3 Access Point alias, you can simplify your data access at scale on Amazon EMR. You can use Amazon S3 Access Points with all versions of Amazon EMR at no additional cost in all AWS regions where Amazon EMR is available. To learn more about Amazon S3 Access Points and Access Point aliases, see [Using a bucket-style alias for your access point](https://docs.aws.amazon.com/AmazonS3/latest/userguide/access-points-alias.html) in the *Amazon S3 User Guide*. + New `DescribeReleaseLabel` and `ListReleaseLabel` API parameters provide Amazon EMR release label details. You can programmatically list releases available in the region where the API request is run, and list the available applications for a specific Amazon EMR release label. The release label parameters also list Amazon EMR releases that support a specified application, such as Spark. This information can be used to programmatically launch Amazon EMR clusters. For example, you can launch a cluster using the latest release version from the `ListReleaseLabel` results. For more information, see [DescribeReleaseLabel](https://docs.aws.amazon.com/emr/latest/APIReference/API_DescribeReleaseLabel.html) and [ListReleaseLabels](https://docs.aws.amazon.com/emr/latest/APIReference/API_ListReleaseLabels.html) in the *Amazon EMR API Reference*. + With Amazon EMR 6.3.0, you can launch a cluster that natively integrates with Apache Ranger. Apache Ranger is an open-source framework to enable, monitor, and manage comprehensive data security across the Hadoop platform. For more information, see [Apache Ranger](https://ranger.apache.org/). With native integration, you can bring your own Apache Ranger to enforce fine-grained data access control on Amazon EMR. See [Integrate Amazon EMR with Apache Ranger](https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-ranger.html) in the Amazon EMR Management Guide. + Scoped managed policies: To align with AWS best practices, Amazon EMR has introduced v2 EMR-scoped default managed policies as replacements for policies that will be deprecated. See [Amazon EMR Managed Policies](https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-managed-iam-policies.html). + Instance Metadata Service (IMDS) V2 support status: For Amazon EMR 6.2 or later, Amazon EMR components use IMDSv2 for all IMDS calls. For IMDS calls in your application code, you can use both IMDSv1 and IMDSv2, or configure the IMDS to use only IMDSv2 for added security. If you disable IMDSv1 in earlier Amazon EMR 6.x releases, it causes cluster startup failure. **Changes, enhancements, and resolved issues** + This is a release to fix issues with Amazon EMR Scaling when it fails to scale up/scale down a cluster successfully or causes application failures. + Fixed an issue where scaling requests failed for a large, highly utilized cluster when Amazon EMR on-cluster daemons were running health checking activities, such as gathering YARN node state and HDFS node state. This was happening because on-cluster daemons were not able to communicate the health status data of a node to internal Amazon EMR components. + Improved EMR on-cluster daemons to correctly track the node states when IP addresses are reused to improve reliability during scaling operations. + [SPARK-29683](https://issues.apache.org/jira/browse/SPARK-29683). Fixed an issue where job failures occurred during cluster scale-down as Spark was assuming all available nodes were deny-listed. + [YARN-9011](https://issues.apache.org/jira/browse/YARN-9011). Fixed an issue where job failures occurred due to a race condition in YARN decommissioning when cluster tried to scale up or down. + Fixed issue with step or job failures during cluster scaling by ensuring that the node states are always consistent between the Amazon EMR on-cluster daemons and YARN/HDFS. + Fixed an issue where cluster operations such as scale down and step submission failed for Amazon EMR clusters enabled with Kerberos authentication. This was because the Amazon EMR on-cluster daemon did not renew the Kerberos ticket, which is required to securely communicate with HDFS/YARN running on the primary node. + Newer Amazon EMR releases fix the issue with a lower "Max open files" limit on older AL2 in Amazon EMR. Amazon EMR releases 5.30.1, 5.30.2, 5.31.1, 5.32.1, 6.0.1, 6.1.1, 6.2.1, 5.33.0, 6.3.0 and later now include a permanent fix with a higher "Max open files" setting. + Spark SQL UI explain mode default changed from `extended` to `formatted` in [Spark 3.1](https://issues.apache.org/jira/browse/SPARK-31325). Amazon EMR reverted it back to `extended` to include logical plan information in the Spark SQL UI. This can be reverted by setting `spark.sql.ui.explainMode` to `formatted`. + The following commits were backported from the Spark master branch. - [[SPARK-34752]](https://issues.apache.org/jira/browse/SPARK-34752)[BUILD] Bump Jetty to 9.4.37 to address CVE-2020-27223. - [[SPARK-34534]](https://issues.apache.org/jira/browse/SPARK-34534) Fix blockIds order when use FetchShuffleBlocks to fetch blocks. - [[SPARK-34681]](https://issues.apache.org/jira/browse/SPARK-34681) [SQL] Fix bug for full outer shuffled hash join when building left side with non-equal condition. - [[SPARK-34497]](https://issues.apache.org/jira/browse/SPARK-34497) [SQL] Fix built-in JDBC connection providers to restore JVM security context changes. + To improve interoperability with Nvidia Spark RAPIDs plugin, Added workaround to address an issue preventing dynamic partition pruning from triggering when using Nvidia Spark RAPIDs with adaptive query execution disabled, see [RAPIDS issue \$11378](https://github.com/NVIDIA/spark-rapids/issues/1378) and [RAPIDS issue \$1\$11386](https://github.com/NVIDIA/spark-rapids/issues/1386). For details of the new configuration `spark.sql.optimizer.dynamicPartitionPruning.enforceBroadcastReuse`, see [RAPIDS issue \$1\$11386](https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-spark-performance.html#emr-spark-performance-dynamic). + The file output committer default algorithm has been changed from the v2 algorithm to the v1 algorithm in open source Spark 3.1. For more information, see this [Amazon EMR optimizing Spark performance - dynamic partition pruning](https://issues.apache.org/jira/browse/SPARK-33019). + Amazon EMR reverted to the v2 algorithm, the default used in prior Amazon EMR 6.x releases, to prevent performance regression. To restore the open source Spark 3.1 behavior, set `spark.hadoop.mapreduce.fileoutputcommitter.algorithm.version` to `1`. Open source Spark made this change because task commit in file output committer algorithm v2 is not atomic, which can cause an output data correctness issue in some cases. However, task commit in algorithm v1 is also not atomic. In some scenarios task commit includes a delete performed before a rename. This can result in a silent data correctness issue. + Fixed Managed Scaling issues in earlier Amazon EMR releases and made improvements so application failure rates are significantly reduced. + Installed the AWS Java SDK Bundle on each new cluster. This is a single jar containing all service SDKs and their dependencies, instead of individual component jars. For more information, see [Java SDK Bundled Dependency](https://aws.amazon.com/blogs/developer/java-sdk-bundle/). **Known issues** + For Amazon EMR 6.3.0 and 6.2.0 private subnet clusters, you cannot access the Ganglia web UI. You will get an "access denied (403)" error. Other web UIs, such as Spark, Hue, JupyterHub, Zeppelin, Livy, and Tez are working normally. Ganglia web UI access on public subnet clusters are also working normally. To resolve this issue, restart httpd service on the primary node with `sudo systemctl restart httpd`. This issue is fixed in Amazon EMR 6.4.0. + When AWS Glue Data Catalog is enabled, using Spark to access a AWS Glue DB with null string location URI may fail. This happens to earlier Amazon EMR releases, but SPARK-31709 (https://issues.apache.org/jira/browse/SPARK-31709) makes it apply to more cases. For example, when creating a table within the default AWS Glue DB whose location URI is a null string, `spark.sql("CREATE TABLE mytest (key string) location '/table_path';")` fails with the message, "Cannot create a Path from an empty string." To work around this, manually set a location URI of your AWS Glue databases, then create tables within these databases using Spark. + In Amazon EMR 6.3.0, PrestoSQL has upgraded from version 343 to version 350. There are two security related changes from the open source that relate to this version change. File-based catalog access control is changed from `deny` to `allow` when table, schema, or session property rules are not defined. Also, file-based system access control is changed to support files without catalog rules defined. In this case, all access to catalogs is allowed. For more information, see [Release 344 (9 Oct 2020)](https://trino.io/docs/current/release/release-344.html#security). + Note that the Hadoop user directory (/home/hadoop) is readable by everyone. It has Unix 755 (drwxr-xr-x) directory permissions to allow read access by frameworks like Hive. You can put files in /home/hadoop and its subdirectories, but be aware of the permissions on those directories to protect sensitive information. + **Lower "Max open files" limit on older AL2 [fixed in newer releases].** Amazon EMR releases: emr-5.30.x, emr-5.31.0, emr-5.32.0, emr-6.0.0, emr-6.1.0, and emr-6.2.0 are based on older versions ofAmazon Linux 2 (AL2), which have a lower ulimit setting for "Max open files" when Amazon EMR clusters are created with the default AMI. Amazon EMR releases 5.30.1, 5.30.2, 5.31.1, 5.32.1, 6.0.1, 6.1.1, 6.2.1, 5.33.0, 6.3.0 and later include a permanent fix with a higher "Max open files" setting. Releases with the lower open file limit causes a "Too many open files" error when submitting Spark job. In the impacted releases, the Amazon EMR default AMI has a default ulimit setting of 4096 for "Max open files," which is lower than the 65536 file limit in the latestAmazon Linux 2 AMI. The lower ulimit setting for "Max open files" causes Spark job failure when the Spark driver and executor try to open more than 4096 files. To fix the issue, Amazon EMR has a bootstrap action (BA) script that adjusts the ulimit setting at cluster creation. If you are using an older Amazon EMR version that doesn't have the permanent fix for this issue, the following workaround lets you to explicitly set the instance-controller ulimit to a maximum of 65536 files. **Explicitly set a ulimit from the command line** 1. Edit `/etc/systemd/system/instance-controller.service` to add the following parameters to Service section. `LimitNOFILE=65536` `LimitNPROC=65536` 1. Restart InstanceController `$ sudo systemctl daemon-reload` `$ sudo systemctl restart instance-controller` **Set a ulimit using bootstrap action (BA)** You can also use a bootstrap action (BA) script to configure the instance-controller ulimit to 65536 files at cluster creation. ``` #!/bin/bash for user in hadoop spark hive; do sudo tee /etc/security/limits.d/$user.conf << EOF $user - nofile 65536 $user - nproc 65536 EOF done for proc in instancecontroller logpusher; do sudo mkdir -p /etc/systemd/system/$proc.service.d/ sudo tee /etc/systemd/system/$proc.service.d/override.conf << EOF [Service] LimitNOFILE=65536 LimitNPROC=65536 EOF pid=$(pgrep -f aws157.$proc.Main) sudo prlimit --pid $pid --nofile=65535:65535 --nproc=65535:65535 done sudo systemctl daemon-reload ``` + **Important** EMR clusters that run Amazon Linux or Amazon Linux 2 Amazon Machine Images (AMIs) use default Amazon Linux behavior, and do not automatically download and install important and critical kernel updates that require a reboot. This is the same behavior as other Amazon EC2 instances that run the default Amazon Linux AMI. If new Amazon Linux software updates that require a reboot (such as kernel, NVIDIA, and CUDA updates) become available after an Amazon EMR release becomes available, EMR cluster instances that run the default AMI do not automatically download and install those updates. To get kernel updates, you can [customize your Amazon EMR AMI](https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-custom-ami.html) to [use the latest Amazon Linux AMI](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/finding-an-ami.html). + To use Spark actions with Apache Oozie, you must add the following configuration to your Oozie `workflow.xml` file. Otherwise, several critical libraries such as Hadoop and EMRFS will be missing from the classpath of the Spark executors that Oozie launches. ``` --conf spark.yarn.populateHadoopClasspath=true ``` + When you use Spark with Hive partition location formatting to read data in Amazon S3, and you run Spark on Amazon EMR releases 5.30.0 to 5.36.0, and 6.2.0 to 6.9.0, you might encounter an issue that prevents your cluster from reading data correctly. This can happen if your partitions have all of the following characteristics: + Two or more partitions are scanned from the same table. + At least one partition directory path is a prefix of at least one other partition directory path, for example, `s3://bucket/table/p=a` is a prefix of `s3://bucket/table/p=a b`. + The first character that follows the prefix in the other partition directory has a UTF-8 value that’s less than than the `/` character (U\$1002F). For example, the space character (U\$10020) that occurs between a and b in `s3://bucket/table/p=a b` falls into this category. Note that there are 14 other non-control characters: `!"#$%&‘()*+,-`. For more information, see [UTF-8 encoding table and Unicode characters](https://www.utf8-chartable.de/). As a workaround to this issue, set the `spark.sql.sources.fastS3PartitionDiscovery.enabled` configuration to `false` in the `spark-defaults` classification. ## 6.3.0 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | aws-sagemaker-spark-sdk | 1.4.1 | Amazon SageMaker Spark SDK | | emr-ddb | 4.16.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 3.2.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.5.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-notebook-env | 1.2.0 | Conda env for emr notebook which includes jupyter enterprise gateway | | emr-s3-dist-cp | 2.18.0 | Distributed copy application optimized for Amazon S3. | | emr-s3-select | 2.1.0 | EMR S3Select Connector | | emrfs | 2.46.0 | Amazon S3 connector for Hadoop ecosystem applications. | | flink-client | 1.12.1 | Apache Flink command line client scripts and applications. | | flink-jobmanager-config | 1.12.1 | Managing resources on EMR nodes for Apache Flink JobManager. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 3.2.1-amzn-3 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 3.2.1-amzn-3 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 3.2.1-amzn-3 | HDFS command-line client and library | | hadoop-hdfs-namenode | 3.2.1-amzn-3 | HDFS service for tracking file names and block locations. | | hadoop-hdfs-journalnode | 3.2.1-amzn-3 | HDFS service for managing the Hadoop filesystem journal on HA clusters. | | hadoop-httpfs-server | 3.2.1-amzn-3 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 3.2.1-amzn-3 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 3.2.1-amzn-3 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 3.2.1-amzn-3 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 3.2.1-amzn-3 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 3.2.1-amzn-3 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 2.2.6-amzn-1 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 2.2.6-amzn-1 | Service for serving one or more HBase regions. | | hbase-client | 2.2.6-amzn-1 | HBase command-line client. | | hbase-rest-server | 2.2.6-amzn-1 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 2.2.6-amzn-1 | Service providing a Thrift endpoint to HBase. | | hcatalog-client | 3.1.2-amzn-4 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 3.1.2-amzn-4 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 3.1.2-amzn-4 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 3.1.2-amzn-4 | Hive command line client. | | hive-hbase | 3.1.2-amzn-4 | Hive-hbase client. | | hive-metastore-server | 3.1.2-amzn-4 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server2 | 3.1.2-amzn-4 | Service for accepting Hive queries as web requests. | | hudi | 0.7.0-amzn-0 | Incremental processing framework to power data pipline at low latency and high efficiency. | | hudi-presto | 0.7.0-amzn-0 | Bundle library for running Presto with Hudi. | | hudi-prestosql | 0.7.0-amzn-0 | Bundle library for running PrestoSQL with Hudi. | | hudi-spark | 0.7.0-amzn-0 | Bundle library for running Spark with Hudi. | | hue-server | 4.9.0 | Web application for analyzing data using Hadoop ecosystem applications | | jupyterhub | 1.2.2 | Multi-user server for Jupyter notebooks | | livy-server | 0.7.0-incubating | REST interface for interacting with Apache Spark | | nginx | 1.12.1 | nginx [engine x] is an HTTP and reverse proxy server | | mxnet | 1.7.0 | A flexible, scalable, and efficient library for deep learning. | | mariadb-server | 5.5.68\$1 | MariaDB database server. | | nvidia-cuda | 10.1.243 | Nvidia drivers and Cuda toolkit | | oozie-client | 5.2.1 | Oozie command-line client. | | oozie-server | 5.2.1 | Service for accepting Oozie workflow requests. | | opencv | 4.5.0 | Open Source Computer Vision Library. | | phoenix-library | 5.0.0-HBase-2.0 | The phoenix libraries for server and client | | phoenix-query-server | 5.0.0-HBase-2.0 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.245.1-amzn-0 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.245.1-amzn-0 | Service for executing pieces of a query. | | presto-client | 0.245.1-amzn-0 | Presto command-line client which is installed on an HA cluster's stand-by masters where Presto server is not started. | | prestosql-coordinator | 350 | Service for accepting queries and managing query execution among prestosql-workers. | | prestosql-worker | 350 | Service for executing pieces of a query. | | prestosql-client | 350 | Presto command-line client which is installed on an HA cluster's stand-by masters where Presto server is not started. | | pig-client | 0.17.0 | Pig command-line client. | | r | 4.0.2 | The R Project for Statistical Computing | | ranger-kms-server | 2.0.0 | Apache Ranger Key Management System | | spark-client | 3.1.1-amzn-0 | Spark command-line clients. | | spark-history-server | 3.1.1-amzn-0 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 3.1.1-amzn-0 | In-memory execution engine for YARN. | | spark-yarn-slave | 3.1.1-amzn-0 | Apache Spark libraries needed by YARN slaves. | | spark-rapids | 0.4.1 | Nvidia Spark RAPIDS plugin that accelerates Apache Spark with GPUs. | | sqoop-client | 1.4.7 | Apache Sqoop command-line client. | | tensorflow | 2.4.1 | TensorFlow open source software library for high performance numerical computation. | | tez-on-yarn | 0.9.2 | The tez YARN application and libraries. | | webserver | 2.4.41\$1 | Apache HTTP server. | | zeppelin-server | 0.9.0 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.4.14 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.4.14 | ZooKeeper command line client. | ## 6.3.0 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). Reconfiguration actions occur when you specify a configuration for instance groups in a running cluster. Amazon EMR only initiates reconfiguration actions for the classifications that you modify. For more information, see [Reconfigure an instance group in a running cluster](emr-configure-apps-running-cluster.md). **emr-6.3.0 classifications** | Classifications | Description | Reconfiguration Actions | | --- | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | Restarts the ResourceManager service. | | container-executor | Change values in Hadoop YARN's container-executor.cfg file. | Not available. | | container-log4j | Change values in Hadoop YARN's container-log4j.properties file. | Not available. | | core-site | Change values in Hadoop's core-site.xml file. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts Hadoop KMS, Ranger KMS, HiveServer2, Hive MetaStore, Hadoop Httpfs, and MapReduce-HistoryServer. | | docker-conf | Change docker related settings. | Not available. | | emrfs-site | Change EMRFS settings. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts HBaseRegionserver, HBaseMaster, HBaseThrift, HBaseRest, HiveServer2, Hive MetaStore, Hadoop Httpfs, and MapReduce-HistoryServer. | | flink-conf | Change flink-conf.yaml settings. | Restarts Flink history server. | | flink-log4j | Change Flink log4j.properties settings. | Restarts Flink history server. | | flink-log4j-session | Change Flink log4j-session.properties settings for Kubernetes/Yarn session. | Restarts Flink history server. | | flink-log4j-cli | Change Flink log4j-cli.properties settings. | Restarts Flink history server. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts PhoenixQueryserver, HiveServer2, Hive MetaStore, and MapReduce-HistoryServer. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | Restarts the Hadoop HDFS services SecondaryNamenode, Datanode, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts Hadoop KMS, Hadoop Httpfs, and MapReduce-HistoryServer. | | hadoop-ssl-server | Change hadoop ssl server configuration | Not available. | | hadoop-ssl-client | Change hadoop ssl client configuration | Not available. | | hbase | Amazon EMR-curated settings for Apache HBase. | Custom EMR specific property. Sets emrfs-site and hbase-site configs. See those for their associated restarts. | | hbase-env | Change values in HBase's environment. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbase.properties file. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | Not available. | | hbase-site | Change values in HBase's hbase-site.xml file. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. Additionally restarts Phoenix QueryServer. | | hdfs-encryption-zones | Configure HDFS encryption zones. | This classification should not be reconfigured. | | hdfs-env | Change values in the HDFS environment. | Restarts Hadoop HDFS services Namenode, Datanode, and ZKFC. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Additionally restarts Hadoop Httpfs. | | hcatalog-env | Change values in HCatalog's environment. | Restarts Hive HCatalog Server. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | Restarts Hive HCatalog Server. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | Restarts Hive HCatalog Server. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | Restarts Hive WebHCat server. | | hcatalog-webhcat-log4j2 | Change values in HCatalog WebHCat's log4j2.properties. | Restarts Hive WebHCat server. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | Restarts Hive WebHCat server. | | hive | Amazon EMR-curated settings for Apache Hive. | Sets configurations to launch Hive LLAP service. | | hive-beeline-log4j2 | Change values in Hive's beeline-log4j2.properties file. | Not available. | | hive-parquet-logging | Change values in Hive's parquet-logging.properties file. | Not available. | | hive-env | Change values in the Hive environment. | Restarts HiveServer2, HiveMetastore, and Hive HCatalog-Server. Runs Hive schemaTool CLI commands to verify hive-metastore. | | hive-exec-log4j2 | Change values in Hive's hive-exec-log4j2.properties file. | Not available. | | hive-llap-daemon-log4j2 | Change values in Hive's llap-daemon-log4j2.properties file. | Not available. | | hive-log4j2 | Change values in Hive's hive-log4j2.properties file. | Not available. | | hive-site | Change values in Hive's hive-site.xml file | Restarts HiveServer2, HiveMetastore, and Hive HCatalog-Server. Runs Hive schemaTool CLI commands to verify hive-metastore. Also restarts Oozie and Zeppelin. | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | Not available. | | hue-ini | Change values in Hue's ini file | Restarts Hue. Also activates Hue config override CLI commands to pick up new configurations. | | httpfs-env | Change values in the HTTPFS environment. | Restarts Hadoop Httpfs service. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | Restarts Hadoop Httpfs service. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | Not available. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | Restarts Hadoop-KMS service. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | Not available. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | Restarts Hadoop-KMS and Ranger-KMS service. | | hudi-env | Change values in the Hudi environment. | Not available. | | jupyter-notebook-conf | Change values in Jupyter Notebook's jupyter\$1notebook\$1config.py file. | Not available. | | jupyter-hub-conf | Change values in JupyterHubs's jupyterhub\$1config.py file. | Not available. | | jupyter-s3-conf | Configure Jupyter Notebook S3 persistence. | Not available. | | jupyter-sparkmagic-conf | Change values in Sparkmagic's config.json file. | Not available. | | livy-conf | Change values in Livy's livy.conf file. | Restarts Livy Server. | | livy-env | Change values in the Livy environment. | Restarts Livy Server. | | livy-log4j | Change Livy log4j.properties settings. | Restarts Livy Server. | | mapred-env | Change values in the MapReduce application's environment. | Restarts Hadoop MapReduce-HistoryServer. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | Restarts Hadoop MapReduce-HistoryServer. | | oozie-env | Change values in Oozie's environment. | Restarts Oozie. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | Restarts Oozie. | | oozie-site | Change values in Oozie's oozie-site.xml file. | Restarts Oozie. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | Not available. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | Not available. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | Restarts Phoenix-QueryServer. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | Not available. | | pig-env | Change values in the Pig environment. | Not available. | | pig-properties | Change values in Pig's pig.properties file. | Restarts Oozie. | | pig-log4j | Change values in Pig's log4j.properties file. | Not available. | | presto-log | Change values in Presto's log.properties file. | Restarts Presto-Server (for PrestoDB) | | presto-config | Change values in Presto's config.properties file. | Restarts Presto-Server (for PrestoDB) | | presto-password-authenticator | Change values in Presto's password-authenticator.properties file. | Not available. | | presto-env | Change values in Presto's presto-env.sh file. | Restarts Presto-Server (for PrestoDB) | | presto-node | Change values in Presto's node.properties file. | Not available. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | Not available. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | Not available. | | presto-connector-hive | Change values in Presto's hive.properties file. | Restarts Presto-Server (for PrestoDB) | | presto-connector-jmx | Change values in Presto's jmx.properties file. | Not available. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | Not available. | | presto-connector-localfile | Change values in Presto's localfile.properties file. | Not available. | | presto-connector-memory | Change values in Presto's memory.properties file. | Not available. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | Not available. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | Not available. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | Not available. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | Not available. | | presto-connector-redis | Change values in Presto's redis.properties file. | Not available. | | presto-connector-redshift | Change values in Presto's redshift.properties file. | Not available. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | Not available. | | presto-connector-tpcds | Change values in Presto's tpcds.properties file. | Not available. | | prestosql-log | Change values in Presto's log.properties file. | Restarts Presto-Server (for PrestoSQL) | | prestosql-config | Change values in Presto's config.properties file. | Restarts Presto-Server (for PrestoSQL) | | prestosql-password-authenticator | Change values in Presto's password-authenticator.properties file. | Restarts Presto-Server (for PrestoSQL) | | prestosql-env | Change values in Presto's presto-env.sh file. | Restarts Presto-Server (for PrestoSQL) | | prestosql-node | Change values in PrestoSQL's node.properties file. | Not available. | | prestosql-connector-blackhole | Change values in PrestoSQL's blackhole.properties file. | Not available. | | prestosql-connector-cassandra | Change values in PrestoSQL's cassandra.properties file. | Not available. | | prestosql-connector-hive | Change values in PrestoSQL's hive.properties file. | Restarts Presto-Server (for PrestoSQL) | | prestosql-connector-jmx | Change values in PrestoSQL's jmx.properties file. | Not available. | | prestosql-connector-kafka | Change values in PrestoSQL's kafka.properties file. | Not available. | | prestosql-connector-localfile | Change values in PrestoSQL's localfile.properties file. | Not available. | | prestosql-connector-memory | Change values in PrestoSQL's memory.properties file. | Not available. | | prestosql-connector-mongodb | Change values in PrestoSQL's mongodb.properties file. | Not available. | | prestosql-connector-mysql | Change values in PrestoSQL's mysql.properties file. | Not available. | | prestosql-connector-postgresql | Change values in PrestoSQL's postgresql.properties file. | Not available. | | prestosql-connector-raptor | Change values in PrestoSQL's raptor.properties file. | Not available. | | prestosql-connector-redis | Change values in PrestoSQL's redis.properties file. | Not available. | | prestosql-connector-redshift | Change values in PrestoSQL's redshift.properties file. | Not available. | | prestosql-connector-tpch | Change values in PrestoSQL's tpch.properties file. | Not available. | | prestosql-connector-tpcds | Change values in PrestoSQL's tpcds.properties file. | Not available. | | ranger-kms-dbks-site | Change values in dbks-site.xml file of Ranger KMS. | Restarts Ranger KMS Server. | | ranger-kms-site | Change values in ranger-kms-site.xml file of Ranger KMS. | Restarts Ranger KMS Server. | | ranger-kms-env | Change values in the Ranger KMS environment. | Restarts Ranger KMS Server. | | ranger-kms-log4j | Change values in kms-log4j.properties file of Ranger KMS. | Not available. | | ranger-kms-db-ca | Change values for CA file on S3 for MySQL SSL connection with Ranger KMS. | Not available. | | spark | Amazon EMR-curated settings for Apache Spark. | This property modifies spark-defaults. See actions there. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | Restarts Spark history server and Spark thrift server. | | spark-env | Change values in the Spark environment. | Restarts Spark history server and Spark thrift server. | | spark-hive-site | Change values in Spark's hive-site.xml file | Not available. | | spark-log4j | Change values in Spark's log4j.properties file. | Restarts Spark history server and Spark thrift server. | | spark-metrics | Change values in Spark's metrics.properties file. | Restarts Spark history server and Spark thrift server. | | sqoop-env | Change values in Sqoop's environment. | Not available. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | Not available. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | Not available. | | tez-site | Change values in Tez's tez-site.xml file. | Restart Oozie and HiveServer2. | | yarn-env | Change values in the YARN environment. | Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts MapReduce-HistoryServer. | | yarn-site | Change values in YARN's yarn-site.xml file. | Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts Livy Server and MapReduce-HistoryServer. | | zeppelin-env | Change values in the Zeppelin environment. | Restarts Zeppelin. | | zeppelin-site | Change configuration settings in zeppelin-site.xml. | Restarts Zeppelin. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | Restarts Zookeeper server. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | Restarts Zookeeper server. | # Amazon EMR release 6.2.1 ## 6.2.1 application versions This release includes the following applications: [https://flink.apache.org/](https://flink.apache.org/), [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [https://hudi.apache.org](https://hudi.apache.org), [http://gethue.com/](http://gethue.com/), [https://jupyter-enterprise-gateway.readthedocs.io/en/latest/](https://jupyter-enterprise-gateway.readthedocs.io/en/latest/), [https://jupyterhub.readthedocs.io/en/latest/#](https://jupyterhub.readthedocs.io/en/latest/#), [https://livy.incubator.apache.org/](https://livy.incubator.apache.org/), [https://mxnet.incubator.apache.org/](https://mxnet.incubator.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://prestosql.io/](https://prestosql.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://www.tensorflow.org/](https://www.tensorflow.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-6.2.1 | emr-6.2.0 | emr-6.1.1 | emr-6.1.0 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.11.880 | 1.11.880 | 1.11.828 | 1.11.828 | | Python | 2.7, 3.7 | 2.7, 3.7 | 2.7, 3.7 | 2.7, 3.7 | | Scala | 2.12.10 | 2.12.10 | 2.12.10 | 2.12.10 | | AmazonCloudWatchAgent | - | - | - | - | | Delta | - | - | - | - | | Flink | 1.11.2 | 1.11.2 | 1.11.0 | 1.11.0 | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 2.2.6-amzn-0 | 2.2.6-amzn-0 | 2.2.5 | 2.2.5 | | HCatalog | 3.1.2-amzn-3 | 3.1.2-amzn-3 | 3.1.2-amzn-2 | 3.1.2-amzn-2 | | Hadoop | 3.2.1-amzn-2.1 | 3.2.1-amzn-2 | 3.2.1-amzn-1.1 | 3.2.1-amzn-1 | | Hive | 3.1.2-amzn-3 | 3.1.2-amzn-3 | 3.1.2-amzn-2 | 3.1.2-amzn-2 | | Hudi | 0.6.0-amzn-1 | 0.6.0-amzn-1 | 0.5.2-incubating-amzn-2 | 0.5.2-incubating-amzn-2 | | Hue | 4.8.0 | 4.8.0 | 4.7.1 | 4.7.1 | | Iceberg | - | - | - | - | | JupyterEnterpriseGateway | 2.1.0 | 2.1.0 | - | - | | JupyterHub | 1.1.0 | 1.1.0 | 1.1.0 | 1.1.0 | | Livy | 0.7.0-incubating | 0.7.0-incubating | 0.7.0-incubating | 0.7.0-incubating | | MXNet | 1.7.0 | 1.7.0 | 1.6.0 | 1.6.0 | | Mahout | - | - | - | - | | Oozie | 5.2.0 | 5.2.0 | 5.2.0 | 5.2.0 | | Phoenix | 5.0.0-HBase-2.0 | 5.0.0-HBase-2.0 | 5.0.0-HBase-2.0 | 5.0.0-HBase-2.0 | | Pig | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | | Presto | 0.238.3-amzn-1 | 0.238.3-amzn-1 | 0.232 | 0.232 | | Spark | 3.0.1-amzn-0.1 | 3.0.1-amzn-0 | 3.0.0-amzn-0.1 | 3.0.0-amzn-0 | | Sqoop | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | | TensorFlow | 2.3.1 | 2.3.1 | 2.1.0 | 2.1.0 | | Tez | 0.9.2 | 0.9.2 | 0.9.2 | 0.9.2 | | Trino (PrestoSQL) | 343 | 343 | 338 | 338 | | Zeppelin | 0.9.0-preview1 | 0.9.0-preview1 | 0.9.0-preview1 | 0.9.0-preview1 | | ZooKeeper | 3.4.14 | 3.4.14 | 3.4.14 | 3.4.14 | ## 6.2.1 release notes This is a release to fix issues with Amazon EMR Scaling when it fails to scale up/scale down a cluster successfully or causes application failures. **Changes, Enhancements, and Resolved Issues** + Fixed an issue where scaling requests failed for a large, highly utilized cluster when Amazon EMR on-cluster daemons were running health checking activities, such as gathering YARN node state and HDFS node state. This was happening because on-cluster daemons were not able to communicate the health status data of a node to internal Amazon EMR components. + Improved EMR on-cluster daemons to correctly track the node states when IP addresses are reused to improve reliability during scaling operations. + [SPARK-29683](https://issues.apache.org/jira/browse/SPARK-29683). Fixed an issue where job failures occurred during cluster scale-down as Spark was assuming all available nodes were deny-listed. + [YARN-9011](https://issues.apache.org/jira/browse/YARN-9011). Fixed an issue where job failures occurred due to a race condition in YARN decommissioning when cluster tried to scale up or down. + Fixed issue with step or job failures during cluster scaling by ensuring that the node states are always consistent between the Amazon EMR on-cluster daemons and YARN/HDFS. + Fixed an issue where cluster operations such as scale down and step submission failed for Amazon EMR clusters enabled with Kerberos authentication. This was because the Amazon EMR on-cluster daemon did not renew the Kerberos ticket, which is required to securely communicate with HDFS/YARN running on the primary node. + Newer Amazon EMR releases fix the issue with a lower "Max open files" limit on older AL2 in Amazon EMR. Amazon EMR releases 5.30.1, 5.30.2, 5.31.1, 5.32.1, 6.0.1, 6.1.1, 6.2.1, 5.33.0, 6.3.0 and later now include a permanent fix with a higher "Max open files" setting. + HTTPS is now enabled by default for Amazon Linux repositories. If you are using an Amazon S3 VPCE policy to restrict access to specific buckets, you must add the new Amazon Linux bucket ARN `arn:aws:s3:::amazonlinux-2-repos-$region/*` to your policy (replace `$region` with the region where the endpoint is). For more information, see this topic in the AWS discussion forums. [Announcement: Amazon Linux 2 now supports the ability to use HTTPS while connecting to package repositories ](https://forums.aws.amazon.com/ann.jspa?annID=8528). **Known issues** + When you use Spark with Hive partition location formatting to read data in Amazon S3, and you run Spark on Amazon EMR releases 5.30.0 to 5.36.0, and 6.2.0 to 6.9.0, you might encounter an issue that prevents your cluster from reading data correctly. This can happen if your partitions have all of the following characteristics: + Two or more partitions are scanned from the same table. + At least one partition directory path is a prefix of at least one other partition directory path, for example, `s3://bucket/table/p=a` is a prefix of `s3://bucket/table/p=a b`. + The first character that follows the prefix in the other partition directory has a UTF-8 value that’s less than than the `/` character (U\$1002F). For example, the space character (U\$10020) that occurs between a and b in `s3://bucket/table/p=a b` falls into this category. Note that there are 14 other non-control characters: `!"#$%&‘()*+,-`. For more information, see [UTF-8 encoding table and Unicode characters](https://www.utf8-chartable.de/). As a workaround to this issue, set the `spark.sql.sources.fastS3PartitionDiscovery.enabled` configuration to `false` in the `spark-defaults` classification. ## 6.2.1 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | aws-sagemaker-spark-sdk | 1.4.1 | Amazon SageMaker Spark SDK | | emr-ddb | 4.16.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 3.1.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.5.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-notebook-env | 1.0.0 | Conda env for emr notebook which includes jupyter enterprise gateway | | emr-s3-dist-cp | 2.16.0 | Distributed copy application optimized for Amazon S3. | | emr-s3-select | 2.0.0 | EMR S3Select Connector | | emrfs | 2.44.0 | Amazon S3 connector for Hadoop ecosystem applications. | | flink-client | 1.11.2 | Apache Flink command line client scripts and applications. | | flink-jobmanager-config | 1.11.2 | Managing resources on EMR nodes for Apache Flink JobManager. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 3.2.1-amzn-2.1 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 3.2.1-amzn-2.1 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 3.2.1-amzn-2.1 | HDFS command-line client and library | | hadoop-hdfs-namenode | 3.2.1-amzn-2.1 | HDFS service for tracking file names and block locations. | | hadoop-hdfs-journalnode | 3.2.1-amzn-2.1 | HDFS service for managing the Hadoop filesystem journal on HA clusters. | | hadoop-httpfs-server | 3.2.1-amzn-2.1 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 3.2.1-amzn-2.1 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 3.2.1-amzn-2.1 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 3.2.1-amzn-2.1 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 3.2.1-amzn-2.1 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 3.2.1-amzn-2.1 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 2.2.6-amzn-0 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 2.2.6-amzn-0 | Service for serving one or more HBase regions. | | hbase-client | 2.2.6-amzn-0 | HBase command-line client. | | hbase-rest-server | 2.2.6-amzn-0 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 2.2.6-amzn-0 | Service providing a Thrift endpoint to HBase. | | hcatalog-client | 3.1.2-amzn-3 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 3.1.2-amzn-3 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 3.1.2-amzn-3 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 3.1.2-amzn-3 | Hive command line client. | | hive-hbase | 3.1.2-amzn-3 | Hive-hbase client. | | hive-metastore-server | 3.1.2-amzn-3 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server2 | 3.1.2-amzn-3 | Service for accepting Hive queries as web requests. | | hudi | 0.6.0-amzn-1 | Incremental processing framework to power data pipline at low latency and high efficiency. | | hudi-presto | 0.6.0-amzn-1 | Bundle library for running Presto with Hudi. | | hudi-prestosql | 0.6.0-amzn-1 | Bundle library for running PrestoSQL with Hudi. | | hudi-spark | 0.6.0-amzn-1 | Bundle library for running Spark with Hudi. | | hue-server | 4.8.0 | Web application for analyzing data using Hadoop ecosystem applications | | jupyterhub | 1.1.0 | Multi-user server for Jupyter notebooks | | livy-server | 0.7.0-incubating | REST interface for interacting with Apache Spark | | nginx | 1.12.1 | nginx [engine x] is an HTTP and reverse proxy server | | mxnet | 1.7.0 | A flexible, scalable, and efficient library for deep learning. | | mariadb-server | 5.5.64\$1 | MariaDB database server. | | nvidia-cuda | 10.1.243 | Nvidia drivers and Cuda toolkit | | oozie-client | 5.2.0 | Oozie command-line client. | | oozie-server | 5.2.0 | Service for accepting Oozie workflow requests. | | opencv | 4.4.0 | Open Source Computer Vision Library. | | phoenix-library | 5.0.0-HBase-2.0 | The phoenix libraries for server and client | | phoenix-query-server | 5.0.0-HBase-2.0 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.238.3-amzn-1 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.238.3-amzn-1 | Service for executing pieces of a query. | | presto-client | 0.238.3-amzn-1 | Presto command-line client which is installed on an HA cluster's stand-by masters where Presto server is not started. | | prestosql-coordinator | 343 | Service for accepting queries and managing query execution among prestosql-workers. | | prestosql-worker | 343 | Service for executing pieces of a query. | | prestosql-client | 343 | Presto command-line client which is installed on an HA cluster's stand-by masters where Presto server is not started. | | pig-client | 0.17.0 | Pig command-line client. | | r | 3.4.3 | The R Project for Statistical Computing | | ranger-kms-server | 2.0.0 | Apache Ranger Key Management System | | spark-client | 3.0.1-amzn-0.1 | Spark command-line clients. | | spark-history-server | 3.0.1-amzn-0.1 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 3.0.1-amzn-0.1 | In-memory execution engine for YARN. | | spark-yarn-slave | 3.0.1-amzn-0.1 | Apache Spark libraries needed by YARN slaves. | | spark-rapids | 0.2.0 | Nvidia Spark RAPIDS plugin that accelerates Apache Spark with GPUs. | | sqoop-client | 1.4.7 | Apache Sqoop command-line client. | | tensorflow | 2.3.1 | TensorFlow open source software library for high performance numerical computation. | | tez-on-yarn | 0.9.2 | The tez YARN application and libraries. | | webserver | 2.4.41\$1 | Apache HTTP server. | | zeppelin-server | 0.9.0-preview1 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.4.14 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.4.14 | ZooKeeper command line client. | ## 6.2.1 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). Reconfiguration actions occur when you specify a configuration for instance groups in a running cluster. Amazon EMR only initiates reconfiguration actions for the classifications that you modify. For more information, see [Reconfigure an instance group in a running cluster](emr-configure-apps-running-cluster.md). **emr-6.2.1 classifications** | Classifications | Description | Reconfiguration Actions | | --- | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | Restarts the ResourceManager service. | | container-executor | Change values in Hadoop YARN's container-executor.cfg file. | Not available. | | container-log4j | Change values in Hadoop YARN's container-log4j.properties file. | Not available. | | core-site | Change values in Hadoop's core-site.xml file. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts Hadoop KMS, Ranger KMS, HiveServer2, Hive MetaStore, Hadoop Httpfs, and MapReduce-HistoryServer. | | docker-conf | Change docker related settings. | Not available. | | emrfs-site | Change EMRFS settings. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts HBaseRegionserver, HBaseMaster, HBaseThrift, HBaseRest, HiveServer2, Hive MetaStore, Hadoop Httpfs, and MapReduce-HistoryServer. | | flink-conf | Change flink-conf.yaml settings. | Not available. | | flink-log4j | Change Flink log4j.properties settings. | Not available. | | flink-log4j-yarn-session | Change Flink log4j-yarn-session.properties settings. | Not available. | | flink-log4j-cli | Change Flink log4j-cli.properties settings. | Not available. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts PhoenixQueryserver, HiveServer2, Hive MetaStore, and MapReduce-HistoryServer. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | Restarts the Hadoop HDFS services SecondaryNamenode, Datanode, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts Hadoop KMS, Hadoop Httpfs, and MapReduce-HistoryServer. | | hadoop-ssl-server | Change hadoop ssl server configuration | Not available. | | hadoop-ssl-client | Change hadoop ssl client configuration | Not available. | | hbase | Amazon EMR-curated settings for Apache HBase. | Custom EMR specific property. Sets emrfs-site and hbase-site configs. See those for their associated restarts. | | hbase-env | Change values in HBase's environment. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbase.properties file. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | Not available. | | hbase-site | Change values in HBase's hbase-site.xml file. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. Additionally restarts Phoenix QueryServer. | | hdfs-encryption-zones | Configure HDFS encryption zones. | This classification should not be reconfigured. | | hdfs-env | Change values in the HDFS environment. | Restarts Hadoop HDFS ZKFC. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Additionally restarts Hadoop Httpfs. | | hcatalog-env | Change values in HCatalog's environment. | Restarts Hive HCatalog Server. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | Restarts Hive HCatalog Server. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | Restarts Hive HCatalog Server. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | Restarts Hive WebHCat server. | | hcatalog-webhcat-log4j2 | Change values in HCatalog WebHCat's log4j2.properties. | Restarts Hive WebHCat server. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | Restarts Hive WebHCat server. | | hive | Amazon EMR-curated settings for Apache Hive. | Sets configurations to launch Hive LLAP service. | | hive-beeline-log4j2 | Change values in Hive's beeline-log4j2.properties file. | Not available. | | hive-parquet-logging | Change values in Hive's parquet-logging.properties file. | Not available. | | hive-env | Change values in the Hive environment. | Restarts HiveServer2, HiveMetastore, and Hive HCatalog-Server. Runs Hive schemaTool CLI commands to verify hive-metastore. | | hive-exec-log4j2 | Change values in Hive's hive-exec-log4j2.properties file. | Not available. | | hive-llap-daemon-log4j2 | Change values in Hive's llap-daemon-log4j2.properties file. | Not available. | | hive-log4j2 | Change values in Hive's hive-log4j2.properties file. | Not available. | | hive-site | Change values in Hive's hive-site.xml file | Restarts HiveServer2, HiveMetastore, and Hive HCatalog-Server. Runs Hive schemaTool CLI commands to verify hive-metastore. Also restarts Oozie and Zeppelin. | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | Not available. | | hue-ini | Change values in Hue's ini file | Restarts Hue. Also activates Hue config override CLI commands to pick up new configurations. | | httpfs-env | Change values in the HTTPFS environment. | Restarts Hadoop Httpfs service. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | Restarts Hadoop Httpfs service. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | Not available. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | Restarts Hadoop-KMS service. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | Not available. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | Restarts Hadoop-KMS and Ranger-KMS service. | | hudi-env | Change values in the Hudi environment. | Not available. | | jupyter-notebook-conf | Change values in Jupyter Notebook's jupyter\$1notebook\$1config.py file. | Not available. | | jupyter-hub-conf | Change values in JupyterHubs's jupyterhub\$1config.py file. | Not available. | | jupyter-s3-conf | Configure Jupyter Notebook S3 persistence. | Not available. | | jupyter-sparkmagic-conf | Change values in Sparkmagic's config.json file. | Not available. | | livy-conf | Change values in Livy's livy.conf file. | Restarts Livy Server. | | livy-env | Change values in the Livy environment. | Restarts Livy Server. | | livy-log4j | Change Livy log4j.properties settings. | Restarts Livy Server. | | mapred-env | Change values in the MapReduce application's environment. | Restarts Hadoop MapReduce-HistoryServer. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | Restarts Hadoop MapReduce-HistoryServer. | | oozie-env | Change values in Oozie's environment. | Restarts Oozie. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | Restarts Oozie. | | oozie-site | Change values in Oozie's oozie-site.xml file. | Restarts Oozie. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | Not available. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | Not available. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | Restarts Phoenix-QueryServer. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | Not available. | | pig-env | Change values in the Pig environment. | Not available. | | pig-properties | Change values in Pig's pig.properties file. | Restarts Oozie. | | pig-log4j | Change values in Pig's log4j.properties file. | Not available. | | presto-log | Change values in Presto's log.properties file. | Restarts Presto-Server (for PrestoDB) | | presto-config | Change values in Presto's config.properties file. | Restarts Presto-Server (for PrestoDB) | | presto-password-authenticator | Change values in Presto's password-authenticator.properties file. | Not available. | | presto-env | Change values in Presto's presto-env.sh file. | Restarts Presto-Server (for PrestoDB) | | presto-node | Change values in Presto's node.properties file. | Not available. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | Not available. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | Not available. | | presto-connector-hive | Change values in Presto's hive.properties file. | Restarts Presto-Server (for PrestoDB) | | presto-connector-jmx | Change values in Presto's jmx.properties file. | Not available. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | Not available. | | presto-connector-localfile | Change values in Presto's localfile.properties file. | Not available. | | presto-connector-memory | Change values in Presto's memory.properties file. | Not available. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | Not available. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | Not available. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | Not available. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | Not available. | | presto-connector-redis | Change values in Presto's redis.properties file. | Not available. | | presto-connector-redshift | Change values in Presto's redshift.properties file. | Not available. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | Not available. | | presto-connector-tpcds | Change values in Presto's tpcds.properties file. | Not available. | | prestosql-log | Change values in Presto's log.properties file. | Restarts Presto-Server (for PrestoSQL) | | prestosql-config | Change values in Presto's config.properties file. | Restarts Presto-Server (for PrestoSQL) | | prestosql-password-authenticator | Change values in Presto's password-authenticator.properties file. | Restarts Presto-Server (for PrestoSQL) | | prestosql-env | Change values in Presto's presto-env.sh file. | Restarts Presto-Server (for PrestoSQL) | | prestosql-node | Change values in PrestoSQL's node.properties file. | Not available. | | prestosql-connector-blackhole | Change values in PrestoSQL's blackhole.properties file. | Not available. | | prestosql-connector-cassandra | Change values in PrestoSQL's cassandra.properties file. | Not available. | | prestosql-connector-hive | Change values in PrestoSQL's hive.properties file. | Restarts Presto-Server (for PrestoSQL) | | prestosql-connector-jmx | Change values in PrestoSQL's jmx.properties file. | Not available. | | prestosql-connector-kafka | Change values in PrestoSQL's kafka.properties file. | Not available. | | prestosql-connector-localfile | Change values in PrestoSQL's localfile.properties file. | Not available. | | prestosql-connector-memory | Change values in PrestoSQL's memory.properties file. | Not available. | | prestosql-connector-mongodb | Change values in PrestoSQL's mongodb.properties file. | Not available. | | prestosql-connector-mysql | Change values in PrestoSQL's mysql.properties file. | Not available. | | prestosql-connector-postgresql | Change values in PrestoSQL's postgresql.properties file. | Not available. | | prestosql-connector-raptor | Change values in PrestoSQL's raptor.properties file. | Not available. | | prestosql-connector-redis | Change values in PrestoSQL's redis.properties file. | Not available. | | prestosql-connector-redshift | Change values in PrestoSQL's redshift.properties file. | Not available. | | prestosql-connector-tpch | Change values in PrestoSQL's tpch.properties file. | Not available. | | prestosql-connector-tpcds | Change values in PrestoSQL's tpcds.properties file. | Not available. | | ranger-kms-dbks-site | Change values in dbks-site.xml file of Ranger KMS. | Restarts Ranger KMS Server. | | ranger-kms-site | Change values in ranger-kms-site.xml file of Ranger KMS. | Restarts Ranger KMS Server. | | ranger-kms-env | Change values in the Ranger KMS environment. | Restarts Ranger KMS Server. | | ranger-kms-log4j | Change values in kms-log4j.properties file of Ranger KMS. | Not available. | | ranger-kms-db-ca | Change values for CA file on S3 for MySQL SSL connection with Ranger KMS. | Not available. | | spark | Amazon EMR-curated settings for Apache Spark. | This property modifies spark-defaults. See actions there. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | Restarts Spark history server and Spark thrift server. | | spark-env | Change values in the Spark environment. | Restarts Spark history server and Spark thrift server. | | spark-hive-site | Change values in Spark's hive-site.xml file | Not available. | | spark-log4j | Change values in Spark's log4j.properties file. | Restarts Spark history server and Spark thrift server. | | spark-metrics | Change values in Spark's metrics.properties file. | Restarts Spark history server and Spark thrift server. | | sqoop-env | Change values in Sqoop's environment. | Not available. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | Not available. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | Not available. | | tez-site | Change values in Tez's tez-site.xml file. | Restart Oozie. | | yarn-env | Change values in the YARN environment. | Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts MapReduce-HistoryServer. | | yarn-site | Change values in YARN's yarn-site.xml file. | Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts Livy Server and MapReduce-HistoryServer. | | zeppelin-env | Change values in the Zeppelin environment. | Restarts Zeppelin. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | Restarts Zookeeper server. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | Restarts Zookeeper server. | # Amazon EMR release 6.2.0 ## 6.2.0 application versions This release includes the following applications: [https://flink.apache.org/](https://flink.apache.org/), [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [https://hudi.apache.org](https://hudi.apache.org), [http://gethue.com/](http://gethue.com/), [https://jupyter-enterprise-gateway.readthedocs.io/en/latest/](https://jupyter-enterprise-gateway.readthedocs.io/en/latest/), [https://jupyterhub.readthedocs.io/en/latest/#](https://jupyterhub.readthedocs.io/en/latest/#), [https://livy.incubator.apache.org/](https://livy.incubator.apache.org/), [https://mxnet.incubator.apache.org/](https://mxnet.incubator.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://prestosql.io/](https://prestosql.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://www.tensorflow.org/](https://www.tensorflow.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-6.2.0 | emr-6.1.1 | emr-6.1.0 | emr-6.0.1 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.11.880 | 1.11.828 | 1.11.828 | 1.11.711 | | Python | 2.7, 3.7 | 2.7, 3.7 | 2.7, 3.7 | 2.7, 3.7 | | Scala | 2.12.10 | 2.12.10 | 2.12.10 | 2.12.10 | | AmazonCloudWatchAgent | - | - | - | - | | Delta | - | - | - | - | | Flink | 1.11.2 | 1.11.0 | 1.11.0 | - | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 2.2.6-amzn-0 | 2.2.5 | 2.2.5 | 2.2.3 | | HCatalog | 3.1.2-amzn-3 | 3.1.2-amzn-2 | 3.1.2-amzn-2 | 3.1.2-amzn-0 | | Hadoop | 3.2.1-amzn-2 | 3.2.1-amzn-1.1 | 3.2.1-amzn-1 | 3.2.1-amzn-0.1 | | Hive | 3.1.2-amzn-3 | 3.1.2-amzn-2 | 3.1.2-amzn-2 | 3.1.2-amzn-0 | | Hudi | 0.6.0-amzn-1 | 0.5.2-incubating-amzn-2 | 0.5.2-incubating-amzn-2 | 0.5.0-incubating-amzn-1 | | Hue | 4.8.0 | 4.7.1 | 4.7.1 | 4.4.0 | | Iceberg | - | - | - | - | | JupyterEnterpriseGateway | 2.1.0 | - | - | - | | JupyterHub | 1.1.0 | 1.1.0 | 1.1.0 | 1.0.0 | | Livy | 0.7.0-incubating | 0.7.0-incubating | 0.7.0-incubating | 0.6.0-incubating | | MXNet | 1.7.0 | 1.6.0 | 1.6.0 | 1.5.1 | | Mahout | - | - | - | - | | Oozie | 5.2.0 | 5.2.0 | 5.2.0 | 5.1.0 | | Phoenix | 5.0.0-HBase-2.0 | 5.0.0-HBase-2.0 | 5.0.0-HBase-2.0 | 5.0.0-HBase-2.0 | | Pig | 0.17.0 | 0.17.0 | 0.17.0 | - | | Presto | 0.238.3-amzn-1 | 0.232 | 0.232 | 0.230 | | Spark | 3.0.1-amzn-0 | 3.0.0-amzn-0.1 | 3.0.0-amzn-0 | 2.4.4 | | Sqoop | 1.4.7 | 1.4.7 | 1.4.7 | - | | TensorFlow | 2.3.1 | 2.1.0 | 2.1.0 | 1.14.0 | | Tez | 0.9.2 | 0.9.2 | 0.9.2 | 0.9.2 | | Trino (PrestoSQL) | 343 | 338 | 338 | - | | Zeppelin | 0.9.0-preview1 | 0.9.0-preview1 | 0.9.0-preview1 | 0.9.0-SNAPSHOT | | ZooKeeper | 3.4.14 | 3.4.14 | 3.4.14 | 3.4.14 | ## 6.2.0 release notes The following release notes include information for Amazon EMR release 6.2.0. Changes are relative to 6.1.0. Initial release date: Dec 09, 2020 Last updated date: Oct 04, 2021 **Supported applications** + AWS SDK for Java version 1.11.828 + emr-record-server version 1.7.0 + Flink version 1.11.2 + Ganglia version 3.7.2 + Hadoop version 3.2.1-amzn-1 + HBase version 2.2.6-amzn-0 + HBase-operator-tools 1.0.0 + HCatalog version 3.1.2-amzn-0 + Hive version 3.1.2-amzn-3 + Hudi version 0.6.0-amzn-1 + Hue version 4.8.0 + JupyterHub version 1.1.0 + Livy version 0.7.0 + MXNet version 1.7.0 + Oozie version 5.2.0 + Phoenix version 5.0.0 + Pig version 0.17.0 + Presto version 0.238.3-amzn-1 + PrestoSQL version 343 + Spark version 3.0.1-amzn-0 + spark-rapids 0.2.0 + TensorFlow version 2.3.1 + Zeppelin version 0.9.0-preview1 + Zookeeper version 3.4.14 + Connectors and drivers: DynamoDB Connector 4.16.0 **New features** + HBase: Removed rename in commit phase and added persistent HFile tracking. See [Persistent HFile Tracking](https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-hbase-s3.html#emr-hbase-s3-hfile-tracking) in the *Amazon EMR Release Guide*. + HBase: Backported [Create a config that forces to cache blocks on compaction](https://issues.apache.org/jira/browse/HBASE-23066). + PrestoDB: Improvements to Dynamic Partition Pruning. Rule-based Join Reorder works on non-partitioned data. + Scoped managed policies: To align with AWS best practices, Amazon EMR has introduced v2 EMR-scoped default managed policies as replacements for policies that will be deprecated. See [Amazon EMR Managed Policies](https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-managed-iam-policies.html). + Instance Metadata Service (IMDS) V2 support status: For Amazon EMR 6.2 or later, Amazon EMR components use IMDSv2 for all IMDS calls. For IMDS calls in your application code, you can use both IMDSv1 and IMDSv2, or configure the IMDS to use only IMDSv2 for added security. If you disable IMDSv1 in earlier Amazon EMR 6.x releases, it causes cluster startup failure. **Changes, enhancements, and resolved issues** + This is a release to fix issues with Amazon EMR Scaling when it fails to scale up/scale down a cluster successfully or causes application failures. + Fixed an issue where scaling requests failed for a large, highly utilized cluster when Amazon EMR on-cluster daemons were running health checking activities, such as gathering YARN node state and HDFS node state. This was happening because on-cluster daemons were not able to communicate the health status data of a node to internal Amazon EMR components. + Improved EMR on-cluster daemons to correctly track the node states when IP addresses are reused to improve reliability during scaling operations. + [SPARK-29683](https://issues.apache.org/jira/browse/SPARK-29683). Fixed an issue where job failures occurred during cluster scale-down as Spark was assuming all available nodes were deny-listed. + [YARN-9011](https://issues.apache.org/jira/browse/YARN-9011). Fixed an issue where job failures occurred due to a race condition in YARN decommissioning when cluster tried to scale up or down. + Fixed issue with step or job failures during cluster scaling by ensuring that the node states are always consistent between the Amazon EMR on-cluster daemons and YARN/HDFS. + Fixed an issue where cluster operations such as scale down and step submission failed for Amazon EMR clusters enabled with Kerberos authentication. This was because the Amazon EMR on-cluster daemon did not renew the Kerberos ticket, which is required to securely communicate with HDFS/YARN running on the primary node. + Newer Amazon EMR releases fix the issue with a lower "Max open files" limit on older AL2 in Amazon EMR. Amazon EMR releases 5.30.1, 5.30.2, 5.31.1, 5.32.1, 6.0.1, 6.1.1, 6.2.1, 5.33.0, 6.3.0 and later now include a permanent fix with a higher "Max open files" setting. + Spark: Performance improvements in Spark runtime. **Known issues** + Amazon EMR 6.2 has incorrect permissions set on the /etc/cron.d/libinstance-controller-java file in EMR 6.2.0. Permissions on the file are 645 (-rw-r--r-x), when they should be 644 (-rw-r--r--). As a result, Amazon EMR version 6.2 does not log instance-state logs, and the /emr/instance-logs directory is empty. This issue is fixed in Amazon EMR 6.3.0 and later. To work around this issue, run the following script as a bootstrap action at cluster launch. ``` #!/bin/bash sudo chmod 644 /etc/cron.d/libinstance-controller-java ``` + For Amazon EMR 6.2.0 and 6.3.0 private subnet clusters, you cannot access the Ganglia web UI. You will get an "access denied (403)" error. Other web UIs, such as Spark, Hue, JupyterHub, Zeppelin, Livy, and Tez are working normally. Ganglia web UI access on public subnet clusters are also working normally. To resolve this issue, restart httpd service on the primary node with `sudo systemctl restart httpd`. This issue is fixed in Amazon EMR 6.4.0. + There is an issue in Amazon EMR 6.2.0 where httpd continuously fails, causing Ganglia to be unavailable. You get a "cannot connect to the server" error. To fix a cluster that is already running with this issue, SSH to the cluster primary node and add the line `Listen 80` to the file `httpd.conf` located at `/etc/httpd/conf/httpd.conf`. This issue is fixed in Amazon EMR 6.3.0. + HTTPD fails on EMR 6.2.0 clusters when you use a security configuration. This makes the Ganglia web application user interface unavailable. To access the Ganglia web application user interface, add `Listen 80` to the `/etc/httpd/conf/httpd.conf` file on the primary node of your cluster. For information about connecting to your cluster, see [Connect to the Primary Node Using SSH](https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-connect-master-node-ssh.html). EMR Notebooks also fail to establish a connection with EMR 6.2.0 clusters when you use a security configuration. The notebook will fail to list kernels and submit Spark jobs. We recommend that you use EMR Notebooks with another version of Amazon EMR instead. + **Lower "Max open files" limit on older AL2 [fixed in newer releases].** Amazon EMR releases: emr-5.30.x, emr-5.31.0, emr-5.32.0, emr-6.0.0, emr-6.1.0, and emr-6.2.0 are based on older versions ofAmazon Linux 2 (AL2), which have a lower ulimit setting for "Max open files" when Amazon EMR clusters are created with the default AMI. Amazon EMR releases 5.30.1, 5.30.2, 5.31.1, 5.32.1, 6.0.1, 6.1.1, 6.2.1, 5.33.0, 6.3.0 and later include a permanent fix with a higher "Max open files" setting. Releases with the lower open file limit causes a "Too many open files" error when submitting Spark job. In the impacted releases, the Amazon EMR default AMI has a default ulimit setting of 4096 for "Max open files," which is lower than the 65536 file limit in the latestAmazon Linux 2 AMI. The lower ulimit setting for "Max open files" causes Spark job failure when the Spark driver and executor try to open more than 4096 files. To fix the issue, Amazon EMR has a bootstrap action (BA) script that adjusts the ulimit setting at cluster creation. If you are using an older Amazon EMR version that doesn't have the permanent fix for this issue, the following workaround lets you to explicitly set the instance-controller ulimit to a maximum of 65536 files. **Explicitly set a ulimit from the command line** 1. Edit `/etc/systemd/system/instance-controller.service` to add the following parameters to Service section. `LimitNOFILE=65536` `LimitNPROC=65536` 1. Restart InstanceController `$ sudo systemctl daemon-reload` `$ sudo systemctl restart instance-controller` **Set a ulimit using bootstrap action (BA)** You can also use a bootstrap action (BA) script to configure the instance-controller ulimit to 65536 files at cluster creation. ``` #!/bin/bash for user in hadoop spark hive; do sudo tee /etc/security/limits.d/$user.conf << EOF $user - nofile 65536 $user - nproc 65536 EOF done for proc in instancecontroller logpusher; do sudo mkdir -p /etc/systemd/system/$proc.service.d/ sudo tee /etc/systemd/system/$proc.service.d/override.conf << EOF [Service] LimitNOFILE=65536 LimitNPROC=65536 EOF pid=$(pgrep -f aws157.$proc.Main) sudo prlimit --pid $pid --nofile=65535:65535 --nproc=65535:65535 done sudo systemctl daemon-reload ``` + **Important** Amazon EMR 6.1.0 and 6.2.0 include a performance issue that can critically affect all Hudi insert, upsert, and delete operations. If you plan to use Hudi with Amazon EMR 6.1.0 or 6.2.0, you should contact AWS support to obtain a patched Hudi RPM. + **Important** EMR clusters that run Amazon Linux or Amazon Linux 2 Amazon Machine Images (AMIs) use default Amazon Linux behavior, and do not automatically download and install important and critical kernel updates that require a reboot. This is the same behavior as other Amazon EC2 instances that run the default Amazon Linux AMI. If new Amazon Linux software updates that require a reboot (such as kernel, NVIDIA, and CUDA updates) become available after an Amazon EMR release becomes available, EMR cluster instances that run the default AMI do not automatically download and install those updates. To get kernel updates, you can [customize your Amazon EMR AMI](https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-custom-ami.html) to [use the latest Amazon Linux AMI](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/finding-an-ami.html). + Amazon EMR 6.2.0 Maven artifacts are not published. They will be published with a future release of Amazon EMR. + Persistent HFile tracking using the HBase storefile system table does not support the HBase region replication feature. For more information about HBase region replication, see [Timeline-consistent High Available Reads](http://hbase.apache.org/book.html#arch.timelineconsistent.reads). + Amazon EMR 6.x and EMR 5.x Hive bucketing version differences EMR 5.x uses OOS Apache Hive 2, while in EMR 6.x uses OOS Apache Hive 3. The open source Hive2 uses Bucketing version 1, while open source Hive3 uses Bucketing version 2. This bucketing version difference between Hive 2 (EMR 5.x) and Hive 3 (EMR 6.x) means Hive bucketing hashing functions differently. See the example below. The following table is an example created in EMR 6.x and EMR 5.x, respectively. ``` -- Using following LOCATION in EMR 6.x CREATE TABLE test_bucketing (id INT, desc STRING) PARTITIONED BY (day STRING) CLUSTERED BY(id) INTO 128 BUCKETS LOCATION 's3://your-own-s3-bucket/emr-6-bucketing/'; -- Using following LOCATION in EMR 5.x LOCATION 's3://your-own-s3-bucket/emr-5-bucketing/'; ``` Inserting the same data in both EMR 6.x and EMR 5.x. ``` INSERT INTO test_bucketing PARTITION (day='01') VALUES(66, 'some_data'); INSERT INTO test_bucketing PARTITION (day='01') VALUES(200, 'some_data'); ``` Checking the S3 location, shows the bucketing file name is different, because the hashing function is different between EMR 6.x (Hive 3) and EMR 5.x (Hive 2). ``` [hadoop@ip-10-0-0-122 ~]$ aws s3 ls s3://your-own-s3-bucket/emr-6-bucketing/day=01/ 2020-10-21 20:35:16 13 000025_0 2020-10-21 20:35:22 14 000121_0 [hadoop@ip-10-0-0-122 ~]$ aws s3 ls s3://your-own-s3-bucket/emr-5-bucketing/day=01/ 2020-10-21 20:32:07 13 000066_0 2020-10-21 20:32:51 14 000072_0 ``` You can also see the version difference by running the following command in Hive CLI in EMR 6.x. Note that it returns bucketing version 2. ``` hive> DESCRIBE FORMATTED test_bucketing; ... Table Parameters: bucketing_version 2 ... ``` + Known issue in clusters with multiple primary nodes and Kerberos authentication If you run clusters with multiple primary nodes and Kerberos authentication in Amazon EMR releases 5.20.0 and later, you may encounter problems with cluster operations such as scale down or step submission, after the cluster has been running for some time. The time period depends on the Kerberos ticket validity period that you defined. The scale-down problem impacts both automatic scale-down and explicit scale down requests that you submitted. Additional cluster operations can also be impacted. Workaround: + SSH as `hadoop` user to the lead primary node of the EMR cluster with multiple primary nodes. + Run the following command to renew Kerberos ticket for `hadoop` user. ``` kinit -kt ``` Typically, the keytab file is located at `/etc/hadoop.keytab` and the principal is in the form of `hadoop/@`. **Note** This workaround will be effective for the time period the Kerberos ticket is valid. This duration is 10 hours by default, but can configured by your Kerberos settings. You must re-run the above command once the Kerberos ticket expires. + When you use Spark with Hive partition location formatting to read data in Amazon S3, and you run Spark on Amazon EMR releases 5.30.0 to 5.36.0, and 6.2.0 to 6.9.0, you might encounter an issue that prevents your cluster from reading data correctly. This can happen if your partitions have all of the following characteristics: + Two or more partitions are scanned from the same table. + At least one partition directory path is a prefix of at least one other partition directory path, for example, `s3://bucket/table/p=a` is a prefix of `s3://bucket/table/p=a b`. + The first character that follows the prefix in the other partition directory has a UTF-8 value that’s less than than the `/` character (U\$1002F). For example, the space character (U\$10020) that occurs between a and b in `s3://bucket/table/p=a b` falls into this category. Note that there are 14 other non-control characters: `!"#$%&‘()*+,-`. For more information, see [UTF-8 encoding table and Unicode characters](https://www.utf8-chartable.de/). As a workaround to this issue, set the `spark.sql.sources.fastS3PartitionDiscovery.enabled` configuration to `false` in the `spark-defaults` classification. ## 6.2.0 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | aws-sagemaker-spark-sdk | 1.4.1 | Amazon SageMaker Spark SDK | | emr-ddb | 4.16.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 3.1.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.5.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-notebook-env | 1.0.0 | Conda env for emr notebook which includes jupyter enterprise gateway | | emr-s3-dist-cp | 2.16.0 | Distributed copy application optimized for Amazon S3. | | emr-s3-select | 2.0.0 | EMR S3Select Connector | | emrfs | 2.44.0 | Amazon S3 connector for Hadoop ecosystem applications. | | flink-client | 1.11.2 | Apache Flink command line client scripts and applications. | | flink-jobmanager-config | 1.11.2 | Managing resources on EMR nodes for Apache Flink JobManager. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 3.2.1-amzn-2 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 3.2.1-amzn-2 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 3.2.1-amzn-2 | HDFS command-line client and library | | hadoop-hdfs-namenode | 3.2.1-amzn-2 | HDFS service for tracking file names and block locations. | | hadoop-hdfs-journalnode | 3.2.1-amzn-2 | HDFS service for managing the Hadoop filesystem journal on HA clusters. | | hadoop-httpfs-server | 3.2.1-amzn-2 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 3.2.1-amzn-2 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 3.2.1-amzn-2 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 3.2.1-amzn-2 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 3.2.1-amzn-2 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 3.2.1-amzn-2 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 2.2.6-amzn-0 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 2.2.6-amzn-0 | Service for serving one or more HBase regions. | | hbase-client | 2.2.6-amzn-0 | HBase command-line client. | | hbase-rest-server | 2.2.6-amzn-0 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 2.2.6-amzn-0 | Service providing a Thrift endpoint to HBase. | | hcatalog-client | 3.1.2-amzn-3 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 3.1.2-amzn-3 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 3.1.2-amzn-3 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 3.1.2-amzn-3 | Hive command line client. | | hive-hbase | 3.1.2-amzn-3 | Hive-hbase client. | | hive-metastore-server | 3.1.2-amzn-3 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server2 | 3.1.2-amzn-3 | Service for accepting Hive queries as web requests. | | hudi | 0.6.0-amzn-1 | Incremental processing framework to power data pipline at low latency and high efficiency. | | hudi-presto | 0.6.0-amzn-1 | Bundle library for running Presto with Hudi. | | hudi-prestosql | 0.6.0-amzn-1 | Bundle library for running PrestoSQL with Hudi. | | hudi-spark | 0.6.0-amzn-1 | Bundle library for running Spark with Hudi. | | hue-server | 4.8.0 | Web application for analyzing data using Hadoop ecosystem applications | | jupyterhub | 1.1.0 | Multi-user server for Jupyter notebooks | | livy-server | 0.7.0-incubating | REST interface for interacting with Apache Spark | | nginx | 1.12.1 | nginx [engine x] is an HTTP and reverse proxy server | | mxnet | 1.7.0 | A flexible, scalable, and efficient library for deep learning. | | mariadb-server | 5.5.64\$1 | MariaDB database server. | | nvidia-cuda | 10.1.243 | Nvidia drivers and Cuda toolkit | | oozie-client | 5.2.0 | Oozie command-line client. | | oozie-server | 5.2.0 | Service for accepting Oozie workflow requests. | | opencv | 4.4.0 | Open Source Computer Vision Library. | | phoenix-library | 5.0.0-HBase-2.0 | The phoenix libraries for server and client | | phoenix-query-server | 5.0.0-HBase-2.0 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.238.3-amzn-1 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.238.3-amzn-1 | Service for executing pieces of a query. | | presto-client | 0.238.3-amzn-1 | Presto command-line client which is installed on an HA cluster's stand-by masters where Presto server is not started. | | prestosql-coordinator | 343 | Service for accepting queries and managing query execution among prestosql-workers. | | prestosql-worker | 343 | Service for executing pieces of a query. | | prestosql-client | 343 | Presto command-line client which is installed on an HA cluster's stand-by masters where Presto server is not started. | | pig-client | 0.17.0 | Pig command-line client. | | r | 3.4.3 | The R Project for Statistical Computing | | ranger-kms-server | 2.0.0 | Apache Ranger Key Management System | | spark-client | 3.0.1-amzn-0 | Spark command-line clients. | | spark-history-server | 3.0.1-amzn-0 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 3.0.1-amzn-0 | In-memory execution engine for YARN. | | spark-yarn-slave | 3.0.1-amzn-0 | Apache Spark libraries needed by YARN slaves. | | spark-rapids | 0.2.0 | Nvidia Spark RAPIDS plugin that accelerates Apache Spark with GPUs. | | sqoop-client | 1.4.7 | Apache Sqoop command-line client. | | tensorflow | 2.3.1 | TensorFlow open source software library for high performance numerical computation. | | tez-on-yarn | 0.9.2 | The tez YARN application and libraries. | | webserver | 2.4.41\$1 | Apache HTTP server. | | zeppelin-server | 0.9.0-preview1 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.4.14 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.4.14 | ZooKeeper command line client. | ## 6.2.0 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). Reconfiguration actions occur when you specify a configuration for instance groups in a running cluster. Amazon EMR only initiates reconfiguration actions for the classifications that you modify. For more information, see [Reconfigure an instance group in a running cluster](emr-configure-apps-running-cluster.md). **emr-6.2.0 classifications** | Classifications | Description | Reconfiguration Actions | | --- | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | Restarts the ResourceManager service. | | container-executor | Change values in Hadoop YARN's container-executor.cfg file. | Not available. | | container-log4j | Change values in Hadoop YARN's container-log4j.properties file. | Not available. | | core-site | Change values in Hadoop's core-site.xml file. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts Hadoop KMS, Ranger KMS, HiveServer2, Hive MetaStore, Hadoop Httpfs, and MapReduce-HistoryServer. | | docker-conf | Change docker related settings. | Not available. | | emrfs-site | Change EMRFS settings. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts HBaseRegionserver, HBaseMaster, HBaseThrift, HBaseRest, HiveServer2, Hive MetaStore, Hadoop Httpfs, and MapReduce-HistoryServer. | | flink-conf | Change flink-conf.yaml settings. | Not available. | | flink-log4j | Change Flink log4j.properties settings. | Not available. | | flink-log4j-yarn-session | Change Flink log4j-yarn-session.properties settings. | Not available. | | flink-log4j-cli | Change Flink log4j-cli.properties settings. | Not available. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts PhoenixQueryserver, HiveServer2, Hive MetaStore, and MapReduce-HistoryServer. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | Restarts the Hadoop HDFS services SecondaryNamenode, Datanode, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts Hadoop KMS, Hadoop Httpfs, and MapReduce-HistoryServer. | | hadoop-ssl-server | Change hadoop ssl server configuration | Not available. | | hadoop-ssl-client | Change hadoop ssl client configuration | Not available. | | hbase | Amazon EMR-curated settings for Apache HBase. | Custom EMR specific property. Sets emrfs-site and hbase-site configs. See those for their associated restarts. | | hbase-env | Change values in HBase's environment. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbase.properties file. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | Not available. | | hbase-site | Change values in HBase's hbase-site.xml file. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. Additionally restarts Phoenix QueryServer. | | hdfs-encryption-zones | Configure HDFS encryption zones. | This classification should not be reconfigured. | | hdfs-env | Change values in the HDFS environment. | Restarts Hadoop HDFS ZKFC. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Additionally restarts Hadoop Httpfs. | | hcatalog-env | Change values in HCatalog's environment. | Restarts Hive HCatalog Server. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | Restarts Hive HCatalog Server. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | Restarts Hive HCatalog Server. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | Restarts Hive WebHCat server. | | hcatalog-webhcat-log4j2 | Change values in HCatalog WebHCat's log4j2.properties. | Restarts Hive WebHCat server. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | Restarts Hive WebHCat server. | | hive | Amazon EMR-curated settings for Apache Hive. | Sets configurations to launch Hive LLAP service. | | hive-beeline-log4j2 | Change values in Hive's beeline-log4j2.properties file. | Not available. | | hive-parquet-logging | Change values in Hive's parquet-logging.properties file. | Not available. | | hive-env | Change values in the Hive environment. | Restarts HiveServer2, HiveMetastore, and Hive HCatalog-Server. Runs Hive schemaTool CLI commands to verify hive-metastore. | | hive-exec-log4j2 | Change values in Hive's hive-exec-log4j2.properties file. | Not available. | | hive-llap-daemon-log4j2 | Change values in Hive's llap-daemon-log4j2.properties file. | Not available. | | hive-log4j2 | Change values in Hive's hive-log4j2.properties file. | Not available. | | hive-site | Change values in Hive's hive-site.xml file | Restarts HiveServer2, HiveMetastore, and Hive HCatalog-Server. Runs Hive schemaTool CLI commands to verify hive-metastore. Also restarts Oozie and Zeppelin. | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | Not available. | | hue-ini | Change values in Hue's ini file | Restarts Hue. Also activates Hue config override CLI commands to pick up new configurations. | | httpfs-env | Change values in the HTTPFS environment. | Restarts Hadoop Httpfs service. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | Restarts Hadoop Httpfs service. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | Not available. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | Restarts Hadoop-KMS service. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | Not available. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | Restarts Hadoop-KMS and Ranger-KMS service. | | hudi-env | Change values in the Hudi environment. | Not available. | | jupyter-notebook-conf | Change values in Jupyter Notebook's jupyter\$1notebook\$1config.py file. | Not available. | | jupyter-hub-conf | Change values in JupyterHubs's jupyterhub\$1config.py file. | Not available. | | jupyter-s3-conf | Configure Jupyter Notebook S3 persistence. | Not available. | | jupyter-sparkmagic-conf | Change values in Sparkmagic's config.json file. | Not available. | | livy-conf | Change values in Livy's livy.conf file. | Restarts Livy Server. | | livy-env | Change values in the Livy environment. | Restarts Livy Server. | | livy-log4j | Change Livy log4j.properties settings. | Restarts Livy Server. | | mapred-env | Change values in the MapReduce application's environment. | Restarts Hadoop MapReduce-HistoryServer. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | Restarts Hadoop MapReduce-HistoryServer. | | oozie-env | Change values in Oozie's environment. | Restarts Oozie. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | Restarts Oozie. | | oozie-site | Change values in Oozie's oozie-site.xml file. | Restarts Oozie. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | Not available. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | Not available. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | Restarts Phoenix-QueryServer. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | Not available. | | pig-env | Change values in the Pig environment. | Not available. | | pig-properties | Change values in Pig's pig.properties file. | Restarts Oozie. | | pig-log4j | Change values in Pig's log4j.properties file. | Not available. | | presto-log | Change values in Presto's log.properties file. | Restarts Presto-Server (for PrestoDB) | | presto-config | Change values in Presto's config.properties file. | Restarts Presto-Server (for PrestoDB) | | presto-password-authenticator | Change values in Presto's password-authenticator.properties file. | Not available. | | presto-env | Change values in Presto's presto-env.sh file. | Restarts Presto-Server (for PrestoDB) | | presto-node | Change values in Presto's node.properties file. | Not available. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | Not available. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | Not available. | | presto-connector-hive | Change values in Presto's hive.properties file. | Restarts Presto-Server (for PrestoDB) | | presto-connector-jmx | Change values in Presto's jmx.properties file. | Not available. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | Not available. | | presto-connector-localfile | Change values in Presto's localfile.properties file. | Not available. | | presto-connector-memory | Change values in Presto's memory.properties file. | Not available. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | Not available. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | Not available. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | Not available. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | Not available. | | presto-connector-redis | Change values in Presto's redis.properties file. | Not available. | | presto-connector-redshift | Change values in Presto's redshift.properties file. | Not available. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | Not available. | | presto-connector-tpcds | Change values in Presto's tpcds.properties file. | Not available. | | prestosql-log | Change values in Presto's log.properties file. | Restarts Presto-Server (for PrestoSQL) | | prestosql-config | Change values in Presto's config.properties file. | Restarts Presto-Server (for PrestoSQL) | | prestosql-password-authenticator | Change values in Presto's password-authenticator.properties file. | Restarts Presto-Server (for PrestoSQL) | | prestosql-env | Change values in Presto's presto-env.sh file. | Restarts Presto-Server (for PrestoSQL) | | prestosql-node | Change values in PrestoSQL's node.properties file. | Not available. | | prestosql-connector-blackhole | Change values in PrestoSQL's blackhole.properties file. | Not available. | | prestosql-connector-cassandra | Change values in PrestoSQL's cassandra.properties file. | Not available. | | prestosql-connector-hive | Change values in PrestoSQL's hive.properties file. | Restarts Presto-Server (for PrestoSQL) | | prestosql-connector-jmx | Change values in PrestoSQL's jmx.properties file. | Not available. | | prestosql-connector-kafka | Change values in PrestoSQL's kafka.properties file. | Not available. | | prestosql-connector-localfile | Change values in PrestoSQL's localfile.properties file. | Not available. | | prestosql-connector-memory | Change values in PrestoSQL's memory.properties file. | Not available. | | prestosql-connector-mongodb | Change values in PrestoSQL's mongodb.properties file. | Not available. | | prestosql-connector-mysql | Change values in PrestoSQL's mysql.properties file. | Not available. | | prestosql-connector-postgresql | Change values in PrestoSQL's postgresql.properties file. | Not available. | | prestosql-connector-raptor | Change values in PrestoSQL's raptor.properties file. | Not available. | | prestosql-connector-redis | Change values in PrestoSQL's redis.properties file. | Not available. | | prestosql-connector-redshift | Change values in PrestoSQL's redshift.properties file. | Not available. | | prestosql-connector-tpch | Change values in PrestoSQL's tpch.properties file. | Not available. | | prestosql-connector-tpcds | Change values in PrestoSQL's tpcds.properties file. | Not available. | | ranger-kms-dbks-site | Change values in dbks-site.xml file of Ranger KMS. | Restarts Ranger KMS Server. | | ranger-kms-site | Change values in ranger-kms-site.xml file of Ranger KMS. | Restarts Ranger KMS Server. | | ranger-kms-env | Change values in the Ranger KMS environment. | Restarts Ranger KMS Server. | | ranger-kms-log4j | Change values in kms-log4j.properties file of Ranger KMS. | Not available. | | ranger-kms-db-ca | Change values for CA file on S3 for MySQL SSL connection with Ranger KMS. | Not available. | | spark | Amazon EMR-curated settings for Apache Spark. | This property modifies spark-defaults. See actions there. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | Restarts Spark history server and Spark thrift server. | | spark-env | Change values in the Spark environment. | Restarts Spark history server and Spark thrift server. | | spark-hive-site | Change values in Spark's hive-site.xml file | Not available. | | spark-log4j | Change values in Spark's log4j.properties file. | Restarts Spark history server and Spark thrift server. | | spark-metrics | Change values in Spark's metrics.properties file. | Restarts Spark history server and Spark thrift server. | | sqoop-env | Change values in Sqoop's environment. | Not available. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | Not available. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | Not available. | | tez-site | Change values in Tez's tez-site.xml file. | Restart Oozie. | | yarn-env | Change values in the YARN environment. | Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts MapReduce-HistoryServer. | | yarn-site | Change values in YARN's yarn-site.xml file. | Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts Livy Server and MapReduce-HistoryServer. | | zeppelin-env | Change values in the Zeppelin environment. | Restarts Zeppelin. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | Restarts Zookeeper server. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | Restarts Zookeeper server. | # Amazon EMR release 6.1.1 ## 6.1.1 application versions This release includes the following applications: [https://flink.apache.org/](https://flink.apache.org/), [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [https://hudi.apache.org](https://hudi.apache.org), [http://gethue.com/](http://gethue.com/), [https://jupyterhub.readthedocs.io/en/latest/#](https://jupyterhub.readthedocs.io/en/latest/#), [https://livy.incubator.apache.org/](https://livy.incubator.apache.org/), [https://mxnet.incubator.apache.org/](https://mxnet.incubator.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://prestosql.io/](https://prestosql.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://www.tensorflow.org/](https://www.tensorflow.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-6.1.1 | emr-6.1.0 | emr-6.0.1 | emr-6.0.0 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.11.828 | 1.11.828 | 1.11.711 | 1.11.711 | | Python | 2.7, 3.7 | 2.7, 3.7 | 2.7, 3.7 | 2.7, 3.7 | | Scala | 2.12.10 | 2.12.10 | 2.12.10 | 2.11.12 | | AmazonCloudWatchAgent | - | - | - | - | | Delta | - | - | - | - | | Flink | 1.11.0 | 1.11.0 | - | - | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 2.2.5 | 2.2.5 | 2.2.3 | 2.2.3 | | HCatalog | 3.1.2-amzn-2 | 3.1.2-amzn-2 | 3.1.2-amzn-0 | 3.1.2-amzn-0 | | Hadoop | 3.2.1-amzn-1.1 | 3.2.1-amzn-1 | 3.2.1-amzn-0.1 | 3.2.1-amzn-0 | | Hive | 3.1.2-amzn-2 | 3.1.2-amzn-2 | 3.1.2-amzn-0 | 3.1.2-amzn-0 | | Hudi | 0.5.2-incubating-amzn-2 | 0.5.2-incubating-amzn-2 | 0.5.0-incubating-amzn-1 | 0.5.0-incubating-amzn-1 | | Hue | 4.7.1 | 4.7.1 | 4.4.0 | 4.4.0 | | Iceberg | - | - | - | - | | JupyterEnterpriseGateway | - | - | - | - | | JupyterHub | 1.1.0 | 1.1.0 | 1.0.0 | 1.0.0 | | Livy | 0.7.0-incubating | 0.7.0-incubating | 0.6.0-incubating | 0.6.0-incubating | | MXNet | 1.6.0 | 1.6.0 | 1.5.1 | 1.5.1 | | Mahout | - | - | - | - | | Oozie | 5.2.0 | 5.2.0 | 5.1.0 | 5.1.0 | | Phoenix | 5.0.0-HBase-2.0 | 5.0.0-HBase-2.0 | 5.0.0-HBase-2.0 | 5.0.0-HBase-2.0 | | Pig | 0.17.0 | 0.17.0 | - | - | | Presto | 0.232 | 0.232 | 0.230 | 0.230 | | Spark | 3.0.0-amzn-0.1 | 3.0.0-amzn-0 | 2.4.4 | 2.4.4 | | Sqoop | 1.4.7 | 1.4.7 | - | - | | TensorFlow | 2.1.0 | 2.1.0 | 1.14.0 | 1.14.0 | | Tez | 0.9.2 | 0.9.2 | 0.9.2 | 0.9.2 | | Trino (PrestoSQL) | 338 | 338 | - | - | | Zeppelin | 0.9.0-preview1 | 0.9.0-preview1 | 0.9.0-SNAPSHOT | 0.9.0-SNAPSHOT | | ZooKeeper | 3.4.14 | 3.4.14 | 3.4.14 | 3.4.14 | ## 6.1.1 release notes This is a release to fix issues with Amazon EMR Scaling when it fails to scale up/scale down a cluster successfully or causes application failures. **Changes, Enhancements, and Resolved Issues** + Fixed an issue where scaling requests failed for a large, highly utilized cluster when Amazon EMR on-cluster daemons were running health checking activities, such as gathering YARN node state and HDFS node state. This was happening because on-cluster daemons were not able to communicate the health status data of a node to internal Amazon EMR components. + Improved EMR on-cluster daemons to correctly track the node states when IP addresses are reused to improve reliability during scaling operations. + [SPARK-29683](https://issues.apache.org/jira/browse/SPARK-29683). Fixed an issue where job failures occurred during cluster scale-down as Spark was assuming all available nodes were deny-listed. + [YARN-9011](https://issues.apache.org/jira/browse/YARN-9011). Fixed an issue where job failures occurred due to a race condition in YARN decommissioning when cluster tried to scale up or down. + Fixed issue with step or job failures during cluster scaling by ensuring that the node states are always consistent between the Amazon EMR on-cluster daemons and YARN/HDFS. + Fixed an issue where cluster operations such as scale down and step submission failed for Amazon EMR clusters enabled with Kerberos authentication. This was because the Amazon EMR on-cluster daemon did not renew the Kerberos ticket, which is required to securely communicate with HDFS/YARN running on the primary node. + Newer Amazon EMR releases fix the issue with a lower "Max open files" limit on older AL2 in Amazon EMR. Amazon EMR releases 5.30.1, 5.30.2, 5.31.1, 5.32.1, 6.0.1, 6.1.1, 6.2.1, 5.33.0, 6.3.0 and later now include a permanent fix with a higher "Max open files" setting. + HTTPS is now enabled by default for Amazon Linux repositories. If you are using an Amazon S3 VPCE policy to restrict access to specific buckets, you must add the new Amazon Linux bucket ARN `arn:aws:s3:::amazonlinux-2-repos-$region/*` to your policy (replace `$region` with the region where the endpoint is). For more information, see this topic in the AWS discussion forums. [Announcement: Amazon Linux 2 now supports the ability to use HTTPS while connecting to package repositories ](https://forums.aws.amazon.com/ann.jspa?annID=8528). ## 6.1.1 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | aws-sagemaker-spark-sdk | 1.3.0 | Amazon SageMaker Spark SDK | | emr-ddb | 4.14.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 3.1.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.5.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-s3-dist-cp | 2.14.0 | Distributed copy application optimized for Amazon S3. | | emr-s3-select | 2.0.0 | EMR S3Select Connector | | emrfs | 2.42.0 | Amazon S3 connector for Hadoop ecosystem applications. | | flink-client | 1.11.0 | Apache Flink command line client scripts and applications. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 3.2.1-amzn-1.1 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 3.2.1-amzn-1.1 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 3.2.1-amzn-1.1 | HDFS command-line client and library | | hadoop-hdfs-namenode | 3.2.1-amzn-1.1 | HDFS service for tracking file names and block locations. | | hadoop-hdfs-journalnode | 3.2.1-amzn-1.1 | HDFS service for managing the Hadoop filesystem journal on HA clusters. | | hadoop-httpfs-server | 3.2.1-amzn-1.1 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 3.2.1-amzn-1.1 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 3.2.1-amzn-1.1 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 3.2.1-amzn-1.1 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 3.2.1-amzn-1.1 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 3.2.1-amzn-1.1 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 2.2.5 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 2.2.5 | Service for serving one or more HBase regions. | | hbase-client | 2.2.5 | HBase command-line client. | | hbase-rest-server | 2.2.5 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 2.2.5 | Service providing a Thrift endpoint to HBase. | | hcatalog-client | 3.1.2-amzn-2 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 3.1.2-amzn-2 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 3.1.2-amzn-2 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 3.1.2-amzn-2 | Hive command line client. | | hive-hbase | 3.1.2-amzn-2 | Hive-hbase client. | | hive-metastore-server | 3.1.2-amzn-2 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server2 | 3.1.2-amzn-2 | Service for accepting Hive queries as web requests. | | hudi | 0.5.2-incubating-amzn-2 | Incremental processing framework to power data pipline at low latency and high efficiency. | | hudi-presto | 0.5.2-incubating-amzn-2 | Bundle library for running Presto with Hudi. | | hudi-prestosql | 0.5.2-incubating-amzn-2 | Bundle library for running PrestoSQL with Hudi. | | hudi-spark | 0.5.2-incubating-amzn-2 | Bundle library for running Spark with Hudi. | | hue-server | 4.7.1 | Web application for analyzing data using Hadoop ecosystem applications | | jupyterhub | 1.1.0 | Multi-user server for Jupyter notebooks | | livy-server | 0.7.0-incubating | REST interface for interacting with Apache Spark | | nginx | 1.12.1 | nginx [engine x] is an HTTP and reverse proxy server | | mxnet | 1.6.0 | A flexible, scalable, and efficient library for deep learning. | | mariadb-server | 5.5.64\$1 | MariaDB database server. | | nvidia-cuda | 9.2.88 | Nvidia drivers and Cuda toolkit | | oozie-client | 5.2.0 | Oozie command-line client. | | oozie-server | 5.2.0 | Service for accepting Oozie workflow requests. | | opencv | 4.3.0 | Open Source Computer Vision Library. | | phoenix-library | 5.0.0-HBase-2.0 | The phoenix libraries for server and client | | phoenix-query-server | 5.0.0-HBase-2.0 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.232 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.232 | Service for executing pieces of a query. | | presto-client | 0.232 | Presto command-line client which is installed on an HA cluster's stand-by masters where Presto server is not started. | | prestosql-coordinator | 338 | Service for accepting queries and managing query execution among prestosql-workers. | | prestosql-worker | 338 | Service for executing pieces of a query. | | prestosql-client | 338 | Presto command-line client which is installed on an HA cluster's stand-by masters where Presto server is not started. | | pig-client | 0.17.0 | Pig command-line client. | | r | 3.4.3 | The R Project for Statistical Computing | | ranger-kms-server | 2.0.0 | Apache Ranger Key Management System | | spark-client | 3.0.0-amzn-0.1 | Spark command-line clients. | | spark-history-server | 3.0.0-amzn-0.1 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 3.0.0-amzn-0.1 | In-memory execution engine for YARN. | | spark-yarn-slave | 3.0.0-amzn-0.1 | Apache Spark libraries needed by YARN slaves. | | sqoop-client | 1.4.7 | Apache Sqoop command-line client. | | tensorflow | 2.1.0 | TensorFlow open source software library for high performance numerical computation. | | tez-on-yarn | 0.9.2 | The tez YARN application and libraries. | | webserver | 2.4.41\$1 | Apache HTTP server. | | zeppelin-server | 0.9.0-preview1 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.4.14 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.4.14 | ZooKeeper command line client. | ## 6.1.1 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). **emr-6.1.1 classifications** | Classifications | Description | | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | | container-executor | Change values in Hadoop YARN's container-executor.cfg file. | | container-log4j | Change values in Hadoop YARN's container-log4j.properties file. | | core-site | Change values in Hadoop's core-site.xml file. | | emrfs-site | Change EMRFS settings. | | flink-conf | Change flink-conf.yaml settings. | | flink-log4j | Change Flink log4j.properties settings. | | flink-log4j-yarn-session | Change Flink log4j-yarn-session.properties settings. | | flink-log4j-cli | Change Flink log4j-cli.properties settings. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | | hadoop-ssl-server | Change hadoop ssl server configuration | | hadoop-ssl-client | Change hadoop ssl client configuration | | hbase | Amazon EMR-curated settings for Apache HBase. | | hbase-env | Change values in HBase's environment. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbase.properties file. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | | hbase-site | Change values in HBase's hbase-site.xml file. | | hdfs-encryption-zones | Configure HDFS encryption zones. | | hdfs-env | Change values in the HDFS environment. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | | hcatalog-env | Change values in HCatalog's environment. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | | hcatalog-webhcat-log4j2 | Change values in HCatalog WebHCat's log4j2.properties. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | | hive | Amazon EMR-curated settings for Apache Hive. | | hive-beeline-log4j2 | Change values in Hive's beeline-log4j2.properties file. | | hive-parquet-logging | Change values in Hive's parquet-logging.properties file. | | hive-env | Change values in the Hive environment. | | hive-exec-log4j2 | Change values in Hive's hive-exec-log4j2.properties file. | | hive-llap-daemon-log4j2 | Change values in Hive's llap-daemon-log4j2.properties file. | | hive-log4j2 | Change values in Hive's hive-log4j2.properties file. | | hive-site | Change values in Hive's hive-site.xml file | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | | hue-ini | Change values in Hue's ini file | | httpfs-env | Change values in the HTTPFS environment. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | | hudi-env | Change values in the Hudi environment. | | jupyter-notebook-conf | Change values in Jupyter Notebook's jupyter\$1notebook\$1config.py file. | | jupyter-hub-conf | Change values in JupyterHubs's jupyterhub\$1config.py file. | | jupyter-s3-conf | Configure Jupyter Notebook S3 persistence. | | jupyter-sparkmagic-conf | Change values in Sparkmagic's config.json file. | | livy-conf | Change values in Livy's livy.conf file. | | livy-env | Change values in the Livy environment. | | livy-log4j | Change Livy log4j.properties settings. | | mapred-env | Change values in the MapReduce application's environment. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | | oozie-env | Change values in Oozie's environment. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | | oozie-site | Change values in Oozie's oozie-site.xml file. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | | pig-env | Change values in the Pig environment. | | pig-properties | Change values in Pig's pig.properties file. | | pig-log4j | Change values in Pig's log4j.properties file. | | presto-log | Change values in Presto's log.properties file. | | presto-config | Change values in Presto's config.properties file. | | presto-password-authenticator | Change values in Presto's password-authenticator.properties file. | | presto-env | Change values in Presto's presto-env.sh file. | | presto-node | Change values in Presto's node.properties file. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | | presto-connector-hive | Change values in Presto's hive.properties file. | | presto-connector-jmx | Change values in Presto's jmx.properties file. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | | presto-connector-localfile | Change values in Presto's localfile.properties file. | | presto-connector-memory | Change values in Presto's memory.properties file. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | | presto-connector-redis | Change values in Presto's redis.properties file. | | presto-connector-redshift | Change values in Presto's redshift.properties file. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | | presto-connector-tpcds | Change values in Presto's tpcds.properties file. | | prestosql-log | Change values in Presto's log.properties file. | | prestosql-config | Change values in Presto's config.properties file. | | prestosql-password-authenticator | Change values in Presto's password-authenticator.properties file. | | prestosql-env | Change values in Presto's presto-env.sh file. | | prestosql-node | Change values in PrestoSQL's node.properties file. | | prestosql-connector-blackhole | Change values in PrestoSQL's blackhole.properties file. | | prestosql-connector-cassandra | Change values in PrestoSQL's cassandra.properties file. | | prestosql-connector-hive | Change values in PrestoSQL's hive.properties file. | | prestosql-connector-jmx | Change values in PrestoSQL's jmx.properties file. | | prestosql-connector-kafka | Change values in PrestoSQL's kafka.properties file. | | prestosql-connector-localfile | Change values in PrestoSQL's localfile.properties file. | | prestosql-connector-memory | Change values in PrestoSQL's memory.properties file. | | prestosql-connector-mongodb | Change values in PrestoSQL's mongodb.properties file. | | prestosql-connector-mysql | Change values in PrestoSQL's mysql.properties file. | | prestosql-connector-postgresql | Change values in PrestoSQL's postgresql.properties file. | | prestosql-connector-raptor | Change values in PrestoSQL's raptor.properties file. | | prestosql-connector-redis | Change values in PrestoSQL's redis.properties file. | | prestosql-connector-redshift | Change values in PrestoSQL's redshift.properties file. | | prestosql-connector-tpch | Change values in PrestoSQL's tpch.properties file. | | prestosql-connector-tpcds | Change values in PrestoSQL's tpcds.properties file. | | ranger-kms-dbks-site | Change values in dbks-site.xml file of Ranger KMS. | | ranger-kms-site | Change values in ranger-kms-site.xml file of Ranger KMS. | | ranger-kms-env | Change values in the Ranger KMS environment. | | ranger-kms-log4j | Change values in kms-log4j.properties file of Ranger KMS. | | ranger-kms-db-ca | Change values for CA file on S3 for MySQL SSL connection with Ranger KMS. | | spark | Amazon EMR-curated settings for Apache Spark. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | | spark-env | Change values in the Spark environment. | | spark-hive-site | Change values in Spark's hive-site.xml file | | spark-log4j | Change values in Spark's log4j.properties file. | | spark-metrics | Change values in Spark's metrics.properties file. | | sqoop-env | Change values in Sqoop's environment. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | | tez-site | Change values in Tez's tez-site.xml file. | | yarn-env | Change values in the YARN environment. | | yarn-site | Change values in YARN's yarn-site.xml file. | | zeppelin-env | Change values in the Zeppelin environment. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | # Amazon EMR release 6.1.0 ## 6.1.0 application versions This release includes the following applications: [https://flink.apache.org/](https://flink.apache.org/), [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [https://hudi.apache.org](https://hudi.apache.org), [http://gethue.com/](http://gethue.com/), [https://jupyterhub.readthedocs.io/en/latest/#](https://jupyterhub.readthedocs.io/en/latest/#), [https://livy.incubator.apache.org/](https://livy.incubator.apache.org/), [https://mxnet.incubator.apache.org/](https://mxnet.incubator.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://prestosql.io/](https://prestosql.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://www.tensorflow.org/](https://www.tensorflow.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-6.1.1 | emr-6.1.0 | emr-6.0.1 | emr-6.0.0 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.11.828 | 1.11.828 | 1.11.711 | 1.11.711 | | Python | 2.7, 3.7 | 2.7, 3.7 | 2.7, 3.7 | 2.7, 3.7 | | Scala | 2.12.10 | 2.12.10 | 2.12.10 | 2.11.12 | | AmazonCloudWatchAgent | - | - | - | - | | Delta | - | - | - | - | | Flink | 1.11.0 | 1.11.0 | - | - | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 2.2.5 | 2.2.5 | 2.2.3 | 2.2.3 | | HCatalog | 3.1.2-amzn-2 | 3.1.2-amzn-2 | 3.1.2-amzn-0 | 3.1.2-amzn-0 | | Hadoop | 3.2.1-amzn-1.1 | 3.2.1-amzn-1 | 3.2.1-amzn-0.1 | 3.2.1-amzn-0 | | Hive | 3.1.2-amzn-2 | 3.1.2-amzn-2 | 3.1.2-amzn-0 | 3.1.2-amzn-0 | | Hudi | 0.5.2-incubating-amzn-2 | 0.5.2-incubating-amzn-2 | 0.5.0-incubating-amzn-1 | 0.5.0-incubating-amzn-1 | | Hue | 4.7.1 | 4.7.1 | 4.4.0 | 4.4.0 | | Iceberg | - | - | - | - | | JupyterEnterpriseGateway | - | - | - | - | | JupyterHub | 1.1.0 | 1.1.0 | 1.0.0 | 1.0.0 | | Livy | 0.7.0-incubating | 0.7.0-incubating | 0.6.0-incubating | 0.6.0-incubating | | MXNet | 1.6.0 | 1.6.0 | 1.5.1 | 1.5.1 | | Mahout | - | - | - | - | | Oozie | 5.2.0 | 5.2.0 | 5.1.0 | 5.1.0 | | Phoenix | 5.0.0-HBase-2.0 | 5.0.0-HBase-2.0 | 5.0.0-HBase-2.0 | 5.0.0-HBase-2.0 | | Pig | 0.17.0 | 0.17.0 | - | - | | Presto | 0.232 | 0.232 | 0.230 | 0.230 | | Spark | 3.0.0-amzn-0.1 | 3.0.0-amzn-0 | 2.4.4 | 2.4.4 | | Sqoop | 1.4.7 | 1.4.7 | - | - | | TensorFlow | 2.1.0 | 2.1.0 | 1.14.0 | 1.14.0 | | Tez | 0.9.2 | 0.9.2 | 0.9.2 | 0.9.2 | | Trino (PrestoSQL) | 338 | 338 | - | - | | Zeppelin | 0.9.0-preview1 | 0.9.0-preview1 | 0.9.0-SNAPSHOT | 0.9.0-SNAPSHOT | | ZooKeeper | 3.4.14 | 3.4.14 | 3.4.14 | 3.4.14 | ## 6.1.0 release notes The following release notes include information for Amazon EMR release 6.1.0. Changes are relative to 6.0.0. Initial release date: Sept 04, 2020 Last updated date: Oct 15, 2020 **Supported applications** + AWS SDK for Java version 1.11.828 + Flink version 1.11.0 + Ganglia version 3.7.2 + Hadoop version 3.2.1-amzn-1 + HBase version 2.2.5 + HBase-operator-tools 1.0.0 + HCatalog version 3.1.2-amzn-0 + Hive version 3.1.2-amzn-1 + Hudi version 0.5.2-incubating + Hue version 4.7.1 + JupyterHub version 1.1.0 + Livy version 0.7.0 + MXNet version 1.6.0 + Oozie version 5.2.0 + Phoenix version 5.0.0 + Presto version 0.232 + PrestoSQL version 338 + Spark version 3.0.0-amzn-0 + TensorFlow version 2.1.0 + Zeppelin version 0.9.0-preview1 + Zookeeper version 3.4.14 + Connectors and drivers: DynamoDB Connector 4.14.0 **New features** + ARM instance types are supported starting with Amazon EMR version 5.30.0 and Amazon EMR version 6.1.0. + M6g general purpose instance types are supported starting with Amazon EMR versions 6.1.0 and 5.30.0. For more information, see [Supported Instance Types](https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-supported-instance-types.html) in the *Amazon EMR Management Guide*. + The EC2 placement group feature is supported starting with Amazon EMR version 5.23.0 as an option for multiple primary node clusters. Currently, only primary node types are supported by the placement group feature, and the `SPREAD` strategy is applied to those primary nodes. The `SPREAD` strategy places a small group of instances across separate underlying hardware to guard against the loss of multiple primary nodes in the event of a hardware failure. For more information, see [EMR Integration with EC2 Placement Group](https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-plan-ha-placementgroup.html) in the *Amazon EMR Management Guide*. + Managed Scaling – With Amazon EMR version 6.1.0, you can enable Amazon EMR managed scaling to automatically increase or decrease the number of instances or units in your cluster based on workload. Amazon EMR continuously evaluates cluster metrics to make scaling decisions that optimize your clusters for cost and speed. Managed Scaling is also available on Amazon EMR version 5.30.0 and later, except 6.0.0. For more information, see [Scaling Cluster Resources](https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-scale-on-demand.html) in the *Amazon EMR Management Guide*. + PrestoSQL version 338 is supported with EMR 6.1.0. For more information, see [Presto](https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-presto.html). + PrestoSQL is supported on EMR 6.1.0 and later versions only, not on EMR 6.0.0 or EMR 5.x. + The application name, `Presto` continues to be used to install PrestoDB on clusters. To install PrestoSQL on clusters, use the application name `PrestoSQL`. + You can install either PrestoDB or PrestoSQL, but you cannot install both on a single cluster. If both PrestoDB and PrestoSQL are specified when attempting to create a cluster, a validation error occurs and the cluster creation request fails. + PrestoSQL is supported on both single-master and muti-master clusters. On multi-master clusters, an external Hive metastore is required to run PrestoSQL or PrestoDB. See [Supported applications in an EMR cluster with multiple primary nodes](https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-plan-ha-applications.html#emr-plan-ha-applications-list). + ECR auto authentication support on Apache Hadoop and Apache Spark with Docker: Spark users can use Docker images from Docker Hub and Amazon Elastic Container Registry (Amazon ECR) to define environment and library dependencies. [Configure Docker](https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-plan-docker.html) and [Run Spark Applications with Docker Using Amazon EMR 6.x](https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-spark-docker.html). + EMR supports Apache Hive ACID transactions: Amazon EMR 6.1.0 adds support for Hive ACID transactions so it complies with the ACID properties of a database. With this feature, you can run `INSERT, UPDATE, DELETE,` and `MERGE` operations in Hive managed tables with data in Amazon Simple Storage Service (Amazon S3). This is a key feature for use cases like streaming ingestion, data restatement, bulk updates using MERGE, and slowly changing dimensions. For more information, including configuration examples and use cases, see [Amazon EMR supports Apache Hive ACID transactions](https://aws.amazon.com/blogs/big-data/amazon-emr-supports-apache-hive-acid-transactions). **Changes, enhancements, and resolved issues** + This is a release to fix issues with Amazon EMR Scaling when it fails to scale up/scale down a cluster successfully or causes application failures. + Fixed an issue where scaling requests failed for a large, highly utilized cluster when Amazon EMR on-cluster daemons were running health checking activities, such as gathering YARN node state and HDFS node state. This was happening because on-cluster daemons were not able to communicate the health status data of a node to internal Amazon EMR components. + Improved EMR on-cluster daemons to correctly track the node states when IP addresses are reused to improve reliability during scaling operations. + [SPARK-29683](https://issues.apache.org/jira/browse/SPARK-29683). Fixed an issue where job failures occurred during cluster scale-down as Spark was assuming all available nodes were deny-listed. + [YARN-9011](https://issues.apache.org/jira/browse/YARN-9011). Fixed an issue where job failures occurred due to a race condition in YARN decommissioning when cluster tried to scale up or down. + Fixed issue with step or job failures during cluster scaling by ensuring that the node states are always consistent between the Amazon EMR on-cluster daemons and YARN/HDFS. + Fixed an issue where cluster operations such as scale down and step submission failed for Amazon EMR clusters enabled with Kerberos authentication. This was because the Amazon EMR on-cluster daemon did not renew the Kerberos ticket, which is required to securely communicate with HDFS/YARN running on the primary node. + Newer Amazon EMR releases fix the issue with a lower "Max open files" limit on older AL2 in Amazon EMR. Amazon EMR releases 5.30.1, 5.30.2, 5.31.1, 5.32.1, 6.0.1, 6.1.1, 6.2.1, 5.33.0, 6.3.0 and later now include a permanent fix with a higher "Max open files" setting. + Apache Flink is not supported on EMR 6.0.0, but it is supported on EMR 6.1.0 with Flink 1.11.0. This is the first version of Flink to officially support Hadoop 3. See [Apache Flink 1.11.0 Release Announcement](https://flink.apache.org/news/2020/07/06/release-1.11.0.html). + Ganglia has been removed from default EMR 6.1.0 package bundles. **Known issues** + **Lower "Max open files" limit on older AL2 [fixed in newer releases].** Amazon EMR releases: emr-5.30.x, emr-5.31.0, emr-5.32.0, emr-6.0.0, emr-6.1.0, and emr-6.2.0 are based on older versions ofAmazon Linux 2 (AL2), which have a lower ulimit setting for "Max open files" when Amazon EMR clusters are created with the default AMI. Amazon EMR releases 5.30.1, 5.30.2, 5.31.1, 5.32.1, 6.0.1, 6.1.1, 6.2.1, 5.33.0, 6.3.0 and later include a permanent fix with a higher "Max open files" setting. Releases with the lower open file limit causes a "Too many open files" error when submitting Spark job. In the impacted releases, the Amazon EMR default AMI has a default ulimit setting of 4096 for "Max open files," which is lower than the 65536 file limit in the latestAmazon Linux 2 AMI. The lower ulimit setting for "Max open files" causes Spark job failure when the Spark driver and executor try to open more than 4096 files. To fix the issue, Amazon EMR has a bootstrap action (BA) script that adjusts the ulimit setting at cluster creation. If you are using an older Amazon EMR version that doesn't have the permanent fix for this issue, the following workaround lets you to explicitly set the instance-controller ulimit to a maximum of 65536 files. **Explicitly set a ulimit from the command line** 1. Edit `/etc/systemd/system/instance-controller.service` to add the following parameters to Service section. `LimitNOFILE=65536` `LimitNPROC=65536` 1. Restart InstanceController `$ sudo systemctl daemon-reload` `$ sudo systemctl restart instance-controller` **Set a ulimit using bootstrap action (BA)** You can also use a bootstrap action (BA) script to configure the instance-controller ulimit to 65536 files at cluster creation. ``` #!/bin/bash for user in hadoop spark hive; do sudo tee /etc/security/limits.d/$user.conf << EOF $user - nofile 65536 $user - nproc 65536 EOF done for proc in instancecontroller logpusher; do sudo mkdir -p /etc/systemd/system/$proc.service.d/ sudo tee /etc/systemd/system/$proc.service.d/override.conf << EOF [Service] LimitNOFILE=65536 LimitNPROC=65536 EOF pid=$(pgrep -f aws157.$proc.Main) sudo prlimit --pid $pid --nofile=65535:65535 --nproc=65535:65535 done sudo systemctl daemon-reload ``` + **Important** Amazon EMR 6.1.0 and 6.2.0 include a performance issue that can critically affect all Hudi insert, upsert, and delete operations. If you plan to use Hudi with Amazon EMR 6.1.0 or 6.2.0, you should contact AWS support to obtain a patched Hudi RPM. + If you set custom garbage collection configuration with `spark.driver.extraJavaOptions` and `spark.executor.extraJavaOptions`, this will result in driver/executor launch failure with EMR 6.1 due to conflicting garbage collection configuration. With EMR Release 6.1.0, you should specify custom Spark garbage collection configuration for drivers and executors with the properties `spark.driver.defaultJavaOptions` and `spark.executor.defaultJavaOptions` instead. Read more in [Apache Spark Runtime Environment](https://spark.apache.org/docs/latest/configuration.html#runtime-environment) and [Configuring Spark Garbage Collection on Amazon EMR 6.1.0](https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-spark-configure.html#spark-gc-config). + Using Pig with Oozie (and within Hue, since Hue uses Oozie actions to run Pig scripts), generates an error that a native-lzo library cannot be loaded. This error message is informational and does not block Pig from running. + Hudi Concurrency Support: Currently Hudi doesn't support concurrent writes to a single Hudi table. In addition, Hudi rolls back any changes being done by in-progress writers before allowing a new writer to start. Concurrent writes can interfere with this mechanism and introduce race conditions, which can lead to data corruption. You should ensure that as part of your data processing workflow, there is only a single Hudi writer operating against a Hudi table at any time. Hudi does support multiple concurrent readers operating against the same Hudi table. + Known issue in clusters with multiple primary nodes and Kerberos authentication If you run clusters with multiple primary nodes and Kerberos authentication in Amazon EMR releases 5.20.0 and later, you may encounter problems with cluster operations such as scale down or step submission, after the cluster has been running for some time. The time period depends on the Kerberos ticket validity period that you defined. The scale-down problem impacts both automatic scale-down and explicit scale down requests that you submitted. Additional cluster operations can also be impacted. Workaround: + SSH as `hadoop` user to the lead primary node of the EMR cluster with multiple primary nodes. + Run the following command to renew Kerberos ticket for `hadoop` user. ``` kinit -kt ``` Typically, the keytab file is located at `/etc/hadoop.keytab` and the principal is in the form of `hadoop/@`. **Note** This workaround will be effective for the time period the Kerberos ticket is valid. This duration is 10 hours by default, but can configured by your Kerberos settings. You must re-run the above command once the Kerberos ticket expires. + There is an issue in Amazon EMR 6.1.0 that affects clusters running Presto. After an extended period of time (days), the cluster may throw errors such as, "su: failed to execute /bin/bash: Resource temporarily unavailable" or "shell request failed on channel 0". This issue is caused by an internal Amazon EMR process (InstanceController) that is spawning too many Light Weight Processes (LWP), which eventually causes the Hadoop user to exceed their nproc limit. This prevents the user from opening additional processes. The solution for this issue is to upgrade to EMR 6.2.0. ## 6.1.0 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | aws-sagemaker-spark-sdk | 1.3.0 | Amazon SageMaker Spark SDK | | emr-ddb | 4.14.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 3.1.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.5.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-s3-dist-cp | 2.14.0 | Distributed copy application optimized for Amazon S3. | | emr-s3-select | 2.0.0 | EMR S3Select Connector | | emrfs | 2.42.0 | Amazon S3 connector for Hadoop ecosystem applications. | | flink-client | 1.11.0 | Apache Flink command line client scripts and applications. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 3.2.1-amzn-1 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 3.2.1-amzn-1 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 3.2.1-amzn-1 | HDFS command-line client and library | | hadoop-hdfs-namenode | 3.2.1-amzn-1 | HDFS service for tracking file names and block locations. | | hadoop-hdfs-journalnode | 3.2.1-amzn-1 | HDFS service for managing the Hadoop filesystem journal on HA clusters. | | hadoop-httpfs-server | 3.2.1-amzn-1 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 3.2.1-amzn-1 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 3.2.1-amzn-1 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 3.2.1-amzn-1 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 3.2.1-amzn-1 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 3.2.1-amzn-1 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 2.2.5 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 2.2.5 | Service for serving one or more HBase regions. | | hbase-client | 2.2.5 | HBase command-line client. | | hbase-rest-server | 2.2.5 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 2.2.5 | Service providing a Thrift endpoint to HBase. | | hcatalog-client | 3.1.2-amzn-2 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 3.1.2-amzn-2 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 3.1.2-amzn-2 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 3.1.2-amzn-2 | Hive command line client. | | hive-hbase | 3.1.2-amzn-2 | Hive-hbase client. | | hive-metastore-server | 3.1.2-amzn-2 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server2 | 3.1.2-amzn-2 | Service for accepting Hive queries as web requests. | | hudi | 0.5.2-incubating-amzn-2 | Incremental processing framework to power data pipline at low latency and high efficiency. | | hudi-presto | 0.5.2-incubating-amzn-2 | Bundle library for running Presto with Hudi. | | hudi-prestosql | 0.5.2-incubating-amzn-2 | Bundle library for running PrestoSQL with Hudi. | | hudi-spark | 0.5.2-incubating-amzn-2 | Bundle library for running Spark with Hudi. | | hue-server | 4.7.1 | Web application for analyzing data using Hadoop ecosystem applications | | jupyterhub | 1.1.0 | Multi-user server for Jupyter notebooks | | livy-server | 0.7.0-incubating | REST interface for interacting with Apache Spark | | nginx | 1.12.1 | nginx [engine x] is an HTTP and reverse proxy server | | mxnet | 1.6.0 | A flexible, scalable, and efficient library for deep learning. | | mariadb-server | 5.5.64\$1 | MariaDB database server. | | nvidia-cuda | 9.2.88 | Nvidia drivers and Cuda toolkit | | oozie-client | 5.2.0 | Oozie command-line client. | | oozie-server | 5.2.0 | Service for accepting Oozie workflow requests. | | opencv | 4.3.0 | Open Source Computer Vision Library. | | phoenix-library | 5.0.0-HBase-2.0 | The phoenix libraries for server and client | | phoenix-query-server | 5.0.0-HBase-2.0 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.232 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.232 | Service for executing pieces of a query. | | presto-client | 0.232 | Presto command-line client which is installed on an HA cluster's stand-by masters where Presto server is not started. | | prestosql-coordinator | 338 | Service for accepting queries and managing query execution among prestosql-workers. | | prestosql-worker | 338 | Service for executing pieces of a query. | | prestosql-client | 338 | Presto command-line client which is installed on an HA cluster's stand-by masters where Presto server is not started. | | pig-client | 0.17.0 | Pig command-line client. | | r | 3.4.3 | The R Project for Statistical Computing | | ranger-kms-server | 2.0.0 | Apache Ranger Key Management System | | spark-client | 3.0.0-amzn-0 | Spark command-line clients. | | spark-history-server | 3.0.0-amzn-0 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 3.0.0-amzn-0 | In-memory execution engine for YARN. | | spark-yarn-slave | 3.0.0-amzn-0 | Apache Spark libraries needed by YARN slaves. | | sqoop-client | 1.4.7 | Apache Sqoop command-line client. | | tensorflow | 2.1.0 | TensorFlow open source software library for high performance numerical computation. | | tez-on-yarn | 0.9.2 | The tez YARN application and libraries. | | webserver | 2.4.41\$1 | Apache HTTP server. | | zeppelin-server | 0.9.0-preview1 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.4.14 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.4.14 | ZooKeeper command line client. | ## 6.1.0 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). **emr-6.1.0 classifications** | Classifications | Description | | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | | container-executor | Change values in Hadoop YARN's container-executor.cfg file. | | container-log4j | Change values in Hadoop YARN's container-log4j.properties file. | | core-site | Change values in Hadoop's core-site.xml file. | | emrfs-site | Change EMRFS settings. | | flink-conf | Change flink-conf.yaml settings. | | flink-log4j | Change Flink log4j.properties settings. | | flink-log4j-yarn-session | Change Flink log4j-yarn-session.properties settings. | | flink-log4j-cli | Change Flink log4j-cli.properties settings. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | | hadoop-ssl-server | Change hadoop ssl server configuration | | hadoop-ssl-client | Change hadoop ssl client configuration | | hbase | Amazon EMR-curated settings for Apache HBase. | | hbase-env | Change values in HBase's environment. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbase.properties file. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | | hbase-site | Change values in HBase's hbase-site.xml file. | | hdfs-encryption-zones | Configure HDFS encryption zones. | | hdfs-env | Change values in the HDFS environment. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | | hcatalog-env | Change values in HCatalog's environment. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | | hcatalog-webhcat-log4j2 | Change values in HCatalog WebHCat's log4j2.properties. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | | hive | Amazon EMR-curated settings for Apache Hive. | | hive-beeline-log4j2 | Change values in Hive's beeline-log4j2.properties file. | | hive-parquet-logging | Change values in Hive's parquet-logging.properties file. | | hive-env | Change values in the Hive environment. | | hive-exec-log4j2 | Change values in Hive's hive-exec-log4j2.properties file. | | hive-llap-daemon-log4j2 | Change values in Hive's llap-daemon-log4j2.properties file. | | hive-log4j2 | Change values in Hive's hive-log4j2.properties file. | | hive-site | Change values in Hive's hive-site.xml file | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | | hue-ini | Change values in Hue's ini file | | httpfs-env | Change values in the HTTPFS environment. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | | hudi-env | Change values in the Hudi environment. | | jupyter-notebook-conf | Change values in Jupyter Notebook's jupyter\$1notebook\$1config.py file. | | jupyter-hub-conf | Change values in JupyterHubs's jupyterhub\$1config.py file. | | jupyter-s3-conf | Configure Jupyter Notebook S3 persistence. | | jupyter-sparkmagic-conf | Change values in Sparkmagic's config.json file. | | livy-conf | Change values in Livy's livy.conf file. | | livy-env | Change values in the Livy environment. | | livy-log4j | Change Livy log4j.properties settings. | | mapred-env | Change values in the MapReduce application's environment. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | | oozie-env | Change values in Oozie's environment. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | | oozie-site | Change values in Oozie's oozie-site.xml file. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | | pig-env | Change values in the Pig environment. | | pig-properties | Change values in Pig's pig.properties file. | | pig-log4j | Change values in Pig's log4j.properties file. | | presto-log | Change values in Presto's log.properties file. | | presto-config | Change values in Presto's config.properties file. | | presto-password-authenticator | Change values in Presto's password-authenticator.properties file. | | presto-env | Change values in Presto's presto-env.sh file. | | presto-node | Change values in Presto's node.properties file. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | | presto-connector-hive | Change values in Presto's hive.properties file. | | presto-connector-jmx | Change values in Presto's jmx.properties file. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | | presto-connector-localfile | Change values in Presto's localfile.properties file. | | presto-connector-memory | Change values in Presto's memory.properties file. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | | presto-connector-redis | Change values in Presto's redis.properties file. | | presto-connector-redshift | Change values in Presto's redshift.properties file. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | | presto-connector-tpcds | Change values in Presto's tpcds.properties file. | | prestosql-log | Change values in Presto's log.properties file. | | prestosql-config | Change values in Presto's config.properties file. | | prestosql-password-authenticator | Change values in Presto's password-authenticator.properties file. | | prestosql-env | Change values in Presto's presto-env.sh file. | | prestosql-node | Change values in PrestoSQL's node.properties file. | | prestosql-connector-blackhole | Change values in PrestoSQL's blackhole.properties file. | | prestosql-connector-cassandra | Change values in PrestoSQL's cassandra.properties file. | | prestosql-connector-hive | Change values in PrestoSQL's hive.properties file. | | prestosql-connector-jmx | Change values in PrestoSQL's jmx.properties file. | | prestosql-connector-kafka | Change values in PrestoSQL's kafka.properties file. | | prestosql-connector-localfile | Change values in PrestoSQL's localfile.properties file. | | prestosql-connector-memory | Change values in PrestoSQL's memory.properties file. | | prestosql-connector-mongodb | Change values in PrestoSQL's mongodb.properties file. | | prestosql-connector-mysql | Change values in PrestoSQL's mysql.properties file. | | prestosql-connector-postgresql | Change values in PrestoSQL's postgresql.properties file. | | prestosql-connector-raptor | Change values in PrestoSQL's raptor.properties file. | | prestosql-connector-redis | Change values in PrestoSQL's redis.properties file. | | prestosql-connector-redshift | Change values in PrestoSQL's redshift.properties file. | | prestosql-connector-tpch | Change values in PrestoSQL's tpch.properties file. | | prestosql-connector-tpcds | Change values in PrestoSQL's tpcds.properties file. | | ranger-kms-dbks-site | Change values in dbks-site.xml file of Ranger KMS. | | ranger-kms-site | Change values in ranger-kms-site.xml file of Ranger KMS. | | ranger-kms-env | Change values in the Ranger KMS environment. | | ranger-kms-log4j | Change values in kms-log4j.properties file of Ranger KMS. | | ranger-kms-db-ca | Change values for CA file on S3 for MySQL SSL connection with Ranger KMS. | | spark | Amazon EMR-curated settings for Apache Spark. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | | spark-env | Change values in the Spark environment. | | spark-hive-site | Change values in Spark's hive-site.xml file | | spark-log4j | Change values in Spark's log4j.properties file. | | spark-metrics | Change values in Spark's metrics.properties file. | | sqoop-env | Change values in Sqoop's environment. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | | tez-site | Change values in Tez's tez-site.xml file. | | yarn-env | Change values in the YARN environment. | | yarn-site | Change values in YARN's yarn-site.xml file. | | zeppelin-env | Change values in the Zeppelin environment. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | # Amazon EMR release 6.0.1 ## 6.0.1 application versions This release includes the following applications: [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [https://hudi.apache.org](https://hudi.apache.org), [http://gethue.com/](http://gethue.com/), [https://jupyterhub.readthedocs.io/en/latest/#](https://jupyterhub.readthedocs.io/en/latest/#), [https://livy.incubator.apache.org/](https://livy.incubator.apache.org/), [https://mxnet.incubator.apache.org/](https://mxnet.incubator.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [https://www.tensorflow.org/](https://www.tensorflow.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-6.1.1 | emr-6.1.0 | emr-6.0.1 | emr-6.0.0 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.11.828 | 1.11.828 | 1.11.711 | 1.11.711 | | Python | 2.7, 3.7 | 2.7, 3.7 | 2.7, 3.7 | 2.7, 3.7 | | Scala | 2.12.10 | 2.12.10 | 2.12.10 | 2.11.12 | | AmazonCloudWatchAgent | - | - | - | - | | Delta | - | - | - | - | | Flink | 1.11.0 | 1.11.0 | - | - | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 2.2.5 | 2.2.5 | 2.2.3 | 2.2.3 | | HCatalog | 3.1.2-amzn-2 | 3.1.2-amzn-2 | 3.1.2-amzn-0 | 3.1.2-amzn-0 | | Hadoop | 3.2.1-amzn-1.1 | 3.2.1-amzn-1 | 3.2.1-amzn-0.1 | 3.2.1-amzn-0 | | Hive | 3.1.2-amzn-2 | 3.1.2-amzn-2 | 3.1.2-amzn-0 | 3.1.2-amzn-0 | | Hudi | 0.5.2-incubating-amzn-2 | 0.5.2-incubating-amzn-2 | 0.5.0-incubating-amzn-1 | 0.5.0-incubating-amzn-1 | | Hue | 4.7.1 | 4.7.1 | 4.4.0 | 4.4.0 | | Iceberg | - | - | - | - | | JupyterEnterpriseGateway | - | - | - | - | | JupyterHub | 1.1.0 | 1.1.0 | 1.0.0 | 1.0.0 | | Livy | 0.7.0-incubating | 0.7.0-incubating | 0.6.0-incubating | 0.6.0-incubating | | MXNet | 1.6.0 | 1.6.0 | 1.5.1 | 1.5.1 | | Mahout | - | - | - | - | | Oozie | 5.2.0 | 5.2.0 | 5.1.0 | 5.1.0 | | Phoenix | 5.0.0-HBase-2.0 | 5.0.0-HBase-2.0 | 5.0.0-HBase-2.0 | 5.0.0-HBase-2.0 | | Pig | 0.17.0 | 0.17.0 | - | - | | Presto | 0.232 | 0.232 | 0.230 | 0.230 | | Spark | 3.0.0-amzn-0.1 | 3.0.0-amzn-0 | 2.4.4 | 2.4.4 | | Sqoop | 1.4.7 | 1.4.7 | - | - | | TensorFlow | 2.1.0 | 2.1.0 | 1.14.0 | 1.14.0 | | Tez | 0.9.2 | 0.9.2 | 0.9.2 | 0.9.2 | | Trino (PrestoSQL) | 338 | 338 | - | - | | Zeppelin | 0.9.0-preview1 | 0.9.0-preview1 | 0.9.0-SNAPSHOT | 0.9.0-SNAPSHOT | | ZooKeeper | 3.4.14 | 3.4.14 | 3.4.14 | 3.4.14 | ## 6.0.1 release notes This is a release to fix issues with Amazon EMR Scaling when it fails to scale up/scale down a cluster successfully or causes application failures. **Changes, Enhancements, and Resolved Issues** + Fixed an issue where scaling requests failed for a large, highly utilized cluster when Amazon EMR on-cluster daemons were running health checking activities, such as gathering YARN node state and HDFS node state. This was happening because on-cluster daemons were not able to communicate the health status data of a node to internal Amazon EMR components. + Improved EMR on-cluster daemons to correctly track the node states when IP addresses are reused to improve reliability during scaling operations. + [SPARK-29683](https://issues.apache.org/jira/browse/SPARK-29683). Fixed an issue where job failures occurred during cluster scale-down as Spark was assuming all available nodes were deny-listed. + [YARN-9011](https://issues.apache.org/jira/browse/YARN-9011). Fixed an issue where job failures occurred due to a race condition in YARN decommissioning when cluster tried to scale up or down. + Fixed issue with step or job failures during cluster scaling by ensuring that the node states are always consistent between the Amazon EMR on-cluster daemons and YARN/HDFS. + Fixed an issue where cluster operations such as scale down and step submission failed for Amazon EMR clusters enabled with Kerberos authentication. This was because the Amazon EMR on-cluster daemon did not renew the Kerberos ticket, which is required to securely communicate with HDFS/YARN running on the primary node. + Newer Amazon EMR releases fix the issue with a lower "Max open files" limit on older AL2 in Amazon EMR. Amazon EMR releases 5.30.1, 5.30.2, 5.31.1, 5.32.1, 6.0.1, 6.1.1, 6.2.1, 5.33.0, 6.3.0 and later now include a permanent fix with a higher "Max open files" setting. + HTTPS is now enabled by default for Amazon Linux repositories. If you are using an Amazon S3 VPCE policy to restrict access to specific buckets, you must add the new Amazon Linux bucket ARN `arn:aws:s3:::amazonlinux-2-repos-$region/*` to your policy (replace `$region` with the region where the endpoint is). For more information, see this topic in the AWS discussion forums. [Announcement: Amazon Linux 2 now supports the ability to use HTTPS while connecting to package repositories ](https://forums.aws.amazon.com/ann.jspa?annID=8528). ## 6.0.1 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | aws-sagemaker-spark-sdk | 1.2.6 | Amazon SageMaker Spark SDK | | emr-ddb | 4.14.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 3.0.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.5.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-s3-dist-cp | 2.14.0 | Distributed copy application optimized for Amazon S3. | | emr-s3-select | 1.5.0 | EMR S3Select Connector | | emrfs | 2.39.0 | Amazon S3 connector for Hadoop ecosystem applications. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 3.2.1-amzn-0.1 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 3.2.1-amzn-0.1 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 3.2.1-amzn-0.1 | HDFS command-line client and library | | hadoop-hdfs-namenode | 3.2.1-amzn-0.1 | HDFS service for tracking file names and block locations. | | hadoop-hdfs-journalnode | 3.2.1-amzn-0.1 | HDFS service for managing the Hadoop filesystem journal on HA clusters. | | hadoop-httpfs-server | 3.2.1-amzn-0.1 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 3.2.1-amzn-0.1 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 3.2.1-amzn-0.1 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 3.2.1-amzn-0.1 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 3.2.1-amzn-0.1 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 3.2.1-amzn-0.1 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 2.2.3 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 2.2.3 | Service for serving one or more HBase regions. | | hbase-client | 2.2.3 | HBase command-line client. | | hbase-rest-server | 2.2.3 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 2.2.3 | Service providing a Thrift endpoint to HBase. | | hcatalog-client | 3.1.2-amzn-0 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 3.1.2-amzn-0 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 3.1.2-amzn-0 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 3.1.2-amzn-0 | Hive command line client. | | hive-hbase | 3.1.2-amzn-0 | Hive-hbase client. | | hive-metastore-server | 3.1.2-amzn-0 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server2 | 3.1.2-amzn-0 | Service for accepting Hive queries as web requests. | | hudi | 0.5.0-incubating-amzn-1 | Incremental processing framework to power data pipline at low latency and high efficiency. | | hudi-presto | 0.5.0-incubating-amzn-1 | Bundle library for running Presto with Hudi. | | hue-server | 4.4.0 | Web application for analyzing data using Hadoop ecosystem applications | | jupyterhub | 1.0.0 | Multi-user server for Jupyter notebooks | | livy-server | 0.6.0-incubating | REST interface for interacting with Apache Spark | | nginx | 1.12.1 | nginx [engine x] is an HTTP and reverse proxy server | | mxnet | 1.5.1 | A flexible, scalable, and efficient library for deep learning. | | mariadb-server | 5.5.64\$1 | MariaDB database server. | | nvidia-cuda | 9.2.88 | Nvidia drivers and Cuda toolkit | | oozie-client | 5.1.0 | Oozie command-line client. | | oozie-server | 5.1.0 | Service for accepting Oozie workflow requests. | | opencv | 3.4.0 | Open Source Computer Vision Library. | | phoenix-library | 5.0.0-HBase-2.0 | The phoenix libraries for server and client | | phoenix-query-server | 5.0.0-HBase-2.0 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.230 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.230 | Service for executing pieces of a query. | | presto-client | 0.230 | Presto command-line client which is installed on an HA cluster's stand-by masters where Presto server is not started. | | r | 3.4.3 | The R Project for Statistical Computing | | spark-client | 2.4.4 | Spark command-line clients. | | spark-history-server | 2.4.4 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 2.4.4 | In-memory execution engine for YARN. | | spark-yarn-slave | 2.4.4 | Apache Spark libraries needed by YARN slaves. | | tensorflow | 1.14.0 | TensorFlow open source software library for high performance numerical computation. | | tez-on-yarn | 0.9.2 | The tez YARN application and libraries. | | webserver | 2.4.41\$1 | Apache HTTP server. | | zeppelin-server | 0.9.0-SNAPSHOT | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.4.14 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.4.14 | ZooKeeper command line client. | ## 6.0.1 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). **emr-6.0.1 classifications** | Classifications | Description | | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | | container-executor | Change values in Hadoop YARN's container-executor.cfg file. | | container-log4j | Change values in Hadoop YARN's container-log4j.properties file. | | core-site | Change values in Hadoop's core-site.xml file. | | emrfs-site | Change EMRFS settings. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | | hadoop-ssl-server | Change hadoop ssl server configuration | | hadoop-ssl-client | Change hadoop ssl client configuration | | hbase | Amazon EMR-curated settings for Apache HBase. | | hbase-env | Change values in HBase's environment. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbase.properties file. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | | hbase-site | Change values in HBase's hbase-site.xml file. | | hdfs-encryption-zones | Configure HDFS encryption zones. | | hdfs-env | Change values in the HDFS environment. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | | hcatalog-env | Change values in HCatalog's environment. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | | hcatalog-webhcat-log4j2 | Change values in HCatalog WebHCat's log4j2.properties. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | | hive | Amazon EMR-curated settings for Apache Hive. | | hive-beeline-log4j2 | Change values in Hive's beeline-log4j2.properties file. | | hive-parquet-logging | Change values in Hive's parquet-logging.properties file. | | hive-env | Change values in the Hive environment. | | hive-exec-log4j2 | Change values in Hive's hive-exec-log4j2.properties file. | | hive-llap-daemon-log4j2 | Change values in Hive's llap-daemon-log4j2.properties file. | | hive-log4j2 | Change values in Hive's hive-log4j2.properties file. | | hive-site | Change values in Hive's hive-site.xml file | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | | hue-ini | Change values in Hue's ini file | | httpfs-env | Change values in the HTTPFS environment. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | | jupyter-notebook-conf | Change values in Jupyter Notebook's jupyter\$1notebook\$1config.py file. | | jupyter-hub-conf | Change values in JupyterHubs's jupyterhub\$1config.py file. | | jupyter-s3-conf | Configure Jupyter Notebook S3 persistence. | | jupyter-sparkmagic-conf | Change values in Sparkmagic's config.json file. | | livy-conf | Change values in Livy's livy.conf file. | | livy-env | Change values in the Livy environment. | | livy-log4j | Change Livy log4j.properties settings. | | mapred-env | Change values in the MapReduce application's environment. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | | oozie-env | Change values in Oozie's environment. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | | oozie-site | Change values in Oozie's oozie-site.xml file. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | | presto-log | Change values in Presto's log.properties file. | | presto-config | Change values in Presto's config.properties file. | | presto-password-authenticator | Change values in Presto's password-authenticator.properties file. | | presto-env | Change values in Presto's presto-env.sh file. | | presto-node | Change values in Presto's node.properties file. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | | presto-connector-hive | Change values in Presto's hive.properties file. | | presto-connector-jmx | Change values in Presto's jmx.properties file. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | | presto-connector-localfile | Change values in Presto's localfile.properties file. | | presto-connector-memory | Change values in Presto's memory.properties file. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | | presto-connector-redis | Change values in Presto's redis.properties file. | | presto-connector-redshift | Change values in Presto's redshift.properties file. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | | presto-connector-tpcds | Change values in Presto's tpcds.properties file. | | ranger-kms-dbks-site | Change values in dbks-site.xml file of Ranger KMS. | | ranger-kms-site | Change values in ranger-kms-site.xml file of Ranger KMS. | | ranger-kms-env | Change values in the Ranger KMS environment. | | ranger-kms-log4j | Change values in kms-log4j.properties file of Ranger KMS. | | ranger-kms-db-ca | Change values for CA file on S3 for MySQL SSL connection with Ranger KMS. | | recordserver-env | Change values in the EMR RecordServer environment. | | recordserver-conf | Change values in EMR RecordServer's erver.properties file. | | recordserver-log4j | Change values in EMR RecordServer's log4j.properties file. | | spark | Amazon EMR-curated settings for Apache Spark. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | | spark-env | Change values in the Spark environment. | | spark-hive-site | Change values in Spark's hive-site.xml file | | spark-log4j | Change values in Spark's log4j.properties file. | | spark-metrics | Change values in Spark's metrics.properties file. | | tez-site | Change values in Tez's tez-site.xml file. | | yarn-env | Change values in the YARN environment. | | yarn-site | Change values in YARN's yarn-site.xml file. | | zeppelin-env | Change values in the Zeppelin environment. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | # Amazon EMR release 6.0.0 ## 6.0.0 application versions This release includes the following applications: [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [https://hudi.apache.org](https://hudi.apache.org), [http://gethue.com/](http://gethue.com/), [https://jupyterhub.readthedocs.io/en/latest/#](https://jupyterhub.readthedocs.io/en/latest/#), [https://livy.incubator.apache.org/](https://livy.incubator.apache.org/), [https://mxnet.incubator.apache.org/](https://mxnet.incubator.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [https://www.tensorflow.org/](https://www.tensorflow.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-6.1.1 | emr-6.1.0 | emr-6.0.1 | emr-6.0.0 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.11.828 | 1.11.828 | 1.11.711 | 1.11.711 | | Python | 2.7, 3.7 | 2.7, 3.7 | 2.7, 3.7 | 2.7, 3.7 | | Scala | 2.12.10 | 2.12.10 | 2.12.10 | 2.12.10 | | AmazonCloudWatchAgent | - | - | - | - | | Delta | - | - | - | - | | Flink | 1.11.0 | 1.11.0 | - | - | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 2.2.5 | 2.2.5 | 2.2.3 | 2.2.3 | | HCatalog | 3.1.2-amzn-2 | 3.1.2-amzn-2 | 3.1.2-amzn-0 | 3.1.2-amzn-0 | | Hadoop | 3.2.1-amzn-1.1 | 3.2.1-amzn-1 | 3.2.1-amzn-0.1 | 3.2.1-amzn-0 | | Hive | 3.1.2-amzn-2 | 3.1.2-amzn-2 | 3.1.2-amzn-0 | 3.1.2-amzn-0 | | Hudi | 0.5.2-incubating-amzn-2 | 0.5.2-incubating-amzn-2 | 0.5.0-incubating-amzn-1 | 0.5.0-incubating-amzn-1 | | Hue | 4.7.1 | 4.7.1 | 4.4.0 | 4.4.0 | | Iceberg | - | - | - | - | | JupyterEnterpriseGateway | - | - | - | - | | JupyterHub | 1.1.0 | 1.1.0 | 1.0.0 | 1.0.0 | | Livy | 0.7.0-incubating | 0.7.0-incubating | 0.6.0-incubating | 0.6.0-incubating | | MXNet | 1.6.0 | 1.6.0 | 1.5.1 | 1.5.1 | | Mahout | - | - | - | - | | Oozie | 5.2.0 | 5.2.0 | 5.1.0 | 5.1.0 | | Phoenix | 5.0.0-HBase-2.0 | 5.0.0-HBase-2.0 | 5.0.0-HBase-2.0 | 5.0.0-HBase-2.0 | | Pig | 0.17.0 | 0.17.0 | - | - | | Presto | 0.232 | 0.232 | 0.230 | 0.230 | | Spark | 3.0.0-amzn-0.1 | 3.0.0-amzn-0 | 2.4.4 | 2.4.4 | | Sqoop | 1.4.7 | 1.4.7 | - | - | | TensorFlow | 2.1.0 | 2.1.0 | 1.14.0 | 1.14.0 | | Tez | 0.9.2 | 0.9.2 | 0.9.2 | 0.9.2 | | Trino (PrestoSQL) | 338 | 338 | - | - | | Zeppelin | 0.9.0-preview1 | 0.9.0-preview1 | 0.9.0-SNAPSHOT | 0.9.0-SNAPSHOT | | ZooKeeper | 3.4.14 | 3.4.14 | 3.4.14 | 3.4.14 | ## 6.0.0 release notes The following release notes include information for Amazon EMR release 6.0.0. Initial release date: March 10, 2020 **Supported applications** + AWS SDK for Java version 1.11.711 + Ganglia version 3.7.2 + Hadoop version 3.2.1 + HBase version 2.2.3 + HCatalog version 3.1.2 + Hive version 3.1.2 + Hudi version 0.5.0-incubating + Hue version 4.4.0 + JupyterHub version 1.0.0 + Livy version 0.6.0 + MXNet version 1.5.1 + Oozie version 5.1.0 + Phoenix version 5.0.0 + Presto version 0.230 + Spark version 2.4.4 + TensorFlow version 1.14.0 + Zeppelin version 0.9.0-SNAPSHOT + Zookeeper version 3.4.14 + Connectors and drivers: DynamoDB Connector 4.14.0 **Note** Flink, Sqoop, Pig, and Mahout are not available in Amazon EMR version 6.0.0. **New features** + YARN Docker Runtime Support - YARN applications, such as Spark jobs, can now run in the context of a Docker container. This allows you to easily define dependencies in a Docker image without the need to install custom libraries on your Amazon EMR cluster. For more information, see [Configure Docker Integration](https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-plan-docker.html) and [Run Spark applications with Docker using Amazon EMR 6.0.0](https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-spark-docker.html). + Hive LLAP Support - Hive now supports the LLAP execution mode for improved query performance. For more information, see [Using Hive LLAP](https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-hive-llap.html). **Changes, enhancements, and resolved issues** + This is a release to fix issues with Amazon EMR Scaling when it fails to scale up/scale down a cluster successfully or causes application failures. + Fixed an issue where scaling requests failed for a large, highly utilized cluster when Amazon EMR on-cluster daemons were running health checking activities, such as gathering YARN node state and HDFS node state. This was happening because on-cluster daemons were not able to communicate the health status data of a node to internal Amazon EMR components. + Improved EMR on-cluster daemons to correctly track the node states when IP addresses are reused to improve reliability during scaling operations. + [SPARK-29683](https://issues.apache.org/jira/browse/SPARK-29683). Fixed an issue where job failures occurred during cluster scale-down as Spark was assuming all available nodes were deny-listed. + [YARN-9011](https://issues.apache.org/jira/browse/YARN-9011). Fixed an issue where job failures occurred due to a race condition in YARN decommissioning when cluster tried to scale up or down. + Fixed issue with step or job failures during cluster scaling by ensuring that the node states are always consistent between the Amazon EMR on-cluster daemons and YARN/HDFS. + Fixed an issue where cluster operations such as scale down and step submission failed for Amazon EMR clusters enabled with Kerberos authentication. This was because the Amazon EMR on-cluster daemon did not renew the Kerberos ticket, which is required to securely communicate with HDFS/YARN running on the primary node. + Newer Amazon EMR releases fix the issue with a lower "Max open files" limit on older AL2 in Amazon EMR. Amazon EMR releases 5.30.1, 5.30.2, 5.31.1, 5.32.1, 6.0.1, 6.1.1, 6.2.1, 5.33.0, 6.3.0 and later now include a permanent fix with a higher "Max open files" setting. + Amazon Linux + Amazon Linux 2 is the operating system for the EMR 6.x release series. + `systemd` is used for service management instead of `upstart` used inAmazon Linux 1. + Java Development Kit (JDK) + Corretto JDK 8 is the default JDK for the EMR 6.x release series. + Scala + Scala 2.12 is used with Apache Spark and Apache Livy. + Python 3 + Python 3 is now the default version of Python in EMR. + YARN node labels + Beginning with Amazon EMR 6.x release series, the YARN node labels feature is disabled by default. The application master processes can run on both core and task nodes by default. You can enable the YARN node labels feature by configuring following properties: `yarn.node-labels.enabled` and `yarn.node-labels.am.default-node-label-expression`. For more information, see [Understanding Primary, Core, and Task Nodes](https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-master-core-task-nodes.html). **Known issues** + **Lower "Max open files" limit on older AL2 [fixed in newer releases].** Amazon EMR releases: emr-5.30.x, emr-5.31.0, emr-5.32.0, emr-6.0.0, emr-6.1.0, and emr-6.2.0 are based on older versions ofAmazon Linux 2 (AL2), which have a lower ulimit setting for "Max open files" when Amazon EMR clusters are created with the default AMI. Amazon EMR releases 5.30.1, 5.30.2, 5.31.1, 5.32.1, 6.0.1, 6.1.1, 6.2.1, 5.33.0, 6.3.0 and later include a permanent fix with a higher "Max open files" setting. Releases with the lower open file limit causes a "Too many open files" error when submitting Spark job. In the impacted releases, the Amazon EMR default AMI has a default ulimit setting of 4096 for "Max open files," which is lower than the 65536 file limit in the latestAmazon Linux 2 AMI. The lower ulimit setting for "Max open files" causes Spark job failure when the Spark driver and executor try to open more than 4096 files. To fix the issue, Amazon EMR has a bootstrap action (BA) script that adjusts the ulimit setting at cluster creation. If you are using an older Amazon EMR version that doesn't have the permanent fix for this issue, the following workaround lets you to explicitly set the instance-controller ulimit to a maximum of 65536 files. **Explicitly set a ulimit from the command line** 1. Edit `/etc/systemd/system/instance-controller.service` to add the following parameters to Service section. `LimitNOFILE=65536` `LimitNPROC=65536` 1. Restart InstanceController `$ sudo systemctl daemon-reload` `$ sudo systemctl restart instance-controller` **Set a ulimit using bootstrap action (BA)** You can also use a bootstrap action (BA) script to configure the instance-controller ulimit to 65536 files at cluster creation. ``` #!/bin/bash for user in hadoop spark hive; do sudo tee /etc/security/limits.d/$user.conf << EOF $user - nofile 65536 $user - nproc 65536 EOF done for proc in instancecontroller logpusher; do sudo mkdir -p /etc/systemd/system/$proc.service.d/ sudo tee /etc/systemd/system/$proc.service.d/override.conf << EOF [Service] LimitNOFILE=65536 LimitNPROC=65536 EOF pid=$(pgrep -f aws157.$proc.Main) sudo prlimit --pid $pid --nofile=65535:65535 --nproc=65535:65535 done sudo systemctl daemon-reload ``` + Spark interactive shell, including PySpark, SparkR, and spark-shell, does not support using Docker with additional libraries. + To use Python 3 with Amazon EMR version 6.0.0, you must add `PATH` to `yarn.nodemanager.env-whitelist`. + The Live Long and Process (LLAP) functionality is not supported when you use the AWS Glue Data Catalog as the metastore for Hive. + When using Amazon EMR 6.0.0 with Spark and Docker integration, you need to configure the instances in your cluster with the same instance type and the same amount of EBS volumes to avoid failure when submitting a Spark job with Docker runtime. + In Amazon EMR 6.0.0, HBase on Amazon S3 storage mode is impacted by the [HBASE-24286](https://issues.apache.org/jira/browse/HBASE-24286). issue. HBase master cannot initialize when the cluster is created using existing S3 data. + Known issue in clusters with multiple primary nodes and Kerberos authentication If you run clusters with multiple primary nodes and Kerberos authentication in Amazon EMR releases 5.20.0 and later, you may encounter problems with cluster operations such as scale down or step submission, after the cluster has been running for some time. The time period depends on the Kerberos ticket validity period that you defined. The scale-down problem impacts both automatic scale-down and explicit scale down requests that you submitted. Additional cluster operations can also be impacted. Workaround: + SSH as `hadoop` user to the lead primary node of the EMR cluster with multiple primary nodes. + Run the following command to renew Kerberos ticket for `hadoop` user. ``` kinit -kt ``` Typically, the keytab file is located at `/etc/hadoop.keytab` and the principal is in the form of `hadoop/@`. **Note** This workaround will be effective for the time period the Kerberos ticket is valid. This duration is 10 hours by default, but can configured by your Kerberos settings. You must re-run the above command once the Kerberos ticket expires. ## 6.0.0 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | aws-sagemaker-spark-sdk | 1.2.6 | Amazon SageMaker Spark SDK | | emr-ddb | 4.14.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 3.0.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.5.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-s3-dist-cp | 2.14.0 | Distributed copy application optimized for Amazon S3. | | emr-s3-select | 1.5.0 | EMR S3Select Connector | | emrfs | 2.39.0 | Amazon S3 connector for Hadoop ecosystem applications. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 3.2.1-amzn-0 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 3.2.1-amzn-0 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 3.2.1-amzn-0 | HDFS command-line client and library | | hadoop-hdfs-namenode | 3.2.1-amzn-0 | HDFS service for tracking file names and block locations. | | hadoop-hdfs-journalnode | 3.2.1-amzn-0 | HDFS service for managing the Hadoop filesystem journal on HA clusters. | | hadoop-httpfs-server | 3.2.1-amzn-0 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 3.2.1-amzn-0 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 3.2.1-amzn-0 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 3.2.1-amzn-0 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 3.2.1-amzn-0 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 3.2.1-amzn-0 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 2.2.3 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 2.2.3 | Service for serving one or more HBase regions. | | hbase-client | 2.2.3 | HBase command-line client. | | hbase-rest-server | 2.2.3 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 2.2.3 | Service providing a Thrift endpoint to HBase. | | hcatalog-client | 3.1.2-amzn-0 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 3.1.2-amzn-0 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 3.1.2-amzn-0 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 3.1.2-amzn-0 | Hive command line client. | | hive-hbase | 3.1.2-amzn-0 | Hive-hbase client. | | hive-metastore-server | 3.1.2-amzn-0 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server2 | 3.1.2-amzn-0 | Service for accepting Hive queries as web requests. | | hudi | 0.5.0-incubating-amzn-1 | Incremental processing framework to power data pipline at low latency and high efficiency. | | hudi-presto | 0.5.0-incubating-amzn-1 | Bundle library for running Presto with Hudi. | | hue-server | 4.4.0 | Web application for analyzing data using Hadoop ecosystem applications | | jupyterhub | 1.0.0 | Multi-user server for Jupyter notebooks | | livy-server | 0.6.0-incubating | REST interface for interacting with Apache Spark | | nginx | 1.12.1 | nginx [engine x] is an HTTP and reverse proxy server | | mxnet | 1.5.1 | A flexible, scalable, and efficient library for deep learning. | | mariadb-server | 5.5.64\$1 | MariaDB database server. | | nvidia-cuda | 9.2.88 | Nvidia drivers and Cuda toolkit | | oozie-client | 5.1.0 | Oozie command-line client. | | oozie-server | 5.1.0 | Service for accepting Oozie workflow requests. | | opencv | 3.4.0 | Open Source Computer Vision Library. | | phoenix-library | 5.0.0-HBase-2.0 | The phoenix libraries for server and client | | phoenix-query-server | 5.0.0-HBase-2.0 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.230 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.230 | Service for executing pieces of a query. | | presto-client | 0.230 | Presto command-line client which is installed on an HA cluster's stand-by masters where Presto server is not started. | | r | 3.4.3 | The R Project for Statistical Computing | | spark-client | 2.4.4 | Spark command-line clients. | | spark-history-server | 2.4.4 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 2.4.4 | In-memory execution engine for YARN. | | spark-yarn-slave | 2.4.4 | Apache Spark libraries needed by YARN slaves. | | tensorflow | 1.14.0 | TensorFlow open source software library for high performance numerical computation. | | tez-on-yarn | 0.9.2 | The tez YARN application and libraries. | | webserver | 2.4.41\$1 | Apache HTTP server. | | zeppelin-server | 0.9.0-SNAPSHOT | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.4.14 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.4.14 | ZooKeeper command line client. | ## 6.0.0 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). **emr-6.0.0 classifications** | Classifications | Description | | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | | container-executor | Change values in Hadoop YARN's container-executor.cfg file. | | container-log4j | Change values in Hadoop YARN's container-log4j.properties file. | | core-site | Change values in Hadoop's core-site.xml file. | | emrfs-site | Change EMRFS settings. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | | hadoop-ssl-server | Change hadoop ssl server configuration | | hadoop-ssl-client | Change hadoop ssl client configuration | | hbase | Amazon EMR-curated settings for Apache HBase. | | hbase-env | Change values in HBase's environment. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbase.properties file. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | | hbase-site | Change values in HBase's hbase-site.xml file. | | hdfs-encryption-zones | Configure HDFS encryption zones. | | hdfs-env | Change values in the HDFS environment. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | | hcatalog-env | Change values in HCatalog's environment. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | | hcatalog-webhcat-log4j2 | Change values in HCatalog WebHCat's log4j2.properties. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | | hive | Amazon EMR-curated settings for Apache Hive. | | hive-beeline-log4j2 | Change values in Hive's beeline-log4j2.properties file. | | hive-parquet-logging | Change values in Hive's parquet-logging.properties file. | | hive-env | Change values in the Hive environment. | | hive-exec-log4j2 | Change values in Hive's hive-exec-log4j2.properties file. | | hive-llap-daemon-log4j2 | Change values in Hive's llap-daemon-log4j2.properties file. | | hive-log4j2 | Change values in Hive's hive-log4j2.properties file. | | hive-site | Change values in Hive's hive-site.xml file | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | | hue-ini | Change values in Hue's ini file | | httpfs-env | Change values in the HTTPFS environment. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | | jupyter-notebook-conf | Change values in Jupyter Notebook's jupyter\$1notebook\$1config.py file. | | jupyter-hub-conf | Change values in JupyterHubs's jupyterhub\$1config.py file. | | jupyter-s3-conf | Configure Jupyter Notebook S3 persistence. | | jupyter-sparkmagic-conf | Change values in Sparkmagic's config.json file. | | livy-conf | Change values in Livy's livy.conf file. | | livy-env | Change values in the Livy environment. | | livy-log4j | Change Livy log4j.properties settings. | | mapred-env | Change values in the MapReduce application's environment. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | | oozie-env | Change values in Oozie's environment. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | | oozie-site | Change values in Oozie's oozie-site.xml file. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | | presto-log | Change values in Presto's log.properties file. | | presto-config | Change values in Presto's config.properties file. | | presto-password-authenticator | Change values in Presto's password-authenticator.properties file. | | presto-env | Change values in Presto's presto-env.sh file. | | presto-node | Change values in Presto's node.properties file. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | | presto-connector-hive | Change values in Presto's hive.properties file. | | presto-connector-jmx | Change values in Presto's jmx.properties file. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | | presto-connector-localfile | Change values in Presto's localfile.properties file. | | presto-connector-memory | Change values in Presto's memory.properties file. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | | presto-connector-redis | Change values in Presto's redis.properties file. | | presto-connector-redshift | Change values in Presto's redshift.properties file. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | | presto-connector-tpcds | Change values in Presto's tpcds.properties file. | | ranger-kms-dbks-site | Change values in dbks-site.xml file of Ranger KMS. | | ranger-kms-site | Change values in ranger-kms-site.xml file of Ranger KMS. | | ranger-kms-env | Change values in the Ranger KMS environment. | | ranger-kms-log4j | Change values in kms-log4j.properties file of Ranger KMS. | | ranger-kms-db-ca | Change values for CA file on S3 for MySQL SSL connection with Ranger KMS. | | recordserver-env | Change values in the EMR RecordServer environment. | | recordserver-conf | Change values in EMR RecordServer's erver.properties file. | | recordserver-log4j | Change values in EMR RecordServer's log4j.properties file. | | spark | Amazon EMR-curated settings for Apache Spark. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | | spark-env | Change values in the Spark environment. | | spark-hive-site | Change values in Spark's hive-site.xml file | | spark-log4j | Change values in Spark's log4j.properties file. | | spark-metrics | Change values in Spark's metrics.properties file. | | tez-site | Change values in Tez's tez-site.xml file. | | yarn-env | Change values in the YARN environment. | | yarn-site | Change values in YARN's yarn-site.xml file. | | zeppelin-env | Change values in the Zeppelin environment. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | # Amazon EMR 5.x release versions This section contains application versions, release notes, component versions, and configuration classifications available in each Amazon EMR 5.x release version. When you launch a cluster, you can choose from multiple releases of Amazon EMR. This allows you to test and use application versions that fit your compatibility requirements. You specify the release number with the *release label*. Release labels are in the form `emr-x.x.x`. For example, `emr-7.12.0`. New Amazon EMR releases are made available in different Regions over a period of several days, beginning with the first Region on the initial release date. The latest release version may not be available in your Region during this period. For a comprehensive table of application versions in every Amazon EMR 5.x release, see [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md). **Topics** + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Amazon EMR release 5.36.2](emr-5362-release.md) + [Amazon EMR release 5.36.1](emr-5361-release.md) + [Amazon EMR release 5.36.0](emr-5360-release.md) + [Amazon EMR release 5.35.0](emr-5350-release.md) + [Amazon EMR release 5.34.0](emr-5340-release.md) + [Amazon EMR release 5.33.1](emr-5331-release.md) + [Amazon EMR release 5.33.0](emr-5330-release.md) + [Amazon EMR release 5.32.1](emr-5321-release.md) + [Amazon EMR release 5.32.0](emr-5320-release.md) + [Amazon EMR release 5.31.1](emr-5311-release.md) + [Amazon EMR release 5.31.0](emr-5310-release.md) + [Amazon EMR release 5.30.2](emr-5302-release.md) + [Amazon EMR release 5.30.1](emr-5301-release.md) + [Amazon EMR release 5.30.0](emr-5300-release.md) + [Amazon EMR release 5.29.0](emr-5290-release.md) + [Amazon EMR release 5.28.1](emr-5281-release.md) + [Amazon EMR release 5.28.0](emr-5280-release.md) + [Amazon EMR release 5.27.1](emr-5271-release.md) + [Amazon EMR release 5.27.0](emr-5270-release.md) + [Amazon EMR release 5.26.0](emr-5260-release.md) + [Amazon EMR release 5.25.0](emr-5250-release.md) + [Amazon EMR release 5.24.1](emr-5241-release.md) + [Amazon EMR release 5.24.0](emr-5240-release.md) + [Amazon EMR release 5.23.1](emr-5231-release.md) + [Amazon EMR release 5.23.0](emr-5230-release.md) + [Amazon EMR release 5.22.0](emr-5220-release.md) + [Amazon EMR release 5.21.2](emr-5212-release.md) + [Amazon EMR release 5.21.1](emr-5211-release.md) + [Amazon EMR release 5.21.0](emr-5210-release.md) + [Amazon EMR release 5.20.1](emr-5201-release.md) + [Amazon EMR release 5.20.0](emr-5200-release.md) + [Amazon EMR release 5.19.1](emr-5191-release.md) + [Amazon EMR release 5.19.0](emr-5190-release.md) + [Amazon EMR release 5.18.1](emr-5181-release.md) + [Amazon EMR release 5.18.0](emr-5180-release.md) + [Amazon EMR release 5.17.2](emr-5172-release.md) + [Amazon EMR release 5.17.1](emr-5171-release.md) + [Amazon EMR release 5.17.0](emr-5170-release.md) + [Amazon EMR release 5.16.1](emr-5161-release.md) + [Amazon EMR release 5.16.0](emr-5160-release.md) + [Amazon EMR release 5.15.1](emr-5151-release.md) + [Amazon EMR release 5.15.0](emr-5150-release.md) + [Amazon EMR release 5.14.2](emr-5142-release.md) + [Amazon EMR release 5.14.1](emr-5141-release.md) + [Amazon EMR release 5.14.0](emr-5140-release.md) + [Amazon EMR release 5.13.1](emr-5131-release.md) + [Amazon EMR release 5.13.0](emr-5130-release.md) + [Amazon EMR release 5.12.3](emr-5123-release.md) + [Amazon EMR release 5.12.2](emr-5122-release.md) + [Amazon EMR release 5.12.1](emr-5121-release.md) + [Amazon EMR release 5.12.0](emr-5120-release.md) + [Amazon EMR release 5.11.4](emr-5114-release.md) + [Amazon EMR release 5.11.3](emr-5113-release.md) + [Amazon EMR release 5.11.2](emr-5112-release.md) + [Amazon EMR release 5.11.1](emr-5111-release.md) + [Amazon EMR release 5.11.0](emr-5110-release.md) + [Amazon EMR release 5.10.1](emr-5101-release.md) + [Amazon EMR release 5.10.0](emr-5100-release.md) + [Amazon EMR release 5.9.1](emr-591-release.md) + [Amazon EMR release 5.9.0](emr-590-release.md) + [Amazon EMR release 5.8.3](emr-583-release.md) + [Amazon EMR release 5.8.2](emr-582-release.md) + [Amazon EMR release 5.8.1](emr-581-release.md) + [Amazon EMR release 5.8.0](emr-580-release.md) + [Amazon EMR release 5.7.1](emr-571-release.md) + [Amazon EMR release 5.7.0](emr-570-release.md) + [Amazon EMR release 5.6.1](emr-561-release.md) + [Amazon EMR release 5.6.0](emr-560-release.md) + [Amazon EMR release 5.5.4](emr-554-release.md) + [Amazon EMR release 5.5.3](emr-553-release.md) + [Amazon EMR release 5.5.2](emr-552-release.md) + [Amazon EMR release 5.5.1](emr-551-release.md) + [Amazon EMR release 5.5.0](emr-550-release.md) + [Amazon EMR release 5.4.1](emr-541-release.md) + [Amazon EMR release 5.4.0](emr-540-release.md) + [Amazon EMR release 5.3.2](emr-532-release.md) + [Amazon EMR release 5.3.1](emr-531-release.md) + [Amazon EMR release 5.3.0](emr-530-release.md) + [Amazon EMR release 5.2.3](emr-523-release.md) + [Amazon EMR release 5.2.2](emr-522-release.md) + [Amazon EMR release 5.2.1](emr-521-release.md) + [Amazon EMR release 5.2.0](emr-520-release.md) + [Amazon EMR release 5.1.1](emr-511-release.md) + [Amazon EMR release 5.1.0](emr-510-release.md) + [Amazon EMR release 5.0.3](emr-503-release.md) + [Amazon EMR release 5.0.2](emr-502-release.md) + [Amazon EMR release 5.0.1](emr-501-release.md) + [Amazon EMR release 5.0.0](emr-500-release.md) # Application versions in Amazon EMR 5.x releases The following table lists the application versions that are available in each Amazon EMR 5.x release. **Application version information** | | emr-5.36.2 | emr-5.36.1 | emr-5.36.0 | emr-5.35.0 | emr-5.34.0 | emr-5.33.1 | emr-5.33.0 | emr-5.32.1 | emr-5.32.0 | emr-5.31.1 | emr-5.31.0 | emr-5.30.2 | emr-5.30.1 | emr-5.30.0 | emr-5.29.0 | emr-5.28.1 | emr-5.28.0 | emr-5.27.1 | emr-5.27.0 | emr-5.26.0 | emr-5.25.0 | emr-5.24.1 | emr-5.24.0 | emr-5.23.1 | emr-5.23.0 | emr-5.22.0 | emr-5.21.2 | emr-5.21.1 | emr-5.21.0 | emr-5.20.1 | emr-5.20.0 | emr-5.19.1 | emr-5.19.0 | emr-5.18.1 | emr-5.18.0 | emr-5.17.2 | emr-5.17.1 | emr-5.17.0 | emr-5.16.1 | emr-5.16.0 | emr-5.15.1 | emr-5.15.0 | emr-5.14.2 | emr-5.14.1 | emr-5.14.0 | emr-5.13.1 | emr-5.13.0 | emr-5.12.3 | emr-5.12.2 | emr-5.12.1 | emr-5.12.0 | emr-5.11.4 | emr-5.11.3 | emr-5.11.2 | emr-5.11.1 | emr-5.11.0 | emr-5.10.1 | emr-5.10.0 | emr-5.9.1 | emr-5.9.0 | emr-5.8.3 | emr-5.8.2 | emr-5.8.1 | emr-5.8.0 | emr-5.7.1 | emr-5.7.0 | emr-5.6.1 | emr-5.6.0 | emr-5.5.4 | emr-5.5.3 | emr-5.5.2 | emr-5.5.1 | emr-5.5.0 | emr-5.4.1 | emr-5.4.0 | emr-5.3.2 | emr-5.3.1 | emr-5.3.0 | emr-5.2.3 | emr-5.2.2 | emr-5.2.1 | emr-5.2.0 | emr-5.1.1 | emr-5.1.0 | emr-5.0.3 | emr-5.0.2 | emr-5.0.1 | emr-5.0.0 | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.12.206 | 1.12.206 | 1.12.206 | 1.12.159 | 1.11.970 | 1.11.970 | 1.11.970 | 1.11.890 | 1.11.890 | 1.11.852 | 1.11.852 | 1.11.759 | 1.11.759 | 1.11.759 | 1.11.682 | 1.11.659 | 1.11.659 | 1.11.615 | 1.11.615 | 1.11.595 | 1.11.566 | 1.11.546 | 1.11.546 | 1.11.519 | 1.11.519 | 1.11.510 | 1.11.479 | 1.11.479 | 1.11.479 | 1.11.461 | 1.11.461 | 1.11.433 | 1.11.433 | 1.11.393 | 1.11.393 | 1.11.336 | 1.11.336 | 1.11.336 | 1.11.336 | 1.11.336 | 1.11.333 | 1.11.333 | 1.11.297 | 1.11.297 | 1.11.297 | 1.11.297 | 1.11.297 | 1.11.267 | 1.11.267 | 1.11.267 | 1.11.267 | 1.11.238 | 1.11.238 | 1.11.238 | 1.11.238 | 1.11.238 | 1.11.221 | 1.11.221 | 1.11.183 | 1.11.183 | 1.11.160 | 1.11.160 | 1.11.160 | 1.11.160 | 1.10.75 | 1.10.75 | 1.10.75 | 1.10.75 | 1.10.75 | 1.10.75 | 1.10.75 | 1.10.75 | 1.10.75 | 1.10.75 | 1.10.75 | 1.10.75 | 1.10.75 | 1.10.75 | 1.10.75 | 1.10.75 | 1.10.75 | 1.10.75 | 1.10.75 | 1.10.75 | 1.10.75 | 1.10.75 | 1.10.75 | 1.10.75 | | Python | 2.7, 3.7 | 2.7, 3.7 | 2.7, 3.7 | 2.7, 3.7 | 2.7, 3.7 | 2.7, 3.7 | 2.7, 3.7 | 2.7, 3.7 | 2.7, 3.7 | 2.7, 3.7 | 2.7, 3.7 | 2.7, 3.7 | 2.7, 3.7 | 2.7, 3.7 | 2.7, 3.6 | 2.7, 3.6 | 2.7, 3.6 | 2.7, 3.6 | 2.7, 3.6 | 2.7, 3.6 | 2.7, 3.6 | 2.7, 3.6 | 2.7, 3.6 | 2.7, 3.6 | 2.7, 3.6 | 2.7, 3.6 | 2.7, 3.6 | 2.7, 3.6 | 2.7, 3.6 | 2.7, 3.6 | 2.7, 3.6 | 2.7, 3.4 | 2.7, 3.4 | 2.7, 3.4 | 2.7, 3.4 | 2.7, 3.4 | 2.7, 3.4 | 2.7, 3.4 | 2.7, 3.4 | 2.7, 3.4 | 2.7, 3.4 | 2.7, 3.4 | 2.7, 3.4 | 2.7, 3.4 | 2.7, 3.4 | 2.7, 3.4 | 2.7, 3.4 | 2.7, 3.4 | 2.7, 3.4 | 2.7, 3.4 | 2.7, 3.4 | 2.7, 3.4 | 2.7, 3.4 | 2.7, 3.4 | 2.7, 3.4 | 2.7, 3.4 | 2.7, 3.4 | 2.7, 3.4 | Not tracked | Not tracked | Not tracked | Not tracked | Not tracked | Not tracked | Not tracked | Not tracked | Not tracked | Not tracked | Not tracked | Not tracked | Not tracked | Not tracked | Not tracked | Not tracked | Not tracked | Not tracked | Not tracked | Not tracked | Not tracked | Not tracked | Not tracked | Not tracked | Not tracked | Not tracked | Not tracked | Not tracked | Not tracked | Not tracked | | Scala | 2.11.12 | 2.11.12 | 2.11.12 | 2.11.12 | 2.11.12 | 2.11.12 | 2.11.12 | 2.11.12 | 2.11.12 | 2.11.12 | 2.11.12 | 2.11.12 | 2.11.12 | 2.11.12 | 2.11.12 | 2.11.12 | 2.11.12 | 2.11.12 | 2.11.12 | 2.11.12 | 2.11.12 | 2.11.12 | 2.11.12 | 2.11.12 | 2.11.12 | 2.11.12 | 2.11.12 | 2.11.12 | 2.11.12 | 2.11.12 | 2.11.12 | 2.11.8 | 2.11.8 | 2.11.8 | 2.11.8 | 2.11.8 | 2.11.8 | 2.11.8 | 2.11.8 | 2.11.8 | 2.11.8 | 2.11.8 | 2.11.8 | 2.11.8 | 2.11.8 | 2.11.8 | 2.11.8 | 2.11.8 | 2.11.8 | 2.11.8 | 2.11.8 | 2.11.8 | 2.11.8 | 2.11.8 | 2.11.8 | 2.11.8 | 2.11.8 | 2.11.8 | 2.11.8 | 2.11.8 | 2.11.8 | 2.11.8 | 2.11.8 | 2.11.8 | 2.11.8 | 2.11.8 | 2.11.8 | 2.11.8 | 2.11.8 | 2.11.8 | 2.11.8 | 2.11.8 | 2.11.8 | 2.11.8 | 2.11.8 | 2.11.8 | 2.11.8 | 2.11.8 | 2.11.8 | 2.11.8 | 2.11.8 | 2.11.8 | 2.11.8 | 2.11.8 | 2.11.8 | 2.11.8 | 2.11.8 | 2.11.8 | | AmazonCloudWatchAgent | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | Delta | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | Flink | 1.14.2 | 1.14.2 | 1.14.2 | 1.14.2 | 1.13.1 | 1.12.1 | 1.12.1 | 1.11.2 | 1.11.2 | 1.11.0 | 1.11.0 | 1.10.0 | 1.10.0 | 1.10.0 | 1.9.1 | 1.9.0 | 1.9.0 | 1.8.1 | 1.8.1 | 1.8.0 | 1.8.0 | 1.8.0 | 1.8.0 | 1.7.1 | 1.7.1 | 1.7.1 | 1.7.0 | 1.7.0 | 1.7.0 | 1.6.2 | 1.6.2 | 1.6.1 | 1.6.1 | 1.6.0 | 1.6.0 | 1.5.2 | 1.5.2 | 1.5.2 | 1.5.0 | 1.5.0 | 1.4.2 | 1.4.2 | 1.4.2 | 1.4.2 | 1.4.2 | 1.4.0 | 1.4.0 | 1.4.0 | 1.4.0 | 1.4.0 | 1.4.0 | 1.3.2 | 1.3.2 | 1.3.2 | 1.3.2 | 1.3.2 | 1.3.2 | 1.3.2 | 1.3.2 | 1.3.2 | 1.3.1 | 1.3.1 | 1.3.1 | 1.3.1 | 1.3.0 | 1.3.0 | 1.2.1 | 1.2.1 | 1.2.0 | 1.2.0 | 1.2.0 | 1.2.0 | 1.2.0 | 1.2.0 | 1.2.0 | 1.1.4 | 1.1.4 | 1.1.4 | 1.1.3 | 1.1.3 | 1.1.3 | 1.1.3 | 1.1.3 | 1.1.3 | - | - | - | - | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 1.4.13 | 1.4.13 | 1.4.13 | 1.4.13 | 1.4.13 | 1.4.13 | 1.4.13 | 1.4.13 | 1.4.13 | 1.4.13 | 1.4.13 | 1.4.13 | 1.4.13 | 1.4.13 | 1.4.10 | 1.4.10 | 1.4.10 | 1.4.10 | 1.4.10 | 1.4.10 | 1.4.9 | 1.4.9 | 1.4.9 | 1.4.9 | 1.4.9 | 1.4.9 | 1.4.8 | 1.4.8 | 1.4.8 | 1.4.8 | 1.4.8 | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.6 | 1.4.6 | 1.4.6 | 1.4.4 | 1.4.4 | 1.4.4 | 1.4.4 | 1.4.2 | 1.4.2 | 1.4.2 | 1.4.2 | 1.4.2 | 1.4.0 | 1.4.0 | 1.4.0 | 1.4.0 | 1.3.1 | 1.3.1 | 1.3.1 | 1.3.1 | 1.3.1 | 1.3.1 | 1.3.1 | 1.3.1 | 1.3.1 | 1.3.1 | 1.3.1 | 1.3.1 | 1.3.1 | 1.3.1 | 1.3.1 | 1.3.0 | 1.3.0 | 1.3.0 | 1.3.0 | 1.3.0 | 1.3.0 | 1.3.0 | 1.3.0 | 1.3.0 | 1.2.3 | 1.2.3 | 1.2.3 | 1.2.3 | 1.2.3 | 1.2.3 | 1.2.3 | 1.2.3 | 1.2.3 | 1.2.2 | 1.2.2 | 1.2.2 | 1.2.2 | | HCatalog | 2.3.9-amzn-2 | 2.3.9-amzn-2 | 2.3.9-amzn-2 | 2.3.9-amzn-0 | 2.3.8-amzn-0 | 2.3.7-amzn-4 | 2.3.7-amzn-4 | 2.3.7-amzn-3 | 2.3.7-amzn-3 | 2.3.7-amzn-1 | 2.3.7-amzn-1 | 2.3.6-amzn-2 | 2.3.6-amzn-2 | 2.3.6-amzn-2 | 2.3.6-amzn-1 | 2.3.6-amzn-0 | 2.3.6-amzn-0 | 2.3.5-amzn-1 | 2.3.5-amzn-1 | 2.3.5-amzn-0 | 2.3.5-amzn-0 | 2.3.4-amzn-2 | 2.3.4-amzn-2 | 2.3.4-amzn-1 | 2.3.4-amzn-1 | 2.3.4-amzn-1 | 2.3.4-amzn-0 | 2.3.4-amzn-0 | 2.3.4-amzn-0 | 2.3.4-amzn-0 | 2.3.4-amzn-0 | 2.3.3-amzn-2 | 2.3.3-amzn-2 | 2.3.3-amzn-2 | 2.3.3-amzn-2 | 2.3.3-amzn-1 | 2.3.3-amzn-1 | 2.3.3-amzn-1 | 2.3.3-amzn-1 | 2.3.3-amzn-1 | 2.3.3-amzn-0 | 2.3.3-amzn-0 | 2.3.2-amzn-2 | 2.3.2-amzn-2 | 2.3.2-amzn-2 | 2.3.2-amzn-2 | 2.3.2-amzn-2 | 2.3.2-amzn-1 | 2.3.2-amzn-1 | 2.3.2-amzn-1 | 2.3.2-amzn-1 | 2.3.2-amzn-0 | 2.3.2-amzn-0 | 2.3.2-amzn-0 | 2.3.2-amzn-0 | 2.3.2-amzn-0 | 2.3.1-amzn-0 | 2.3.1-amzn-0 | 2.3.0-amzn-0 | 2.3.0-amzn-0 | 2.3.0-amzn-0 | 2.3.0-amzn-0 | 2.3.0-amzn-0 | 2.3.0-amzn-0 | 2.1.1-amzn-0 | 2.1.1-amzn-0 | 2.1.1-amzn-0 | 2.1.1-amzn-0 | 2.1.1-amzn-0 | 2.1.1-amzn-0 | 2.1.1-amzn-0 | 2.1.1-amzn-0 | 2.1.1-amzn-0 | 2.1.1-amzn-0 | 2.1.1-amzn-0 | 2.1.1-amzn-0 | 2.1.1-amzn-0 | 2.1.1-amzn-0 | 2.1.0-amzn-0 | 2.1.0-amzn-0 | 2.1.0-amzn-0 | 2.1.0-amzn-0 | 2.1.0-amzn-0 | 2.1.0-amzn-0 | 2.1.0-amzn-0 | 2.1.0-amzn-0 | 2.1.0-amzn-0 | 2.1.0-amzn-0 | | Hadoop | 2.10.1-amzn-4 | 2.10.1-amzn-4 | 2.10.1-amzn-4 | 2.10.1-amzn-3 | 2.10.1-amzn-2 | 2.10.1-amzn-1.1 | 2.10.1-amzn-1 | 2.10.1-amzn-0.1 | 2.10.1-amzn-0 | 2.10.0-amzn-0.1 | 2.10.0-amzn-0 | 2.8.5-amzn-6.1 | 2.8.5-amzn-6 | 2.8.5-amzn-6 | 2.8.5-amzn-5 | 2.8.5-amzn-5 | 2.8.5-amzn-5 | 2.8.5-amzn-4 | 2.8.5-amzn-4 | 2.8.5-amzn-4 | 2.8.5-amzn-4 | 2.8.5-amzn-4 | 2.8.5-amzn-4 | 2.8.5-amzn-3 | 2.8.5-amzn-3 | 2.8.5-amzn-2 | 2.8.5-amzn-1 | 2.8.5-amzn-1 | 2.8.5-amzn-1 | 2.8.5-amzn-1 | 2.8.5-amzn-1 | 2.8.5-amzn-0 | 2.8.5-amzn-0 | 2.8.4-amzn-1 | 2.8.4-amzn-1 | 2.8.4-amzn-1 | 2.8.4-amzn-1 | 2.8.4-amzn-1 | 2.8.4-amzn-0 | 2.8.4-amzn-0 | 2.8.3-amzn-0 | 2.8.3-amzn-0 | 2.8.3-amzn-0 | 2.8.3-amzn-0 | 2.8.3-amzn-0 | 2.8.3-amzn-0 | 2.8.3-amzn-0 | 2.8.3-amzn-0 | 2.8.3-amzn-0 | 2.8.3-amzn-0 | 2.8.3-amzn-0 | 2.7.3-amzn-6 | 2.7.3-amzn-6 | 2.7.3-amzn-6 | 2.7.3-amzn-6 | 2.7.3-amzn-6 | 2.7.3-amzn-5 | 2.7.3-amzn-5 | 2.7.3-amzn-4 | 2.7.3-amzn-4 | 2.7.3-amzn-3 | 2.7.3-amzn-3 | 2.7.3-amzn-3 | 2.7.3-amzn-3 | 2.7.3-amzn-2 | 2.7.3-amzn-2 | 2.7.3-amzn-2 | 2.7.3-amzn-2 | 2.7.3-amzn-2 | 2.7.3-amzn-2 | 2.7.3-amzn-2 | 2.7.3-amzn-2 | 2.7.3-amzn-2 | 2.7.3-amzn-1 | 2.7.3-amzn-1 | 2.7.3-amzn-1 | 2.7.3-amzn-1 | 2.7.3-amzn-1 | 2.7.3-amzn-1 | 2.7.3-amzn-1 | 2.7.3-amzn-1 | 2.7.3-amzn-0 | 2.7.3-amzn-0 | 2.7.3-amzn-0 | 2.7.3-amzn-0 | 2.7.2-amzn-3 | 2.7.2-amzn-3 | 2.7.2-amzn-3 | | Hive | 2.3.9-amzn-2 | 2.3.9-amzn-2 | 2.3.9-amzn-2 | 2.3.9-amzn-0 | 2.3.8-amzn-0 | 2.3.7-amzn-4 | 2.3.7-amzn-4 | 2.3.7-amzn-3 | 2.3.7-amzn-3 | 2.3.7-amzn-1 | 2.3.7-amzn-1 | 2.3.6-amzn-2 | 2.3.6-amzn-2 | 2.3.6-amzn-2 | 2.3.6-amzn-1 | 2.3.6-amzn-0 | 2.3.6-amzn-0 | 2.3.5-amzn-1 | 2.3.5-amzn-1 | 2.3.5-amzn-0 | 2.3.5-amzn-0 | 2.3.4-amzn-2 | 2.3.4-amzn-2 | 2.3.4-amzn-1 | 2.3.4-amzn-1 | 2.3.4-amzn-1 | 2.3.4-amzn-0 | 2.3.4-amzn-0 | 2.3.4-amzn-0 | 2.3.4-amzn-0 | 2.3.4-amzn-0 | 2.3.3-amzn-1 | 2.3.3-amzn-1 | 2.3.3-amzn-1 | 2.3.3-amzn-1 | 2.3.3-amzn-1 | 2.3.3-amzn-1 | 2.3.3-amzn-1 | 2.3.3-amzn-1 | 2.3.3-amzn-1 | 2.3.3-amzn-0 | 2.3.3-amzn-0 | 2.3.2-amzn-2 | 2.3.2-amzn-2 | 2.3.2-amzn-2 | 2.3.2-amzn-2 | 2.3.2-amzn-2 | 2.3.2-amzn-1 | 2.3.2-amzn-1 | 2.3.2-amzn-1 | 2.3.2-amzn-1 | 2.3.2-amzn-0 | 2.3.2-amzn-0 | 2.3.2-amzn-0 | 2.3.2-amzn-0 | 2.3.2-amzn-0 | 2.3.1-amzn-0 | 2.3.1-amzn-0 | 2.3.0-amzn-0 | 2.3.0-amzn-0 | 2.3.0-amzn-0 | 2.3.0-amzn-0 | 2.3.0-amzn-0 | 2.3.0-amzn-0 | 2.1.1-amzn-0 | 2.1.1-amzn-0 | 2.1.1-amzn-0 | 2.1.1-amzn-0 | 2.1.1-amzn-0 | 2.1.1-amzn-0 | 2.1.1-amzn-0 | 2.1.1-amzn-0 | 2.1.1-amzn-0 | 2.1.1-amzn-0 | 2.1.1-amzn-0 | 2.1.1-amzn-0 | 2.1.1-amzn-0 | 2.1.1-amzn-0 | 2.1.0-amzn-0 | 2.1.0-amzn-0 | 2.1.0-amzn-0 | 2.1.0-amzn-0 | 2.1.0-amzn-0 | 2.1.0-amzn-0 | 2.1.0-amzn-0 | 2.1.0-amzn-0 | 2.1.0-amzn-0 | 2.1.0-amzn-0 | | Hudi | 0.10.1-amzn-1 | 0.10.1-amzn-1 | 0.10.1-amzn-1 | 0.9.0-amzn-2 | 0.9.0-amzn-0 | 0.7.0-amzn-1 | 0.7.0-amzn-1 | 0.6.0-amzn-0 | 0.6.0-amzn-0 | 0.6.0-amzn-0 | 0.6.0-amzn-0 | 0.5.2-incubating | 0.5.2-incubating | 0.5.2-incubating | 0.5.0-incubating | 0.5.0-incubating | 0.5.0-incubating | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | Hue | 4.10.0 | 4.10.0 | 4.10.0 | 4.10.0 | 4.9.0 | 4.9.0 | 4.9.0 | 4.8.0 | 4.8.0 | 4.7.1 | 4.7.1 | 4.6.0 | 4.6.0 | 4.6.0 | 4.4.0 | 4.4.0 | 4.4.0 | 4.4.0 | 4.4.0 | 4.4.0 | 4.4.0 | 4.4.0 | 4.4.0 | 4.3.0 | 4.3.0 | 4.3.0 | 4.3.0 | 4.3.0 | 4.3.0 | 4.3.0 | 4.3.0 | 4.2.0 | 4.2.0 | 4.2.0 | 4.2.0 | 4.2.0 | 4.2.0 | 4.2.0 | 4.2.0 | 4.2.0 | 4.2.0 | 4.2.0 | 4.1.0 | 4.1.0 | 4.1.0 | 4.1.0 | 4.1.0 | 4.1.0 | 4.1.0 | 4.1.0 | 4.1.0 | 4.0.1 | 4.0.1 | 4.0.1 | 4.0.1 | 4.0.1 | 4.0.1 | 4.0.1 | 4.0.1 | 4.0.1 | 3.12.0 | 3.12.0 | 3.12.0 | 3.12.0 | 3.12.0 | 3.12.0 | 3.12.0 | 3.12.0 | 3.12.0 | 3.12.0 | 3.12.0 | 3.12.0 | 3.12.0 | 3.11.0 | 3.11.0 | 3.11.0 | 3.11.0 | 3.11.0 | 3.10.0-amzn-0 | 3.10.0-amzn-0 | 3.10.0-amzn-0 | 3.10.0-amzn-0 | 3.10.0-amzn-0 | 3.10.0-amzn-0 | 3.10.0-amzn-0 | 3.10.0-amzn-0 | 3.10.0-amzn-0 | 3.10.0-amzn-0 | | Iceberg | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | JupyterEnterpriseGateway | 2.6.0 | 2.6.0 | 2.1.0 | 2.1.0 | 2.1.0 | 2.1.0 | 2.1.0 | 2.1.0 | 2.1.0 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | JupyterHub | 1.4.1 | 1.4.1 | 1.4.1 | 1.4.1 | 1.4.1 | 1.2.2 | 1.2.2 | 1.1.0 | 1.1.0 | 1.1.0 | 1.1.0 | 1.1.0 | 1.1.0 | 1.1.0 | 1.0.0 | 1.0.0 | 1.0.0 | 1.0.0 | 1.0.0 | 0.9.6 | 0.9.6 | 0.9.6 | 0.9.6 | 0.9.4 | 0.9.4 | 0.9.4 | 0.9.4 | 0.9.4 | 0.9.4 | 0.9.4 | 0.9.4 | 0.9.4 | 0.9.4 | 0.8.1 | 0.8.1 | 0.8.1 | 0.8.1 | 0.8.1 | 0.8.1 | 0.8.1 | 0.8.1 | 0.8.1 | 0.8.1 | 0.8.1 | 0.8.1 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | Livy | 0.7.1-incubating | 0.7.1-incubating | 0.7.1-incubating | 0.7.1-incubating | 0.7.1-incubating | 0.7.0-incubating | 0.7.0-incubating | 0.7.0-incubating | 0.7.0-incubating | 0.7.0-incubating | 0.7.0-incubating | 0.7.0-incubating | 0.7.0-incubating | 0.7.0-incubating | 0.6.0-incubating | 0.6.0-incubating | 0.6.0-incubating | 0.6.0-incubating | 0.6.0-incubating | 0.6.0-incubating | 0.6.0-incubating | 0.6.0-incubating | 0.6.0-incubating | 0.5.0-incubating | 0.5.0-incubating | 0.5.0-incubating | 0.5.0-incubating | 0.5.0-incubating | 0.5.0-incubating | 0.5.0-incubating | 0.5.0-incubating | 0.5.0-incubating | 0.5.0-incubating | 0.5.0-incubating | 0.5.0-incubating | 0.5.0-incubating | 0.5.0-incubating | 0.5.0-incubating | 0.5.0-incubating | 0.5.0-incubating | 0.4.0-incubating | 0.4.0-incubating | 0.4.0-incubating | 0.4.0-incubating | 0.4.0-incubating | 0.4.0-incubating | 0.4.0-incubating | 0.4.0-incubating | 0.4.0-incubating | 0.4.0-incubating | 0.4.0-incubating | 0.4.0-incubating | 0.4.0-incubating | 0.4.0-incubating | 0.4.0-incubating | 0.4.0-incubating | 0.4.0-incubating | 0.4.0-incubating | 0.4.0-incubating | 0.4.0-incubating | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | MXNet | 1.8.0 | 1.8.0 | 1.8.0 | 1.8.0 | 1.8.0 | 1.7.0 | 1.7.0 | 1.7.0 | 1.7.0 | 1.6.0 | 1.6.0 | 1.5.1 | 1.5.1 | 1.5.1 | 1.5.1 | 1.5.1 | 1.5.1 | 1.4.0 | 1.4.0 | 1.4.0 | 1.4.0 | 1.4.0 | 1.4.0 | 1.3.1 | 1.3.1 | 1.3.1 | 1.3.1 | 1.3.1 | 1.3.1 | 1.3.1 | 1.3.1 | 1.3.0 | 1.3.0 | 1.2.0 | 1.2.0 | 1.2.0 | 1.2.0 | 1.2.0 | 1.2.0 | 1.2.0 | 1.1.0 | 1.1.0 | 1.1.0 | 1.1.0 | 1.1.0 | 1.0.0 | 1.0.0 | 1.0.0 | 1.0.0 | 1.0.0 | 1.0.0 | 0.12.0 | 0.12.0 | 0.12.0 | 0.12.0 | 0.12.0 | 0.12.0 | 0.12.0 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | Mahout | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | 0.12.2 | 0.12.2 | 0.12.2 | 0.12.2 | 0.12.2 | 0.12.2 | 0.12.2 | 0.12.2 | 0.12.2 | 0.12.2 | 0.12.2 | 0.12.2 | 0.12.2 | 0.12.2 | 0.12.2 | 0.12.2 | 0.12.2 | 0.12.2 | 0.12.2 | 0.12.2 | | Oozie | 5.2.1 | 5.2.1 | 5.2.1 | 5.2.1 | 5.2.1 | 5.2.0 | 5.2.0 | 5.2.0 | 5.2.0 | 5.2.0 | 5.2.0 | 5.2.0 | 5.2.0 | 5.2.0 | 5.1.0 | 5.1.0 | 5.1.0 | 5.1.0 | 5.1.0 | 5.1.0 | 5.1.0 | 5.1.0 | 5.1.0 | 5.1.0 | 5.1.0 | 5.1.0 | 5.0.0 | 5.0.0 | 5.0.0 | 5.0.0 | 5.0.0 | 5.0.0 | 5.0.0 | 5.0.0 | 5.0.0 | 5.0.0 | 5.0.0 | 5.0.0 | 5.0.0 | 5.0.0 | 5.0.0 | 5.0.0 | 4.3.0 | 4.3.0 | 4.3.0 | 4.3.0 | 4.3.0 | 4.3.0 | 4.3.0 | 4.3.0 | 4.3.0 | 4.3.0 | 4.3.0 | 4.3.0 | 4.3.0 | 4.3.0 | 4.3.0 | 4.3.0 | 4.3.0 | 4.3.0 | 4.3.0 | 4.3.0 | 4.3.0 | 4.3.0 | 4.3.0 | 4.3.0 | 4.3.0 | 4.3.0 | 4.3.0 | 4.3.0 | 4.3.0 | 4.3.0 | 4.3.0 | 4.3.0 | 4.3.0 | 4.3.0 | 4.3.0 | 4.3.0 | 4.2.0 | 4.2.0 | 4.2.0 | 4.2.0 | 4.2.0 | 4.2.0 | 4.2.0 | 4.2.0 | 4.2.0 | 4.2.0 | | Phoenix | 4.14.3-HBase-1.4 | 4.14.3-HBase-1.4 | 4.14.3-HBase-1.4 | 4.14.3-HBase-1.4 | 4.14.3-HBase-1.4 | 4.14.3-HBase-1.4 | 4.14.3-HBase-1.4 | 4.14.3-HBase-1.4 | 4.14.3-HBase-1.4 | 4.14.3-HBase-1.4 | 4.14.3-HBase-1.4 | 4.14.3-HBase-1.4 | 4.14.3-HBase-1.4 | 4.14.3-HBase-1.4 | 4.14.3-HBase-1.4 | 4.14.3-HBase-1.4 | 4.14.3-HBase-1.4 | 4.14.2-HBase-1.4 | 4.14.2-HBase-1.4 | 4.14.2-HBase-1.4 | 4.14.1-HBase-1.4 | 4.14.1-HBase-1.4 | 4.14.1-HBase-1.4 | 4.14.1-HBase-1.4 | 4.14.1-HBase-1.4 | 4.14.1-HBase-1.4 | 4.14.0-HBase-1.4 | 4.14.0-HBase-1.4 | 4.14.0-HBase-1.4 | 4.14.0-HBase-1.4 | 4.14.0-HBase-1.4 | 4.14.0-HBase-1.4 | 4.14.0-HBase-1.4 | 4.14.0-HBase-1.4 | 4.14.0-HBase-1.4 | 4.14.0-HBase-1.4 | 4.14.0-HBase-1.4 | 4.14.0-HBase-1.4 | 4.14.0-HBase-1.4 | 4.14.0-HBase-1.4 | 4.13.0-HBase-1.4 | 4.13.0-HBase-1.4 | 4.13.0-HBase-1.4 | 4.13.0-HBase-1.4 | 4.13.0-HBase-1.4 | 4.13.0-HBase-1.4 | 4.13.0-HBase-1.4 | 4.13.0-HBase-1.4 | 4.13.0-HBase-1.4 | 4.13.0-HBase-1.4 | 4.13.0-HBase-1.4 | 4.11.0-HBase-1.3 | 4.11.0-HBase-1.3 | 4.11.0-HBase-1.3 | 4.11.0-HBase-1.3 | 4.11.0-HBase-1.3 | 4.11.0-HBase-1.3 | 4.11.0-HBase-1.3 | 4.11.0-HBase-1.3 | 4.11.0-HBase-1.3 | 4.11.0-HBase-1.3 | 4.11.0-HBase-1.3 | 4.11.0-HBase-1.3 | 4.11.0-HBase-1.3 | 4.11.0-HBase-1.3 | 4.11.0-HBase-1.3 | 4.9.0-HBase-1.2 | 4.9.0-HBase-1.2 | 4.9.0-HBase-1.2 | 4.9.0-HBase-1.2 | 4.9.0-HBase-1.2 | 4.9.0-HBase-1.2 | 4.9.0-HBase-1.2 | 4.9.0-HBase-1.2 | 4.9.0-HBase-1.2 | 4.7.0-HBase-1.2 | 4.7.0-HBase-1.2 | 4.7.0-HBase-1.2 | 4.7.0-HBase-1.2 | 4.7.0-HBase-1.2 | 4.7.0-HBase-1.2 | 4.7.0-HBase-1.2 | 4.7.0-HBase-1.2 | 4.7.0-HBase-1.2 | 4.7.0-HBase-1.2 | 4.7.0-HBase-1.2 | 4.7.0-HBase-1.2 | 4.7.0-HBase-1.2 | | Pig | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | 0.16.0-amzn-1 | 0.16.0-amzn-1 | 0.16.0-amzn-1 | 0.16.0-amzn-1 | 0.16.0-amzn-0 | 0.16.0-amzn-0 | 0.16.0-amzn-0 | 0.16.0-amzn-0 | 0.16.0-amzn-0 | 0.16.0-amzn-0 | 0.16.0-amzn-0 | 0.16.0-amzn-0 | 0.16.0-amzn-0 | 0.16.0-amzn-0 | 0.16.0-amzn-0 | 0.16.0-amzn-0 | 0.16.0-amzn-0 | 0.16.0-amzn-0 | 0.16.0-amzn-0 | 0.16.0-amzn-0 | 0.16.0-amzn-0 | 0.16.0-amzn-0 | 0.16.0-amzn-0 | 0.16.0-amzn-0 | 0.16.0-amzn-0 | 0.16.0-amzn-0 | 0.16.0-amzn-0 | 0.16.0-amzn-0 | | Presto | 0.267-amzn-1 | 0.267-amzn-1 | 0.267-amzn-1 | 0.266-amzn-0 | 0.261-amzn-0 | 0.245.1-amzn-0 | 0.245.1-amzn-0 | 0.240.1-amzn-0 | 0.240.1-amzn-0 | 0.238.3-amzn-0 | 0.238.3-amzn-0 | 0.232 | 0.232 | 0.232 | 0.227 | 0.227 | 0.227 | 0.224 | 0.224 | 0.220 | 0.220 | 0.219 | 0.219 | 0.215 | 0.215 | 0.215 | 0.215 | 0.215 | 0.215 | 0.214 | 0.214 | 0.212 | 0.212 | 0.210 | 0.210 | 0.206 | 0.206 | 0.206 | 0.203 | 0.203 | 0.194 | 0.194 | 0.194 | 0.194 | 0.194 | 0.194 | 0.194 | 0.188 | 0.188 | 0.188 | 0.188 | 0.187 | 0.187 | 0.187 | 0.187 | 0.187 | 0.187 | 0.187 | 0.184 | 0.184 | 0.170 | 0.170 | 0.170 | 0.170 | 0.170 | 0.170 | 0.170 | 0.170 | 0.170 | 0.170 | 0.170 | 0.170 | 0.170 | 0.166 | 0.166 | 0.157.1 | 0.157.1 | 0.157.1 | 0.157.1 | 0.157.1 | 0.157.1 | 0.152.3 | 0.152.3 | 0.152.3 | 0.152.3 | 0.150 | 0.150 | 0.150 | | Spark | 2.4.8-amzn-2 | 2.4.8-amzn-2 | 2.4.8-amzn-2 | 2.4.8-amzn-1 | 2.4.8-amzn-0 | 2.4.7-amzn-1.1 | 2.4.7-amzn-1 | 2.4.7-amzn-0.1 | 2.4.7-amzn-0 | 2.4.6-amzn-0.1 | 2.4.6-amzn-0 | 2.4.5-amzn-0.1 | 2.4.5-amzn-0 | 2.4.5-amzn-0 | 2.4.4 | 2.4.4 | 2.4.4 | 2.4.4 | 2.4.4 | 2.4.3 | 2.4.3 | 2.4.2 | 2.4.2 | 2.4.0 | 2.4.0 | 2.4.0 | 2.4.0 | 2.4.0 | 2.4.0 | 2.4.0 | 2.4.0 | 2.3.2 | 2.3.2 | 2.3.2 | 2.3.2 | 2.3.1 | 2.3.1 | 2.3.1 | 2.3.1 | 2.3.1 | 2.3.0 | 2.3.0 | 2.3.0 | 2.3.0 | 2.3.0 | 2.3.0 | 2.3.0 | 2.2.1 | 2.2.1 | 2.2.1 | 2.2.1 | 2.2.1 | 2.2.1 | 2.2.1 | 2.2.1 | 2.2.1 | 2.2.0 | 2.2.0 | 2.2.0 | 2.2.0 | 2.2.0 | 2.2.0 | 2.2.0 | 2.2.0 | 2.1.1 | 2.1.1 | 2.1.1 | 2.1.1 | 2.1.0 | 2.1.0 | 2.1.0 | 2.1.0 | 2.1.0 | 2.1.0 | 2.1.0 | 2.1.0 | 2.1.0 | 2.1.0 | 2.0.2 | 2.0.2 | 2.0.2 | 2.0.2 | 2.0.1 | 2.0.1 | 2.0.1 | 2.0.0 | 2.0.0 | 2.0.0 | | Sqoop | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.6 | 1.4.6 | 1.4.6 | 1.4.6 | 1.4.6 | 1.4.6 | 1.4.6 | 1.4.6 | 1.4.6 | 1.4.6 | 1.4.6 | 1.4.6 | 1.4.6 | 1.4.6 | 1.4.6 | 1.4.6 | 1.4.6 | 1.4.6 | 1.4.6 | 1.4.6 | 1.4.6 | 1.4.6 | 1.4.6 | 1.4.6 | 1.4.6 | 1.4.6 | 1.4.6 | 1.4.6 | 1.4.6 | 1.4.6 | 1.4.6 | 1.4.6 | 1.4.6 | 1.4.6 | 1.4.6 | 1.4.6 | 1.4.6 | 1.4.6 | 1.4.6 | 1.4.6 | 1.4.6 | 1.4.6 | 1.4.6 | | TensorFlow | 2.4.1 | 2.4.1 | 2.4.1 | 2.4.1 | 2.4.1 | 2.4.1 | 2.4.1 | 2.3.1 | 2.3.1 | 2.1.0 | 2.1.0 | 1.14.0 | 1.14.0 | 1.14.0 | 1.14.0 | 1.14.0 | 1.14.0 | 1.14.0 | 1.14.0 | 1.13.1 | 1.13.1 | 1.12.0 | 1.12.0 | 1.12.0 | 1.12.0 | 1.12.0 | 1.12.0 | 1.12.0 | 1.12.0 | 1.12.0 | 1.12.0 | 1.11.0 | 1.11.0 | 1.9.0 | 1.9.0 | 1.9.0 | 1.9.0 | 1.9.0 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | Tez | 0.9.2 | 0.9.2 | 0.9.2 | 0.9.2 | 0.9.2 | 0.9.2 | 0.9.2 | 0.9.2 | 0.9.2 | 0.9.2 | 0.9.2 | 0.9.2 | 0.9.2 | 0.9.2 | 0.9.2 | 0.9.2 | 0.9.2 | 0.9.2 | 0.9.2 | 0.9.2 | 0.9.2 | 0.9.1 | 0.9.1 | 0.9.1 | 0.9.1 | 0.9.1 | 0.9.1 | 0.9.1 | 0.9.1 | 0.9.1 | 0.9.1 | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | | Trino (PrestoSQL) | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | Zeppelin | 0.10.0 | 0.10.0 | 0.10.0 | 0.10.0 | 0.10.0 | 0.9.0 | 0.9.0 | 0.8.2 | 0.8.2 | 0.8.2 | 0.8.2 | 0.8.2 | 0.8.2 | 0.8.2 | 0.8.2 | 0.8.2 | 0.8.2 | 0.8.1 | 0.8.1 | 0.8.1 | 0.8.1 | 0.8.1 | 0.8.1 | 0.8.1 | 0.8.1 | 0.8.1 | 0.8.0 | 0.8.0 | 0.8.0 | 0.8.0 | 0.8.0 | 0.8.0 | 0.8.0 | 0.8.0 | 0.8.0 | 0.7.3 | 0.7.3 | 0.7.3 | 0.7.3 | 0.7.3 | 0.7.3 | 0.7.3 | 0.7.3 | 0.7.3 | 0.7.3 | 0.7.3 | 0.7.3 | 0.7.3 | 0.7.3 | 0.7.3 | 0.7.3 | 0.7.3 | 0.7.3 | 0.7.3 | 0.7.3 | 0.7.3 | 0.7.3 | 0.7.3 | 0.7.2 | 0.7.2 | 0.7.2 | 0.7.2 | 0.7.2 | 0.7.2 | 0.7.2 | 0.7.2 | 0.7.1 | 0.7.1 | 0.7.1 | 0.7.1 | 0.7.1 | 0.7.1 | 0.7.1 | 0.7.0 | 0.7.0 | 0.6.2 | 0.6.2 | 0.6.2 | 0.6.2 | 0.6.2 | 0.6.2 | 0.6.2 | 0.6.2 | 0.6.2 | 0.6.1 | 0.6.1-SNAPSHOT | 0.6.1-SNAPSHOT | 0.6.1-SNAPSHOT | | ZooKeeper | 3.4.14 | 3.4.14 | 3.4.14 | 3.4.14 | 3.4.14 | 3.4.14 | 3.4.14 | 3.4.14 | 3.4.14 | 3.4.14 | 3.4.14 | 3.4.14 | 3.4.14 | 3.4.14 | 3.4.14 | 3.4.14 | 3.4.14 | 3.4.14 | 3.4.14 | 3.4.14 | 3.4.14 | 3.4.13 | 3.4.13 | 3.4.13 | 3.4.13 | 3.4.13 | 3.4.13 | 3.4.13 | 3.4.13 | 3.4.13 | 3.4.13 | 3.4.13 | 3.4.13 | 3.4.12 | 3.4.12 | 3.4.12 | 3.4.12 | 3.4.12 | 3.4.12 | 3.4.12 | 3.4.12 | 3.4.12 | 3.4.10 | 3.4.10 | 3.4.10 | 3.4.10 | 3.4.10 | 3.4.10 | 3.4.10 | 3.4.10 | 3.4.10 | 3.4.10 | 3.4.10 | 3.4.10 | 3.4.10 | 3.4.10 | 3.4.10 | 3.4.10 | 3.4.10 | 3.4.10 | 3.4.10 | 3.4.10 | 3.4.10 | 3.4.10 | 3.4.10 | 3.4.10 | 3.4.10 | 3.4.10 | 3.4.10 | 3.4.10 | 3.4.10 | 3.4.10 | 3.4.10 | 3.4.9 | 3.4.9 | 3.4.9 | 3.4.9 | 3.4.9 | 3.4.9 | 3.4.9 | 3.4.9 | 3.4.8 | 3.4.8 | 3.4.8 | 3.4.8 | 3.4.8 | 3.4.8 | 3.4.8 | # Amazon EMR release 5.36.2 ## 5.36.2 application versions This release includes the following applications: [https://flink.apache.org/](https://flink.apache.org/), [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [https://hudi.apache.org](https://hudi.apache.org), [http://gethue.com/](http://gethue.com/), [https://jupyter-enterprise-gateway.readthedocs.io/en/latest/](https://jupyter-enterprise-gateway.readthedocs.io/en/latest/), [https://jupyterhub.readthedocs.io/en/latest/#](https://jupyterhub.readthedocs.io/en/latest/#), [https://livy.incubator.apache.org/](https://livy.incubator.apache.org/), [https://mxnet.incubator.apache.org/](https://mxnet.incubator.apache.org/), [http://mahout.apache.org/](http://mahout.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://www.tensorflow.org/](https://www.tensorflow.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-5.36.2 | emr-5.36.1 | emr-5.36.0 | emr-5.35.0 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.12.206 | 1.12.206 | 1.12.206 | 1.12.159 | | Python | 2.7, 3.7 | 2.7, 3.7 | 2.7, 3.7 | 2.7, 3.7 | | Scala | 2.11.12 | 2.11.12 | 2.11.12 | 2.11.12 | | AmazonCloudWatchAgent | - | - | - | - | | Delta | - | - | - | - | | Flink | 1.14.2 | 1.14.2 | 1.14.2 | 1.14.2 | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 1.4.13 | 1.4.13 | 1.4.13 | 1.4.13 | | HCatalog | 2.3.9-amzn-2 | 2.3.9-amzn-2 | 2.3.9-amzn-2 | 2.3.9-amzn-0 | | Hadoop | 2.10.1-amzn-4 | 2.10.1-amzn-4 | 2.10.1-amzn-4 | 2.10.1-amzn-3 | | Hive | 2.3.9-amzn-2 | 2.3.9-amzn-2 | 2.3.9-amzn-2 | 2.3.9-amzn-0 | | Hudi | 0.10.1-amzn-1 | 0.10.1-amzn-1 | 0.10.1-amzn-1 | 0.9.0-amzn-2 | | Hue | 4.10.0 | 4.10.0 | 4.10.0 | 4.10.0 | | Iceberg | - | - | - | - | | JupyterEnterpriseGateway | 2.6.0 | 2.6.0 | 2.1.0 | 2.1.0 | | JupyterHub | 1.4.1 | 1.4.1 | 1.4.1 | 1.4.1 | | Livy | 0.7.1-incubating | 0.7.1-incubating | 0.7.1-incubating | 0.7.1-incubating | | MXNet | 1.8.0 | 1.8.0 | 1.8.0 | 1.8.0 | | Mahout | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | | Oozie | 5.2.1 | 5.2.1 | 5.2.1 | 5.2.1 | | Phoenix | 4.14.3-HBase-1.4 | 4.14.3-HBase-1.4 | 4.14.3-HBase-1.4 | 4.14.3-HBase-1.4 | | Pig | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | | Presto | 0.267-amzn-1 | 0.267-amzn-1 | 0.267-amzn-1 | 0.266-amzn-0 | | Spark | 2.4.8-amzn-2 | 2.4.8-amzn-2 | 2.4.8-amzn-2 | 2.4.8-amzn-1 | | Sqoop | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | | TensorFlow | 2.4.1 | 2.4.1 | 2.4.1 | 2.4.1 | | Tez | 0.9.2 | 0.9.2 | 0.9.2 | 0.9.2 | | Trino (PrestoSQL) | - | - | - | - | | Zeppelin | 0.10.0 | 0.10.0 | 0.10.0 | 0.10.0 | | ZooKeeper | 3.4.14 | 3.4.14 | 3.4.14 | 3.4.14 | ## 5.36.2 release notes The following release notes include information for Amazon EMR release 5.36.2. Changes are relative to 5.36.1. For information on the release timeline, see the [change log](https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-5362-release.html#5362-changelog). **Changes, enhancements, and resolved issues** + This releases improves cluster scale-down logic so that Amazon EMR doesn't scale-down core nodes below the HDFS replication factor setting for the cluster. This improvement fulfills data redundancy requirements, and reduces the chance that a scaling operation might stall. + This release adds a new retry mechanism to the cluster scaling workflow for EMR clusters that run Presto or Trino. This improvement reduces the risk that cluster resize runs indefinitely due to a single failed resize operation. It also improves cluster utilization, because your cluster scales up and down faster. + Fixes an issue where cluster scale-down operations might stall while Amazon EMR gracefully decommissions a core node and it turns unhealthy before it is fully decommissioned. + Improves the stability of a node in a high-availability cluster with multiple primary nodes when Amazon EMR restarts a single node. + Optimizes log management with Amazon EMR running on Amazon EC2. As a result, you might see a slight reduction in storage costs for your cluster logs. + Improves the management of ZooKeeper transaction log files that are maintained on primary nodes to minimize scenarios where the log files grow out of bounds and interrupt cluster operations. + Fixes a rare bug which can cause a high-availability cluster with multiple primary nodes to fail due to not being able to communicate with the Yarn ResourceManager. + When you launch a cluster with *the latest patch release* of Amazon EMR 5.36 or higher, 6.6 or higher, or 7.0 or higher, Amazon EMR uses the latest Amazon Linux 2023 or Amazon Linux 2 release for the default Amazon EMR AMI. For more information, see [Using the default Amazon Linux AMI for Amazon EMR](https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-default-ami.html). [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-5362-release.html) ## 5.36.2 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | aws-sagemaker-spark-sdk | 1.4.1 | Amazon SageMaker Spark SDK | | emr-ddb | 4.16.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 2.16.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.5.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-notebook-env | 1.5.0 | Conda env for emr notebook which includes jupyter enterprise gateway | | emr-s3-dist-cp | 2.21.0 | Distributed copy application optimized for Amazon S3. | | emr-s3-select | 1.7.0 | EMR S3Select Connector | | emrfs | 2.51.0 | Amazon S3 connector for Hadoop ecosystem applications. | | flink-client | 1.14.2 | Apache Flink command line client scripts and applications. | | flink-jobmanager-config | 1.14.2 | Managing resources on EMR nodes for Apache Flink JobManager. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 2.10.1-amzn-4 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 2.10.1-amzn-4 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 2.10.1-amzn-4 | HDFS command-line client and library | | hadoop-hdfs-namenode | 2.10.1-amzn-4 | HDFS service for tracking file names and block locations. | | hadoop-hdfs-journalnode | 2.10.1-amzn-4 | HDFS service for managing the Hadoop filesystem journal on HA clusters. | | hadoop-httpfs-server | 2.10.1-amzn-4 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 2.10.1-amzn-4 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 2.10.1-amzn-4 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 2.10.1-amzn-4 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 2.10.1-amzn-4 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 2.10.1-amzn-4 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 1.4.13 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 1.4.13 | Service for serving one or more HBase regions. | | hbase-client | 1.4.13 | HBase command-line client. | | hbase-rest-server | 1.4.13 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 1.4.13 | Service providing a Thrift endpoint to HBase. | | hcatalog-client | 2.3.9-amzn-2 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 2.3.9-amzn-2 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 2.3.9-amzn-2 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 2.3.9-amzn-2 | Hive command line client. | | hive-hbase | 2.3.9-amzn-2 | Hive-hbase client. | | hive-metastore-server | 2.3.9-amzn-2 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server2 | 2.3.9-amzn-2 | Service for accepting Hive queries as web requests. | | hudi | 0.10.1-amzn-1 | Incremental processing framework to power data pipeline at low latency and high efficiency. | | hudi-spark | 0.10.1-amzn-1 | Bundle library for running Spark with Hudi. | | hudi-presto | 0.10.1-amzn-1 | Bundle library for running Presto with Hudi. | | hue-server | 4.10.0 | Web application for analyzing data using Hadoop ecosystem applications | | jupyterhub | 1.4.1 | Multi-user server for Jupyter notebooks | | livy-server | 0.7.1-incubating | REST interface for interacting with Apache Spark | | nginx | 1.13.1 | nginx [engine x] is an HTTP and reverse proxy server | | mahout-client | 0.13.0 | Library for machine learning. | | mxnet | 1.8.0 | A flexible, scalable, and efficient library for deep learning. | | mariadb-server | 5.5.68 | MySQL database server. | | nvidia-cuda | 11.0.194 | Nvidia drivers and Cuda toolkit | | oozie-client | 5.2.1 | Oozie command-line client. | | oozie-server | 5.2.1 | Service for accepting Oozie workflow requests. | | opencv | 4.5.0 | Open Source Computer Vision Library. | | phoenix-library | 4.14.3-HBase-1.4 | The phoenix libraries for server and client | | phoenix-query-server | 4.14.3-HBase-1.4 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.267-amzn-1 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.267-amzn-1 | Service for executing pieces of a query. | | presto-client | 0.267-amzn-1 | Presto command-line client which is installed on an HA cluster's stand-by masters where Presto server is not started. | | pig-client | 0.17.0 | Pig command-line client. | | r | 4.0.2 | The R Project for Statistical Computing | | ranger-kms-server | 1.2.0 | Apache Ranger Key Management System | | spark-client | 2.4.8-amzn-2 | Spark command-line clients. | | spark-history-server | 2.4.8-amzn-2 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 2.4.8-amzn-2 | In-memory execution engine for YARN. | | spark-yarn-slave | 2.4.8-amzn-2 | Apache Spark libraries needed by YARN slaves. | | sqoop-client | 1.4.7 | Apache Sqoop command-line client. | | tensorflow | 2.4.1 | TensorFlow open source software library for high performance numerical computation. | | tez-on-yarn | 0.9.2 | The tez YARN application and libraries. | | webserver | 2.4.25\$1 | Apache HTTP server. | | zeppelin-server | 0.10.0 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.4.14 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.4.14 | ZooKeeper command line client. | ## 5.36.2 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). Reconfiguration actions occur when you specify a configuration for instance groups in a running cluster. Amazon EMR only initiates reconfiguration actions for the classifications that you modify. For more information, see [Reconfigure an instance group in a running cluster](emr-configure-apps-running-cluster.md). **emr-5.36.2 classifications** | Classifications | Description | Reconfiguration Actions | | --- | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | Restarts the ResourceManager service. | | container-executor | Change values in Hadoop YARN's container-executor.cfg file. | Not available. | | container-log4j | Change values in Hadoop YARN's container-log4j.properties file. | Not available. | | core-site | Change values in Hadoop's core-site.xml file. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts Hadoop KMS, Ranger KMS, HiveServer2, Hive MetaStore, Hadoop Httpfs, and MapReduce-HistoryServer. | | docker-conf | Change docker related settings. | Not available. | | emrfs-site | Change EMRFS settings. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts HBaseRegionserver, HBaseMaster, HBaseThrift, HBaseRest, HiveServer2, Hive MetaStore, Hadoop Httpfs, and MapReduce-HistoryServer. | | flink-conf | Change flink-conf.yaml settings. | Restarts Flink history server. | | flink-log4j | Change Flink log4j.properties settings. | Restarts Flink history server. | | flink-log4j-session | Change Flink log4j-session.properties settings for Kubernetes/Yarn session. | Not available. | | flink-log4j-cli | Change Flink log4j-cli.properties settings. | Restarts Flink history server. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts PhoenixQueryserver, HiveServer2, Hive MetaStore, and MapReduce-HistoryServer. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts Hadoop KMS, Hadoop Httpfs, and MapReduce-HistoryServer. | | hadoop-ssl-server | Change hadoop ssl server configuration | Not available. | | hadoop-ssl-client | Change hadoop ssl client configuration | Not available. | | hbase | Amazon EMR-curated settings for Apache HBase. | Custom EMR specific property. Sets emrfs-site and hbase-site configs. See those for their associated restarts. | | hbase-env | Change values in HBase's environment. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbase.properties file. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | Not available. | | hbase-site | Change values in HBase's hbase-site.xml file. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. Additionally restarts Phoenix QueryServer. | | hdfs-encryption-zones | Configure HDFS encryption zones. | Should not be reconfigured. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Additionally restarts Hadoop Httpfs. | | hcatalog-env | Change values in HCatalog's environment. | Restarts Hive HCatalog Server. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | Restarts Hive HCatalog Server. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | Restarts Hive HCatalog Server. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | Restarts Hive WebHCat Server. | | hcatalog-webhcat-log4j2 | Change values in HCatalog WebHCat's log4j2.properties. | Restarts Hive WebHCat Server. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | Restarts Hive WebHCat Server. | | hive-beeline-log4j2 | Change values in Hive's beeline-log4j2.properties file. | Not available. | | hive-parquet-logging | Change values in Hive's parquet-logging.properties file. | Not available. | | hive-env | Change values in the Hive environment. | Restarts HiveServer2 and HiveMetastore. Runs Hive schemaTool CLI commands to verify hive-metastore. | | hive-exec-log4j2 | Change values in Hive's hive-exec-log4j2.properties file. | Restarts HiveServer2 and HiveMetastore. | | hive-llap-daemon-log4j2 | Change values in Hive's llap-daemon-log4j2.properties file. | Not available. | | hive-log4j2 | Change values in Hive's hive-log4j2.properties file. | Not available. | | hive-site | Change values in Hive's hive-site.xml file | Restarts HiveServer2 and HiveMetastore. Runs Hive schemaTool CLI commands to verify hive-metastore. Also restarts Oozie and Zeppelin. | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | Not available. | | hue-ini | Change values in Hue's ini file | Restarts Hue. Also activates Hue config override CLI commands to pick up new configurations. | | httpfs-env | Change values in the HTTPFS environment. | Restarts Hadoop Httpfs service. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | Restarts Hadoop Httpfs service. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | Not available. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | Restarts Hadoop-KMS service. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | Not available. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | Restarts Hadoop-KMS and Ranger-KMS service. | | hudi-env | Change values in the Hudi environment. | Not available. | | hudi-defaults | Change values in Hudi's hudi-defaults.conf file. | Not available. | | jupyter-notebook-conf | Change values in Jupyter Notebook's jupyter\$1notebook\$1config.py file. | Not available. | | jupyter-hub-conf | Change values in JupyterHubs's jupyterhub\$1config.py file. | Not available. | | jupyter-s3-conf | Configure Jupyter Notebook S3 persistence. | Not available. | | jupyter-sparkmagic-conf | Change values in Sparkmagic's config.json file. | Not available. | | livy-conf | Change values in Livy's livy.conf file. | Restarts Livy Server. | | livy-env | Change values in the Livy environment. | Restarts Livy Server. | | livy-log4j | Change Livy log4j.properties settings. | Restarts Livy Server. | | mapred-env | Change values in the MapReduce application's environment. | Restarts Hadoop MapReduce-HistoryServer. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | Restarts Hadoop MapReduce-HistoryServer. | | oozie-env | Change values in Oozie's environment. | Restarts Oozie. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | Restarts Oozie. | | oozie-site | Change values in Oozie's oozie-site.xml file. | Restarts Oozie. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | Not available. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | Not available. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | Restarts Phoenix-QueryServer. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | Not available. | | pig-env | Change values in the Pig environment. | Not available. | | pig-properties | Change values in Pig's pig.properties file. | Restarts Oozie. | | pig-log4j | Change values in Pig's log4j.properties file. | Not available. | | presto-log | Change values in Presto's log.properties file. | Restarts Presto-Server. | | presto-config | Change values in Presto's config.properties file. | Restarts Presto-Server. | | presto-password-authenticator | Change values in Presto's password-authenticator.properties file. | Not available. | | presto-env | Change values in Presto's presto-env.sh file. | Restarts Presto-Server. | | presto-node | Change values in Presto's node.properties file. | Not available. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | Not available. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | Not available. | | presto-connector-hive | Change values in Presto's hive.properties file. | Restarts Presto-Server. | | presto-connector-jmx | Change values in Presto's jmx.properties file. | Not available. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | Not available. | | presto-connector-localfile | Change values in Presto's localfile.properties file. | Not available. | | presto-connector-memory | Change values in Presto's memory.properties file. | Not available. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | Not available. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | Not available. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | Not available. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | Not available. | | presto-connector-redis | Change values in Presto's redis.properties file. | Not available. | | presto-connector-redshift | Change values in Presto's redshift.properties file. | Not available. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | Not available. | | presto-connector-tpcds | Change values in Presto's tpcds.properties file. | Not available. | | ranger-kms-dbks-site | Change values in dbks-site.xml file of Ranger KMS. | Restarts Ranger KMS Server. | | ranger-kms-site | Change values in ranger-kms-site.xml file of Ranger KMS. | Restarts Ranger KMS Server. | | ranger-kms-env | Change values in the Ranger KMS environment. | Restarts Ranger KMS Server. | | ranger-kms-log4j | Change values in kms-log4j.properties file of Ranger KMS. | Not available. | | ranger-kms-db-ca | Change values for CA file on S3 for MySQL SSL connection with Ranger KMS. | Not available. | | recordserver-env | Change values in the EMR RecordServer environment. | Restarts EMR record server. | | recordserver-conf | Change values in EMR RecordServer's erver.properties file. | Restarts EMR record server. | | recordserver-log4j | Change values in EMR RecordServer's log4j.properties file. | Restarts EMR record server. | | spark | Amazon EMR-curated settings for Apache Spark. | This property modifies spark-defaults. See actions there. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | Restarts Spark history server and Spark thrift server. | | spark-env | Change values in the Spark environment. | Restarts Spark history server and Spark thrift server. | | spark-hive-site | Change values in Spark's hive-site.xml file | Not available. | | spark-log4j | Change values in Spark's log4j.properties file. | Restarts Spark history server and Spark thrift server. | | spark-metrics | Change values in Spark's metrics.properties file. | Restarts Spark history server and Spark thrift server. | | sqoop-env | Change values in Sqoop's environment. | Not available. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | Not available. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | Not available. | | tez-site | Change values in Tez's tez-site.xml file. | Restarts Oozie. | | yarn-env | Change values in the YARN environment. | Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts MapReduce-HistoryServer. | | yarn-site | Change values in YARN's yarn-site.xml file. | Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts Livy Server and MapReduce-HistoryServer. | | zeppelin-env | Change values in the Zeppelin environment. | Restarts Zeppelin. | | zeppelin-site | Change configuration settings in zeppelin-site.xml. | Restarts Zeppelin. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | Restarts Zookeeper server. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | Restarts Zookeeper server. | ## 5.36.2 change log **Change log for 5.36.2 release and release notes** | Date | Event | Description | | --- | --- | --- | | 2024-05-28 | Deployment complete | Amazon EMR 5.36.2 fully deployed to all [supported Regions](https://aws.amazon.com/about-aws/global-infrastructure/regional-product-services/) | | 2024-05-28 | Docs publication | Amazon EMR 5.36.2 release notes first published | | 2024-05-20 | Initial release | Amazon EMR 5.36.2 first deployed to limited commercial Regions | # Amazon EMR release 5.36.1 ## 5.36.1 application versions This release includes the following applications: [https://flink.apache.org/](https://flink.apache.org/), [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [https://hudi.apache.org](https://hudi.apache.org), [http://gethue.com/](http://gethue.com/), [https://jupyter-enterprise-gateway.readthedocs.io/en/latest/](https://jupyter-enterprise-gateway.readthedocs.io/en/latest/), [https://jupyterhub.readthedocs.io/en/latest/#](https://jupyterhub.readthedocs.io/en/latest/#), [https://livy.incubator.apache.org/](https://livy.incubator.apache.org/), [https://mxnet.incubator.apache.org/](https://mxnet.incubator.apache.org/), [http://mahout.apache.org/](http://mahout.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://www.tensorflow.org/](https://www.tensorflow.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-5.36.1 | emr-5.36.0 | emr-5.35.0 | emr-5.34.0 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.12.206 | 1.12.206 | 1.12.159 | 1.11.970 | | Python | 2.7, 3.7 | 2.7, 3.7 | 2.7, 3.7 | 2.7, 3.7 | | Scala | 2.11.12 | 2.11.12 | 2.11.12 | 2.11.12 | | AmazonCloudWatchAgent | - | - | - | - | | Delta | - | - | - | - | | Flink | 1.14.2 | 1.14.2 | 1.14.2 | 1.13.1 | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 1.4.13 | 1.4.13 | 1.4.13 | 1.4.13 | | HCatalog | 2.3.9-amzn-2 | 2.3.9-amzn-2 | 2.3.9-amzn-0 | 2.3.8-amzn-0 | | Hadoop | 2.10.1-amzn-4 | 2.10.1-amzn-4 | 2.10.1-amzn-3 | 2.10.1-amzn-2 | | Hive | 2.3.9-amzn-2 | 2.3.9-amzn-2 | 2.3.9-amzn-0 | 2.3.8-amzn-0 | | Hudi | 0.10.1-amzn-1 | 0.10.1-amzn-1 | 0.9.0-amzn-2 | 0.9.0-amzn-0 | | Hue | 4.10.0 | 4.10.0 | 4.10.0 | 4.9.0 | | Iceberg | - | - | - | - | | JupyterEnterpriseGateway | 2.6.0 | 2.1.0 | 2.1.0 | 2.1.0 | | JupyterHub | 1.4.1 | 1.4.1 | 1.4.1 | 1.4.1 | | Livy | 0.7.1-incubating | 0.7.1-incubating | 0.7.1-incubating | 0.7.1-incubating | | MXNet | 1.8.0 | 1.8.0 | 1.8.0 | 1.8.0 | | Mahout | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | | Oozie | 5.2.1 | 5.2.1 | 5.2.1 | 5.2.1 | | Phoenix | 4.14.3-HBase-1.4 | 4.14.3-HBase-1.4 | 4.14.3-HBase-1.4 | 4.14.3-HBase-1.4 | | Pig | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | | Presto | 0.267-amzn-1 | 0.267-amzn-1 | 0.266-amzn-0 | 0.261-amzn-0 | | Spark | 2.4.8-amzn-2 | 2.4.8-amzn-2 | 2.4.8-amzn-1 | 2.4.8-amzn-0 | | Sqoop | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | | TensorFlow | 2.4.1 | 2.4.1 | 2.4.1 | 2.4.1 | | Tez | 0.9.2 | 0.9.2 | 0.9.2 | 0.9.2 | | Trino (PrestoSQL) | - | - | - | - | | Zeppelin | 0.10.0 | 0.10.0 | 0.10.0 | 0.10.0 | | ZooKeeper | 3.4.14 | 3.4.14 | 3.4.14 | 3.4.14 | ## 5.36.1 release notes The following release notes include information for Amazon EMR release 5.36.1. Changes are relative to 5.36.0. For information on the release timeline, see the [change log](https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-5361-release.html#5361-changelog). **Changes, enhancements, and resolved issues** + Amazon EMR release 5.36.1 adds support for archiving logs to Amazon S3 during cluster scale-down. In previous 5.x releases, you could only archive log files to Amazon S3 during cluster termination. This improvement ensures that log files generated on the cluster persist on Amazon S3 even after the node is terminated. For more information, see [Configure cluster logging and debugging](https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-plan-debugging.html). + The 5.36.1 release improves the on-cluster log management daemon to monitor additional log folders in your EMR cluster. This improvement minimizes disk over-utilization scenarios. + The 5.36.1 release automatically restarts the on-cluster log management daemon when it stops. This improvement reduces the risk for nodes to appear unhealthy due to disk over-utilization. + The 5.36.1 release fixes an issue where Amazon EMR daemons on the primary node would maintain stale metadata for terminated instances in the cluster. Maintaining stale data might cause on-cluster CPU and memory usage to grow without bounds, and ultimately cause cluster failures. + For clusters that are launched with multiple primary nodes, the 5.36.1 release fixes an issue where an Amazon EC2 hardware failure on one of the primary nodes could cause a second primary node to fail and render your cluster unstable. + For clusters that are configured with in-transit encryption, Managed Scaling is now *Spark shuffle data* aware. Spark shuffle data is data that Spark redistributes across partitions to perform specific operations. During scale down, Managed Scaling ignores the instances with shuffle data. This prevents job re-attempts and re-computations, which are costly for price and performance. For more information on shuffle operations, see the [Spark Programming Guide](https://spark.apache.org/docs/latest/rdd-programming-guide.html#shuffle-operations). + When you launch a cluster with *the latest patch release* of Amazon EMR 5.36 or higher, 6.6 or higher, or 7.0 or higher, Amazon EMR uses the latest Amazon Linux 2023 or Amazon Linux 2 release for the default Amazon EMR AMI. For more information, see [Using the default Amazon Linux AMI for Amazon EMR](https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-default-ami.html). [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-5361-release.html) ## 5.36.1 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | aws-sagemaker-spark-sdk | 1.4.1 | Amazon SageMaker Spark SDK | | emr-ddb | 4.16.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 2.16.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.5.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-notebook-env | 1.5.0 | Conda env for emr notebook which includes jupyter enterprise gateway | | emr-s3-dist-cp | 2.21.0 | Distributed copy application optimized for Amazon S3. | | emr-s3-select | 1.7.0 | EMR S3Select Connector | | emrfs | 2.51.0 | Amazon S3 connector for Hadoop ecosystem applications. | | flink-client | 1.14.2 | Apache Flink command line client scripts and applications. | | flink-jobmanager-config | 1.14.2 | Managing resources on EMR nodes for Apache Flink JobManager. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 2.10.1-amzn-4 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 2.10.1-amzn-4 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 2.10.1-amzn-4 | HDFS command-line client and library | | hadoop-hdfs-namenode | 2.10.1-amzn-4 | HDFS service for tracking file names and block locations. | | hadoop-hdfs-journalnode | 2.10.1-amzn-4 | HDFS service for managing the Hadoop filesystem journal on HA clusters. | | hadoop-httpfs-server | 2.10.1-amzn-4 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 2.10.1-amzn-4 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 2.10.1-amzn-4 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 2.10.1-amzn-4 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 2.10.1-amzn-4 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 2.10.1-amzn-4 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 1.4.13 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 1.4.13 | Service for serving one or more HBase regions. | | hbase-client | 1.4.13 | HBase command-line client. | | hbase-rest-server | 1.4.13 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 1.4.13 | Service providing a Thrift endpoint to HBase. | | hcatalog-client | 2.3.9-amzn-2 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 2.3.9-amzn-2 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 2.3.9-amzn-2 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 2.3.9-amzn-2 | Hive command line client. | | hive-hbase | 2.3.9-amzn-2 | Hive-hbase client. | | hive-metastore-server | 2.3.9-amzn-2 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server2 | 2.3.9-amzn-2 | Service for accepting Hive queries as web requests. | | hudi | 0.10.1-amzn-1 | Incremental processing framework to power data pipeline at low latency and high efficiency. | | hudi-spark | 0.10.1-amzn-1 | Bundle library for running Spark with Hudi. | | hudi-presto | 0.10.1-amzn-1 | Bundle library for running Presto with Hudi. | | hue-server | 4.10.0 | Web application for analyzing data using Hadoop ecosystem applications | | jupyterhub | 1.4.1 | Multi-user server for Jupyter notebooks | | livy-server | 0.7.1-incubating | REST interface for interacting with Apache Spark | | nginx | 1.13.1 | nginx [engine x] is an HTTP and reverse proxy server | | mahout-client | 0.13.0 | Library for machine learning. | | mxnet | 1.8.0 | A flexible, scalable, and efficient library for deep learning. | | mariadb-server | 5.5.68 | MySQL database server. | | nvidia-cuda | 11.0.194 | Nvidia drivers and Cuda toolkit | | oozie-client | 5.2.1 | Oozie command-line client. | | oozie-server | 5.2.1 | Service for accepting Oozie workflow requests. | | opencv | 4.5.0 | Open Source Computer Vision Library. | | phoenix-library | 4.14.3-HBase-1.4 | The phoenix libraries for server and client | | phoenix-query-server | 4.14.3-HBase-1.4 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.267-amzn-1 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.267-amzn-1 | Service for executing pieces of a query. | | presto-client | 0.267-amzn-1 | Presto command-line client which is installed on an HA cluster's stand-by masters where Presto server is not started. | | pig-client | 0.17.0 | Pig command-line client. | | r | 4.0.2 | The R Project for Statistical Computing | | ranger-kms-server | 1.2.0 | Apache Ranger Key Management System | | spark-client | 2.4.8-amzn-2 | Spark command-line clients. | | spark-history-server | 2.4.8-amzn-2 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 2.4.8-amzn-2 | In-memory execution engine for YARN. | | spark-yarn-slave | 2.4.8-amzn-2 | Apache Spark libraries needed by YARN slaves. | | sqoop-client | 1.4.7 | Apache Sqoop command-line client. | | tensorflow | 2.4.1 | TensorFlow open source software library for high performance numerical computation. | | tez-on-yarn | 0.9.2 | The tez YARN application and libraries. | | webserver | 2.4.25\$1 | Apache HTTP server. | | zeppelin-server | 0.10.0 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.4.14 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.4.14 | ZooKeeper command line client. | ## 5.36.1 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). Reconfiguration actions occur when you specify a configuration for instance groups in a running cluster. Amazon EMR only initiates reconfiguration actions for the classifications that you modify. For more information, see [Reconfigure an instance group in a running cluster](emr-configure-apps-running-cluster.md). **emr-5.36.1 classifications** | Classifications | Description | Reconfiguration Actions | | --- | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | Restarts the ResourceManager service. | | container-executor | Change values in Hadoop YARN's container-executor.cfg file. | Not available. | | container-log4j | Change values in Hadoop YARN's container-log4j.properties file. | Not available. | | core-site | Change values in Hadoop's core-site.xml file. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts Hadoop KMS, Ranger KMS, HiveServer2, Hive MetaStore, Hadoop Httpfs, and MapReduce-HistoryServer. | | docker-conf | Change docker related settings. | Not available. | | emrfs-site | Change EMRFS settings. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts HBaseRegionserver, HBaseMaster, HBaseThrift, HBaseRest, HiveServer2, Hive MetaStore, Hadoop Httpfs, and MapReduce-HistoryServer. | | flink-conf | Change flink-conf.yaml settings. | Restarts Flink history server. | | flink-log4j | Change Flink log4j.properties settings. | Restarts Flink history server. | | flink-log4j-session | Change Flink log4j-session.properties settings for Kubernetes/Yarn session. | Not available. | | flink-log4j-cli | Change Flink log4j-cli.properties settings. | Restarts Flink history server. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts PhoenixQueryserver, HiveServer2, Hive MetaStore, and MapReduce-HistoryServer. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts Hadoop KMS, Hadoop Httpfs, and MapReduce-HistoryServer. | | hadoop-ssl-server | Change hadoop ssl server configuration | Not available. | | hadoop-ssl-client | Change hadoop ssl client configuration | Not available. | | hbase | Amazon EMR-curated settings for Apache HBase. | Custom EMR specific property. Sets emrfs-site and hbase-site configs. See those for their associated restarts. | | hbase-env | Change values in HBase's environment. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbase.properties file. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | Not available. | | hbase-site | Change values in HBase's hbase-site.xml file. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. Additionally restarts Phoenix QueryServer. | | hdfs-encryption-zones | Configure HDFS encryption zones. | Should not be reconfigured. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Additionally restarts Hadoop Httpfs. | | hcatalog-env | Change values in HCatalog's environment. | Restarts Hive HCatalog Server. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | Restarts Hive HCatalog Server. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | Restarts Hive HCatalog Server. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | Restarts Hive WebHCat Server. | | hcatalog-webhcat-log4j2 | Change values in HCatalog WebHCat's log4j2.properties. | Restarts Hive WebHCat Server. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | Restarts Hive WebHCat Server. | | hive-beeline-log4j2 | Change values in Hive's beeline-log4j2.properties file. | Not available. | | hive-parquet-logging | Change values in Hive's parquet-logging.properties file. | Not available. | | hive-env | Change values in the Hive environment. | Restarts HiveServer2 and HiveMetastore. Runs Hive schemaTool CLI commands to verify hive-metastore. | | hive-exec-log4j2 | Change values in Hive's hive-exec-log4j2.properties file. | Restarts HiveServer2 and HiveMetastore. | | hive-llap-daemon-log4j2 | Change values in Hive's llap-daemon-log4j2.properties file. | Not available. | | hive-log4j2 | Change values in Hive's hive-log4j2.properties file. | Not available. | | hive-site | Change values in Hive's hive-site.xml file | Restarts HiveServer2 and HiveMetastore. Runs Hive schemaTool CLI commands to verify hive-metastore. Also restarts Oozie and Zeppelin. | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | Not available. | | hue-ini | Change values in Hue's ini file | Restarts Hue. Also activates Hue config override CLI commands to pick up new configurations. | | httpfs-env | Change values in the HTTPFS environment. | Restarts Hadoop Httpfs service. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | Restarts Hadoop Httpfs service. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | Not available. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | Restarts Hadoop-KMS service. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | Not available. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | Restarts Hadoop-KMS and Ranger-KMS service. | | hudi-env | Change values in the Hudi environment. | Not available. | | hudi-defaults | Change values in Hudi's hudi-defaults.conf file. | Not available. | | jupyter-notebook-conf | Change values in Jupyter Notebook's jupyter\$1notebook\$1config.py file. | Not available. | | jupyter-hub-conf | Change values in JupyterHubs's jupyterhub\$1config.py file. | Not available. | | jupyter-s3-conf | Configure Jupyter Notebook S3 persistence. | Not available. | | jupyter-sparkmagic-conf | Change values in Sparkmagic's config.json file. | Not available. | | livy-conf | Change values in Livy's livy.conf file. | Restarts Livy Server. | | livy-env | Change values in the Livy environment. | Restarts Livy Server. | | livy-log4j | Change Livy log4j.properties settings. | Restarts Livy Server. | | mapred-env | Change values in the MapReduce application's environment. | Restarts Hadoop MapReduce-HistoryServer. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | Restarts Hadoop MapReduce-HistoryServer. | | oozie-env | Change values in Oozie's environment. | Restarts Oozie. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | Restarts Oozie. | | oozie-site | Change values in Oozie's oozie-site.xml file. | Restarts Oozie. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | Not available. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | Not available. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | Restarts Phoenix-QueryServer. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | Not available. | | pig-env | Change values in the Pig environment. | Not available. | | pig-properties | Change values in Pig's pig.properties file. | Restarts Oozie. | | pig-log4j | Change values in Pig's log4j.properties file. | Not available. | | presto-log | Change values in Presto's log.properties file. | Restarts Presto-Server. | | presto-config | Change values in Presto's config.properties file. | Restarts Presto-Server. | | presto-password-authenticator | Change values in Presto's password-authenticator.properties file. | Not available. | | presto-env | Change values in Presto's presto-env.sh file. | Restarts Presto-Server. | | presto-node | Change values in Presto's node.properties file. | Not available. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | Not available. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | Not available. | | presto-connector-hive | Change values in Presto's hive.properties file. | Restarts Presto-Server. | | presto-connector-jmx | Change values in Presto's jmx.properties file. | Not available. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | Not available. | | presto-connector-localfile | Change values in Presto's localfile.properties file. | Not available. | | presto-connector-memory | Change values in Presto's memory.properties file. | Not available. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | Not available. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | Not available. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | Not available. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | Not available. | | presto-connector-redis | Change values in Presto's redis.properties file. | Not available. | | presto-connector-redshift | Change values in Presto's redshift.properties file. | Not available. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | Not available. | | presto-connector-tpcds | Change values in Presto's tpcds.properties file. | Not available. | | ranger-kms-dbks-site | Change values in dbks-site.xml file of Ranger KMS. | Restarts Ranger KMS Server. | | ranger-kms-site | Change values in ranger-kms-site.xml file of Ranger KMS. | Restarts Ranger KMS Server. | | ranger-kms-env | Change values in the Ranger KMS environment. | Restarts Ranger KMS Server. | | ranger-kms-log4j | Change values in kms-log4j.properties file of Ranger KMS. | Not available. | | ranger-kms-db-ca | Change values for CA file on S3 for MySQL SSL connection with Ranger KMS. | Not available. | | recordserver-env | Change values in the EMR RecordServer environment. | Restarts EMR record server. | | recordserver-conf | Change values in EMR RecordServer's erver.properties file. | Restarts EMR record server. | | recordserver-log4j | Change values in EMR RecordServer's log4j.properties file. | Restarts EMR record server. | | spark | Amazon EMR-curated settings for Apache Spark. | This property modifies spark-defaults. See actions there. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | Restarts Spark history server and Spark thrift server. | | spark-env | Change values in the Spark environment. | Restarts Spark history server and Spark thrift server. | | spark-hive-site | Change values in Spark's hive-site.xml file | Not available. | | spark-log4j | Change values in Spark's log4j.properties file. | Restarts Spark history server and Spark thrift server. | | spark-metrics | Change values in Spark's metrics.properties file. | Restarts Spark history server and Spark thrift server. | | sqoop-env | Change values in Sqoop's environment. | Not available. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | Not available. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | Not available. | | tez-site | Change values in Tez's tez-site.xml file. | Restarts Oozie. | | yarn-env | Change values in the YARN environment. | Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts MapReduce-HistoryServer. | | yarn-site | Change values in YARN's yarn-site.xml file. | Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts Livy Server and MapReduce-HistoryServer. | | zeppelin-env | Change values in the Zeppelin environment. | Restarts Zeppelin. | | zeppelin-site | Change configuration settings in zeppelin-site.xml. | Restarts Zeppelin. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | Restarts Zookeeper server. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | Restarts Zookeeper server. | ## 5.36.1 change log **Change log for 5.36.1 release and release notes** | Date | Event | Description | | --- | --- | --- | | 2023-07-26 | Update | New OS release labels 2.0.20230612.0 and 2.0.20230628.0. | | 2023-05-25 | Deployment complete | Amazon EMR 5.36.1 fully deployed to all [supported Regions](https://aws.amazon.com/about-aws/global-infrastructure/regional-product-services/) | | 2023-05-09 | Docs publication | Amazon EMR 5.36.1 release notes first published | | 2023-05-04 | Initial release | Amazon EMR 5.36.1 first deployed to limited commercial Regions | # Amazon EMR release 5.36.0 ## 5.36.0 application versions This release includes the following applications: [https://flink.apache.org/](https://flink.apache.org/), [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [https://hudi.apache.org](https://hudi.apache.org), [http://gethue.com/](http://gethue.com/), [https://jupyter-enterprise-gateway.readthedocs.io/en/latest/](https://jupyter-enterprise-gateway.readthedocs.io/en/latest/), [https://jupyterhub.readthedocs.io/en/latest/#](https://jupyterhub.readthedocs.io/en/latest/#), [https://livy.incubator.apache.org/](https://livy.incubator.apache.org/), [https://mxnet.incubator.apache.org/](https://mxnet.incubator.apache.org/), [http://mahout.apache.org/](http://mahout.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://www.tensorflow.org/](https://www.tensorflow.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-5.36.0 | emr-5.35.0 | emr-5.34.0 | emr-5.33.1 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.12.206 | 1.12.159 | 1.11.970 | 1.11.970 | | Python | 2.7, 3.7 | 2.7, 3.7 | 2.7, 3.7 | 2.7, 3.7 | | Scala | 2.11.12 | 2.11.12 | 2.11.12 | 2.11.12 | | AmazonCloudWatchAgent | - | - | - | - | | Delta | - | - | - | - | | Flink | 1.14.2 | 1.14.2 | 1.13.1 | 1.12.1 | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 1.4.13 | 1.4.13 | 1.4.13 | 1.4.13 | | HCatalog | 2.3.9-amzn-2 | 2.3.9-amzn-0 | 2.3.8-amzn-0 | 2.3.7-amzn-4 | | Hadoop | 2.10.1-amzn-4 | 2.10.1-amzn-3 | 2.10.1-amzn-2 | 2.10.1-amzn-1.1 | | Hive | 2.3.9-amzn-2 | 2.3.9-amzn-0 | 2.3.8-amzn-0 | 2.3.7-amzn-4 | | Hudi | 0.10.1-amzn-1 | 0.9.0-amzn-2 | 0.9.0-amzn-0 | 0.7.0-amzn-1 | | Hue | 4.10.0 | 4.10.0 | 4.9.0 | 4.9.0 | | Iceberg | - | - | - | - | | JupyterEnterpriseGateway | 2.1.0 | 2.1.0 | 2.1.0 | 2.1.0 | | JupyterHub | 1.4.1 | 1.4.1 | 1.4.1 | 1.2.2 | | Livy | 0.7.1-incubating | 0.7.1-incubating | 0.7.1-incubating | 0.7.0-incubating | | MXNet | 1.8.0 | 1.8.0 | 1.8.0 | 1.7.0 | | Mahout | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | | Oozie | 5.2.1 | 5.2.1 | 5.2.1 | 5.2.0 | | Phoenix | 4.14.3-HBase-1.4 | 4.14.3-HBase-1.4 | 4.14.3-HBase-1.4 | 4.14.3-HBase-1.4 | | Pig | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | | Presto | 0.267-amzn-1 | 0.266-amzn-0 | 0.261-amzn-0 | 0.245.1-amzn-0 | | Spark | 2.4.8-amzn-2 | 2.4.8-amzn-1 | 2.4.8-amzn-0 | 2.4.7-amzn-1.1 | | Sqoop | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | | TensorFlow | 2.4.1 | 2.4.1 | 2.4.1 | 2.4.1 | | Tez | 0.9.2 | 0.9.2 | 0.9.2 | 0.9.2 | | Trino (PrestoSQL) | - | - | - | - | | Zeppelin | 0.10.0 | 0.10.0 | 0.10.0 | 0.9.0 | | ZooKeeper | 3.4.14 | 3.4.14 | 3.4.14 | 3.4.14 | ## 5.36.0 release notes The following release notes include information for Amazon EMR release 5.36.0. Changes are relative to 5.35.0. Initial release date: June 15, 2022 **New Features** + Amazon EMR release 5.36.0 adds support for data definition language (DDL) with Apache Spark on Apache Ranger enabled clusters. This allows you to use Apache Ranger for managing access for operations like creating, altering and dropping databases and tables from an Amazon EMR cluster. + When you launch a cluster with *the latest patch release* of Amazon EMR 5.36 or higher, 6.6 or higher, or 7.0 or higher, Amazon EMR uses the latest Amazon Linux 2023 or Amazon Linux 2 release for the default Amazon EMR AMI. For more information, see [Using the default Amazon Linux AMI for Amazon EMR](https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-default-ami.html). **Note** This release no longer gets automatic AMI updates since it has been succeeded by 1 more more patch releases. The patch release is denoted by the number after the second decimal point (`6.8.1`). To see if you're using the latest patch release, check the available releases in the [https://docs.aws.amazon.com/emr/latest/ReleaseGuide](https://docs.aws.amazon.com/emr/latest/ReleaseGuide), or check the **Amazon EMR release** dropdown when you create a cluster in the console, or use the [https://docs.aws.amazon.com/emr/latest/APIReference/API_ListReleaseLabels.html](https://docs.aws.amazon.com/emr/latest/APIReference/API_ListReleaseLabels.html) API or [https://docs.aws.amazon.com/cli/latest/reference/emr/list-release-labels.html](https://docs.aws.amazon.com/cli/latest/reference/emr/list-release-labels.html) CLI action. To get updates about new releases, subscribe to the RSS feed on the [What's new?](https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-whatsnew.html) page. | OsReleaseLabel (Amazon Linux Version) | Amazon Linux Kernel Version | Available Date | Supported Regions | | --- | --- | --- | --- | | 2.0.20260302.0 | 4.14.355-280.714.amzn2 | March 13, 2026 | Africa (Cape Town), Asia Pacific (Hong Kong), Asia Pacific (Tokyo), Asia Pacific (Seoul), Asia Pacific (Osaka), Asia Pacific (Mumbai), Asia Pacific (Hyderabad), Asia Pacific (Singapore), Asia Pacific (Sydney), Asia Pacific (Jakarta), Asia Pacific (Melbourne), Canada (Central), Europe (Frankfurt), Europe (Zurich), Europe (Stockholm), Europe (Milan), Europe (Spain), Europe (Ireland), Europe (London), Europe (Paris), Israel (Tel Aviv), Middle East (UAE), Middle East (Bahrain), South America (São Paulo), US East (N. Virginia), US East (Ohio), US West (N. California), US West (Oregon) | | 2.0.20260216.0 | 4.14.355-280.714.amzn2 | February 25, 2026 | Africa (Cape Town), Asia Pacific (Hong Kong), Asia Pacific (Tokyo), Asia Pacific (Seoul), Asia Pacific (Osaka), Asia Pacific (Mumbai), Asia Pacific (Hyderabad), Asia Pacific (Singapore), Asia Pacific (Sydney), Asia Pacific (Jakarta), Asia Pacific (Melbourne), Canada (Central), Europe (Frankfurt), Europe (Zurich), Europe (Stockholm), Europe (Milan), Europe (Spain), Europe (Ireland), Europe (London), Europe (Paris), Israel (Tel Aviv), Middle East (UAE), Middle East (Bahrain), South America (São Paulo), US East (N. Virginia), US East (Ohio), US West (N. California), US West (Oregon) | | 2.0.20260120.1 | 4.14.355-280.713.amzn2 | February 18, 2026 | Africa (Cape Town), Asia Pacific (Hong Kong), Asia Pacific (Tokyo), Asia Pacific (Seoul), Asia Pacific (Osaka), Asia Pacific (Mumbai), Asia Pacific (Hyderabad), Asia Pacific (Singapore), Asia Pacific (Sydney), Asia Pacific (Jakarta), Asia Pacific (Melbourne), Canada (Central), Europe (Frankfurt), Europe (Zurich), Europe (Stockholm), Europe (Milan), Europe (Spain), Europe (Ireland), Europe (London), Europe (Paris), Israel (Tel Aviv), Middle East (UAE), Middle East (Bahrain), South America (São Paulo), US East (N. Virginia), US East (Ohio), US West (N. California), US West (Oregon) | | 2.0.20251208.0 | 4.14.355-280.710.amzn2 | January 13, 2026 | Africa (Cape Town), Asia Pacific (Hong Kong), Asia Pacific (Tokyo), Asia Pacific (Seoul), Asia Pacific (Osaka), Asia Pacific (Mumbai), Asia Pacific (Hyderabad), Asia Pacific (Singapore), Asia Pacific (Sydney), Asia Pacific (Jakarta), Asia Pacific (Melbourne), Canada (Central), Europe (Frankfurt), Europe (Zurich), Europe (Stockholm), Europe (Milan), Europe (Spain), Europe (Ireland), Europe (London), Europe (Paris), Israel (Tel Aviv), Middle East (UAE), Middle East (Bahrain), South America (São Paulo), US East (N. Virginia), US East (Ohio), US West (N. California), US West (Oregon) | | 2.0.20251121.0 | 4.14.355-280.708.amzn2 | December 16, 2025 | Africa (Cape Town), Asia Pacific (Hong Kong), Asia Pacific (Tokyo), Asia Pacific (Seoul), Asia Pacific (Osaka), Asia Pacific (Mumbai), Asia Pacific (Hyderabad), Asia Pacific (Singapore), Asia Pacific (Sydney), Asia Pacific (Jakarta), Asia Pacific (Melbourne), Canada (Central), Europe (Frankfurt), Europe (Zurich), Europe (Stockholm), Europe (Milan), Europe (Spain), Europe (Ireland), Europe (London), Europe (Paris), Israel (Tel Aviv), Middle East (UAE), Middle East (Bahrain), South America (São Paulo), US East (N. Virginia), US East (Ohio), US West (N. California), US West (Oregon) | | 2.0.20251027.1 | 4.14.355-280.706.amzn2 | November 10, 2025 | Africa (Cape Town), Asia Pacific (Hong Kong), Asia Pacific (Tokyo), Asia Pacific (Seoul), Asia Pacific (Osaka), Asia Pacific (Mumbai), Asia Pacific (Hyderabad), Asia Pacific (Singapore), Asia Pacific (Sydney), Asia Pacific (Jakarta), Asia Pacific (Melbourne), Canada (Central), Europe (Frankfurt), Europe (Zurich), Europe (Stockholm), Europe (Milan), Europe (Spain), Europe (Ireland), Europe (London), Europe (Paris), Israel (Tel Aviv), Middle East (UAE), Middle East (Bahrain), South America (São Paulo), US East (N. Virginia), US East (Ohio), US West (N. California), US West (Oregon) | | 2.0.20250929.2 | 4.14.355-280.695.amzn2 | October 13, 2025 | Africa (Cape Town), Asia Pacific (Hong Kong), Asia Pacific (Tokyo), Asia Pacific (Seoul), Asia Pacific (Osaka), Asia Pacific (Mumbai), Asia Pacific (Hyderabad), Asia Pacific (Singapore), Asia Pacific (Sydney), Asia Pacific (Jakarta), Asia Pacific (Melbourne), Canada (Central), Europe (Frankfurt), Europe (Zurich), Europe (Stockholm), Europe (Milan), Europe (Spain), Europe (Ireland), Europe (London), Europe (Paris), Israel (Tel Aviv), Middle East (UAE), Middle East (Bahrain), South America (São Paulo), US East (N. Virginia), US East (Ohio), US West (N. California), US West (Oregon) | | 2.0.20250818.2 | 4.14.355-280.672.amzn2 | September 17, 2025 | Africa (Cape Town), Asia Pacific (Hong Kong), Asia Pacific (Tokyo), Asia Pacific (Seoul), Asia Pacific (Osaka), Asia Pacific (Mumbai), Asia Pacific (Hyderabad), Asia Pacific (Singapore), Asia Pacific (Sydney), Asia Pacific (Jakarta), Asia Pacific (Melbourne), Canada (Central), Europe (Frankfurt), Europe (Zurich), Europe (Stockholm), Europe (Milan), Europe (Spain), Europe (Ireland), Europe (London), Europe (Paris), Israel (Tel Aviv), Middle East (UAE), Middle East (Bahrain), South America (São Paulo), US East (N. Virginia), US East (Ohio), US West (N. California), US West (Oregon) | | 2.0.20250808.1 | 4.14.355-280.664.amzn2 | August 28, 2025 | Africa (Cape Town), Asia Pacific (Hong Kong), Asia Pacific (Tokyo), Asia Pacific (Seoul), Asia Pacific (Osaka), Asia Pacific (Mumbai), Asia Pacific (Hyderabad), Asia Pacific (Singapore), Asia Pacific (Sydney), Asia Pacific (Jakarta), Asia Pacific (Melbourne), Canada (Central), Europe (Frankfurt), Europe (Zurich), Europe (Stockholm), Europe (Milan), Europe (Spain), Europe (Ireland), Europe (London), Europe (Paris), Israel (Tel Aviv), Middle East (UAE), Middle East (Bahrain), South America (São Paulo), US East (N. Virginia), US East (Ohio), US West (N. California), US West (Oregon) | | 2.0.20250721.2 | 4.14.355-280.652.amzn2 | August 14, 2025 | Africa (Cape Town), Asia Pacific (Hong Kong), Asia Pacific (Tokyo), Asia Pacific (Seoul), Asia Pacific (Osaka), Asia Pacific (Mumbai), Asia Pacific (Hyderabad), Asia Pacific (Singapore), Asia Pacific (Sydney), Asia Pacific (Jakarta), Asia Pacific (Melbourne), Canada (Central), Europe (Frankfurt), Europe (Zurich), Europe (Stockholm), Europe (Milan), Europe (Spain), Europe (Ireland), Europe (London), Europe (Paris), Israel (Tel Aviv), Middle East (UAE), Middle East (Bahrain), South America (São Paulo), US East (N. Virginia), US East (Ohio), US West (N. California), US West (Oregon) | | 2.0.20250623.0 | 4.14.355-277.647.amzn2 | July 21, 2025 | Africa (Cape Town), Asia Pacific (Hong Kong), Asia Pacific (Tokyo), Asia Pacific (Seoul), Asia Pacific (Osaka), Asia Pacific (Mumbai), Asia Pacific (Hyderabad), Asia Pacific (Singapore), Asia Pacific (Sydney), Asia Pacific (Jakarta), Asia Pacific (Melbourne), Canada (Central), Europe (Frankfurt), Europe (Zurich), Europe (Stockholm), Europe (Milan), Europe (Spain), Europe (Ireland), Europe (London), Europe (Paris), Israel (Tel Aviv), Middle East (UAE), Middle East (Bahrain), South America (São Paulo), US East (N. Virginia), US East (Ohio), US West (N. California), US West (Oregon) | | 2.0.20250610.0 | 4.14.355-277.647.amzn2 | July 14, 2025 | Europe (Stockholm), Middle East (Bahrain), China (Beijing), Asia Pacific (Mumbai), Europe (Paris), Asia Pacific (Jakarta), US East (Ohio), Africa (Cape Town), Europe (Ireland), Middle East (UAE), Europe (Frankfurt), South America (São Paulo), Asia Pacific (Hong Kong), Asia Pacific (Hyderabad), US East (N. Virginia), Asia Pacific (Seoul), Asia Pacific (Osaka), Europe (London), Europe (Milan), Asia Pacific (Tokyo), AWS GovCloud (US-East), AWS GovCloud (US-West), US West (Oregon), US West (N. California), Asia Pacific (Singapore), Asia Pacific (Sydney), Israel (Tel Aviv), Canada (Central), Europe (Spain), China (Ningxia), Europe (Zurich) | | 2.0.20250527.1 | 4.14.355-277.647.amzn2 | Jun 19, 2025 | Europe (Stockholm), Middle East (Bahrain), Asia Pacific (Mumbai), Europe (Paris), Asia Pacific (Jakarta), US East (Ohio), Africa (Cape Town), Europe (Ireland), Middle East (UAE), Europe (Frankfurt), South America (São Paulo), Asia Pacific (Hong Kong), Asia Pacific (Hyderabad), US East (N. Virginia), Asia Pacific (Seoul), Asia Pacific (Osaka), Europe (London), Asia Pacific (Melbourne), Europe (Milan), Asia Pacific (Tokyo), US West (Oregon), US West (N. California), Asia Pacific (Singapore), Asia Pacific (Sydney), Israel (Tel Aviv), Canada (Central), Europe (Spain), Europe (Zurich) | | 2.0.20250512.0 | 4.14.355-277.643.amzn2 | Jun 04, 2025 | US East (N. Virginia), US East (Ohio), US West (N. California), US West (Oregon), Europe (Frankfurt), Europe (Ireland), Europe (London), Europe (Milan), Europe (Paris), Europe (Stockholm), Asia Pacific (Hong Kong), Asia Pacific (Jakarta), Asia Pacific (Mumbai), Asia Pacific (Osaka), Asia Pacific (Seoul), Asia Pacific (Singapore), Asia Pacific (Sydney), Asia Pacific (Tokyo), Africa (Cape Town), South America (São Paulo), Middle East (Bahrain), Canada (Central), AWS GovCloud (US-East), AWS GovCloud (US-West), China (Beijing), China (Ningxia) | | 2.0.20250428.0 | 4.14.355-276.639.amzn2 | May 23, 2025 | US East (N. Virginia), US East (Ohio), US West (N. California), US West (Oregon), Europe (Frankfurt), Europe (Ireland), Europe (London), Europe (Milan), Europe (Paris), Europe (Spain), Europe (Stockholm), Europe (Zurich), Asia Pacific (Hong Kong), Asia Pacific (Hyderabad), Asia Pacific (Jakarta), Asia Pacific (Melbourne), Asia Pacific (Mumbai), Asia Pacific (Osaka), Asia Pacific (Seoul), Asia Pacific (Singapore), Asia Pacific (Sydney), Asia Pacific (Tokyo), Africa (Cape Town), South America (São Paulo), Middle East (Bahrain), Middle East (UAE), Canada (Central), Israel (Tel Aviv) | | 2.0.20250414.0 | 4.14.355-276.618.amzn2 | May 12, 2025 | US East (N. Virginia), US East (Ohio), US West (N. California), US West (Oregon), Europe (Stockholm), Europe (Milan), Europe (Frankfurt), Europe (Ireland), Europe (London), Europe (Paris), Asia Pacific (Hong Kong), Asia Pacific (Mumbai), Asia Pacific (Tokyo), Asia Pacific (Seoul), Asia Pacific (Osaka), Asia Pacific (Singapore), Asia Pacific (Sydney), Asia Pacific (Jakarta), Africa (Cape Town), South America (São Paulo), Middle East (Bahrain), Canada (Central), AWS GovCloud (US-West), AWS GovCloud (US-East), China (Beijing), China (Ningxia) | | 2.0.20250321.0 | 4.14.355 | April 09, 2025 | US East (N. Virginia), US East (Ohio), US West (N. California), US West (Oregon), Europe (Stockholm), Europe (Milan), Europe (Frankfurt), Europe (Ireland), Europe (London), Europe (Paris), Asia Pacific (Hong Kong), Asia Pacific (Mumbai), Asia Pacific (Tokyo), Asia Pacific (Seoul), Asia Pacific (Osaka), Asia Pacific (Singapore), Asia Pacific (Sydney), Asia Pacific (Jakarta), Africa (Cape Town), South America (São Paulo), Middle East (Bahrain), Canada (Central), AWS GovCloud (US-West), AWS GovCloud (US-East), China (Beijing), China (Ningxia) | | 2.0.20250305.0 | 4.14.355 | March 18, 2025 | US East (N. Virginia), US East (Ohio), US West (N. California), US West (Oregon), Europe (Stockholm), Europe (Milan), Europe (Frankfurt), Europe (Ireland), Europe (London), Europe (Paris), Asia Pacific (Hong Kong), Asia Pacific (Mumbai), Asia Pacific (Tokyo), Asia Pacific (Seoul), Asia Pacific (Osaka), Asia Pacific (Singapore), Asia Pacific (Sydney), Asia Pacific (Jakarta), Africa (Cape Town), South America (São Paulo), Middle East (Bahrain), Canada (Central), AWS GovCloud (US-West), AWS GovCloud (US-East), China (Beijing), China (Ningxia) | | 2.0.20250220.0 | 4.14.355 | March 08, 2025 | US East (N. Virginia), US East (Ohio), US West (N. California), US West (Oregon), Europe (Stockholm), Europe (Milan), Europe (Frankfurt), Europe (Ireland), Europe (London), Europe (Paris), Asia Pacific (Hong Kong), Asia Pacific (Mumbai), Asia Pacific (Tokyo), Asia Pacific (Seoul), Asia Pacific (Osaka), Asia Pacific (Singapore), Asia Pacific (Sydney), Asia Pacific (Jakarta), Africa (Cape Town), South America (São Paulo), Middle East (Bahrain), Canada (Central), AWS GovCloud (US-West), AWS GovCloud (US-East), China (Beijing), China (Ningxia) | | 2.0.20250201.0 | 4.14.355 | February 28, 2025 | US East (N. Virginia), US East (Ohio), US West (N. California), US West (Oregon), Europe (Stockholm), Europe (Milan), Europe (Frankfurt), Europe (Ireland), Europe (London), Europe (Paris), Asia Pacific (Hong Kong), Asia Pacific (Mumbai), Asia Pacific (Tokyo), Asia Pacific (Seoul), Asia Pacific (Osaka), Asia Pacific (Singapore), Asia Pacific (Sydney), Asia Pacific (Jakarta), Africa (Cape Town), South America (São Paulo), Middle East (Bahrain), Canada (Central), AWS GovCloud (US-West), AWS GovCloud (US-East), China (Beijing), China (Ningxia) | | 2.0.20250123.4 | 4.14.355 | January 27, 2025 | US East (N. Virginia), US East (Ohio), US West (N. California), US West (Oregon), Europe (Stockholm), Europe (Milan), Europe (Frankfurt), Europe (Ireland), Europe (London), Europe (Paris), Asia Pacific (Hong Kong), Asia Pacific (Mumbai), Asia Pacific (Tokyo), Asia Pacific (Seoul), Asia Pacific (Osaka), Asia Pacific (Singapore), Asia Pacific (Sydney), Asia Pacific (Jakarta), Africa (Cape Town), South America (São Paulo), Middle East (Bahrain), Canada (Central), AWS GovCloud (US-West), AWS GovCloud (US-East), China (Beijing), China (Ningxia) | | 2.0.20250116.0 | 4.14.355 | January 23, 2025 | US East (N. Virginia), US East (Ohio), US West (N. California), US West (Oregon), Europe (Stockholm), Europe (Milan), Europe (Frankfurt), Europe (Ireland), Europe (London), Europe (Paris), Asia Pacific (Hong Kong), Asia Pacific (Mumbai), Asia Pacific (Tokyo), Asia Pacific (Seoul), Asia Pacific (Osaka), Asia Pacific (Singapore), Asia Pacific (Sydney), Asia Pacific (Jakarta), Africa (Cape Town), South America (São Paulo), Middle East (Bahrain), Canada (Central), AWS GovCloud (US-West), AWS GovCloud (US-East), China (Beijing), China (Ningxia) | | 2.0.20241217.0 | 4.14.355 | January 8, 2025 | US East (N. Virginia), US East (Ohio), US West (N. California), US West (Oregon), Europe (Stockholm), Europe (Milan), Europe (Frankfurt), Europe (Ireland), Europe (London), Europe (Paris), Asia Pacific (Hong Kong), Asia Pacific (Mumbai), Asia Pacific (Tokyo), Asia Pacific (Seoul), Asia Pacific (Osaka), Asia Pacific (Singapore), Asia Pacific (Sydney), Asia Pacific (Jakarta), Africa (Cape Town), South America (São Paulo), Middle East (Bahrain), Canada (Central), AWS GovCloud (US-West), AWS GovCloud (US-East), China (Beijing), China (Ningxia), Asia Pacific (Hyderabad), Middle East (UAE), Europe (Spain), Europe (Zurich), Asia Pacific (Melbourne), Israel (Tel Aviv) | | 2.0.20240709.1 | 4.14.348 | July 23, 2024 | US East (N. Virginia), US East (Ohio), US West (N. California), US West (Oregon), Europe (Stockholm), Europe (Milan), Europe (Frankfurt), Europe (Ireland), Europe (London), Europe (Paris), Asia Pacific (Hong Kong), Asia Pacific (Mumbai), Asia Pacific (Tokyo), Asia Pacific (Seoul), Asia Pacific (Osaka), Asia Pacific (Singapore), Asia Pacific (Sydney), Asia Pacific (Jakarta), Africa (Cape Town), South America (São Paulo), Middle East (Bahrain), Canada (Central), AWS GovCloud (US-West), AWS GovCloud (US-East), China (Beijing), China (Ningxia), Asia Pacific (Hyderabad), Middle East (UAE), Europe (Spain), Europe (Zurich), Asia Pacific (Melbourne), Israel (Tel Aviv) | | 2.0.20230504.1 | 4.14.313 | May 16, 2023 | US East (N. Virginia), US East (Ohio), US West (N. California), US West (Oregon), Canada (Central), Europe (Stockholm), Europe (Ireland), Europe (London), Europe (Paris), Europe (Frankfurt), Europe (Milan), Asia Pacific (Hong Kong), Asia Pacific (Mumbai), Asia Pacific (Jakarta), Asia Pacific (Tokyo), Asia Pacific (Seoul), Asia Pacific (Osaka), Asia Pacific (Singapore), Asia Pacific (Sydney), Africa (Cape Town), South America (São Paulo), Middle East (Bahrain) | | 2.0.20230418.0 | 4.14.311 | May 3, 2023 | US East (N. Virginia), US East (Ohio), US West (N. California), US West (Oregon), Canada (Central), Europe (Stockholm), Europe (Ireland), Europe (London), Europe (Paris), Europe (Frankfurt), Europe (Milan), Asia Pacific (Hong Kong), Asia Pacific (Mumbai), Asia Pacific (Jakarta), Asia Pacific (Tokyo), Asia Pacific (Seoul), Asia Pacific (Osaka), Asia Pacific (Singapore), Asia Pacific (Sydney), Africa (Cape Town), South America (São Paulo), Middle East (Bahrain) | | 2.0.20230404.1 | 4.14.311 | April 18, 2023 | US East (N. Virginia), US East (Ohio), US West (N. California), US West (Oregon), Canada (Central), Europe (Stockholm), Europe (Ireland), Europe (London), Europe (Paris), Europe (Frankfurt), Europe (Milan), Asia Pacific (Hong Kong), Asia Pacific (Mumbai), Asia Pacific (Jakarta), Asia Pacific (Tokyo), Asia Pacific (Seoul), Asia Pacific (Osaka), Asia Pacific (Singapore), Asia Pacific (Sydney), Africa (Cape Town), South America (São Paulo), Middle East (Bahrain) | | 2.0.20230404.0 | 4.14.311 | April 10, 2023 | US East (N. Virginia), Europe (Paris) | | 2.0.20230320.0 | 4.14.309 | March 30, 2023 | US East (N. Virginia), US East (Ohio), US West (N. California), US West (Oregon), Canada (Central), Europe (Stockholm), Europe (Ireland), Europe (London), Europe (Paris), Europe (Frankfurt), Europe (Milan), Asia Pacific (Hong Kong), Asia Pacific (Mumbai), Asia Pacific (Jakarta), Asia Pacific (Tokyo), Asia Pacific (Seoul), Asia Pacific (Osaka), Asia Pacific (Singapore), Asia Pacific (Sydney), Africa (Cape Town), South America (São Paulo), Middle East (Bahrain) | | 2.0.20230307.0 | 4.14.305 | March 15, 2023 | US East (N. Virginia), US East (Ohio), US West (N. California), US West (Oregon), Canada (Central), Europe (Stockholm), Europe (Ireland), Europe (London), Europe (Paris), Europe (Frankfurt), Europe (Milan), Asia Pacific (Hong Kong), Asia Pacific (Mumbai), Asia Pacific (Jakarta), Asia Pacific (Tokyo), Asia Pacific (Seoul), Asia Pacific (Osaka), Asia Pacific (Singapore), Asia Pacific (Sydney), Africa (Cape Town), South America (São Paulo), Middle East (Bahrain) | | 2.0.20230207.0 | 4.14.304 | February 22, 2023 | US East (N. Virginia), US East (Ohio), US West (N. California), US West (Oregon), Canada (Central), Europe (Stockholm), Europe (Ireland), Europe (London), Europe (Paris), Europe (Frankfurt), Europe (Milan), Asia Pacific (Hong Kong), Asia Pacific (Mumbai), Asia Pacific (Jakarta), Asia Pacific (Tokyo), Asia Pacific (Seoul), Asia Pacific (Osaka), Asia Pacific (Singapore), Asia Pacific (Sydney), Africa (Cape Town), South America (São Paulo), Middle East (Bahrain) | | 2.0.20230119.1 | 4.14.301 | February 3, 2023 | US East (N. Virginia), US East (Ohio), US West (N. California), US West (Oregon), Canada (Central), Europe (Stockholm), Europe (Ireland), Europe (London), Europe (Paris), Europe (Frankfurt), Europe (Milan), Asia Pacific (Hong Kong), Asia Pacific (Mumbai), Asia Pacific (Jakarta), Asia Pacific (Tokyo), Asia Pacific (Seoul), Asia Pacific (Osaka), Asia Pacific (Singapore), Asia Pacific (Sydney), Africa (Cape Town), South America (São Paulo), Middle East (Bahrain) | | 2.0.20221210.1 | 4.14.301 | December 22, 2023 | US East (N. Virginia), US East (Ohio), US West (N. California), US West (Oregon), Canada (Central), Europe (Stockholm), Europe (Ireland), Europe (London), Europe (Paris), Europe (Frankfurt), Europe (Milan), Asia Pacific (Hong Kong), Asia Pacific (Mumbai), Asia Pacific (Jakarta), Asia Pacific (Tokyo), Asia Pacific (Seoul), Asia Pacific (Osaka), Asia Pacific (Singapore), Asia Pacific (Sydney), Africa (Cape Town), South America (São Paulo), Middle East (Bahrain) | | 2.0.20221103.3 | 4.14.296 | December 5, 2022 | US East (N. Virginia), US East (Ohio), US West (N. California), US West (Oregon), Canada (Central), Europe (Stockholm), Europe (Ireland), Europe (London), Europe (Paris), Europe (Frankfurt), Europe (Milan), Asia Pacific (Hong Kong), Asia Pacific (Mumbai), Asia Pacific (Jakarta), Asia Pacific (Tokyo), Asia Pacific (Seoul), Asia Pacific (Osaka), Asia Pacific (Singapore), Asia Pacific (Sydney), Africa (Cape Town), South America (São Paulo), Middle East (Bahrain) | | 2.0.20221004.0 | 4.14.294 | November 2, 2022 | US East (N. Virginia), US East (Ohio), US West (N. California), US West (Oregon), Canada (Central), Europe (Stockholm), Europe (Ireland), Europe (London), Europe (Paris), Europe (Frankfurt), Europe (Milan), Asia Pacific (Hong Kong), Asia Pacific (Mumbai), Asia Pacific (Jakarta), Asia Pacific (Tokyo), Asia Pacific (Seoul), Asia Pacific (Osaka), Asia Pacific (Singapore), Asia Pacific (Sydney), Africa (Cape Town), South America (São Paulo), Middle East (Bahrain) | | 2.0.20220912.1 | 4.14.291 | October 7, 2022 | US East (N. Virginia), US East (Ohio), US West (N. California), US West (Oregon), Canada (Central), Europe (Stockholm), Europe (Ireland), Europe (London), Europe (Paris), Europe (Frankfurt), Europe (Milan), Asia Pacific (Hong Kong), Asia Pacific (Mumbai), Asia Pacific (Jakarta), Asia Pacific (Tokyo), Asia Pacific (Seoul), Asia Pacific (Osaka), Asia Pacific (Singapore), Asia Pacific (Sydney), Africa (Cape Town), South America (São Paulo), Middle East (Bahrain) | | 2.0.20220719.0 | 4.14.287 | August 10, 2022 | US West (N. California), Europe (Paris), Europe (Stockholm), Europe (Frankfurt), Asia Pacific (Mumbai), Middle East (Bahrain) | | 2.0.20220426.0 | 4.14.281 | June 14, 2022 | US East (N. Virginia), US East (Ohio), US West (N. California), US West (Oregon), Canada (Central), Europe (Stockholm), Europe (Ireland), Europe (London), Europe (Paris), Europe (Frankfurt), Europe (Milan), Asia Pacific (Hong Kong), Asia Pacific (Mumbai), Asia Pacific (Jakarta), Asia Pacific (Tokyo), Asia Pacific (Seoul), Asia Pacific (Osaka), Asia Pacific (Singapore), Asia Pacific (Sydney), Africa (Cape Town), South America (São Paulo), Middle East (Bahrain) | **Changes, Enhancements, and Resolved Issues** + Amazon EMR 5.36.0 upgrades now support: aws-sdk 1.12.206, Hadoop 2.10.1-amzn-4, Hive 2.3.9-amzn-2, Hudi 0.10.1-amzn-1, Spark 2.4.8-amzn-2, Presto 0.267-amzn-1, Amazon Glue connector 1.18.0, EMRFS 2.51.0. **Known issues** + When you use Spark with Hive partition location formatting to read data in Amazon S3, and you run Spark on Amazon EMR releases 5.30.0 to 5.36.0, and 6.2.0 to 6.9.0, you might encounter an issue that prevents your cluster from reading data correctly. This can happen if your partitions have all of the following characteristics: + Two or more partitions are scanned from the same table. + At least one partition directory path is a prefix of at least one other partition directory path, for example, `s3://bucket/table/p=a` is a prefix of `s3://bucket/table/p=a b`. + The first character that follows the prefix in the other partition directory has a UTF-8 value that’s less than than the `/` character (U\$1002F). For example, the space character (U\$10020) that occurs between a and b in `s3://bucket/table/p=a b` falls into this category. Note that there are 14 other non-control characters: `!"#$%&‘()*+,-`. For more information, see [UTF-8 encoding table and Unicode characters](https://www.utf8-chartable.de/). As a workaround to this issue, set the `spark.sql.sources.fastS3PartitionDiscovery.enabled` configuration to `false` in the `spark-defaults` classification. + With Amazon EMR releases 5.36.0 and 6.6.0 through 6.9.0, `SecretAgent` and `RecordServer` service components may experience log data loss due to an incorrect file name pattern configuration in Log4j2 properties. The incorrect configuration causes the components to generate only one log file per day. When the rotation strategy occurs, it overwrites the existing file instead of generating a new log file as expected. As a workaround, use a bootstrap action to generate log files each hour and append an auto-increment integer in the file name to handle the rotation. For Amazon EMR 6.6.0 through 6.9.0 releases, use the following bootstrap action when you launch a cluster. ``` ‑‑bootstrap‑actions "Path=s3://emr-data-access-control-us-east-1/customer-bootstrap-actions/log-rotation-emr-6x/replace-puppet.sh,Args=[]" ``` For Amazon EMR 5.36.0, use the following bootstrap action when you launch a cluster. ``` ‑‑bootstrap‑actions "Path=s3://emr-data-access-control-us-east-1/customer-bootstrap-actions/log-rotation-emr-5x/replace-puppet.sh,Args=[]" ``` ## 5.36.0 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | aws-sagemaker-spark-sdk | 1.4.1 | Amazon SageMaker Spark SDK | | emr-ddb | 4.16.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 2.16.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.5.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-notebook-env | 1.5.0 | Conda env for emr notebook which includes jupyter enterprise gateway | | emr-s3-dist-cp | 2.21.0 | Distributed copy application optimized for Amazon S3. | | emr-s3-select | 1.7.0 | EMR S3Select Connector | | emrfs | 2.51.0 | Amazon S3 connector for Hadoop ecosystem applications. | | flink-client | 1.14.2 | Apache Flink command line client scripts and applications. | | flink-jobmanager-config | 1.14.2 | Managing resources on EMR nodes for Apache Flink JobManager. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 2.10.1-amzn-4 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 2.10.1-amzn-4 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 2.10.1-amzn-4 | HDFS command-line client and library | | hadoop-hdfs-namenode | 2.10.1-amzn-4 | HDFS service for tracking file names and block locations. | | hadoop-hdfs-journalnode | 2.10.1-amzn-4 | HDFS service for managing the Hadoop filesystem journal on HA clusters. | | hadoop-httpfs-server | 2.10.1-amzn-4 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 2.10.1-amzn-4 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 2.10.1-amzn-4 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 2.10.1-amzn-4 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 2.10.1-amzn-4 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 2.10.1-amzn-4 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 1.4.13 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 1.4.13 | Service for serving one or more HBase regions. | | hbase-client | 1.4.13 | HBase command-line client. | | hbase-rest-server | 1.4.13 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 1.4.13 | Service providing a Thrift endpoint to HBase. | | hcatalog-client | 2.3.9-amzn-2 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 2.3.9-amzn-2 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 2.3.9-amzn-2 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 2.3.9-amzn-2 | Hive command line client. | | hive-hbase | 2.3.9-amzn-2 | Hive-hbase client. | | hive-metastore-server | 2.3.9-amzn-2 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server2 | 2.3.9-amzn-2 | Service for accepting Hive queries as web requests. | | hudi | 0.10.1-amzn-1 | Incremental processing framework to power data pipeline at low latency and high efficiency. | | hudi-spark | 0.10.1-amzn-1 | Bundle library for running Spark with Hudi. | | hudi-presto | 0.10.1-amzn-1 | Bundle library for running Presto with Hudi. | | hue-server | 4.10.0 | Web application for analyzing data using Hadoop ecosystem applications | | jupyterhub | 1.4.1 | Multi-user server for Jupyter notebooks | | livy-server | 0.7.1-incubating | REST interface for interacting with Apache Spark | | nginx | 1.13.1 | nginx [engine x] is an HTTP and reverse proxy server | | mahout-client | 0.13.0 | Library for machine learning. | | mxnet | 1.8.0 | A flexible, scalable, and efficient library for deep learning. | | mariadb-server | 5.5.68 | MySQL database server. | | nvidia-cuda | 11.0.194 | Nvidia drivers and Cuda toolkit | | oozie-client | 5.2.1 | Oozie command-line client. | | oozie-server | 5.2.1 | Service for accepting Oozie workflow requests. | | opencv | 4.5.0 | Open Source Computer Vision Library. | | phoenix-library | 4.14.3-HBase-1.4 | The phoenix libraries for server and client | | phoenix-query-server | 4.14.3-HBase-1.4 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.267-amzn-1 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.267-amzn-1 | Service for executing pieces of a query. | | presto-client | 0.267-amzn-1 | Presto command-line client which is installed on an HA cluster's stand-by masters where Presto server is not started. | | pig-client | 0.17.0 | Pig command-line client. | | r | 4.0.2 | The R Project for Statistical Computing | | ranger-kms-server | 1.2.0 | Apache Ranger Key Management System | | spark-client | 2.4.8-amzn-2 | Spark command-line clients. | | spark-history-server | 2.4.8-amzn-2 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 2.4.8-amzn-2 | In-memory execution engine for YARN. | | spark-yarn-slave | 2.4.8-amzn-2 | Apache Spark libraries needed by YARN slaves. | | sqoop-client | 1.4.7 | Apache Sqoop command-line client. | | tensorflow | 2.4.1 | TensorFlow open source software library for high performance numerical computation. | | tez-on-yarn | 0.9.2 | The tez YARN application and libraries. | | webserver | 2.4.25\$1 | Apache HTTP server. | | zeppelin-server | 0.10.0 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.4.14 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.4.14 | ZooKeeper command line client. | ## 5.36.0 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). Reconfiguration actions occur when you specify a configuration for instance groups in a running cluster. Amazon EMR only initiates reconfiguration actions for the classifications that you modify. For more information, see [Reconfigure an instance group in a running cluster](emr-configure-apps-running-cluster.md). **emr-5.36.0 classifications** | Classifications | Description | Reconfiguration Actions | | --- | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | Restarts the ResourceManager service. | | container-executor | Change values in Hadoop YARN's container-executor.cfg file. | Not available. | | container-log4j | Change values in Hadoop YARN's container-log4j.properties file. | Not available. | | core-site | Change values in Hadoop's core-site.xml file. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts Hadoop KMS, Ranger KMS, HiveServer2, Hive MetaStore, Hadoop Httpfs, and MapReduce-HistoryServer. | | docker-conf | Change docker related settings. | Not available. | | emrfs-site | Change EMRFS settings. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts HBaseRegionserver, HBaseMaster, HBaseThrift, HBaseRest, HiveServer2, Hive MetaStore, Hadoop Httpfs, and MapReduce-HistoryServer. | | flink-conf | Change flink-conf.yaml settings. | Restarts Flink history server. | | flink-log4j | Change Flink log4j.properties settings. | Restarts Flink history server. | | flink-log4j-session | Change Flink log4j-session.properties settings for Kubernetes/Yarn session. | Not available. | | flink-log4j-cli | Change Flink log4j-cli.properties settings. | Restarts Flink history server. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts PhoenixQueryserver, HiveServer2, Hive MetaStore, and MapReduce-HistoryServer. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts Hadoop KMS, Hadoop Httpfs, and MapReduce-HistoryServer. | | hadoop-ssl-server | Change hadoop ssl server configuration | Not available. | | hadoop-ssl-client | Change hadoop ssl client configuration | Not available. | | hbase | Amazon EMR-curated settings for Apache HBase. | Custom EMR specific property. Sets emrfs-site and hbase-site configs. See those for their associated restarts. | | hbase-env | Change values in HBase's environment. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbase.properties file. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | Not available. | | hbase-site | Change values in HBase's hbase-site.xml file. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. Additionally restarts Phoenix QueryServer. | | hdfs-encryption-zones | Configure HDFS encryption zones. | Should not be reconfigured. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Additionally restarts Hadoop Httpfs. | | hcatalog-env | Change values in HCatalog's environment. | Restarts Hive HCatalog Server. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | Restarts Hive HCatalog Server. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | Restarts Hive HCatalog Server. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | Restarts Hive WebHCat Server. | | hcatalog-webhcat-log4j2 | Change values in HCatalog WebHCat's log4j2.properties. | Restarts Hive WebHCat Server. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | Restarts Hive WebHCat Server. | | hive-beeline-log4j2 | Change values in Hive's beeline-log4j2.properties file. | Not available. | | hive-parquet-logging | Change values in Hive's parquet-logging.properties file. | Not available. | | hive-env | Change values in the Hive environment. | Restarts HiveServer2 and HiveMetastore. Runs Hive schemaTool CLI commands to verify hive-metastore. | | hive-exec-log4j2 | Change values in Hive's hive-exec-log4j2.properties file. | Restarts HiveServer2 and HiveMetastore. | | hive-llap-daemon-log4j2 | Change values in Hive's llap-daemon-log4j2.properties file. | Not available. | | hive-log4j2 | Change values in Hive's hive-log4j2.properties file. | Not available. | | hive-site | Change values in Hive's hive-site.xml file | Restarts HiveServer2 and HiveMetastore. Runs Hive schemaTool CLI commands to verify hive-metastore. Also restarts Oozie and Zeppelin. | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | Not available. | | hue-ini | Change values in Hue's ini file | Restarts Hue. Also activates Hue config override CLI commands to pick up new configurations. | | httpfs-env | Change values in the HTTPFS environment. | Restarts Hadoop Httpfs service. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | Restarts Hadoop Httpfs service. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | Not available. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | Restarts Hadoop-KMS service. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | Not available. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | Restarts Hadoop-KMS and Ranger-KMS service. | | hudi-env | Change values in the Hudi environment. | Not available. | | jupyter-notebook-conf | Change values in Jupyter Notebook's jupyter\$1notebook\$1config.py file. | Not available. | | jupyter-hub-conf | Change values in JupyterHubs's jupyterhub\$1config.py file. | Not available. | | jupyter-s3-conf | Configure Jupyter Notebook S3 persistence. | Not available. | | jupyter-sparkmagic-conf | Change values in Sparkmagic's config.json file. | Not available. | | livy-conf | Change values in Livy's livy.conf file. | Restarts Livy Server. | | livy-env | Change values in the Livy environment. | Restarts Livy Server. | | livy-log4j | Change Livy log4j.properties settings. | Restarts Livy Server. | | mapred-env | Change values in the MapReduce application's environment. | Restarts Hadoop MapReduce-HistoryServer. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | Restarts Hadoop MapReduce-HistoryServer. | | oozie-env | Change values in Oozie's environment. | Restarts Oozie. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | Restarts Oozie. | | oozie-site | Change values in Oozie's oozie-site.xml file. | Restarts Oozie. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | Not available. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | Not available. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | Restarts Phoenix-QueryServer. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | Not available. | | pig-env | Change values in the Pig environment. | Not available. | | pig-properties | Change values in Pig's pig.properties file. | Restarts Oozie. | | pig-log4j | Change values in Pig's log4j.properties file. | Not available. | | presto-log | Change values in Presto's log.properties file. | Restarts Presto-Server. | | presto-config | Change values in Presto's config.properties file. | Restarts Presto-Server. | | presto-password-authenticator | Change values in Presto's password-authenticator.properties file. | Not available. | | presto-env | Change values in Presto's presto-env.sh file. | Restarts Presto-Server. | | presto-node | Change values in Presto's node.properties file. | Not available. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | Not available. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | Not available. | | presto-connector-hive | Change values in Presto's hive.properties file. | Restarts Presto-Server. | | presto-connector-jmx | Change values in Presto's jmx.properties file. | Not available. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | Not available. | | presto-connector-localfile | Change values in Presto's localfile.properties file. | Not available. | | presto-connector-memory | Change values in Presto's memory.properties file. | Not available. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | Not available. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | Not available. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | Not available. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | Not available. | | presto-connector-redis | Change values in Presto's redis.properties file. | Not available. | | presto-connector-redshift | Change values in Presto's redshift.properties file. | Not available. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | Not available. | | presto-connector-tpcds | Change values in Presto's tpcds.properties file. | Not available. | | ranger-kms-dbks-site | Change values in dbks-site.xml file of Ranger KMS. | Restarts Ranger KMS Server. | | ranger-kms-site | Change values in ranger-kms-site.xml file of Ranger KMS. | Restarts Ranger KMS Server. | | ranger-kms-env | Change values in the Ranger KMS environment. | Restarts Ranger KMS Server. | | ranger-kms-log4j | Change values in kms-log4j.properties file of Ranger KMS. | Not available. | | ranger-kms-db-ca | Change values for CA file on S3 for MySQL SSL connection with Ranger KMS. | Not available. | | recordserver-env | Change values in the EMR RecordServer environment. | Restarts EMR record server. | | recordserver-conf | Change values in EMR RecordServer's erver.properties file. | Restarts EMR record server. | | recordserver-log4j | Change values in EMR RecordServer's log4j.properties file. | Restarts EMR record server. | | spark | Amazon EMR-curated settings for Apache Spark. | This property modifies spark-defaults. See actions there. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | Restarts Spark history server and Spark thrift server. | | spark-env | Change values in the Spark environment. | Restarts Spark history server and Spark thrift server. | | spark-hive-site | Change values in Spark's hive-site.xml file | Not available. | | spark-log4j | Change values in Spark's log4j.properties file. | Restarts Spark history server and Spark thrift server. | | spark-metrics | Change values in Spark's metrics.properties file. | Restarts Spark history server and Spark thrift server. | | sqoop-env | Change values in Sqoop's environment. | Not available. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | Not available. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | Not available. | | tez-site | Change values in Tez's tez-site.xml file. | Restarts Oozie. | | yarn-env | Change values in the YARN environment. | Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts MapReduce-HistoryServer. | | yarn-site | Change values in YARN's yarn-site.xml file. | Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts Livy Server and MapReduce-HistoryServer. | | zeppelin-env | Change values in the Zeppelin environment. | Restarts Zeppelin. | | zeppelin-site | Change configuration settings in zeppelin-site.xml. | Restarts Zeppelin. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | Restarts Zookeeper server. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | Restarts Zookeeper server. | # Amazon EMR release 5.35.0 ## 5.35.0 application versions This release includes the following applications: [https://flink.apache.org/](https://flink.apache.org/), [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [https://hudi.apache.org](https://hudi.apache.org), [http://gethue.com/](http://gethue.com/), [https://jupyter-enterprise-gateway.readthedocs.io/en/latest/](https://jupyter-enterprise-gateway.readthedocs.io/en/latest/), [https://jupyterhub.readthedocs.io/en/latest/#](https://jupyterhub.readthedocs.io/en/latest/#), [https://livy.incubator.apache.org/](https://livy.incubator.apache.org/), [https://mxnet.incubator.apache.org/](https://mxnet.incubator.apache.org/), [http://mahout.apache.org/](http://mahout.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://www.tensorflow.org/](https://www.tensorflow.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-5.35.0 | emr-5.34.0 | emr-5.33.1 | emr-5.33.0 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.12.159 | 1.11.970 | 1.11.970 | 1.11.970 | | Python | 2.7, 3.7 | 2.7, 3.7 | 2.7, 3.7 | 2.7, 3.7 | | Scala | 2.11.12 | 2.11.12 | 2.11.12 | 2.11.12 | | AmazonCloudWatchAgent | - | - | - | - | | Delta | - | - | - | - | | Flink | 1.14.2 | 1.13.1 | 1.12.1 | 1.12.1 | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 1.4.13 | 1.4.13 | 1.4.13 | 1.4.13 | | HCatalog | 2.3.9-amzn-0 | 2.3.8-amzn-0 | 2.3.7-amzn-4 | 2.3.7-amzn-4 | | Hadoop | 2.10.1-amzn-3 | 2.10.1-amzn-2 | 2.10.1-amzn-1.1 | 2.10.1-amzn-1 | | Hive | 2.3.9-amzn-0 | 2.3.8-amzn-0 | 2.3.7-amzn-4 | 2.3.7-amzn-4 | | Hudi | 0.9.0-amzn-2 | 0.9.0-amzn-0 | 0.7.0-amzn-1 | 0.7.0-amzn-1 | | Hue | 4.10.0 | 4.9.0 | 4.9.0 | 4.9.0 | | Iceberg | - | - | - | - | | JupyterEnterpriseGateway | 2.1.0 | 2.1.0 | 2.1.0 | 2.1.0 | | JupyterHub | 1.4.1 | 1.4.1 | 1.2.2 | 1.2.2 | | Livy | 0.7.1-incubating | 0.7.1-incubating | 0.7.0-incubating | 0.7.0-incubating | | MXNet | 1.8.0 | 1.8.0 | 1.7.0 | 1.7.0 | | Mahout | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | | Oozie | 5.2.1 | 5.2.1 | 5.2.0 | 5.2.0 | | Phoenix | 4.14.3-HBase-1.4 | 4.14.3-HBase-1.4 | 4.14.3-HBase-1.4 | 4.14.3-HBase-1.4 | | Pig | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | | Presto | 0.266-amzn-0 | 0.261-amzn-0 | 0.245.1-amzn-0 | 0.245.1-amzn-0 | | Spark | 2.4.8-amzn-1 | 2.4.8-amzn-0 | 2.4.7-amzn-1.1 | 2.4.7-amzn-1 | | Sqoop | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | | TensorFlow | 2.4.1 | 2.4.1 | 2.4.1 | 2.4.1 | | Tez | 0.9.2 | 0.9.2 | 0.9.2 | 0.9.2 | | Trino (PrestoSQL) | - | - | - | - | | Zeppelin | 0.10.0 | 0.10.0 | 0.9.0 | 0.9.0 | | ZooKeeper | 3.4.14 | 3.4.14 | 3.4.14 | 3.4.14 | ## 5.35.0 release notes This is the Amazon EMR release 5.35.0 release note. The following release notes include information for Amazon EMR release 5.35.0. Changes are relative to 5.34.0. Initial release date: March 30, 2022 **New Features** + Amazon EMR release 5.35 applications that use Log4j 1.x and Log4j 2.x are upgraded to use Log4j 1.2.17 (or higher) and Log4j 2.17.1 (or higher) respectively, and do not require using bootstrap actions to mitigate the CVE issues in previous releases. See [Approach to mitigate CVE-2021-44228](emr-log4j-vulnerability.md). **Changes, Enhancements, and Resolved Issues** **Flink changes** | Change type | Description | | --- | --- | | Upgrades | [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-5350-release.html) | **Hadoop changes** | Change type | Description | | --- | --- | | Hadoop open source backports since EMR 5.34.0 | [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-5350-release.html) | | Hadoop changes and fixes | [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-5350-release.html) | **Hive changes** | Change type | Description | | --- | --- | | Hive upgraded to open source [release 2.3.9](https://www.mail-archive.com/user@hive.apache.org/msg22311.html), including these JIRA fixes | [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-5350-release.html) | | Hive open source backports since EMR 5.34.0 | [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-5350-release.html) | | Hive upgrades and fixes | [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-5350-release.html) | | New features | [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-5350-release.html) | **Oozie changes** | Change type | Description | | --- | --- | | Oozie open source backports since EMR 5.34.0 | [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-5350-release.html) | **Pig changes** | Change type | Description | | --- | --- | | Upgrades | [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-5350-release.html) | **Known issues** + When you use Spark with Hive partition location formatting to read data in Amazon S3, and you run Spark on Amazon EMR releases 5.30.0 to 5.36.0, and 6.2.0 to 6.9.0, you might encounter an issue that prevents your cluster from reading data correctly. This can happen if your partitions have all of the following characteristics: + Two or more partitions are scanned from the same table. + At least one partition directory path is a prefix of at least one other partition directory path, for example, `s3://bucket/table/p=a` is a prefix of `s3://bucket/table/p=a b`. + The first character that follows the prefix in the other partition directory has a UTF-8 value that’s less than than the `/` character (U\$1002F). For example, the space character (U\$10020) that occurs between a and b in `s3://bucket/table/p=a b` falls into this category. Note that there are 14 other non-control characters: `!"#$%&‘()*+,-`. For more information, see [UTF-8 encoding table and Unicode characters](https://www.utf8-chartable.de/). As a workaround to this issue, set the `spark.sql.sources.fastS3PartitionDiscovery.enabled` configuration to `false` in the `spark-defaults` classification. ## 5.35.0 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | aws-sagemaker-spark-sdk | 1.4.1 | Amazon SageMaker Spark SDK | | emr-ddb | 4.16.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 2.15.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.5.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-notebook-env | 1.5.0 | Conda env for emr notebook which includes jupyter enterprise gateway | | emr-s3-dist-cp | 2.20.0 | Distributed copy application optimized for Amazon S3. | | emr-s3-select | 1.7.0 | EMR S3Select Connector | | emrfs | 2.49.0 | Amazon S3 connector for Hadoop ecosystem applications. | | flink-client | 1.14.2 | Apache Flink command line client scripts and applications. | | flink-jobmanager-config | 1.14.2 | Managing resources on EMR nodes for Apache Flink JobManager. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 2.10.1-amzn-3 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 2.10.1-amzn-3 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 2.10.1-amzn-3 | HDFS command-line client and library | | hadoop-hdfs-namenode | 2.10.1-amzn-3 | HDFS service for tracking file names and block locations. | | hadoop-hdfs-journalnode | 2.10.1-amzn-3 | HDFS service for managing the Hadoop filesystem journal on HA clusters. | | hadoop-httpfs-server | 2.10.1-amzn-3 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 2.10.1-amzn-3 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 2.10.1-amzn-3 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 2.10.1-amzn-3 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 2.10.1-amzn-3 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 2.10.1-amzn-3 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 1.4.13 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 1.4.13 | Service for serving one or more HBase regions. | | hbase-client | 1.4.13 | HBase command-line client. | | hbase-rest-server | 1.4.13 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 1.4.13 | Service providing a Thrift endpoint to HBase. | | hcatalog-client | 2.3.9-amzn-0 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 2.3.9-amzn-0 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 2.3.9-amzn-0 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 2.3.9-amzn-0 | Hive command line client. | | hive-hbase | 2.3.9-amzn-0 | Hive-hbase client. | | hive-metastore-server | 2.3.9-amzn-0 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server2 | 2.3.9-amzn-0 | Service for accepting Hive queries as web requests. | | hudi | 0.9.0-amzn-2 | Incremental processing framework to power data pipeline at low latency and high efficiency. | | hudi-spark | 0.9.0-amzn-2 | Bundle library for running Spark with Hudi. | | hudi-presto | 0.9.0-amzn-2 | Bundle library for running Presto with Hudi. | | hue-server | 4.10.0 | Web application for analyzing data using Hadoop ecosystem applications | | jupyterhub | 1.4.1 | Multi-user server for Jupyter notebooks | | livy-server | 0.7.1-incubating | REST interface for interacting with Apache Spark | | nginx | 1.13.1 | nginx [engine x] is an HTTP and reverse proxy server | | mahout-client | 0.13.0 | Library for machine learning. | | mxnet | 1.8.0 | A flexible, scalable, and efficient library for deep learning. | | mariadb-server | 5.5.68 | MySQL database server. | | nvidia-cuda | 10.1.243 | Nvidia drivers and Cuda toolkit | | oozie-client | 5.2.1 | Oozie command-line client. | | oozie-server | 5.2.1 | Service for accepting Oozie workflow requests. | | opencv | 4.5.0 | Open Source Computer Vision Library. | | phoenix-library | 4.14.3-HBase-1.4 | The phoenix libraries for server and client | | phoenix-query-server | 4.14.3-HBase-1.4 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.266-amzn-0 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.266-amzn-0 | Service for executing pieces of a query. | | presto-client | 0.266-amzn-0 | Presto command-line client which is installed on an HA cluster's stand-by masters where Presto server is not started. | | pig-client | 0.17.0 | Pig command-line client. | | r | 4.0.2 | The R Project for Statistical Computing | | ranger-kms-server | 1.2.0 | Apache Ranger Key Management System | | spark-client | 2.4.8-amzn-1 | Spark command-line clients. | | spark-history-server | 2.4.8-amzn-1 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 2.4.8-amzn-1 | In-memory execution engine for YARN. | | spark-yarn-slave | 2.4.8-amzn-1 | Apache Spark libraries needed by YARN slaves. | | sqoop-client | 1.4.7 | Apache Sqoop command-line client. | | tensorflow | 2.4.1 | TensorFlow open source software library for high performance numerical computation. | | tez-on-yarn | 0.9.2 | The tez YARN application and libraries. | | webserver | 2.4.25\$1 | Apache HTTP server. | | zeppelin-server | 0.10.0 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.4.14 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.4.14 | ZooKeeper command line client. | ## 5.35.0 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). Reconfiguration actions occur when you specify a configuration for instance groups in a running cluster. Amazon EMR only initiates reconfiguration actions for the classifications that you modify. For more information, see [Reconfigure an instance group in a running cluster](emr-configure-apps-running-cluster.md). **emr-5.35.0 classifications** | Classifications | Description | Reconfiguration Actions | | --- | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | Restarts the ResourceManager service. | | container-executor | Change values in Hadoop YARN's container-executor.cfg file. | Not available. | | container-log4j | Change values in Hadoop YARN's container-log4j.properties file. | Not available. | | core-site | Change values in Hadoop's core-site.xml file. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts Hadoop KMS, Ranger KMS, HiveServer2, Hive MetaStore, Hadoop Httpfs, and MapReduce-HistoryServer. | | docker-conf | Change docker related settings. | Not available. | | emrfs-site | Change EMRFS settings. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts HBaseRegionserver, HBaseMaster, HBaseThrift, HBaseRest, HiveServer2, Hive MetaStore, Hadoop Httpfs, and MapReduce-HistoryServer. | | flink-conf | Change flink-conf.yaml settings. | Restarts Flink history server. | | flink-log4j | Change Flink log4j.properties settings. | Restarts Flink history server. | | flink-log4j-session | Change Flink log4j-session.properties settings for Kubernetes/Yarn session. | Not available. | | flink-log4j-cli | Change Flink log4j-cli.properties settings. | Restarts Flink history server. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts PhoenixQueryserver, HiveServer2, Hive MetaStore, and MapReduce-HistoryServer. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts Hadoop KMS, Hadoop Httpfs, and MapReduce-HistoryServer. | | hadoop-ssl-server | Change hadoop ssl server configuration | Not available. | | hadoop-ssl-client | Change hadoop ssl client configuration | Not available. | | hbase | Amazon EMR-curated settings for Apache HBase. | Custom EMR specific property. Sets emrfs-site and hbase-site configs. See those for their associated restarts. | | hbase-env | Change values in HBase's environment. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbase.properties file. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | Not available. | | hbase-site | Change values in HBase's hbase-site.xml file. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. Additionally restarts Phoenix QueryServer. | | hdfs-encryption-zones | Configure HDFS encryption zones. | Should not be reconfigured. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Additionally restarts Hadoop Httpfs. | | hcatalog-env | Change values in HCatalog's environment. | Restarts Hive HCatalog Server. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | Restarts Hive HCatalog Server. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | Restarts Hive HCatalog Server. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | Restarts Hive WebHCat Server. | | hcatalog-webhcat-log4j2 | Change values in HCatalog WebHCat's log4j2.properties. | Restarts Hive WebHCat Server. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | Restarts Hive WebHCat Server. | | hive-beeline-log4j2 | Change values in Hive's beeline-log4j2.properties file. | Not available. | | hive-parquet-logging | Change values in Hive's parquet-logging.properties file. | Not available. | | hive-env | Change values in the Hive environment. | Restarts HiveServer2 and HiveMetastore. Runs Hive schemaTool CLI commands to verify hive-metastore. | | hive-exec-log4j2 | Change values in Hive's hive-exec-log4j2.properties file. | Restarts HiveServer2 and HiveMetastore. | | hive-llap-daemon-log4j2 | Change values in Hive's llap-daemon-log4j2.properties file. | Not available. | | hive-log4j2 | Change values in Hive's hive-log4j2.properties file. | Not available. | | hive-site | Change values in Hive's hive-site.xml file | Restarts HiveServer2 and HiveMetastore. Runs Hive schemaTool CLI commands to verify hive-metastore. Also restarts Oozie and Zeppelin. | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | Not available. | | hue-ini | Change values in Hue's ini file | Restarts Hue. Also activates Hue config override CLI commands to pick up new configurations. | | httpfs-env | Change values in the HTTPFS environment. | Restarts Hadoop Httpfs service. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | Restarts Hadoop Httpfs service. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | Not available. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | Restarts Hadoop-KMS service. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | Not available. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | Restarts Hadoop-KMS and Ranger-KMS service. | | hudi-env | Change values in the Hudi environment. | Not available. | | jupyter-notebook-conf | Change values in Jupyter Notebook's jupyter\$1notebook\$1config.py file. | Not available. | | jupyter-hub-conf | Change values in JupyterHubs's jupyterhub\$1config.py file. | Not available. | | jupyter-s3-conf | Configure Jupyter Notebook S3 persistence. | Not available. | | jupyter-sparkmagic-conf | Change values in Sparkmagic's config.json file. | Not available. | | livy-conf | Change values in Livy's livy.conf file. | Restarts Livy Server. | | livy-env | Change values in the Livy environment. | Restarts Livy Server. | | livy-log4j | Change Livy log4j.properties settings. | Restarts Livy Server. | | mapred-env | Change values in the MapReduce application's environment. | Restarts Hadoop MapReduce-HistoryServer. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | Restarts Hadoop MapReduce-HistoryServer. | | oozie-env | Change values in Oozie's environment. | Restarts Oozie. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | Restarts Oozie. | | oozie-site | Change values in Oozie's oozie-site.xml file. | Restarts Oozie. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | Not available. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | Not available. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | Restarts Phoenix-QueryServer. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | Not available. | | pig-env | Change values in the Pig environment. | Not available. | | pig-properties | Change values in Pig's pig.properties file. | Restarts Oozie. | | pig-log4j | Change values in Pig's log4j.properties file. | Not available. | | presto-log | Change values in Presto's log.properties file. | Restarts Presto-Server. | | presto-config | Change values in Presto's config.properties file. | Restarts Presto-Server. | | presto-password-authenticator | Change values in Presto's password-authenticator.properties file. | Not available. | | presto-env | Change values in Presto's presto-env.sh file. | Restarts Presto-Server. | | presto-node | Change values in Presto's node.properties file. | Not available. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | Not available. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | Not available. | | presto-connector-hive | Change values in Presto's hive.properties file. | Restarts Presto-Server. | | presto-connector-jmx | Change values in Presto's jmx.properties file. | Not available. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | Not available. | | presto-connector-localfile | Change values in Presto's localfile.properties file. | Not available. | | presto-connector-memory | Change values in Presto's memory.properties file. | Not available. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | Not available. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | Not available. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | Not available. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | Not available. | | presto-connector-redis | Change values in Presto's redis.properties file. | Not available. | | presto-connector-redshift | Change values in Presto's redshift.properties file. | Not available. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | Not available. | | presto-connector-tpcds | Change values in Presto's tpcds.properties file. | Not available. | | ranger-kms-dbks-site | Change values in dbks-site.xml file of Ranger KMS. | Restarts Ranger KMS Server. | | ranger-kms-site | Change values in ranger-kms-site.xml file of Ranger KMS. | Restarts Ranger KMS Server. | | ranger-kms-env | Change values in the Ranger KMS environment. | Restarts Ranger KMS Server. | | ranger-kms-log4j | Change values in kms-log4j.properties file of Ranger KMS. | Not available. | | ranger-kms-db-ca | Change values for CA file on S3 for MySQL SSL connection with Ranger KMS. | Not available. | | recordserver-env | Change values in the EMR RecordServer environment. | Restarts EMR record server. | | recordserver-conf | Change values in EMR RecordServer's erver.properties file. | Restarts EMR record server. | | recordserver-log4j | Change values in EMR RecordServer's log4j.properties file. | Restarts EMR record server. | | spark | Amazon EMR-curated settings for Apache Spark. | This property modifies spark-defaults. See actions there. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | Restarts Spark history server and Spark thrift server. | | spark-env | Change values in the Spark environment. | Restarts Spark history server and Spark thrift server. | | spark-hive-site | Change values in Spark's hive-site.xml file | Not available. | | spark-log4j | Change values in Spark's log4j.properties file. | Restarts Spark history server and Spark thrift server. | | spark-metrics | Change values in Spark's metrics.properties file. | Restarts Spark history server and Spark thrift server. | | sqoop-env | Change values in Sqoop's environment. | Not available. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | Not available. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | Not available. | | tez-site | Change values in Tez's tez-site.xml file. | Restarts Oozie. | | yarn-env | Change values in the YARN environment. | Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts MapReduce-HistoryServer. | | yarn-site | Change values in YARN's yarn-site.xml file. | Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts Livy Server and MapReduce-HistoryServer. | | zeppelin-env | Change values in the Zeppelin environment. | Restarts Zeppelin. | | zeppelin-site | Change configuration settings in zeppelin-site.xml. | Restarts Zeppelin. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | Restarts Zookeeper server. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | Restarts Zookeeper server. | # Amazon EMR release 5.34.0 ## 5.34.0 application versions This release includes the following applications: [https://flink.apache.org/](https://flink.apache.org/), [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [https://hudi.apache.org](https://hudi.apache.org), [http://gethue.com/](http://gethue.com/), [https://jupyter-enterprise-gateway.readthedocs.io/en/latest/](https://jupyter-enterprise-gateway.readthedocs.io/en/latest/), [https://jupyterhub.readthedocs.io/en/latest/#](https://jupyterhub.readthedocs.io/en/latest/#), [https://livy.incubator.apache.org/](https://livy.incubator.apache.org/), [https://mxnet.incubator.apache.org/](https://mxnet.incubator.apache.org/), [http://mahout.apache.org/](http://mahout.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://www.tensorflow.org/](https://www.tensorflow.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-5.34.0 | emr-5.33.1 | emr-5.33.0 | emr-5.32.1 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.11.970 | 1.11.970 | 1.11.970 | 1.11.890 | | Python | 2.7, 3.7 | 2.7, 3.7 | 2.7, 3.7 | 2.7, 3.7 | | Scala | 2.11.12 | 2.11.12 | 2.11.12 | 2.11.12 | | AmazonCloudWatchAgent | - | - | - | - | | Delta | - | - | - | - | | Flink | 1.13.1 | 1.12.1 | 1.12.1 | 1.11.2 | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 1.4.13 | 1.4.13 | 1.4.13 | 1.4.13 | | HCatalog | 2.3.8-amzn-0 | 2.3.7-amzn-4 | 2.3.7-amzn-4 | 2.3.7-amzn-3 | | Hadoop | 2.10.1-amzn-2 | 2.10.1-amzn-1.1 | 2.10.1-amzn-1 | 2.10.1-amzn-0.1 | | Hive | 2.3.8-amzn-0 | 2.3.7-amzn-4 | 2.3.7-amzn-4 | 2.3.7-amzn-3 | | Hudi | 0.9.0-amzn-0 | 0.7.0-amzn-1 | 0.7.0-amzn-1 | 0.6.0-amzn-0 | | Hue | 4.9.0 | 4.9.0 | 4.9.0 | 4.8.0 | | Iceberg | - | - | - | - | | JupyterEnterpriseGateway | 2.1.0 | 2.1.0 | 2.1.0 | 2.1.0 | | JupyterHub | 1.4.1 | 1.2.2 | 1.2.2 | 1.1.0 | | Livy | 0.7.1-incubating | 0.7.0-incubating | 0.7.0-incubating | 0.7.0-incubating | | MXNet | 1.8.0 | 1.7.0 | 1.7.0 | 1.7.0 | | Mahout | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | | Oozie | 5.2.1 | 5.2.0 | 5.2.0 | 5.2.0 | | Phoenix | 4.14.3-HBase-1.4 | 4.14.3-HBase-1.4 | 4.14.3-HBase-1.4 | 4.14.3-HBase-1.4 | | Pig | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | | Presto | 0.261-amzn-0 | 0.245.1-amzn-0 | 0.245.1-amzn-0 | 0.240.1-amzn-0 | | Spark | 2.4.8-amzn-0 | 2.4.7-amzn-1.1 | 2.4.7-amzn-1 | 2.4.7-amzn-0.1 | | Sqoop | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | | TensorFlow | 2.4.1 | 2.4.1 | 2.4.1 | 2.3.1 | | Tez | 0.9.2 | 0.9.2 | 0.9.2 | 0.9.2 | | Trino (PrestoSQL) | - | - | - | - | | Zeppelin | 0.10.0 | 0.9.0 | 0.9.0 | 0.8.2 | | ZooKeeper | 3.4.14 | 3.4.14 | 3.4.14 | 3.4.14 | ## 5.34.0 release notes The following release notes include information for Amazon EMR release 5.34.0. Changes are relative to 5.33.1. Initial release date: January 20, 2022 Updated release date: March 21, 2022 **New Features** + **[Managed scaling] Spark shuffle data managed scaling optimization** - For Amazon EMR versions 5.34.0 and later, and EMR versions 6.4.0 and later, managed scaling is now Spark shuffle data aware (data that Spark redistributes across partitions to perform specific operations). For more information on shuffle operations, see [Using EMR managed scaling in Amazon EMR](https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-managed-scaling.html) in the *Amazon EMR Management Guide* and [Spark Programming Guide](https://spark.apache.org/docs/latest/rdd-programming-guide.html#shuffle-operations). + [Hudi] Improvements to simplify Hudi configuration. Disabled optimistic concurrency control by default. **Changes, Enhancements, and Resolved Issues** + This is a release to fix issues with Amazon EMR Scaling when it fails to scale up/scale down a cluster successfully or causes application failures. + Previously, manual restart of the resource manager on a multi-master cluster caused Amazon EMR on-cluster daemons, like Zookeeper, to reload all previously decommissioned or lost nodes in the Zookeeper znode file. This caused default limits to be exceeded in certain situations. Amazon EMR now removes the decommissioned or lost node records older than one hour from the Zookeeper file and the internal limits have been increased. + Fixed an issue where scaling requests failed for a large, highly utilized cluster when Amazon EMR on-cluster daemons were running health checking activities, such as gathering YARN node state and HDFS node state. This was happening because on-cluster daemons were not able to communicate the health status data of a node to internal Amazon EMR components. + Improved EMR on-cluster daemons to correctly track the node states when IP addresses are reused to improve reliability during scaling operations. + [SPARK-29683](https://issues.apache.org/jira/browse/SPARK-29683). Fixed an issue where job failures occurred during cluster scale-down as Spark was assuming all available nodes were deny-listed. + [YARN-9011](https://issues.apache.org/jira/browse/YARN-9011). Fixed an issue where job failures occurred due to a race condition in YARN decommissioning when cluster tried to scale up or down. + Fixed issue with step or job failures during cluster scaling by ensuring that the node states are always consistent between the Amazon EMR on-cluster daemons and YARN/HDFS. + Fixed an issue where cluster operations such as scale down and step submission failed for Amazon EMR clusters enabled with Kerberos authentication. This was because the Amazon EMR on-cluster daemon did not renew the Kerberos ticket, which is required to securely communicate with HDFS/YARN running on the primary node. + Zeppelin upgraded to version 0.10.0. + Livy Fix - upgraded to 0.7.1 + Spark performance improvement - heterogeneous executors are disabled when certain Spark configuration values are overridden in EMR 5.34.0. + WebHDFS and HttpFS server are disabled by default. You can re-enable WebHDFS using the Hadoop configuration, `dfs.webhdfs.enabled`. HttpFS server can be started by using `sudo systemctl start hadoop-httpfs`. **Known Issues** + The Amazon EMR Notebooks feature used with Livy user impersonation does not work because HttpFS is disabled by default. In this case, the EMR notebook cannot connect to the cluster that has Livy impersonation enabled. The workaround is to start HttpFS server before connecting the EMR notebook to the cluster using `sudo systemctl start hadoop-httpfs`. + Hue queries do not work in Amazon EMR 6.4.0 because Apache Hadoop HttpFS server is disabled by default. To use Hue on Amazon EMR 6.4.0, either manually start HttpFS server on the Amazon EMR primary node using `sudo systemctl start hadoop-httpfs`, or [use an Amazon EMR step](https://docs.aws.amazon.com/emr/latest/ManagementGuide/add-step-cli.html). + The Amazon EMR Notebooks feature used with Livy user impersonation does not work because HttpFS is disabled by default. In this case, the EMR notebook cannot connect to the cluster that has Livy impersonation enabled. The workaround is to start HttpFS server before connecting the EMR notebook to the cluster using `sudo systemctl start hadoop-httpfs`. + When you use Spark with Hive partition location formatting to read data in Amazon S3, and you run Spark on Amazon EMR releases 5.30.0 to 5.36.0, and 6.2.0 to 6.9.0, you might encounter an issue that prevents your cluster from reading data correctly. This can happen if your partitions have all of the following characteristics: + Two or more partitions are scanned from the same table. + At least one partition directory path is a prefix of at least one other partition directory path, for example, `s3://bucket/table/p=a` is a prefix of `s3://bucket/table/p=a b`. + The first character that follows the prefix in the other partition directory has a UTF-8 value that’s less than than the `/` character (U\$1002F). For example, the space character (U\$10020) that occurs between a and b in `s3://bucket/table/p=a b` falls into this category. Note that there are 14 other non-control characters: `!"#$%&‘()*+,-`. For more information, see [UTF-8 encoding table and Unicode characters](https://www.utf8-chartable.de/). As a workaround to this issue, set the `spark.sql.sources.fastS3PartitionDiscovery.enabled` configuration to `false` in the `spark-defaults` classification. ## 5.34.0 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | aws-sagemaker-spark-sdk | 1.4.1 | Amazon SageMaker Spark SDK | | emr-ddb | 4.16.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 2.14.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.5.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-notebook-env | 1.4.0 | Conda env for emr notebook which includes jupyter enterprise gateway | | emr-s3-dist-cp | 2.18.0 | Distributed copy application optimized for Amazon S3. | | emr-s3-select | 1.7.0 | EMR S3Select Connector | | emrfs | 2.48.0 | Amazon S3 connector for Hadoop ecosystem applications. | | flink-client | 1.13.1 | Apache Flink command line client scripts and applications. | | flink-jobmanager-config | 1.13.1 | Managing resources on EMR nodes for Apache Flink JobManager. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 2.10.1-amzn-2 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 2.10.1-amzn-2 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 2.10.1-amzn-2 | HDFS command-line client and library | | hadoop-hdfs-namenode | 2.10.1-amzn-2 | HDFS service for tracking file names and block locations. | | hadoop-hdfs-journalnode | 2.10.1-amzn-2 | HDFS service for managing the Hadoop filesystem journal on HA clusters. | | hadoop-httpfs-server | 2.10.1-amzn-2 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 2.10.1-amzn-2 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 2.10.1-amzn-2 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 2.10.1-amzn-2 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 2.10.1-amzn-2 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 2.10.1-amzn-2 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 1.4.13 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 1.4.13 | Service for serving one or more HBase regions. | | hbase-client | 1.4.13 | HBase command-line client. | | hbase-rest-server | 1.4.13 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 1.4.13 | Service providing a Thrift endpoint to HBase. | | hcatalog-client | 2.3.8-amzn-0 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 2.3.8-amzn-0 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 2.3.8-amzn-0 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 2.3.8-amzn-0 | Hive command line client. | | hive-hbase | 2.3.8-amzn-0 | Hive-hbase client. | | hive-metastore-server | 2.3.8-amzn-0 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server2 | 2.3.8-amzn-0 | Service for accepting Hive queries as web requests. | | hudi | 0.9.0-amzn-0 | Incremental processing framework to power data pipeline at low latency and high efficiency. | | hudi-spark | 0.9.0-amzn-0 | Bundle library for running Spark with Hudi. | | hudi-presto | 0.9.0-amzn-0 | Bundle library for running Presto with Hudi. | | hue-server | 4.9.0 | Web application for analyzing data using Hadoop ecosystem applications | | jupyterhub | 1.4.1 | Multi-user server for Jupyter notebooks | | livy-server | 0.7.1-incubating | REST interface for interacting with Apache Spark | | nginx | 1.13.1 | nginx [engine x] is an HTTP and reverse proxy server | | mahout-client | 0.13.0 | Library for machine learning. | | mxnet | 1.8.0 | A flexible, scalable, and efficient library for deep learning. | | mariadb-server | 5.5.68 | MySQL database server. | | nvidia-cuda | 10.1.243 | Nvidia drivers and Cuda toolkit | | oozie-client | 5.2.1 | Oozie command-line client. | | oozie-server | 5.2.1 | Service for accepting Oozie workflow requests. | | opencv | 4.5.0 | Open Source Computer Vision Library. | | phoenix-library | 4.14.3-HBase-1.4 | The phoenix libraries for server and client | | phoenix-query-server | 4.14.3-HBase-1.4 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.261-amzn-0 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.261-amzn-0 | Service for executing pieces of a query. | | presto-client | 0.261-amzn-0 | Presto command-line client which is installed on an HA cluster's stand-by masters where Presto server is not started. | | pig-client | 0.17.0 | Pig command-line client. | | r | 4.0.2 | The R Project for Statistical Computing | | ranger-kms-server | 1.2.0 | Apache Ranger Key Management System | | spark-client | 2.4.8-amzn-0 | Spark command-line clients. | | spark-history-server | 2.4.8-amzn-0 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 2.4.8-amzn-0 | In-memory execution engine for YARN. | | spark-yarn-slave | 2.4.8-amzn-0 | Apache Spark libraries needed by YARN slaves. | | sqoop-client | 1.4.7 | Apache Sqoop command-line client. | | tensorflow | 2.4.1 | TensorFlow open source software library for high performance numerical computation. | | tez-on-yarn | 0.9.2 | The tez YARN application and libraries. | | webserver | 2.4.25\$1 | Apache HTTP server. | | zeppelin-server | 0.10.0 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.4.14 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.4.14 | ZooKeeper command line client. | ## 5.34.0 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). Reconfiguration actions occur when you specify a configuration for instance groups in a running cluster. Amazon EMR only initiates reconfiguration actions for the classifications that you modify. For more information, see [Reconfigure an instance group in a running cluster](emr-configure-apps-running-cluster.md). **emr-5.34.0 classifications** | Classifications | Description | Reconfiguration Actions | | --- | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | Restarts the ResourceManager service. | | container-executor | Change values in Hadoop YARN's container-executor.cfg file. | Not available. | | container-log4j | Change values in Hadoop YARN's container-log4j.properties file. | Not available. | | core-site | Change values in Hadoop's core-site.xml file. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts Hadoop KMS, Ranger KMS, HiveServer2, Hive MetaStore, Hadoop Httpfs, and MapReduce-HistoryServer. | | docker-conf | Change docker related settings. | Not available. | | emrfs-site | Change EMRFS settings. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts HBaseRegionserver, HBaseMaster, HBaseThrift, HBaseRest, HiveServer2, Hive MetaStore, Hadoop Httpfs, and MapReduce-HistoryServer. | | flink-conf | Change flink-conf.yaml settings. | Restarts Flink history server. | | flink-log4j | Change Flink log4j.properties settings. | Restarts Flink history server. | | flink-log4j-session | Change Flink log4j-session.properties settings for Kubernetes/Yarn session. | Not available. | | flink-log4j-cli | Change Flink log4j-cli.properties settings. | Restarts Flink history server. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts PhoenixQueryserver, HiveServer2, Hive MetaStore, and MapReduce-HistoryServer. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts Hadoop KMS, Hadoop Httpfs, and MapReduce-HistoryServer. | | hadoop-ssl-server | Change hadoop ssl server configuration | Not available. | | hadoop-ssl-client | Change hadoop ssl client configuration | Not available. | | hbase | Amazon EMR-curated settings for Apache HBase. | Custom EMR specific property. Sets emrfs-site and hbase-site configs. See those for their associated restarts. | | hbase-env | Change values in HBase's environment. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbase.properties file. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | Not available. | | hbase-site | Change values in HBase's hbase-site.xml file. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. Additionally restarts Phoenix QueryServer. | | hdfs-encryption-zones | Configure HDFS encryption zones. | Should not be reconfigured. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Additionally restarts Hadoop Httpfs. | | hcatalog-env | Change values in HCatalog's environment. | Restarts Hive HCatalog Server. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | Restarts Hive HCatalog Server. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | Restarts Hive HCatalog Server. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | Restarts Hive WebHCat Server. | | hcatalog-webhcat-log4j2 | Change values in HCatalog WebHCat's log4j2.properties. | Restarts Hive WebHCat Server. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | Restarts Hive WebHCat Server. | | hive-beeline-log4j2 | Change values in Hive's beeline-log4j2.properties file. | Not available. | | hive-parquet-logging | Change values in Hive's parquet-logging.properties file. | Not available. | | hive-env | Change values in the Hive environment. | Restarts HiveServer2 and HiveMetastore. Runs Hive schemaTool CLI commands to verify hive-metastore. | | hive-exec-log4j2 | Change values in Hive's hive-exec-log4j2.properties file. | Restarts HiveServer2 and HiveMetastore. | | hive-llap-daemon-log4j2 | Change values in Hive's llap-daemon-log4j2.properties file. | Not available. | | hive-log4j2 | Change values in Hive's hive-log4j2.properties file. | Not available. | | hive-site | Change values in Hive's hive-site.xml file | Restarts HiveServer2 and HiveMetastore. Runs Hive schemaTool CLI commands to verify hive-metastore. Also restarts Oozie and Zeppelin. | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | Not available. | | hue-ini | Change values in Hue's ini file | Restarts Hue. Also activates Hue config override CLI commands to pick up new configurations. | | httpfs-env | Change values in the HTTPFS environment. | Restarts Hadoop Httpfs service. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | Restarts Hadoop Httpfs service. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | Not available. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | Restarts Hadoop-KMS service. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | Not available. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | Restarts Hadoop-KMS and Ranger-KMS service. | | hudi-env | Change values in the Hudi environment. | Not available. | | jupyter-notebook-conf | Change values in Jupyter Notebook's jupyter\$1notebook\$1config.py file. | Not available. | | jupyter-hub-conf | Change values in JupyterHubs's jupyterhub\$1config.py file. | Not available. | | jupyter-s3-conf | Configure Jupyter Notebook S3 persistence. | Not available. | | jupyter-sparkmagic-conf | Change values in Sparkmagic's config.json file. | Not available. | | livy-conf | Change values in Livy's livy.conf file. | Restarts Livy Server. | | livy-env | Change values in the Livy environment. | Restarts Livy Server. | | livy-log4j | Change Livy log4j.properties settings. | Restarts Livy Server. | | mapred-env | Change values in the MapReduce application's environment. | Restarts Hadoop MapReduce-HistoryServer. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | Restarts Hadoop MapReduce-HistoryServer. | | oozie-env | Change values in Oozie's environment. | Restarts Oozie. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | Restarts Oozie. | | oozie-site | Change values in Oozie's oozie-site.xml file. | Restarts Oozie. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | Not available. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | Not available. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | Restarts Phoenix-QueryServer. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | Not available. | | pig-env | Change values in the Pig environment. | Not available. | | pig-properties | Change values in Pig's pig.properties file. | Restarts Oozie. | | pig-log4j | Change values in Pig's log4j.properties file. | Not available. | | presto-log | Change values in Presto's log.properties file. | Restarts Presto-Server. | | presto-config | Change values in Presto's config.properties file. | Restarts Presto-Server. | | presto-password-authenticator | Change values in Presto's password-authenticator.properties file. | Not available. | | presto-env | Change values in Presto's presto-env.sh file. | Restarts Presto-Server. | | presto-node | Change values in Presto's node.properties file. | Not available. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | Not available. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | Not available. | | presto-connector-hive | Change values in Presto's hive.properties file. | Restarts Presto-Server. | | presto-connector-jmx | Change values in Presto's jmx.properties file. | Not available. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | Not available. | | presto-connector-localfile | Change values in Presto's localfile.properties file. | Not available. | | presto-connector-memory | Change values in Presto's memory.properties file. | Not available. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | Not available. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | Not available. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | Not available. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | Not available. | | presto-connector-redis | Change values in Presto's redis.properties file. | Not available. | | presto-connector-redshift | Change values in Presto's redshift.properties file. | Not available. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | Not available. | | presto-connector-tpcds | Change values in Presto's tpcds.properties file. | Not available. | | ranger-kms-dbks-site | Change values in dbks-site.xml file of Ranger KMS. | Restarts Ranger KMS Server. | | ranger-kms-site | Change values in ranger-kms-site.xml file of Ranger KMS. | Restarts Ranger KMS Server. | | ranger-kms-env | Change values in the Ranger KMS environment. | Restarts Ranger KMS Server. | | ranger-kms-log4j | Change values in kms-log4j.properties file of Ranger KMS. | Not available. | | ranger-kms-db-ca | Change values for CA file on S3 for MySQL SSL connection with Ranger KMS. | Not available. | | recordserver-env | Change values in the EMR RecordServer environment. | Restarts EMR record server. | | recordserver-conf | Change values in EMR RecordServer's erver.properties file. | Restarts EMR record server. | | recordserver-log4j | Change values in EMR RecordServer's log4j.properties file. | Restarts EMR record server. | | spark | Amazon EMR-curated settings for Apache Spark. | This property modifies spark-defaults. See actions there. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | Restarts Spark history server and Spark thrift server. | | spark-env | Change values in the Spark environment. | Restarts Spark history server and Spark thrift server. | | spark-hive-site | Change values in Spark's hive-site.xml file | Not available. | | spark-log4j | Change values in Spark's log4j.properties file. | Restarts Spark history server and Spark thrift server. | | spark-metrics | Change values in Spark's metrics.properties file. | Restarts Spark history server and Spark thrift server. | | sqoop-env | Change values in Sqoop's environment. | Not available. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | Not available. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | Not available. | | tez-site | Change values in Tez's tez-site.xml file. | Restarts Oozie. | | yarn-env | Change values in the YARN environment. | Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts MapReduce-HistoryServer. | | yarn-site | Change values in YARN's yarn-site.xml file. | Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts Livy Server and MapReduce-HistoryServer. | | zeppelin-env | Change values in the Zeppelin environment. | Restarts Zeppelin. | | zeppelin-site | Change configuration settings in zeppelin-site.xml. | Restarts Zeppelin. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | Restarts Zookeeper server. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | Restarts Zookeeper server. | # Amazon EMR release 5.33.1 ## 5.33.1 application versions This release includes the following applications: [https://flink.apache.org/](https://flink.apache.org/), [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [https://hudi.apache.org](https://hudi.apache.org), [http://gethue.com/](http://gethue.com/), [https://jupyter-enterprise-gateway.readthedocs.io/en/latest/](https://jupyter-enterprise-gateway.readthedocs.io/en/latest/), [https://jupyterhub.readthedocs.io/en/latest/#](https://jupyterhub.readthedocs.io/en/latest/#), [https://livy.incubator.apache.org/](https://livy.incubator.apache.org/), [https://mxnet.incubator.apache.org/](https://mxnet.incubator.apache.org/), [http://mahout.apache.org/](http://mahout.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://www.tensorflow.org/](https://www.tensorflow.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-5.33.1 | emr-5.33.0 | emr-5.32.1 | emr-5.32.0 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.11.970 | 1.11.970 | 1.11.890 | 1.11.890 | | Python | 2.7, 3.7 | 2.7, 3.7 | 2.7, 3.7 | 2.7, 3.7 | | Scala | 2.11.12 | 2.11.12 | 2.11.12 | 2.11.12 | | AmazonCloudWatchAgent | - | - | - | - | | Delta | - | - | - | - | | Flink | 1.12.1 | 1.12.1 | 1.11.2 | 1.11.2 | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 1.4.13 | 1.4.13 | 1.4.13 | 1.4.13 | | HCatalog | 2.3.7-amzn-4 | 2.3.7-amzn-4 | 2.3.7-amzn-3 | 2.3.7-amzn-3 | | Hadoop | 2.10.1-amzn-1.1 | 2.10.1-amzn-1 | 2.10.1-amzn-0.1 | 2.10.1-amzn-0 | | Hive | 2.3.7-amzn-4 | 2.3.7-amzn-4 | 2.3.7-amzn-3 | 2.3.7-amzn-3 | | Hudi | 0.7.0-amzn-1 | 0.7.0-amzn-1 | 0.6.0-amzn-0 | 0.6.0-amzn-0 | | Hue | 4.9.0 | 4.9.0 | 4.8.0 | 4.8.0 | | Iceberg | - | - | - | - | | JupyterEnterpriseGateway | 2.1.0 | 2.1.0 | 2.1.0 | 2.1.0 | | JupyterHub | 1.2.2 | 1.2.2 | 1.1.0 | 1.1.0 | | Livy | 0.7.0-incubating | 0.7.0-incubating | 0.7.0-incubating | 0.7.0-incubating | | MXNet | 1.7.0 | 1.7.0 | 1.7.0 | 1.7.0 | | Mahout | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | | Oozie | 5.2.0 | 5.2.0 | 5.2.0 | 5.2.0 | | Phoenix | 4.14.3-HBase-1.4 | 4.14.3-HBase-1.4 | 4.14.3-HBase-1.4 | 4.14.3-HBase-1.4 | | Pig | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | | Presto | 0.245.1-amzn-0 | 0.245.1-amzn-0 | 0.240.1-amzn-0 | 0.240.1-amzn-0 | | Spark | 2.4.7-amzn-1.1 | 2.4.7-amzn-1 | 2.4.7-amzn-0.1 | 2.4.7-amzn-0 | | Sqoop | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | | TensorFlow | 2.4.1 | 2.4.1 | 2.3.1 | 2.3.1 | | Tez | 0.9.2 | 0.9.2 | 0.9.2 | 0.9.2 | | Trino (PrestoSQL) | - | - | - | - | | Zeppelin | 0.9.0 | 0.9.0 | 0.8.2 | 0.8.2 | | ZooKeeper | 3.4.14 | 3.4.14 | 3.4.14 | 3.4.14 | ## 5.33.1 release notes The following release notes include information for Amazon EMR release 5.33.0/5.33.1. Changes are relative to 5.32.0. Initial release date: April 19, 2021 Last updated date: August 9, 2021 **Upgrades** + Upgraded Amazon Glue connector to version 1.15.0 + Upgraded AWS SDK for Java to version 1.11.970 + Upgraded EMRFS to version 2.46.0 + Upgraded EMR Goodies to version 2.14.0 + Upgraded EMR Record Server to version 1.9.0 + Upgraded EMR S3 Dist CP to version 2.18.0 + Upgraded EMR Secret Agent to version 1.8.0 + Upgraded Flink to version 1.12.1 + Upgraded Hadoop to version 2.10.1-amzn-1 + Upgraded Hive to version 2.3.7-amzn-4 + Upgraded Hudi to version 0.7.0 + Upgraded Hue to version 4.9.0 + Upgraded OpenCV to version 4.5.0 + Upgraded Presto to version 0.245.1-amzn-0 + Upgraded R to version 4.0.2 + Upgraded Spark to version 2.4.7-amzn-1 + Upgraded TensorFlow to version 2.4.1 + Upgraded Zeppelin to version 0.9.0 **Changes, enhancements, and resolved issues** + This is a release to fix issues with Amazon EMR Scaling when it fails to scale up/scale down a cluster successfully or causes application failures. + Fixed an issue where scaling requests failed for a large, highly utilized cluster when Amazon EMR on-cluster daemons were running health checking activities, such as gathering YARN node state and HDFS node state. This was happening because on-cluster daemons were not able to communicate the health status data of a node to internal Amazon EMR components. + Improved EMR on-cluster daemons to correctly track the node states when IP addresses are reused to improve reliability during scaling operations. + [SPARK-29683](https://issues.apache.org/jira/browse/SPARK-29683). Fixed an issue where job failures occurred during cluster scale-down as Spark was assuming all available nodes were deny-listed. + [YARN-9011](https://issues.apache.org/jira/browse/YARN-9011). Fixed an issue where job failures occurred due to a race condition in YARN decommissioning when cluster tried to scale up or down. + Fixed issue with step or job failures during cluster scaling by ensuring that the node states are always consistent between the Amazon EMR on-cluster daemons and YARN/HDFS. + Fixed an issue where cluster operations such as scale down and step submission failed for Amazon EMR clusters enabled with Kerberos authentication. This was because the Amazon EMR on-cluster daemon did not renew the Kerberos ticket, which is required to securely communicate with HDFS/YARN running on the primary node. + Newer Amazon EMR releases fix the issue with a lower "Max open files" limit on older AL2 in Amazon EMR. Amazon EMR releases 5.30.1, 5.30.2, 5.31.1, 5.32.1, 6.0.1, 6.1.1, 6.2.1, 5.33.0, 6.3.0 and later now include a permanent fix with a higher "Max open files" setting. + **Configuring a cluster to fix Apache YARN Timeline Server version 1 and 1.5 performance issues** Apache YARN Timeline Server version 1 and 1.5 can cause performance issues with very active, large EMR clusters, particularly with `yarn.resourcemanager.system-metrics-publisher.enabled=true`, which is the default setting in Amazon EMR. An open source YARN Timeline Server v2 solves the performance issue related to YARN Timeline Server scalability. Other workarounds for this issue include: + Configuring yarn.resourcemanager.system-metrics-publisher.enabled=false in yarn-site.xml. + Enabling the fix for this issue when creating a cluster, as described below. The following Amazon EMR releases contain a fix for this YARN Timeline Server performance issue. EMR 5.30.2, 5.31.1, 5.32.1, 5.33.1, 5.34.x, 6.0.1, 6.1.1, 6.2.1, 6.3.1, 6.4.x To enable the fix on any of the above specified Amazon EMR releases, set these properties to `true` in a configurations JSON file that is passed in using the [`aws emr create-cluster` command parameter](https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-configure-apps-create-cluster.html): `--configurations file://./configurations.json`. Or enable the fix using the [reconfiguration console UI](https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-configure-apps-running-cluster.html). Example of the configurations.json file contents: ``` [ { "Classification": "yarn-site", "Properties": { "yarn.resourcemanager.system-metrics-publisher.timeline-server-v1.enable-batch": "true", "yarn.resourcemanager.system-metrics-publisher.enabled": "true" }, "Configurations": [] } ] ``` + Spark runtime is now faster when fetching partition locations from Hive Metastore for Spark insert queries. + Upgraded component versions. For a list of component versions, see [About Amazon EMR Releases](https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-release-components.html) in this guide. + Installed the AWS Java SDK Bundle on each new cluster. This is a single jar containing all service SDKs and their dependencies, instead of individual component jars. For more information, see [Java SDK Bundled Dependency](https://aws.amazon.com/blogs/developer/java-sdk-bundle/). + Fixed Managed Scaling issues in earlier Amazon EMR releases and made improvements so application failure rates are significantly reduced. + HTTPS is now enabled by default for Amazon Linux repositories. If you are using an Amazon S3 VPCE policy to restrict access to specific buckets, you must add the new Amazon Linux bucket ARN `arn:aws:s3:::amazonlinux-2-repos-$region/*` to your policy (replace `$region` with the region where the endpoint is). For more information, see this topic in the AWS discussion forums. [Announcement: Amazon Linux 2 now supports the ability to use HTTPS while connecting to package repositories ](https://forums.aws.amazon.com/ann.jspa?annID=8528). **New features** + Amazon EMR supports Amazon S3 Access Points, a feature of Amazon S3 that allows you to easily manage access for shared data lakes. Using your Amazon S3 Access Point alias, you can simplify your data access at scale on Amazon EMR. You can use Amazon S3 Access Points with all versions of Amazon EMR at no additional cost in all AWS regions where Amazon EMR is available. To learn more about Amazon S3 Access Points and Access Point aliases, see [Using a bucket-style alias for your access point](https://docs.aws.amazon.com/AmazonS3/latest/userguide/access-points-alias.html) in the *Amazon S3 User Guide*. + Amazon EMR-5.33 supports new Amazon EC2 instance types: c5a, c5ad, c6gn, c6gd, m6gd, d3, d3en, m5zn, r5b, r6gd. See [Supported Instance Types](https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-supported-instance-types.html). **Known issues** + **Lower "Max open files" limit on older AL2 [fixed in newer releases].** Amazon EMR releases: emr-5.30.x, emr-5.31.0, emr-5.32.0, emr-6.0.0, emr-6.1.0, and emr-6.2.0 are based on older versions ofAmazon Linux 2 (AL2), which have a lower ulimit setting for "Max open files" when Amazon EMR clusters are created with the default AMI. Amazon EMR releases 5.30.1, 5.30.2, 5.31.1, 5.32.1, 6.0.1, 6.1.1, 6.2.1, 5.33.0, 6.3.0 and later include a permanent fix with a higher "Max open files" setting. Releases with the lower open file limit causes a "Too many open files" error when submitting Spark job. In the impacted releases, the Amazon EMR default AMI has a default ulimit setting of 4096 for "Max open files," which is lower than the 65536 file limit in the latestAmazon Linux 2 AMI. The lower ulimit setting for "Max open files" causes Spark job failure when the Spark driver and executor try to open more than 4096 files. To fix the issue, Amazon EMR has a bootstrap action (BA) script that adjusts the ulimit setting at cluster creation. If you are using an older Amazon EMR version that doesn't have the permanent fix for this issue, the following workaround lets you to explicitly set the instance-controller ulimit to a maximum of 65536 files. **Explicitly set a ulimit from the command line** 1. Edit `/etc/systemd/system/instance-controller.service` to add the following parameters to Service section. `LimitNOFILE=65536` `LimitNPROC=65536` 1. Restart InstanceController `$ sudo systemctl daemon-reload` `$ sudo systemctl restart instance-controller` **Set a ulimit using bootstrap action (BA)** You can also use a bootstrap action (BA) script to configure the instance-controller ulimit to 65536 files at cluster creation. ``` #!/bin/bash for user in hadoop spark hive; do sudo tee /etc/security/limits.d/$user.conf << EOF $user - nofile 65536 $user - nproc 65536 EOF done for proc in instancecontroller logpusher; do sudo mkdir -p /etc/systemd/system/$proc.service.d/ sudo tee /etc/systemd/system/$proc.service.d/override.conf << EOF [Service] LimitNOFILE=65536 LimitNPROC=65536 EOF pid=$(pgrep -f aws157.$proc.Main) sudo prlimit --pid $pid --nofile=65535:65535 --nproc=65535:65535 done sudo systemctl daemon-reload ``` + For Amazon EMR 6.3.0 and 6.2.0 private subnet clusters, you cannot access the Ganglia web UI. You will get an "access denied (403)" error. Other web UIs, such as Spark, Hue, JupyterHub, Zeppelin, Livy, and Tez are working normally. Ganglia web UI access on public subnet clusters are also working normally. To resolve this issue, restart httpd service on the primary node with `sudo systemctl restart httpd`. This issue is fixed in Amazon EMR 6.4.0. + **Important** EMR clusters that run Amazon Linux or Amazon Linux 2 Amazon Machine Images (AMIs) use default Amazon Linux behavior, and do not automatically download and install important and critical kernel updates that require a reboot. This is the same behavior as other Amazon EC2 instances that run the default Amazon Linux AMI. If new Amazon Linux software updates that require a reboot (such as kernel, NVIDIA, and CUDA updates) become available after an Amazon EMR release becomes available, EMR cluster instances that run the default AMI do not automatically download and install those updates. To get kernel updates, you can [customize your Amazon EMR AMI](https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-custom-ami.html) to [use the latest Amazon Linux AMI](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/finding-an-ami.html). + Console support to create a security configuration that specifies the AWS Ranger integration option is currently not supported in the GovCloud Region. Security configuration can be done using the CLI. See [Create the EMR Security Configuration](https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-ranger-security-config.html) in the *Amazon EMR Management Guide*. + Scoped managed policies: To align with AWS best practices, Amazon EMR has introduced v2 EMR-scoped default managed policies as replacements for policies that will be deprecated. See [Amazon EMR Managed Policies](https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-managed-iam-policies.html). + When you use Spark with Hive partition location formatting to read data in Amazon S3, and you run Spark on Amazon EMR releases 5.30.0 to 5.36.0, and 6.2.0 to 6.9.0, you might encounter an issue that prevents your cluster from reading data correctly. This can happen if your partitions have all of the following characteristics: + Two or more partitions are scanned from the same table. + At least one partition directory path is a prefix of at least one other partition directory path, for example, `s3://bucket/table/p=a` is a prefix of `s3://bucket/table/p=a b`. + The first character that follows the prefix in the other partition directory has a UTF-8 value that’s less than than the `/` character (U\$1002F). For example, the space character (U\$10020) that occurs between a and b in `s3://bucket/table/p=a b` falls into this category. Note that there are 14 other non-control characters: `!"#$%&‘()*+,-`. For more information, see [UTF-8 encoding table and Unicode characters](https://www.utf8-chartable.de/). As a workaround to this issue, set the `spark.sql.sources.fastS3PartitionDiscovery.enabled` configuration to `false` in the `spark-defaults` classification. ## 5.33.1 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | aws-sagemaker-spark-sdk | 1.4.1 | Amazon SageMaker Spark SDK | | emr-ddb | 4.16.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 2.14.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.5.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-notebook-env | 1.2.0 | Conda env for emr notebook which includes jupyter enterprise gateway | | emr-s3-dist-cp | 2.18.0 | Distributed copy application optimized for Amazon S3. | | emr-s3-select | 1.6.0 | EMR S3Select Connector | | emrfs | 2.46.0 | Amazon S3 connector for Hadoop ecosystem applications. | | flink-client | 1.12.1 | Apache Flink command line client scripts and applications. | | flink-jobmanager-config | 1.12.1 | Managing resources on EMR nodes for Apache Flink JobManager. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 2.10.1-amzn-1.1 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 2.10.1-amzn-1.1 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 2.10.1-amzn-1.1 | HDFS command-line client and library | | hadoop-hdfs-namenode | 2.10.1-amzn-1.1 | HDFS service for tracking file names and block locations. | | hadoop-hdfs-journalnode | 2.10.1-amzn-1.1 | HDFS service for managing the Hadoop filesystem journal on HA clusters. | | hadoop-httpfs-server | 2.10.1-amzn-1.1 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 2.10.1-amzn-1.1 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 2.10.1-amzn-1.1 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 2.10.1-amzn-1.1 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 2.10.1-amzn-1.1 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 2.10.1-amzn-1.1 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 1.4.13 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 1.4.13 | Service for serving one or more HBase regions. | | hbase-client | 1.4.13 | HBase command-line client. | | hbase-rest-server | 1.4.13 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 1.4.13 | Service providing a Thrift endpoint to HBase. | | hcatalog-client | 2.3.7-amzn-4 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 2.3.7-amzn-4 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 2.3.7-amzn-4 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 2.3.7-amzn-4 | Hive command line client. | | hive-hbase | 2.3.7-amzn-4 | Hive-hbase client. | | hive-metastore-server | 2.3.7-amzn-4 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server2 | 2.3.7-amzn-4 | Service for accepting Hive queries as web requests. | | hudi | 0.7.0-amzn-1 | Incremental processing framework to power data pipline at low latency and high efficiency. | | hudi-spark | 0.7.0-amzn-1 | Bundle library for running Spark with Hudi. | | hudi-presto | 0.7.0-amzn-1 | Bundle library for running Presto with Hudi. | | hue-server | 4.9.0 | Web application for analyzing data using Hadoop ecosystem applications | | jupyterhub | 1.2.2 | Multi-user server for Jupyter notebooks | | livy-server | 0.7.0-incubating | REST interface for interacting with Apache Spark | | nginx | 1.12.1 | nginx [engine x] is an HTTP and reverse proxy server | | mahout-client | 0.13.0 | Library for machine learning. | | mxnet | 1.7.0 | A flexible, scalable, and efficient library for deep learning. | | mariadb-server | 5.5.68\$1 | MySQL database server. | | nvidia-cuda | 10.1.243 | Nvidia drivers and Cuda toolkit | | oozie-client | 5.2.0 | Oozie command-line client. | | oozie-server | 5.2.0 | Service for accepting Oozie workflow requests. | | opencv | 4.5.0 | Open Source Computer Vision Library. | | phoenix-library | 4.14.3-HBase-1.4 | The phoenix libraries for server and client | | phoenix-query-server | 4.14.3-HBase-1.4 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.245.1-amzn-0 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.245.1-amzn-0 | Service for executing pieces of a query. | | presto-client | 0.245.1-amzn-0 | Presto command-line client which is installed on an HA cluster's stand-by masters where Presto server is not started. | | pig-client | 0.17.0 | Pig command-line client. | | r | 4.0.2 | The R Project for Statistical Computing | | ranger-kms-server | 1.2.0 | Apache Ranger Key Management System | | spark-client | 2.4.7-amzn-1.1 | Spark command-line clients. | | spark-history-server | 2.4.7-amzn-1.1 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 2.4.7-amzn-1.1 | In-memory execution engine for YARN. | | spark-yarn-slave | 2.4.7-amzn-1.1 | Apache Spark libraries needed by YARN slaves. | | sqoop-client | 1.4.7 | Apache Sqoop command-line client. | | tensorflow | 2.4.1 | TensorFlow open source software library for high performance numerical computation. | | tez-on-yarn | 0.9.2 | The tez YARN application and libraries. | | webserver | 2.4.25\$1 | Apache HTTP server. | | zeppelin-server | 0.9.0 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.4.14 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.4.14 | ZooKeeper command line client. | ## 5.33.1 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). Reconfiguration actions occur when you specify a configuration for instance groups in a running cluster. Amazon EMR only initiates reconfiguration actions for the classifications that you modify. For more information, see [Reconfigure an instance group in a running cluster](emr-configure-apps-running-cluster.md). **emr-5.33.1 classifications** | Classifications | Description | Reconfiguration Actions | | --- | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | Restarts the ResourceManager service. | | container-executor | Change values in Hadoop YARN's container-executor.cfg file. | Not available. | | container-log4j | Change values in Hadoop YARN's container-log4j.properties file. | Not available. | | core-site | Change values in Hadoop's core-site.xml file. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts Hadoop KMS, Ranger KMS, HiveServer2, Hive MetaStore, Hadoop Httpfs, and MapReduce-HistoryServer. | | docker-conf | Change docker related settings. | Not available. | | emrfs-site | Change EMRFS settings. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts HBaseRegionserver, HBaseMaster, HBaseThrift, HBaseRest, HiveServer2, Hive MetaStore, Hadoop Httpfs, and MapReduce-HistoryServer. | | flink-conf | Change flink-conf.yaml settings. | Restarts Flink history server. | | flink-log4j | Change Flink log4j.properties settings. | Restarts Flink history server. | | flink-log4j-session | Change Flink log4j-session.properties settings for Kubernetes/Yarn session. | Not available. | | flink-log4j-cli | Change Flink log4j-cli.properties settings. | Restarts Flink history server. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts PhoenixQueryserver, HiveServer2, Hive MetaStore, and MapReduce-HistoryServer. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts Hadoop KMS, Hadoop Httpfs, and MapReduce-HistoryServer. | | hadoop-ssl-server | Change hadoop ssl server configuration | Not available. | | hadoop-ssl-client | Change hadoop ssl client configuration | Not available. | | hbase | Amazon EMR-curated settings for Apache HBase. | Custom EMR specific property. Sets emrfs-site and hbase-site configs. See those for their associated restarts. | | hbase-env | Change values in HBase's environment. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbase.properties file. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | Not available. | | hbase-site | Change values in HBase's hbase-site.xml file. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. Additionally restarts Phoenix QueryServer. | | hdfs-encryption-zones | Configure HDFS encryption zones. | Should not be reconfigured. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Additionally restarts Hadoop Httpfs. | | hcatalog-env | Change values in HCatalog's environment. | Restarts Hive HCatalog Server. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | Restarts Hive HCatalog Server. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | Restarts Hive HCatalog Server. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | Restarts Hive WebHCat Server. | | hcatalog-webhcat-log4j2 | Change values in HCatalog WebHCat's log4j2.properties. | Restarts Hive WebHCat Server. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | Restarts Hive WebHCat Server. | | hive-beeline-log4j2 | Change values in Hive's beeline-log4j2.properties file. | Not available. | | hive-parquet-logging | Change values in Hive's parquet-logging.properties file. | Not available. | | hive-env | Change values in the Hive environment. | Restarts HiveServer2 and HiveMetastore. Runs Hive schemaTool CLI commands to verify hive-metastore. | | hive-exec-log4j2 | Change values in Hive's hive-exec-log4j2.properties file. | Restarts HiveServer2 and HiveMetastore. | | hive-llap-daemon-log4j2 | Change values in Hive's llap-daemon-log4j2.properties file. | Not available. | | hive-log4j2 | Change values in Hive's hive-log4j2.properties file. | Not available. | | hive-site | Change values in Hive's hive-site.xml file | Restarts HiveServer2 and HiveMetastore. Runs Hive schemaTool CLI commands to verify hive-metastore. Also restarts Oozie and Zeppelin. | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | Not available. | | hue-ini | Change values in Hue's ini file | Restarts Hue. Also activates Hue config override CLI commands to pick up new configurations. | | httpfs-env | Change values in the HTTPFS environment. | Restarts Hadoop Httpfs service. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | Restarts Hadoop Httpfs service. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | Not available. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | Restarts Hadoop-KMS service. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | Not available. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | Restarts Hadoop-KMS and Ranger-KMS service. | | hudi-env | Change values in the Hudi environment. | Not available. | | jupyter-notebook-conf | Change values in Jupyter Notebook's jupyter\$1notebook\$1config.py file. | Not available. | | jupyter-hub-conf | Change values in JupyterHubs's jupyterhub\$1config.py file. | Not available. | | jupyter-s3-conf | Configure Jupyter Notebook S3 persistence. | Not available. | | jupyter-sparkmagic-conf | Change values in Sparkmagic's config.json file. | Not available. | | livy-conf | Change values in Livy's livy.conf file. | Restarts Livy Server. | | livy-env | Change values in the Livy environment. | Restarts Livy Server. | | livy-log4j | Change Livy log4j.properties settings. | Restarts Livy Server. | | mapred-env | Change values in the MapReduce application's environment. | Restarts Hadoop MapReduce-HistoryServer. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | Restarts Hadoop MapReduce-HistoryServer. | | oozie-env | Change values in Oozie's environment. | Restarts Oozie. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | Restarts Oozie. | | oozie-site | Change values in Oozie's oozie-site.xml file. | Restarts Oozie. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | Not available. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | Not available. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | Restarts Phoenix-QueryServer. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | Not available. | | pig-env | Change values in the Pig environment. | Not available. | | pig-properties | Change values in Pig's pig.properties file. | Restarts Oozie. | | pig-log4j | Change values in Pig's log4j.properties file. | Not available. | | presto-log | Change values in Presto's log.properties file. | Restarts Presto-Server. | | presto-config | Change values in Presto's config.properties file. | Restarts Presto-Server. | | presto-password-authenticator | Change values in Presto's password-authenticator.properties file. | Not available. | | presto-env | Change values in Presto's presto-env.sh file. | Restarts Presto-Server. | | presto-node | Change values in Presto's node.properties file. | Not available. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | Not available. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | Not available. | | presto-connector-hive | Change values in Presto's hive.properties file. | Restarts Presto-Server. | | presto-connector-jmx | Change values in Presto's jmx.properties file. | Not available. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | Not available. | | presto-connector-localfile | Change values in Presto's localfile.properties file. | Not available. | | presto-connector-memory | Change values in Presto's memory.properties file. | Not available. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | Not available. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | Not available. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | Not available. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | Not available. | | presto-connector-redis | Change values in Presto's redis.properties file. | Not available. | | presto-connector-redshift | Change values in Presto's redshift.properties file. | Not available. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | Not available. | | presto-connector-tpcds | Change values in Presto's tpcds.properties file. | Not available. | | ranger-kms-dbks-site | Change values in dbks-site.xml file of Ranger KMS. | Restarts Ranger KMS Server. | | ranger-kms-site | Change values in ranger-kms-site.xml file of Ranger KMS. | Restarts Ranger KMS Server. | | ranger-kms-env | Change values in the Ranger KMS environment. | Restarts Ranger KMS Server. | | ranger-kms-log4j | Change values in kms-log4j.properties file of Ranger KMS. | Not available. | | ranger-kms-db-ca | Change values for CA file on S3 for MySQL SSL connection with Ranger KMS. | Not available. | | recordserver-env | Change values in the EMR RecordServer environment. | Restarts EMR record server. | | recordserver-conf | Change values in EMR RecordServer's erver.properties file. | Restarts EMR record server. | | recordserver-log4j | Change values in EMR RecordServer's log4j.properties file. | Restarts EMR record server. | | spark | Amazon EMR-curated settings for Apache Spark. | This property modifies spark-defaults. See actions there. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | Restarts Spark history server and Spark thrift server. | | spark-env | Change values in the Spark environment. | Restarts Spark history server and Spark thrift server. | | spark-hive-site | Change values in Spark's hive-site.xml file | Not available. | | spark-log4j | Change values in Spark's log4j.properties file. | Restarts Spark history server and Spark thrift server. | | spark-metrics | Change values in Spark's metrics.properties file. | Restarts Spark history server and Spark thrift server. | | sqoop-env | Change values in Sqoop's environment. | Not available. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | Not available. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | Not available. | | tez-site | Change values in Tez's tez-site.xml file. | Restarts Oozie. | | yarn-env | Change values in the YARN environment. | Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts MapReduce-HistoryServer. | | yarn-site | Change values in YARN's yarn-site.xml file. | Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts Livy Server and MapReduce-HistoryServer. | | zeppelin-env | Change values in the Zeppelin environment. | Restarts Zeppelin. | | zeppelin-site | Change configuration settings in zeppelin-site.xml. | Restarts Zeppelin. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | Restarts Zookeeper server. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | Restarts Zookeeper server. | # Amazon EMR release 5.33.0 ## 5.33.0 application versions This release includes the following applications: [https://flink.apache.org/](https://flink.apache.org/), [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [https://hudi.apache.org](https://hudi.apache.org), [http://gethue.com/](http://gethue.com/), [https://jupyter-enterprise-gateway.readthedocs.io/en/latest/](https://jupyter-enterprise-gateway.readthedocs.io/en/latest/), [https://jupyterhub.readthedocs.io/en/latest/#](https://jupyterhub.readthedocs.io/en/latest/#), [https://livy.incubator.apache.org/](https://livy.incubator.apache.org/), [https://mxnet.incubator.apache.org/](https://mxnet.incubator.apache.org/), [http://mahout.apache.org/](http://mahout.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://www.tensorflow.org/](https://www.tensorflow.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-5.33.0 | emr-5.32.1 | emr-5.32.0 | emr-5.31.1 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.11.970 | 1.11.890 | 1.11.890 | 1.11.852 | | Python | 2.7, 3.7 | 2.7, 3.7 | 2.7, 3.7 | 2.7, 3.7 | | Scala | 2.11.12 | 2.11.12 | 2.11.12 | 2.11.12 | | AmazonCloudWatchAgent | - | - | - | - | | Delta | - | - | - | - | | Flink | 1.12.1 | 1.11.2 | 1.11.2 | 1.11.0 | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 1.4.13 | 1.4.13 | 1.4.13 | 1.4.13 | | HCatalog | 2.3.7-amzn-4 | 2.3.7-amzn-3 | 2.3.7-amzn-3 | 2.3.7-amzn-1 | | Hadoop | 2.10.1-amzn-1 | 2.10.1-amzn-0.1 | 2.10.1-amzn-0 | 2.10.0-amzn-0.1 | | Hive | 2.3.7-amzn-4 | 2.3.7-amzn-3 | 2.3.7-amzn-3 | 2.3.7-amzn-1 | | Hudi | 0.7.0-amzn-1 | 0.6.0-amzn-0 | 0.6.0-amzn-0 | 0.6.0-amzn-0 | | Hue | 4.9.0 | 4.8.0 | 4.8.0 | 4.7.1 | | Iceberg | - | - | - | - | | JupyterEnterpriseGateway | 2.1.0 | 2.1.0 | 2.1.0 | - | | JupyterHub | 1.2.2 | 1.1.0 | 1.1.0 | 1.1.0 | | Livy | 0.7.0-incubating | 0.7.0-incubating | 0.7.0-incubating | 0.7.0-incubating | | MXNet | 1.7.0 | 1.7.0 | 1.7.0 | 1.6.0 | | Mahout | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | | Oozie | 5.2.0 | 5.2.0 | 5.2.0 | 5.2.0 | | Phoenix | 4.14.3-HBase-1.4 | 4.14.3-HBase-1.4 | 4.14.3-HBase-1.4 | 4.14.3-HBase-1.4 | | Pig | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | | Presto | 0.245.1-amzn-0 | 0.240.1-amzn-0 | 0.240.1-amzn-0 | 0.238.3-amzn-0 | | Spark | 2.4.7-amzn-1 | 2.4.7-amzn-0.1 | 2.4.7-amzn-0 | 2.4.6-amzn-0.1 | | Sqoop | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | | TensorFlow | 2.4.1 | 2.3.1 | 2.3.1 | 2.1.0 | | Tez | 0.9.2 | 0.9.2 | 0.9.2 | 0.9.2 | | Trino (PrestoSQL) | - | - | - | - | | Zeppelin | 0.9.0 | 0.8.2 | 0.8.2 | 0.8.2 | | ZooKeeper | 3.4.14 | 3.4.14 | 3.4.14 | 3.4.14 | ## 5.33.0 release notes ## 5.33.0 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | aws-sagemaker-spark-sdk | 1.4.1 | Amazon SageMaker Spark SDK | | emr-ddb | 4.16.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 2.14.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.5.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-notebook-env | 1.2.0 | Conda env for emr notebook which includes jupyter enterprise gateway | | emr-s3-dist-cp | 2.18.0 | Distributed copy application optimized for Amazon S3. | | emr-s3-select | 1.6.0 | EMR S3Select Connector | | emrfs | 2.46.0 | Amazon S3 connector for Hadoop ecosystem applications. | | flink-client | 1.12.1 | Apache Flink command line client scripts and applications. | | flink-jobmanager-config | 1.12.1 | Managing resources on EMR nodes for Apache Flink JobManager. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 2.10.1-amzn-1 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 2.10.1-amzn-1 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 2.10.1-amzn-1 | HDFS command-line client and library | | hadoop-hdfs-namenode | 2.10.1-amzn-1 | HDFS service for tracking file names and block locations. | | hadoop-hdfs-journalnode | 2.10.1-amzn-1 | HDFS service for managing the Hadoop filesystem journal on HA clusters. | | hadoop-httpfs-server | 2.10.1-amzn-1 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 2.10.1-amzn-1 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 2.10.1-amzn-1 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 2.10.1-amzn-1 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 2.10.1-amzn-1 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 2.10.1-amzn-1 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 1.4.13 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 1.4.13 | Service for serving one or more HBase regions. | | hbase-client | 1.4.13 | HBase command-line client. | | hbase-rest-server | 1.4.13 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 1.4.13 | Service providing a Thrift endpoint to HBase. | | hcatalog-client | 2.3.7-amzn-4 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 2.3.7-amzn-4 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 2.3.7-amzn-4 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 2.3.7-amzn-4 | Hive command line client. | | hive-hbase | 2.3.7-amzn-4 | Hive-hbase client. | | hive-metastore-server | 2.3.7-amzn-4 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server2 | 2.3.7-amzn-4 | Service for accepting Hive queries as web requests. | | hudi | 0.7.0-amzn-1 | Incremental processing framework to power data pipline at low latency and high efficiency. | | hudi-spark | 0.7.0-amzn-1 | Bundle library for running Spark with Hudi. | | hudi-presto | 0.7.0-amzn-1 | Bundle library for running Presto with Hudi. | | hue-server | 4.9.0 | Web application for analyzing data using Hadoop ecosystem applications | | jupyterhub | 1.2.2 | Multi-user server for Jupyter notebooks | | livy-server | 0.7.0-incubating | REST interface for interacting with Apache Spark | | nginx | 1.12.1 | nginx [engine x] is an HTTP and reverse proxy server | | mahout-client | 0.13.0 | Library for machine learning. | | mxnet | 1.7.0 | A flexible, scalable, and efficient library for deep learning. | | mariadb-server | 5.5.68 | MySQL database server. | | nvidia-cuda | 10.1.243 | Nvidia drivers and Cuda toolkit | | oozie-client | 5.2.0 | Oozie command-line client. | | oozie-server | 5.2.0 | Service for accepting Oozie workflow requests. | | opencv | 4.5.0 | Open Source Computer Vision Library. | | phoenix-library | 4.14.3-HBase-1.4 | The phoenix libraries for server and client | | phoenix-query-server | 4.14.3-HBase-1.4 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.245.1-amzn-0 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.245.1-amzn-0 | Service for executing pieces of a query. | | presto-client | 0.245.1-amzn-0 | Presto command-line client which is installed on an HA cluster's stand-by masters where Presto server is not started. | | pig-client | 0.17.0 | Pig command-line client. | | r | 4.0.2 | The R Project for Statistical Computing | | ranger-kms-server | 1.2.0 | Apache Ranger Key Management System | | spark-client | 2.4.7-amzn-1 | Spark command-line clients. | | spark-history-server | 2.4.7-amzn-1 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 2.4.7-amzn-1 | In-memory execution engine for YARN. | | spark-yarn-slave | 2.4.7-amzn-1 | Apache Spark libraries needed by YARN slaves. | | sqoop-client | 1.4.7 | Apache Sqoop command-line client. | | tensorflow | 2.4.1 | TensorFlow open source software library for high performance numerical computation. | | tez-on-yarn | 0.9.2 | The tez YARN application and libraries. | | webserver | 2.4.25\$1 | Apache HTTP server. | | zeppelin-server | 0.9.0 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.4.14 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.4.14 | ZooKeeper command line client. | ## 5.33.0 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). Reconfiguration actions occur when you specify a configuration for instance groups in a running cluster. Amazon EMR only initiates reconfiguration actions for the classifications that you modify. For more information, see [Reconfigure an instance group in a running cluster](emr-configure-apps-running-cluster.md). **emr-5.33.0 classifications** | Classifications | Description | Reconfiguration Actions | | --- | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | Restarts the ResourceManager service. | | container-executor | Change values in Hadoop YARN's container-executor.cfg file. | Not available. | | container-log4j | Change values in Hadoop YARN's container-log4j.properties file. | Not available. | | core-site | Change values in Hadoop's core-site.xml file. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts Hadoop KMS, Ranger KMS, HiveServer2, Hive MetaStore, Hadoop Httpfs, and MapReduce-HistoryServer. | | docker-conf | Change docker related settings. | Not available. | | emrfs-site | Change EMRFS settings. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts HBaseRegionserver, HBaseMaster, HBaseThrift, HBaseRest, HiveServer2, Hive MetaStore, Hadoop Httpfs, and MapReduce-HistoryServer. | | flink-conf | Change flink-conf.yaml settings. | Restarts Flink history server. | | flink-log4j | Change Flink log4j.properties settings. | Restarts Flink history server. | | flink-log4j-session | Change Flink log4j-session.properties settings for Kubernetes/Yarn session. | Not available. | | flink-log4j-cli | Change Flink log4j-cli.properties settings. | Restarts Flink history server. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts PhoenixQueryserver, HiveServer2, Hive MetaStore, and MapReduce-HistoryServer. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts Hadoop KMS, Hadoop Httpfs, and MapReduce-HistoryServer. | | hadoop-ssl-server | Change hadoop ssl server configuration | Not available. | | hadoop-ssl-client | Change hadoop ssl client configuration | Not available. | | hbase | Amazon EMR-curated settings for Apache HBase. | Custom EMR specific property. Sets emrfs-site and hbase-site configs. See those for their associated restarts. | | hbase-env | Change values in HBase's environment. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbase.properties file. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | Not available. | | hbase-site | Change values in HBase's hbase-site.xml file. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. Additionally restarts Phoenix QueryServer. | | hdfs-encryption-zones | Configure HDFS encryption zones. | Should not be reconfigured. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Additionally restarts Hadoop Httpfs. | | hcatalog-env | Change values in HCatalog's environment. | Restarts Hive HCatalog Server. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | Restarts Hive HCatalog Server. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | Restarts Hive HCatalog Server. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | Restarts Hive WebHCat Server. | | hcatalog-webhcat-log4j2 | Change values in HCatalog WebHCat's log4j2.properties. | Restarts Hive WebHCat Server. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | Restarts Hive WebHCat Server. | | hive-beeline-log4j2 | Change values in Hive's beeline-log4j2.properties file. | Not available. | | hive-parquet-logging | Change values in Hive's parquet-logging.properties file. | Not available. | | hive-env | Change values in the Hive environment. | Restarts HiveServer2 and HiveMetastore. Runs Hive schemaTool CLI commands to verify hive-metastore. | | hive-exec-log4j2 | Change values in Hive's hive-exec-log4j2.properties file. | Restarts HiveServer2 and HiveMetastore. | | hive-llap-daemon-log4j2 | Change values in Hive's llap-daemon-log4j2.properties file. | Not available. | | hive-log4j2 | Change values in Hive's hive-log4j2.properties file. | Not available. | | hive-site | Change values in Hive's hive-site.xml file | Restarts HiveServer2 and HiveMetastore. Runs Hive schemaTool CLI commands to verify hive-metastore. Also restarts Oozie and Zeppelin. | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | Not available. | | hue-ini | Change values in Hue's ini file | Restarts Hue. Also activates Hue config override CLI commands to pick up new configurations. | | httpfs-env | Change values in the HTTPFS environment. | Restarts Hadoop Httpfs service. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | Restarts Hadoop Httpfs service. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | Not available. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | Restarts Hadoop-KMS service. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | Not available. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | Restarts Hadoop-KMS and Ranger-KMS service. | | hudi-env | Change values in the Hudi environment. | Not available. | | jupyter-notebook-conf | Change values in Jupyter Notebook's jupyter\$1notebook\$1config.py file. | Not available. | | jupyter-hub-conf | Change values in JupyterHubs's jupyterhub\$1config.py file. | Not available. | | jupyter-s3-conf | Configure Jupyter Notebook S3 persistence. | Not available. | | jupyter-sparkmagic-conf | Change values in Sparkmagic's config.json file. | Not available. | | livy-conf | Change values in Livy's livy.conf file. | Restarts Livy Server. | | livy-env | Change values in the Livy environment. | Restarts Livy Server. | | livy-log4j | Change Livy log4j.properties settings. | Restarts Livy Server. | | mapred-env | Change values in the MapReduce application's environment. | Restarts Hadoop MapReduce-HistoryServer. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | Restarts Hadoop MapReduce-HistoryServer. | | oozie-env | Change values in Oozie's environment. | Restarts Oozie. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | Restarts Oozie. | | oozie-site | Change values in Oozie's oozie-site.xml file. | Restarts Oozie. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | Not available. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | Not available. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | Restarts Phoenix-QueryServer. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | Not available. | | pig-env | Change values in the Pig environment. | Not available. | | pig-properties | Change values in Pig's pig.properties file. | Restarts Oozie. | | pig-log4j | Change values in Pig's log4j.properties file. | Not available. | | presto-log | Change values in Presto's log.properties file. | Restarts Presto-Server. | | presto-config | Change values in Presto's config.properties file. | Restarts Presto-Server. | | presto-password-authenticator | Change values in Presto's password-authenticator.properties file. | Not available. | | presto-env | Change values in Presto's presto-env.sh file. | Restarts Presto-Server. | | presto-node | Change values in Presto's node.properties file. | Not available. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | Not available. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | Not available. | | presto-connector-hive | Change values in Presto's hive.properties file. | Restarts Presto-Server. | | presto-connector-jmx | Change values in Presto's jmx.properties file. | Not available. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | Not available. | | presto-connector-localfile | Change values in Presto's localfile.properties file. | Not available. | | presto-connector-memory | Change values in Presto's memory.properties file. | Not available. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | Not available. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | Not available. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | Not available. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | Not available. | | presto-connector-redis | Change values in Presto's redis.properties file. | Not available. | | presto-connector-redshift | Change values in Presto's redshift.properties file. | Not available. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | Not available. | | presto-connector-tpcds | Change values in Presto's tpcds.properties file. | Not available. | | ranger-kms-dbks-site | Change values in dbks-site.xml file of Ranger KMS. | Restarts Ranger KMS Server. | | ranger-kms-site | Change values in ranger-kms-site.xml file of Ranger KMS. | Restarts Ranger KMS Server. | | ranger-kms-env | Change values in the Ranger KMS environment. | Restarts Ranger KMS Server. | | ranger-kms-log4j | Change values in kms-log4j.properties file of Ranger KMS. | Not available. | | ranger-kms-db-ca | Change values for CA file on S3 for MySQL SSL connection with Ranger KMS. | Not available. | | recordserver-env | Change values in the EMR RecordServer environment. | Restarts EMR record server. | | recordserver-conf | Change values in EMR RecordServer's erver.properties file. | Restarts EMR record server. | | recordserver-log4j | Change values in EMR RecordServer's log4j.properties file. | Restarts EMR record server. | | spark | Amazon EMR-curated settings for Apache Spark. | This property modifies spark-defaults. See actions there. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | Restarts Spark history server and Spark thrift server. | | spark-env | Change values in the Spark environment. | Restarts Spark history server and Spark thrift server. | | spark-hive-site | Change values in Spark's hive-site.xml file | Not available. | | spark-log4j | Change values in Spark's log4j.properties file. | Restarts Spark history server and Spark thrift server. | | spark-metrics | Change values in Spark's metrics.properties file. | Restarts Spark history server and Spark thrift server. | | sqoop-env | Change values in Sqoop's environment. | Not available. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | Not available. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | Not available. | | tez-site | Change values in Tez's tez-site.xml file. | Restarts Oozie. | | yarn-env | Change values in the YARN environment. | Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts MapReduce-HistoryServer. | | yarn-site | Change values in YARN's yarn-site.xml file. | Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts Livy Server and MapReduce-HistoryServer. | | zeppelin-env | Change values in the Zeppelin environment. | Restarts Zeppelin. | | zeppelin-site | Change configuration settings in zeppelin-site.xml. | Restarts Zeppelin. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | Restarts Zookeeper server. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | Restarts Zookeeper server. | # Amazon EMR release 5.32.1 ## 5.32.1 application versions This release includes the following applications: [https://flink.apache.org/](https://flink.apache.org/), [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [https://hudi.apache.org](https://hudi.apache.org), [http://gethue.com/](http://gethue.com/), [https://jupyter-enterprise-gateway.readthedocs.io/en/latest/](https://jupyter-enterprise-gateway.readthedocs.io/en/latest/), [https://jupyterhub.readthedocs.io/en/latest/#](https://jupyterhub.readthedocs.io/en/latest/#), [https://livy.incubator.apache.org/](https://livy.incubator.apache.org/), [https://mxnet.incubator.apache.org/](https://mxnet.incubator.apache.org/), [http://mahout.apache.org/](http://mahout.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://www.tensorflow.org/](https://www.tensorflow.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-5.32.1 | emr-5.32.0 | emr-5.31.1 | emr-5.31.0 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.11.890 | 1.11.890 | 1.11.852 | 1.11.852 | | Python | 2.7, 3.7 | 2.7, 3.7 | 2.7, 3.7 | 2.7, 3.7 | | Scala | 2.11.12 | 2.11.12 | 2.11.12 | 2.11.12 | | AmazonCloudWatchAgent | - | - | - | - | | Delta | - | - | - | - | | Flink | 1.11.2 | 1.11.2 | 1.11.0 | 1.11.0 | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 1.4.13 | 1.4.13 | 1.4.13 | 1.4.13 | | HCatalog | 2.3.7-amzn-3 | 2.3.7-amzn-3 | 2.3.7-amzn-1 | 2.3.7-amzn-1 | | Hadoop | 2.10.1-amzn-0.1 | 2.10.1-amzn-0 | 2.10.0-amzn-0.1 | 2.10.0-amzn-0 | | Hive | 2.3.7-amzn-3 | 2.3.7-amzn-3 | 2.3.7-amzn-1 | 2.3.7-amzn-1 | | Hudi | 0.6.0-amzn-0 | 0.6.0-amzn-0 | 0.6.0-amzn-0 | 0.6.0-amzn-0 | | Hue | 4.8.0 | 4.8.0 | 4.7.1 | 4.7.1 | | Iceberg | - | - | - | - | | JupyterEnterpriseGateway | 2.1.0 | 2.1.0 | - | - | | JupyterHub | 1.1.0 | 1.1.0 | 1.1.0 | 1.1.0 | | Livy | 0.7.0-incubating | 0.7.0-incubating | 0.7.0-incubating | 0.7.0-incubating | | MXNet | 1.7.0 | 1.7.0 | 1.6.0 | 1.6.0 | | Mahout | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | | Oozie | 5.2.0 | 5.2.0 | 5.2.0 | 5.2.0 | | Phoenix | 4.14.3-HBase-1.4 | 4.14.3-HBase-1.4 | 4.14.3-HBase-1.4 | 4.14.3-HBase-1.4 | | Pig | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | | Presto | 0.240.1-amzn-0 | 0.240.1-amzn-0 | 0.238.3-amzn-0 | 0.238.3-amzn-0 | | Spark | 2.4.7-amzn-0.1 | 2.4.7-amzn-0 | 2.4.6-amzn-0.1 | 2.4.6-amzn-0 | | Sqoop | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | | TensorFlow | 2.3.1 | 2.3.1 | 2.1.0 | 2.1.0 | | Tez | 0.9.2 | 0.9.2 | 0.9.2 | 0.9.2 | | Trino (PrestoSQL) | - | - | - | - | | Zeppelin | 0.8.2 | 0.8.2 | 0.8.2 | 0.8.2 | | ZooKeeper | 3.4.14 | 3.4.14 | 3.4.14 | 3.4.14 | ## 5.32.1 release notes This is a release to fix issues with Amazon EMR Scaling when it fails to scale up/scale down a cluster successfully or causes application failures. **Changes, Enhancements, and Resolved Issues** + Fixed an issue where scaling requests failed for a large, highly utilized cluster when Amazon EMR on-cluster daemons were running health checking activities, such as gathering YARN node state and HDFS node state. This was happening because on-cluster daemons were not able to communicate the health status data of a node to internal Amazon EMR components. + Improved EMR on-cluster daemons to correctly track the node states when IP addresses are reused to improve reliability during scaling operations. + [SPARK-29683](https://issues.apache.org/jira/browse/SPARK-29683). Fixed an issue where job failures occurred during cluster scale-down as Spark was assuming all available nodes were deny-listed. + [YARN-9011](https://issues.apache.org/jira/browse/YARN-9011). Fixed an issue where job failures occurred due to a race condition in YARN decommissioning when cluster tried to scale up or down. + Fixed issue with step or job failures during cluster scaling by ensuring that the node states are always consistent between the Amazon EMR on-cluster daemons and YARN/HDFS. + Fixed an issue where cluster operations such as scale down and step submission failed for Amazon EMR clusters enabled with Kerberos authentication. This was because the Amazon EMR on-cluster daemon did not renew the Kerberos ticket, which is required to securely communicate with HDFS/YARN running on the primary node. + Newer Amazon EMR releases fix the issue with a lower "Max open files" limit on older AL2 in Amazon EMR. Amazon EMR releases 5.30.1, 5.30.2, 5.31.1, 5.32.1, 6.0.1, 6.1.1, 6.2.1, 5.33.0, 6.3.0 and later now include a permanent fix with a higher "Max open files" setting. + HTTPS is now enabled by default for Amazon Linux repositories. If you are using an Amazon S3 VPCE policy to restrict access to specific buckets, you must add the new Amazon Linux bucket ARN `arn:aws:s3:::amazonlinux-2-repos-$region/*` to your policy (replace `$region` with the region where the endpoint is). For more information, see this topic in the AWS discussion forums. [Announcement: Amazon Linux 2 now supports the ability to use HTTPS while connecting to package repositories ](https://forums.aws.amazon.com/ann.jspa?annID=8528). **Known issues** + When you use Spark with Hive partition location formatting to read data in Amazon S3, and you run Spark on Amazon EMR releases 5.30.0 to 5.36.0, and 6.2.0 to 6.9.0, you might encounter an issue that prevents your cluster from reading data correctly. This can happen if your partitions have all of the following characteristics: + Two or more partitions are scanned from the same table. + At least one partition directory path is a prefix of at least one other partition directory path, for example, `s3://bucket/table/p=a` is a prefix of `s3://bucket/table/p=a b`. + The first character that follows the prefix in the other partition directory has a UTF-8 value that’s less than than the `/` character (U\$1002F). For example, the space character (U\$10020) that occurs between a and b in `s3://bucket/table/p=a b` falls into this category. Note that there are 14 other non-control characters: `!"#$%&‘()*+,-`. For more information, see [UTF-8 encoding table and Unicode characters](https://www.utf8-chartable.de/). As a workaround to this issue, set the `spark.sql.sources.fastS3PartitionDiscovery.enabled` configuration to `false` in the `spark-defaults` classification. ## 5.32.1 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | aws-sagemaker-spark-sdk | 1.4.1 | Amazon SageMaker Spark SDK | | emr-ddb | 4.16.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 2.13.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.5.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-notebook-env | 1.1.0 | Conda env for emr notebook which includes jupyter enterprise gateway | | emr-s3-dist-cp | 2.17.0 | Distributed copy application optimized for Amazon S3. | | emr-s3-select | 1.6.0 | EMR S3Select Connector | | emrfs | 2.45.0 | Amazon S3 connector for Hadoop ecosystem applications. | | flink-client | 1.11.2 | Apache Flink command line client scripts and applications. | | flink-jobmanager-config | 1.11.2 | Managing resources on EMR nodes for Apache Flink JobManager. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 2.10.1-amzn-0.1 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 2.10.1-amzn-0.1 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 2.10.1-amzn-0.1 | HDFS command-line client and library | | hadoop-hdfs-namenode | 2.10.1-amzn-0.1 | HDFS service for tracking file names and block locations. | | hadoop-hdfs-journalnode | 2.10.1-amzn-0.1 | HDFS service for managing the Hadoop filesystem journal on HA clusters. | | hadoop-httpfs-server | 2.10.1-amzn-0.1 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 2.10.1-amzn-0.1 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 2.10.1-amzn-0.1 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 2.10.1-amzn-0.1 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 2.10.1-amzn-0.1 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 2.10.1-amzn-0.1 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 1.4.13 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 1.4.13 | Service for serving one or more HBase regions. | | hbase-client | 1.4.13 | HBase command-line client. | | hbase-rest-server | 1.4.13 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 1.4.13 | Service providing a Thrift endpoint to HBase. | | hcatalog-client | 2.3.7-amzn-3 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 2.3.7-amzn-3 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 2.3.7-amzn-3 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 2.3.7-amzn-3 | Hive command line client. | | hive-hbase | 2.3.7-amzn-3 | Hive-hbase client. | | hive-metastore-server | 2.3.7-amzn-3 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server2 | 2.3.7-amzn-3 | Service for accepting Hive queries as web requests. | | hudi | 0.6.0-amzn-0 | Incremental processing framework to power data pipline at low latency and high efficiency. | | hudi-spark | 0.6.0-amzn-0 | Bundle library for running Spark with Hudi. | | hudi-presto | 0.6.0-amzn-0 | Bundle library for running Presto with Hudi. | | hue-server | 4.8.0 | Web application for analyzing data using Hadoop ecosystem applications | | jupyterhub | 1.1.0 | Multi-user server for Jupyter notebooks | | livy-server | 0.7.0-incubating | REST interface for interacting with Apache Spark | | nginx | 1.12.1 | nginx [engine x] is an HTTP and reverse proxy server | | mahout-client | 0.13.0 | Library for machine learning. | | mxnet | 1.7.0 | A flexible, scalable, and efficient library for deep learning. | | mariadb-server | 5.5.68\$1 | MySQL database server. | | nvidia-cuda | 10.1.243 | Nvidia drivers and Cuda toolkit | | oozie-client | 5.2.0 | Oozie command-line client. | | oozie-server | 5.2.0 | Service for accepting Oozie workflow requests. | | opencv | 4.4.0 | Open Source Computer Vision Library. | | phoenix-library | 4.14.3-HBase-1.4 | The phoenix libraries for server and client | | phoenix-query-server | 4.14.3-HBase-1.4 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.240.1-amzn-0 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.240.1-amzn-0 | Service for executing pieces of a query. | | presto-client | 0.240.1-amzn-0 | Presto command-line client which is installed on an HA cluster's stand-by masters where Presto server is not started. | | pig-client | 0.17.0 | Pig command-line client. | | r | 3.4.3 | The R Project for Statistical Computing | | ranger-kms-server | 1.2.0 | Apache Ranger Key Management System | | spark-client | 2.4.7-amzn-0.1 | Spark command-line clients. | | spark-history-server | 2.4.7-amzn-0.1 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 2.4.7-amzn-0.1 | In-memory execution engine for YARN. | | spark-yarn-slave | 2.4.7-amzn-0.1 | Apache Spark libraries needed by YARN slaves. | | sqoop-client | 1.4.7 | Apache Sqoop command-line client. | | tensorflow | 2.3.1 | TensorFlow open source software library for high performance numerical computation. | | tez-on-yarn | 0.9.2 | The tez YARN application and libraries. | | webserver | 2.4.25\$1 | Apache HTTP server. | | zeppelin-server | 0.8.2 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.4.14 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.4.14 | ZooKeeper command line client. | ## 5.32.1 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). Reconfiguration actions occur when you specify a configuration for instance groups in a running cluster. Amazon EMR only initiates reconfiguration actions for the classifications that you modify. For more information, see [Reconfigure an instance group in a running cluster](emr-configure-apps-running-cluster.md). **emr-5.32.1 classifications** | Classifications | Description | Reconfiguration Actions | | --- | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | Restarts the ResourceManager service. | | container-executor | Change values in Hadoop YARN's container-executor.cfg file. | Not available. | | container-log4j | Change values in Hadoop YARN's container-log4j.properties file. | Not available. | | core-site | Change values in Hadoop's core-site.xml file. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts Hadoop KMS, Ranger KMS, HiveServer2, Hive MetaStore, Hadoop Httpfs, and MapReduce-HistoryServer. | | docker-conf | Change docker related settings. | Not available. | | emrfs-site | Change EMRFS settings. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts HBaseRegionserver, HBaseMaster, HBaseThrift, HBaseRest, HiveServer2, Hive MetaStore, Hadoop Httpfs, and MapReduce-HistoryServer. | | flink-conf | Change flink-conf.yaml settings. | Restarts Flink history server. | | flink-log4j | Change Flink log4j.properties settings. | Restarts Flink history server. | | flink-log4j-yarn-session | Change Flink log4j-yarn-session.properties settings. | Not available. | | flink-log4j-cli | Change Flink log4j-cli.properties settings. | Restarts Flink history server. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts PhoenixQueryserver, HiveServer2, Hive MetaStore, and MapReduce-HistoryServer. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts Hadoop KMS, Hadoop Httpfs, and MapReduce-HistoryServer. | | hadoop-ssl-server | Change hadoop ssl server configuration | Not available. | | hadoop-ssl-client | Change hadoop ssl client configuration | Not available. | | hbase | Amazon EMR-curated settings for Apache HBase. | Custom EMR specific property. Sets emrfs-site and hbase-site configs. See those for their associated restarts. | | hbase-env | Change values in HBase's environment. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbase.properties file. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | Not available. | | hbase-site | Change values in HBase's hbase-site.xml file. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. Additionally restarts Phoenix QueryServer. | | hdfs-encryption-zones | Configure HDFS encryption zones. | Should not be reconfigured. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Additionally restarts Hadoop Httpfs. | | hcatalog-env | Change values in HCatalog's environment. | Restarts Hive HCatalog Server. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | Restarts Hive HCatalog Server. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | Restarts Hive HCatalog Server. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | Restarts Hive WebHCat Server. | | hcatalog-webhcat-log4j2 | Change values in HCatalog WebHCat's log4j2.properties. | Restarts Hive WebHCat Server. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | Restarts Hive WebHCat Server. | | hive-beeline-log4j2 | Change values in Hive's beeline-log4j2.properties file. | Not available. | | hive-parquet-logging | Change values in Hive's parquet-logging.properties file. | Not available. | | hive-env | Change values in the Hive environment. | Restarts HiveServer2 and HiveMetastore. Runs Hive schemaTool CLI commands to verify hive-metastore. | | hive-exec-log4j2 | Change values in Hive's hive-exec-log4j2.properties file. | Restarts HiveServer2 and HiveMetastore. | | hive-llap-daemon-log4j2 | Change values in Hive's llap-daemon-log4j2.properties file. | Not available. | | hive-log4j2 | Change values in Hive's hive-log4j2.properties file. | Not available. | | hive-site | Change values in Hive's hive-site.xml file | Restarts HiveServer2 and HiveMetastore. Runs Hive schemaTool CLI commands to verify hive-metastore. Also restarts Oozie and Zeppelin. | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | Not available. | | hue-ini | Change values in Hue's ini file | Restarts Hue. Also activates Hue config override CLI commands to pick up new configurations. | | httpfs-env | Change values in the HTTPFS environment. | Restarts Hadoop Httpfs service. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | Restarts Hadoop Httpfs service. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | Not available. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | Restarts Hadoop-KMS service. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | Not available. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | Restarts Hadoop-KMS and Ranger-KMS service. | | hudi-env | Change values in the Hudi environment. | Not available. | | jupyter-notebook-conf | Change values in Jupyter Notebook's jupyter\$1notebook\$1config.py file. | Not available. | | jupyter-hub-conf | Change values in JupyterHubs's jupyterhub\$1config.py file. | Not available. | | jupyter-s3-conf | Configure Jupyter Notebook S3 persistence. | Not available. | | jupyter-sparkmagic-conf | Change values in Sparkmagic's config.json file. | Not available. | | livy-conf | Change values in Livy's livy.conf file. | Restarts Livy Server. | | livy-env | Change values in the Livy environment. | Restarts Livy Server. | | livy-log4j | Change Livy log4j.properties settings. | Restarts Livy Server. | | mapred-env | Change values in the MapReduce application's environment. | Restarts Hadoop MapReduce-HistoryServer. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | Restarts Hadoop MapReduce-HistoryServer. | | oozie-env | Change values in Oozie's environment. | Restarts Oozie. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | Restarts Oozie. | | oozie-site | Change values in Oozie's oozie-site.xml file. | Restarts Oozie. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | Not available. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | Not available. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | Restarts Phoenix-QueryServer. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | Not available. | | pig-env | Change values in the Pig environment. | Not available. | | pig-properties | Change values in Pig's pig.properties file. | Restarts Oozie. | | pig-log4j | Change values in Pig's log4j.properties file. | Not available. | | presto-log | Change values in Presto's log.properties file. | Restarts Presto-Server. | | presto-config | Change values in Presto's config.properties file. | Restarts Presto-Server. | | presto-password-authenticator | Change values in Presto's password-authenticator.properties file. | Not available. | | presto-env | Change values in Presto's presto-env.sh file. | Restarts Presto-Server. | | presto-node | Change values in Presto's node.properties file. | Not available. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | Not available. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | Not available. | | presto-connector-hive | Change values in Presto's hive.properties file. | Restarts Presto-Server. | | presto-connector-jmx | Change values in Presto's jmx.properties file. | Not available. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | Not available. | | presto-connector-localfile | Change values in Presto's localfile.properties file. | Not available. | | presto-connector-memory | Change values in Presto's memory.properties file. | Not available. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | Not available. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | Not available. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | Not available. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | Not available. | | presto-connector-redis | Change values in Presto's redis.properties file. | Not available. | | presto-connector-redshift | Change values in Presto's redshift.properties file. | Not available. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | Not available. | | presto-connector-tpcds | Change values in Presto's tpcds.properties file. | Not available. | | ranger-kms-dbks-site | Change values in dbks-site.xml file of Ranger KMS. | Restarts Ranger KMS Server. | | ranger-kms-site | Change values in ranger-kms-site.xml file of Ranger KMS. | Restarts Ranger KMS Server. | | ranger-kms-env | Change values in the Ranger KMS environment. | Restarts Ranger KMS Server. | | ranger-kms-log4j | Change values in kms-log4j.properties file of Ranger KMS. | Not available. | | ranger-kms-db-ca | Change values for CA file on S3 for MySQL SSL connection with Ranger KMS. | Not available. | | recordserver-env | Change values in the EMR RecordServer environment. | Restarts EMR record server. | | recordserver-conf | Change values in EMR RecordServer's erver.properties file. | Restarts EMR record server. | | recordserver-log4j | Change values in EMR RecordServer's log4j.properties file. | Restarts EMR record server. | | spark | Amazon EMR-curated settings for Apache Spark. | This property modifies spark-defaults. See actions there. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | Restarts Spark history server and Spark thrift server. | | spark-env | Change values in the Spark environment. | Restarts Spark history server and Spark thrift server. | | spark-hive-site | Change values in Spark's hive-site.xml file | Not available. | | spark-log4j | Change values in Spark's log4j.properties file. | Restarts Spark history server and Spark thrift server. | | spark-metrics | Change values in Spark's metrics.properties file. | Restarts Spark history server and Spark thrift server. | | sqoop-env | Change values in Sqoop's environment. | Not available. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | Not available. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | Not available. | | tez-site | Change values in Tez's tez-site.xml file. | Restarts Oozie. | | yarn-env | Change values in the YARN environment. | Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts MapReduce-HistoryServer. | | yarn-site | Change values in YARN's yarn-site.xml file. | Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts Livy Server and MapReduce-HistoryServer. | | zeppelin-env | Change values in the Zeppelin environment. | Restarts Zeppelin. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | Restarts Zookeeper server. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | Restarts Zookeeper server. | # Amazon EMR release 5.32.0 ## 5.32.0 application versions This release includes the following applications: [https://flink.apache.org/](https://flink.apache.org/), [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [https://hudi.apache.org](https://hudi.apache.org), [http://gethue.com/](http://gethue.com/), [https://jupyter-enterprise-gateway.readthedocs.io/en/latest/](https://jupyter-enterprise-gateway.readthedocs.io/en/latest/), [https://jupyterhub.readthedocs.io/en/latest/#](https://jupyterhub.readthedocs.io/en/latest/#), [https://livy.incubator.apache.org/](https://livy.incubator.apache.org/), [https://mxnet.incubator.apache.org/](https://mxnet.incubator.apache.org/), [http://mahout.apache.org/](http://mahout.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://www.tensorflow.org/](https://www.tensorflow.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-5.32.0 | emr-5.31.1 | emr-5.31.0 | emr-5.30.2 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.11.890 | 1.11.852 | 1.11.852 | 1.11.759 | | Python | 2.7, 3.7 | 2.7, 3.7 | 2.7, 3.7 | 2.7, 3.7 | | Scala | 2.11.12 | 2.11.12 | 2.11.12 | 2.11.12 | | AmazonCloudWatchAgent | - | - | - | - | | Delta | - | - | - | - | | Flink | 1.11.2 | 1.11.0 | 1.11.0 | 1.10.0 | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 1.4.13 | 1.4.13 | 1.4.13 | 1.4.13 | | HCatalog | 2.3.7-amzn-3 | 2.3.7-amzn-1 | 2.3.7-amzn-1 | 2.3.6-amzn-2 | | Hadoop | 2.10.1-amzn-0 | 2.10.0-amzn-0.1 | 2.10.0-amzn-0 | 2.8.5-amzn-6.1 | | Hive | 2.3.7-amzn-3 | 2.3.7-amzn-1 | 2.3.7-amzn-1 | 2.3.6-amzn-2 | | Hudi | 0.6.0-amzn-0 | 0.6.0-amzn-0 | 0.6.0-amzn-0 | 0.5.2-incubating | | Hue | 4.8.0 | 4.7.1 | 4.7.1 | 4.6.0 | | Iceberg | - | - | - | - | | JupyterEnterpriseGateway | 2.1.0 | - | - | - | | JupyterHub | 1.1.0 | 1.1.0 | 1.1.0 | 1.1.0 | | Livy | 0.7.0-incubating | 0.7.0-incubating | 0.7.0-incubating | 0.7.0-incubating | | MXNet | 1.7.0 | 1.6.0 | 1.6.0 | 1.5.1 | | Mahout | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | | Oozie | 5.2.0 | 5.2.0 | 5.2.0 | 5.2.0 | | Phoenix | 4.14.3-HBase-1.4 | 4.14.3-HBase-1.4 | 4.14.3-HBase-1.4 | 4.14.3-HBase-1.4 | | Pig | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | | Presto | 0.240.1-amzn-0 | 0.238.3-amzn-0 | 0.238.3-amzn-0 | 0.232 | | Spark | 2.4.7-amzn-0 | 2.4.6-amzn-0.1 | 2.4.6-amzn-0 | 2.4.5-amzn-0.1 | | Sqoop | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | | TensorFlow | 2.3.1 | 2.1.0 | 2.1.0 | 1.14.0 | | Tez | 0.9.2 | 0.9.2 | 0.9.2 | 0.9.2 | | Trino (PrestoSQL) | - | - | - | - | | Zeppelin | 0.8.2 | 0.8.2 | 0.8.2 | 0.8.2 | | ZooKeeper | 3.4.14 | 3.4.14 | 3.4.14 | 3.4.14 | ## 5.32.0 release notes The following release notes include information for Amazon EMR release 5.32.0. Changes are relative to 5.31.0. Initial release date: Jan 8, 2021 **Upgrades** + Upgraded Amazon Glue connector to version 1.14.0 + Upgraded Amazon SageMaker Spark SDK to version 1.4.1 + Upgraded AWS SDK for Java to version 1.11.890 + Upgraded EMR DynamoDB Connector version 4.16.0 + Upgraded EMRFS to version 2.45.0 + Upgraded EMR Log Analytics Metrics to version 1.18.0 + Upgraded EMR MetricsAndEventsApiGateway Client to version 1.5.0 + Upgraded EMR Record Server to version 1.8.0 + Upgraded EMR S3 Dist CP to version 2.17.0 + Upgraded EMR Secret Agent to version 1.7.0 + Upgraded Flink to version 1.11.2 + Upgraded Hadoop to version 2.10.1-amzn-0 + Upgraded Hive to version 2.3.7-amzn-3 + Upgraded Hue to version 4.8.0 + Upgraded Mxnet to version 1.7.0 + Upgraded OpenCV to version 4.4.0 + Upgraded Presto to version 0.240.1-amzn-0 + Upgraded Spark to version 2.4.7-amzn-0 + Upgraded TensorFlow to version 2.3.1 **Changes, enhancements, and resolved issues** + This is a release to fix issues with Amazon EMR Scaling when it fails to scale up/scale down a cluster successfully or causes application failures. + Fixed an issue where scaling requests failed for a large, highly utilized cluster when Amazon EMR on-cluster daemons were running health checking activities, such as gathering YARN node state and HDFS node state. This was happening because on-cluster daemons were not able to communicate the health status data of a node to internal Amazon EMR components. + Improved EMR on-cluster daemons to correctly track the node states when IP addresses are reused to improve reliability during scaling operations. + [SPARK-29683](https://issues.apache.org/jira/browse/SPARK-29683). Fixed an issue where job failures occurred during cluster scale-down as Spark was assuming all available nodes were deny-listed. + [YARN-9011](https://issues.apache.org/jira/browse/YARN-9011). Fixed an issue where job failures occurred due to a race condition in YARN decommissioning when cluster tried to scale up or down. + Fixed issue with step or job failures during cluster scaling by ensuring that the node states are always consistent between the Amazon EMR on-cluster daemons and YARN/HDFS. + Fixed an issue where cluster operations such as scale down and step submission failed for Amazon EMR clusters enabled with Kerberos authentication. This was because the Amazon EMR on-cluster daemon did not renew the Kerberos ticket, which is required to securely communicate with HDFS/YARN running on the primary node. + Newer Amazon EMR releases fix the issue with a lower "Max open files" limit on older AL2 in Amazon EMR. Amazon EMR releases 5.30.1, 5.30.2, 5.31.1, 5.32.1, 6.0.1, 6.1.1, 6.2.1, 5.33.0, 6.3.0 and later now include a permanent fix with a higher "Max open files" setting. + Upgraded component versions. + For a list of component versions, see [About Amazon EMR Releases](https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-release-components.html) in this guide. **New features** + Starting with Amazon EMR 5.32.0 and 6.5.0, dynamic executor sizing for Apache Spark is enabled by default. To turn this feature on or off, you can use the `spark.yarn.heterogeneousExecutors.enabled` configuration parameter. + Instance Metadata Service (IMDS) V2 support status: Amazon EMR 5.23.1, 5.27.1 and 5.32 or later components use IMDSv2 for all IMDS calls. For IMDS calls in your application code, you can use both IMDSv1 and IMDSv2, or configure the IMDS to use only IMDSv2 for added security. For other 5.x EMR releases, disabling IMDSv1 causes cluster startup failure. + Beginning with Amazon EMR 5.32.0, you can launch a cluster that natively integrates with Apache Ranger. Apache Ranger is an open-source framework to enable, monitor, and manage comprehensive data security across the Hadoop platform. For more information, see [Apache Ranger](https://ranger.apache.org/). With native integration, you can bring your own Apache Ranger to enforce fine-grained data access control on Amazon EMR. See [Integrate Amazon EMR with Apache Ranger](https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-ranger.html) in the *Amazon EMR Release Guide*. + Amazon EMR Release 5.32.0 supports Amazon EMR on EKS. For more details on getting started with EMR on EKS, see [What is Amazon EMR on EKS](https://docs.aws.amazon.com/emr/latest/EMR-on-EKS-DevelopmentGuide/emr-eks.html). + Amazon EMR Release 5.32.0 supports Amazon EMR Studio (Preview). For more details on getting started with EMR Studio, see [Amazon EMR Studio (Preview)](https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-studio.html). + Scoped managed policies: To align with AWS best practices, Amazon EMR has introduced v2 EMR-scoped default managed policies as replacements for policies that will be deprecated. See [Amazon EMR Managed Policies](https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-managed-iam-policies.html). **Known issues** + For Amazon EMR 6.3.0 and 6.2.0 private subnet clusters, you cannot access the Ganglia web UI. You will get an "access denied (403)" error. Other web UIs, such as Spark, Hue, JupyterHub, Zeppelin, Livy, and Tez are working normally. Ganglia web UI access on public subnet clusters are also working normally. To resolve this issue, restart httpd service on the primary node with `sudo systemctl restart httpd`. This issue is fixed in Amazon EMR 6.4.0. + **Lower "Max open files" limit on older AL2 [fixed in newer releases].** Amazon EMR releases: emr-5.30.x, emr-5.31.0, emr-5.32.0, emr-6.0.0, emr-6.1.0, and emr-6.2.0 are based on older versions ofAmazon Linux 2 (AL2), which have a lower ulimit setting for "Max open files" when Amazon EMR clusters are created with the default AMI. Amazon EMR releases 5.30.1, 5.30.2, 5.31.1, 5.32.1, 6.0.1, 6.1.1, 6.2.1, 5.33.0, 6.3.0 and later include a permanent fix with a higher "Max open files" setting. Releases with the lower open file limit causes a "Too many open files" error when submitting Spark job. In the impacted releases, the Amazon EMR default AMI has a default ulimit setting of 4096 for "Max open files," which is lower than the 65536 file limit in the latestAmazon Linux 2 AMI. The lower ulimit setting for "Max open files" causes Spark job failure when the Spark driver and executor try to open more than 4096 files. To fix the issue, Amazon EMR has a bootstrap action (BA) script that adjusts the ulimit setting at cluster creation. If you are using an older Amazon EMR version that doesn't have the permanent fix for this issue, the following workaround lets you to explicitly set the instance-controller ulimit to a maximum of 65536 files. **Explicitly set a ulimit from the command line** 1. Edit `/etc/systemd/system/instance-controller.service` to add the following parameters to Service section. `LimitNOFILE=65536` `LimitNPROC=65536` 1. Restart InstanceController `$ sudo systemctl daemon-reload` `$ sudo systemctl restart instance-controller` **Set a ulimit using bootstrap action (BA)** You can also use a bootstrap action (BA) script to configure the instance-controller ulimit to 65536 files at cluster creation. ``` #!/bin/bash for user in hadoop spark hive; do sudo tee /etc/security/limits.d/$user.conf << EOF $user - nofile 65536 $user - nproc 65536 EOF done for proc in instancecontroller logpusher; do sudo mkdir -p /etc/systemd/system/$proc.service.d/ sudo tee /etc/systemd/system/$proc.service.d/override.conf << EOF [Service] LimitNOFILE=65536 LimitNPROC=65536 EOF pid=$(pgrep -f aws157.$proc.Main) sudo prlimit --pid $pid --nofile=65535:65535 --nproc=65535:65535 done sudo systemctl daemon-reload ``` + **Important** EMR clusters that run Amazon Linux or Amazon Linux 2 Amazon Machine Images (AMIs) use default Amazon Linux behavior, and do not automatically download and install important and critical kernel updates that require a reboot. This is the same behavior as other Amazon EC2 instances that run the default Amazon Linux AMI. If new Amazon Linux software updates that require a reboot (such as kernel, NVIDIA, and CUDA updates) become available after an Amazon EMR release becomes available, EMR cluster instances that run the default AMI do not automatically download and install those updates. To get kernel updates, you can [customize your Amazon EMR AMI](https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-custom-ami.html) to [use the latest Amazon Linux AMI](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/finding-an-ami.html). + Console support to create a security configuration that specifies the AWS Ranger integration option is currently not supported in the GovCloud Region. Security configuration can be done using the CLI. See [Create the EMR Security Configuration](https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-ranger-security-config.html) in the *Amazon EMR Management Guide*. + When AtRestEncryption or HDFS encryption is enabled on a cluster that uses Amazon EMR 5.31.0 or 5.32.0, Hive queries result in the following runtime exception. ``` TaskAttempt 3 failed, info=[Error: Error while running task ( failure ) : attempt_1604112648850_0001_1_01_000000_3:java.lang.RuntimeException: java.lang.RuntimeException: Hive Runtime Error while closing operators: java.io.IOException: java.util.ServiceConfigurationError: org.apache.hadoop.security.token.TokenIdentifier: Provider org.apache.hadoop.hbase.security.token.AuthenticationTokenIdentifier not found ``` + When you use Spark with Hive partition location formatting to read data in Amazon S3, and you run Spark on Amazon EMR releases 5.30.0 to 5.36.0, and 6.2.0 to 6.9.0, you might encounter an issue that prevents your cluster from reading data correctly. This can happen if your partitions have all of the following characteristics: + Two or more partitions are scanned from the same table. + At least one partition directory path is a prefix of at least one other partition directory path, for example, `s3://bucket/table/p=a` is a prefix of `s3://bucket/table/p=a b`. + The first character that follows the prefix in the other partition directory has a UTF-8 value that’s less than than the `/` character (U\$1002F). For example, the space character (U\$10020) that occurs between a and b in `s3://bucket/table/p=a b` falls into this category. Note that there are 14 other non-control characters: `!"#$%&‘()*+,-`. For more information, see [UTF-8 encoding table and Unicode characters](https://www.utf8-chartable.de/). As a workaround to this issue, set the `spark.sql.sources.fastS3PartitionDiscovery.enabled` configuration to `false` in the `spark-defaults` classification. ## 5.32.0 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | aws-sagemaker-spark-sdk | 1.4.1 | Amazon SageMaker Spark SDK | | emr-ddb | 4.16.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 2.13.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.5.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-notebook-env | 1.1.0 | Conda env for emr notebook which includes jupyter enterprise gateway | | emr-s3-dist-cp | 2.17.0 | Distributed copy application optimized for Amazon S3. | | emr-s3-select | 1.6.0 | EMR S3Select Connector | | emrfs | 2.45.0 | Amazon S3 connector for Hadoop ecosystem applications. | | flink-client | 1.11.2 | Apache Flink command line client scripts and applications. | | flink-jobmanager-config | 1.11.2 | Managing resources on EMR nodes for Apache Flink JobManager. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 2.10.1-amzn-0 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 2.10.1-amzn-0 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 2.10.1-amzn-0 | HDFS command-line client and library | | hadoop-hdfs-namenode | 2.10.1-amzn-0 | HDFS service for tracking file names and block locations. | | hadoop-hdfs-journalnode | 2.10.1-amzn-0 | HDFS service for managing the Hadoop filesystem journal on HA clusters. | | hadoop-httpfs-server | 2.10.1-amzn-0 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 2.10.1-amzn-0 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 2.10.1-amzn-0 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 2.10.1-amzn-0 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 2.10.1-amzn-0 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 2.10.1-amzn-0 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 1.4.13 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 1.4.13 | Service for serving one or more HBase regions. | | hbase-client | 1.4.13 | HBase command-line client. | | hbase-rest-server | 1.4.13 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 1.4.13 | Service providing a Thrift endpoint to HBase. | | hcatalog-client | 2.3.7-amzn-3 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 2.3.7-amzn-3 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 2.3.7-amzn-3 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 2.3.7-amzn-3 | Hive command line client. | | hive-hbase | 2.3.7-amzn-3 | Hive-hbase client. | | hive-metastore-server | 2.3.7-amzn-3 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server2 | 2.3.7-amzn-3 | Service for accepting Hive queries as web requests. | | hudi | 0.6.0-amzn-0 | Incremental processing framework to power data pipline at low latency and high efficiency. | | hudi-spark | 0.6.0-amzn-0 | Bundle library for running Spark with Hudi. | | hudi-presto | 0.6.0-amzn-0 | Bundle library for running Presto with Hudi. | | hue-server | 4.8.0 | Web application for analyzing data using Hadoop ecosystem applications | | jupyterhub | 1.1.0 | Multi-user server for Jupyter notebooks | | livy-server | 0.7.0-incubating | REST interface for interacting with Apache Spark | | nginx | 1.12.1 | nginx [engine x] is an HTTP and reverse proxy server | | mahout-client | 0.13.0 | Library for machine learning. | | mxnet | 1.7.0 | A flexible, scalable, and efficient library for deep learning. | | mariadb-server | 5.5.68 | MySQL database server. | | nvidia-cuda | 10.1.243 | Nvidia drivers and Cuda toolkit | | oozie-client | 5.2.0 | Oozie command-line client. | | oozie-server | 5.2.0 | Service for accepting Oozie workflow requests. | | opencv | 4.4.0 | Open Source Computer Vision Library. | | phoenix-library | 4.14.3-HBase-1.4 | The phoenix libraries for server and client | | phoenix-query-server | 4.14.3-HBase-1.4 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.240.1-amzn-0 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.240.1-amzn-0 | Service for executing pieces of a query. | | presto-client | 0.240.1-amzn-0 | Presto command-line client which is installed on an HA cluster's stand-by masters where Presto server is not started. | | pig-client | 0.17.0 | Pig command-line client. | | r | 3.4.3 | The R Project for Statistical Computing | | ranger-kms-server | 1.2.0 | Apache Ranger Key Management System | | spark-client | 2.4.7-amzn-0 | Spark command-line clients. | | spark-history-server | 2.4.7-amzn-0 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 2.4.7-amzn-0 | In-memory execution engine for YARN. | | spark-yarn-slave | 2.4.7-amzn-0 | Apache Spark libraries needed by YARN slaves. | | sqoop-client | 1.4.7 | Apache Sqoop command-line client. | | tensorflow | 2.3.1 | TensorFlow open source software library for high performance numerical computation. | | tez-on-yarn | 0.9.2 | The tez YARN application and libraries. | | webserver | 2.4.25\$1 | Apache HTTP server. | | zeppelin-server | 0.8.2 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.4.14 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.4.14 | ZooKeeper command line client. | ## 5.32.0 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). Reconfiguration actions occur when you specify a configuration for instance groups in a running cluster. Amazon EMR only initiates reconfiguration actions for the classifications that you modify. For more information, see [Reconfigure an instance group in a running cluster](emr-configure-apps-running-cluster.md). **emr-5.32.0 classifications** | Classifications | Description | Reconfiguration Actions | | --- | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | Restarts the ResourceManager service. | | container-executor | Change values in Hadoop YARN's container-executor.cfg file. | Not available. | | container-log4j | Change values in Hadoop YARN's container-log4j.properties file. | Not available. | | core-site | Change values in Hadoop's core-site.xml file. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts Hadoop KMS, Ranger KMS, HiveServer2, Hive MetaStore, Hadoop Httpfs, and MapReduce-HistoryServer. | | docker-conf | Change docker related settings. | Not available. | | emrfs-site | Change EMRFS settings. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts HBaseRegionserver, HBaseMaster, HBaseThrift, HBaseRest, HiveServer2, Hive MetaStore, Hadoop Httpfs, and MapReduce-HistoryServer. | | flink-conf | Change flink-conf.yaml settings. | Restarts Flink history server. | | flink-log4j | Change Flink log4j.properties settings. | Restarts Flink history server. | | flink-log4j-yarn-session | Change Flink log4j-yarn-session.properties settings. | Not available. | | flink-log4j-cli | Change Flink log4j-cli.properties settings. | Restarts Flink history server. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts PhoenixQueryserver, HiveServer2, Hive MetaStore, and MapReduce-HistoryServer. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts Hadoop KMS, Hadoop Httpfs, and MapReduce-HistoryServer. | | hadoop-ssl-server | Change hadoop ssl server configuration | Not available. | | hadoop-ssl-client | Change hadoop ssl client configuration | Not available. | | hbase | Amazon EMR-curated settings for Apache HBase. | Custom EMR specific property. Sets emrfs-site and hbase-site configs. See those for their associated restarts. | | hbase-env | Change values in HBase's environment. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbase.properties file. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | Not available. | | hbase-site | Change values in HBase's hbase-site.xml file. | Restarts the HBase services RegionServer, HBaseMaster, ThriftServer, RestServer. Additionally restarts Phoenix QueryServer. | | hdfs-encryption-zones | Configure HDFS encryption zones. | Should not be reconfigured. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | Restarts the Hadoop HDFS services Namenode, SecondaryNamenode, Datanode, ZKFC, and Journalnode. Additionally restarts Hadoop Httpfs. | | hcatalog-env | Change values in HCatalog's environment. | Restarts Hive HCatalog Server. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | Restarts Hive HCatalog Server. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | Restarts Hive HCatalog Server. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | Restarts Hive WebHCat Server. | | hcatalog-webhcat-log4j2 | Change values in HCatalog WebHCat's log4j2.properties. | Restarts Hive WebHCat Server. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | Restarts Hive WebHCat Server. | | hive-beeline-log4j2 | Change values in Hive's beeline-log4j2.properties file. | Not available. | | hive-parquet-logging | Change values in Hive's parquet-logging.properties file. | Not available. | | hive-env | Change values in the Hive environment. | Restarts HiveServer2 and HiveMetastore. Runs Hive schemaTool CLI commands to verify hive-metastore. | | hive-exec-log4j2 | Change values in Hive's hive-exec-log4j2.properties file. | Restarts HiveServer2 and HiveMetastore. | | hive-llap-daemon-log4j2 | Change values in Hive's llap-daemon-log4j2.properties file. | Not available. | | hive-log4j2 | Change values in Hive's hive-log4j2.properties file. | Not available. | | hive-site | Change values in Hive's hive-site.xml file | Restarts HiveServer2 and HiveMetastore. Runs Hive schemaTool CLI commands to verify hive-metastore. Also restarts Oozie and Zeppelin. | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | Not available. | | hue-ini | Change values in Hue's ini file | Restarts Hue. Also activates Hue config override CLI commands to pick up new configurations. | | httpfs-env | Change values in the HTTPFS environment. | Restarts Hadoop Httpfs service. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | Restarts Hadoop Httpfs service. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | Not available. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | Restarts Hadoop-KMS service. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | Not available. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | Restarts Hadoop-KMS and Ranger-KMS service. | | hudi-env | Change values in the Hudi environment. | Not available. | | jupyter-notebook-conf | Change values in Jupyter Notebook's jupyter\$1notebook\$1config.py file. | Not available. | | jupyter-hub-conf | Change values in JupyterHubs's jupyterhub\$1config.py file. | Not available. | | jupyter-s3-conf | Configure Jupyter Notebook S3 persistence. | Not available. | | jupyter-sparkmagic-conf | Change values in Sparkmagic's config.json file. | Not available. | | livy-conf | Change values in Livy's livy.conf file. | Restarts Livy Server. | | livy-env | Change values in the Livy environment. | Restarts Livy Server. | | livy-log4j | Change Livy log4j.properties settings. | Restarts Livy Server. | | mapred-env | Change values in the MapReduce application's environment. | Restarts Hadoop MapReduce-HistoryServer. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | Restarts Hadoop MapReduce-HistoryServer. | | oozie-env | Change values in Oozie's environment. | Restarts Oozie. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | Restarts Oozie. | | oozie-site | Change values in Oozie's oozie-site.xml file. | Restarts Oozie. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | Not available. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | Not available. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | Restarts Phoenix-QueryServer. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | Not available. | | pig-env | Change values in the Pig environment. | Not available. | | pig-properties | Change values in Pig's pig.properties file. | Restarts Oozie. | | pig-log4j | Change values in Pig's log4j.properties file. | Not available. | | presto-log | Change values in Presto's log.properties file. | Restarts Presto-Server. | | presto-config | Change values in Presto's config.properties file. | Restarts Presto-Server. | | presto-password-authenticator | Change values in Presto's password-authenticator.properties file. | Not available. | | presto-env | Change values in Presto's presto-env.sh file. | Restarts Presto-Server. | | presto-node | Change values in Presto's node.properties file. | Not available. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | Not available. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | Not available. | | presto-connector-hive | Change values in Presto's hive.properties file. | Restarts Presto-Server. | | presto-connector-jmx | Change values in Presto's jmx.properties file. | Not available. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | Not available. | | presto-connector-localfile | Change values in Presto's localfile.properties file. | Not available. | | presto-connector-memory | Change values in Presto's memory.properties file. | Not available. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | Not available. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | Not available. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | Not available. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | Not available. | | presto-connector-redis | Change values in Presto's redis.properties file. | Not available. | | presto-connector-redshift | Change values in Presto's redshift.properties file. | Not available. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | Not available. | | presto-connector-tpcds | Change values in Presto's tpcds.properties file. | Not available. | | ranger-kms-dbks-site | Change values in dbks-site.xml file of Ranger KMS. | Restarts Ranger KMS Server. | | ranger-kms-site | Change values in ranger-kms-site.xml file of Ranger KMS. | Restarts Ranger KMS Server. | | ranger-kms-env | Change values in the Ranger KMS environment. | Restarts Ranger KMS Server. | | ranger-kms-log4j | Change values in kms-log4j.properties file of Ranger KMS. | Not available. | | ranger-kms-db-ca | Change values for CA file on S3 for MySQL SSL connection with Ranger KMS. | Not available. | | recordserver-env | Change values in the EMR RecordServer environment. | Restarts EMR record server. | | recordserver-conf | Change values in EMR RecordServer's erver.properties file. | Restarts EMR record server. | | recordserver-log4j | Change values in EMR RecordServer's log4j.properties file. | Restarts EMR record server. | | spark | Amazon EMR-curated settings for Apache Spark. | This property modifies spark-defaults. See actions there. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | Restarts Spark history server and Spark thrift server. | | spark-env | Change values in the Spark environment. | Restarts Spark history server and Spark thrift server. | | spark-hive-site | Change values in Spark's hive-site.xml file | Not available. | | spark-log4j | Change values in Spark's log4j.properties file. | Restarts Spark history server and Spark thrift server. | | spark-metrics | Change values in Spark's metrics.properties file. | Restarts Spark history server and Spark thrift server. | | sqoop-env | Change values in Sqoop's environment. | Not available. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | Not available. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | Not available. | | tez-site | Change values in Tez's tez-site.xml file. | Restarts Oozie. | | yarn-env | Change values in the YARN environment. | Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts MapReduce-HistoryServer. | | yarn-site | Change values in YARN's yarn-site.xml file. | Restarts the Hadoop YARN services ResourceManager, NodeManager, ProxyServer, and TimelineServer. Additionally restarts Livy Server and MapReduce-HistoryServer. | | zeppelin-env | Change values in the Zeppelin environment. | Restarts Zeppelin. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | Restarts Zookeeper server. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | Restarts Zookeeper server. | # Amazon EMR release 5.31.1 ## 5.31.1 application versions This release includes the following applications: [https://flink.apache.org/](https://flink.apache.org/), [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [https://hudi.apache.org](https://hudi.apache.org), [http://gethue.com/](http://gethue.com/), [https://jupyterhub.readthedocs.io/en/latest/#](https://jupyterhub.readthedocs.io/en/latest/#), [https://livy.incubator.apache.org/](https://livy.incubator.apache.org/), [https://mxnet.incubator.apache.org/](https://mxnet.incubator.apache.org/), [http://mahout.apache.org/](http://mahout.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://www.tensorflow.org/](https://www.tensorflow.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-5.31.1 | emr-5.31.0 | emr-5.30.2 | emr-5.30.1 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.11.852 | 1.11.852 | 1.11.759 | 1.11.759 | | Python | 2.7, 3.7 | 2.7, 3.7 | 2.7, 3.7 | 2.7, 3.7 | | Scala | 2.11.12 | 2.11.12 | 2.11.12 | 2.11.12 | | AmazonCloudWatchAgent | - | - | - | - | | Delta | - | - | - | - | | Flink | 1.11.0 | 1.11.0 | 1.10.0 | 1.10.0 | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 1.4.13 | 1.4.13 | 1.4.13 | 1.4.13 | | HCatalog | 2.3.7-amzn-1 | 2.3.7-amzn-1 | 2.3.6-amzn-2 | 2.3.6-amzn-2 | | Hadoop | 2.10.0-amzn-0.1 | 2.10.0-amzn-0 | 2.8.5-amzn-6.1 | 2.8.5-amzn-6 | | Hive | 2.3.7-amzn-1 | 2.3.7-amzn-1 | 2.3.6-amzn-2 | 2.3.6-amzn-2 | | Hudi | 0.6.0-amzn-0 | 0.6.0-amzn-0 | 0.5.2-incubating | 0.5.2-incubating | | Hue | 4.7.1 | 4.7.1 | 4.6.0 | 4.6.0 | | Iceberg | - | - | - | - | | JupyterEnterpriseGateway | - | - | - | - | | JupyterHub | 1.1.0 | 1.1.0 | 1.1.0 | 1.1.0 | | Livy | 0.7.0-incubating | 0.7.0-incubating | 0.7.0-incubating | 0.7.0-incubating | | MXNet | 1.6.0 | 1.6.0 | 1.5.1 | 1.5.1 | | Mahout | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | | Oozie | 5.2.0 | 5.2.0 | 5.2.0 | 5.2.0 | | Phoenix | 4.14.3-HBase-1.4 | 4.14.3-HBase-1.4 | 4.14.3-HBase-1.4 | 4.14.3-HBase-1.4 | | Pig | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | | Presto | 0.238.3-amzn-0 | 0.238.3-amzn-0 | 0.232 | 0.232 | | Spark | 2.4.6-amzn-0.1 | 2.4.6-amzn-0 | 2.4.5-amzn-0.1 | 2.4.5-amzn-0 | | Sqoop | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | | TensorFlow | 2.1.0 | 2.1.0 | 1.14.0 | 1.14.0 | | Tez | 0.9.2 | 0.9.2 | 0.9.2 | 0.9.2 | | Trino (PrestoSQL) | - | - | - | - | | Zeppelin | 0.8.2 | 0.8.2 | 0.8.2 | 0.8.2 | | ZooKeeper | 3.4.14 | 3.4.14 | 3.4.14 | 3.4.14 | ## 5.31.1 release notes This is a release to fix issues with Amazon EMR Scaling when it fails to scale up/scale down a cluster successfully or causes application failures. **Changes, Enhancements, and Resolved Issues** + Fixed an issue where scaling requests failed for a large, highly utilized cluster when Amazon EMR on-cluster daemons were running health checking activities, such as gathering YARN node state and HDFS node state. This was happening because on-cluster daemons were not able to communicate the health status data of a node to internal Amazon EMR components. + Improved EMR on-cluster daemons to correctly track the node states when IP addresses are reused to improve reliability during scaling operations. + [SPARK-29683](https://issues.apache.org/jira/browse/SPARK-29683). Fixed an issue where job failures occurred during cluster scale-down as Spark was assuming all available nodes were deny-listed. + [YARN-9011](https://issues.apache.org/jira/browse/YARN-9011). Fixed an issue where job failures occurred due to a race condition in YARN decommissioning when cluster tried to scale up or down. + Fixed issue with step or job failures during cluster scaling by ensuring that the node states are always consistent between the Amazon EMR on-cluster daemons and YARN/HDFS. + Fixed an issue where cluster operations such as scale down and step submission failed for Amazon EMR clusters enabled with Kerberos authentication. This was because the Amazon EMR on-cluster daemon did not renew the Kerberos ticket, which is required to securely communicate with HDFS/YARN running on the primary node. + Newer Amazon EMR releases fix the issue with a lower "Max open files" limit on older AL2 in Amazon EMR. Amazon EMR releases 5.30.1, 5.30.2, 5.31.1, 5.32.1, 6.0.1, 6.1.1, 6.2.1, 5.33.0, 6.3.0 and later now include a permanent fix with a higher "Max open files" setting. + HTTPS is now enabled by default for Amazon Linux repositories. If you are using an Amazon S3 VPCE policy to restrict access to specific buckets, you must add the new Amazon Linux bucket ARN `arn:aws:s3:::amazonlinux-2-repos-$region/*` to your policy (replace `$region` with the region where the endpoint is). For more information, see this topic in the AWS discussion forums. [Announcement: Amazon Linux 2 now supports the ability to use HTTPS while connecting to package repositories ](https://forums.aws.amazon.com/ann.jspa?annID=8528). **Known issues** + When you use Spark with Hive partition location formatting to read data in Amazon S3, and you run Spark on Amazon EMR releases 5.30.0 to 5.36.0, and 6.2.0 to 6.9.0, you might encounter an issue that prevents your cluster from reading data correctly. This can happen if your partitions have all of the following characteristics: + Two or more partitions are scanned from the same table. + At least one partition directory path is a prefix of at least one other partition directory path, for example, `s3://bucket/table/p=a` is a prefix of `s3://bucket/table/p=a b`. + The first character that follows the prefix in the other partition directory has a UTF-8 value that’s less than than the `/` character (U\$1002F). For example, the space character (U\$10020) that occurs between a and b in `s3://bucket/table/p=a b` falls into this category. Note that there are 14 other non-control characters: `!"#$%&‘()*+,-`. For more information, see [UTF-8 encoding table and Unicode characters](https://www.utf8-chartable.de/). As a workaround to this issue, set the `spark.sql.sources.fastS3PartitionDiscovery.enabled` configuration to `false` in the `spark-defaults` classification. ## 5.31.1 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | aws-sagemaker-spark-sdk | 1.4.0 | Amazon SageMaker Spark SDK | | emr-ddb | 4.15.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 2.13.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.5.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-s3-dist-cp | 2.15.0 | Distributed copy application optimized for Amazon S3. | | emr-s3-select | 1.6.0 | EMR S3Select Connector | | emrfs | 2.43.0 | Amazon S3 connector for Hadoop ecosystem applications. | | flink-client | 1.11.0 | Apache Flink command line client scripts and applications. | | flink-jobmanager-config | 1.11.0 | Managing resources on EMR nodes for Apache Flink JobManager. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 2.10.0-amzn-0.1 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 2.10.0-amzn-0.1 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 2.10.0-amzn-0.1 | HDFS command-line client and library | | hadoop-hdfs-namenode | 2.10.0-amzn-0.1 | HDFS service for tracking file names and block locations. | | hadoop-hdfs-journalnode | 2.10.0-amzn-0.1 | HDFS service for managing the Hadoop filesystem journal on HA clusters. | | hadoop-httpfs-server | 2.10.0-amzn-0.1 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 2.10.0-amzn-0.1 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 2.10.0-amzn-0.1 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 2.10.0-amzn-0.1 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 2.10.0-amzn-0.1 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 2.10.0-amzn-0.1 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 1.4.13 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 1.4.13 | Service for serving one or more HBase regions. | | hbase-client | 1.4.13 | HBase command-line client. | | hbase-rest-server | 1.4.13 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 1.4.13 | Service providing a Thrift endpoint to HBase. | | hcatalog-client | 2.3.7-amzn-1 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 2.3.7-amzn-1 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 2.3.7-amzn-1 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 2.3.7-amzn-1 | Hive command line client. | | hive-hbase | 2.3.7-amzn-1 | Hive-hbase client. | | hive-metastore-server | 2.3.7-amzn-1 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server2 | 2.3.7-amzn-1 | Service for accepting Hive queries as web requests. | | hudi | 0.6.0-amzn-0 | Incremental processing framework to power data pipline at low latency and high efficiency. | | hudi-spark | 0.6.0-amzn-0 | Bundle library for running Spark with Hudi. | | hudi-presto | 0.6.0-amzn-0 | Bundle library for running Presto with Hudi. | | hue-server | 4.7.1 | Web application for analyzing data using Hadoop ecosystem applications | | jupyterhub | 1.1.0 | Multi-user server for Jupyter notebooks | | livy-server | 0.7.0-incubating | REST interface for interacting with Apache Spark | | nginx | 1.12.1 | nginx [engine x] is an HTTP and reverse proxy server | | mahout-client | 0.13.0 | Library for machine learning. | | mxnet | 1.6.0 | A flexible, scalable, and efficient library for deep learning. | | mariadb-server | 5.5.64\$1 | MySQL database server. | | nvidia-cuda | 9.2.88 | Nvidia drivers and Cuda toolkit | | oozie-client | 5.2.0 | Oozie command-line client. | | oozie-server | 5.2.0 | Service for accepting Oozie workflow requests. | | opencv | 4.3.0 | Open Source Computer Vision Library. | | phoenix-library | 4.14.3-HBase-1.4 | The phoenix libraries for server and client | | phoenix-query-server | 4.14.3-HBase-1.4 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.238.3-amzn-0 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.238.3-amzn-0 | Service for executing pieces of a query. | | presto-client | 0.238.3-amzn-0 | Presto command-line client which is installed on an HA cluster's stand-by masters where Presto server is not started. | | pig-client | 0.17.0 | Pig command-line client. | | r | 3.4.3 | The R Project for Statistical Computing | | ranger-kms-server | 1.2.0 | Apache Ranger Key Management System | | spark-client | 2.4.6-amzn-0.1 | Spark command-line clients. | | spark-history-server | 2.4.6-amzn-0.1 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 2.4.6-amzn-0.1 | In-memory execution engine for YARN. | | spark-yarn-slave | 2.4.6-amzn-0.1 | Apache Spark libraries needed by YARN slaves. | | sqoop-client | 1.4.7 | Apache Sqoop command-line client. | | tensorflow | 2.1.0 | TensorFlow open source software library for high performance numerical computation. | | tez-on-yarn | 0.9.2 | The tez YARN application and libraries. | | webserver | 2.4.25\$1 | Apache HTTP server. | | zeppelin-server | 0.8.2 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.4.14 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.4.14 | ZooKeeper command line client. | ## 5.31.1 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). **emr-5.31.1 classifications** | Classifications | Description | | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | | container-log4j | Change values in Hadoop YARN's container-log4j.properties file. | | core-site | Change values in Hadoop's core-site.xml file. | | emrfs-site | Change EMRFS settings. | | flink-conf | Change flink-conf.yaml settings. | | flink-log4j | Change Flink log4j.properties settings. | | flink-log4j-yarn-session | Change Flink log4j-yarn-session.properties settings. | | flink-log4j-cli | Change Flink log4j-cli.properties settings. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | | hadoop-ssl-server | Change hadoop ssl server configuration | | hadoop-ssl-client | Change hadoop ssl client configuration | | hbase | Amazon EMR-curated settings for Apache HBase. | | hbase-env | Change values in HBase's environment. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbase.properties file. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | | hbase-site | Change values in HBase's hbase-site.xml file. | | hdfs-encryption-zones | Configure HDFS encryption zones. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | | hcatalog-env | Change values in HCatalog's environment. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | | hcatalog-webhcat-log4j2 | Change values in HCatalog WebHCat's log4j2.properties. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | | hive-beeline-log4j2 | Change values in Hive's beeline-log4j2.properties file. | | hive-parquet-logging | Change values in Hive's parquet-logging.properties file. | | hive-env | Change values in the Hive environment. | | hive-exec-log4j2 | Change values in Hive's hive-exec-log4j2.properties file. | | hive-llap-daemon-log4j2 | Change values in Hive's llap-daemon-log4j2.properties file. | | hive-log4j2 | Change values in Hive's hive-log4j2.properties file. | | hive-site | Change values in Hive's hive-site.xml file | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | | hue-ini | Change values in Hue's ini file | | httpfs-env | Change values in the HTTPFS environment. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | | hudi-env | Change values in the Hudi environment. | | jupyter-notebook-conf | Change values in Jupyter Notebook's jupyter\$1notebook\$1config.py file. | | jupyter-hub-conf | Change values in JupyterHubs's jupyterhub\$1config.py file. | | jupyter-s3-conf | Configure Jupyter Notebook S3 persistence. | | jupyter-sparkmagic-conf | Change values in Sparkmagic's config.json file. | | livy-conf | Change values in Livy's livy.conf file. | | livy-env | Change values in the Livy environment. | | livy-log4j | Change Livy log4j.properties settings. | | mapred-env | Change values in the MapReduce application's environment. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | | oozie-env | Change values in Oozie's environment. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | | oozie-site | Change values in Oozie's oozie-site.xml file. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | | pig-env | Change values in the Pig environment. | | pig-properties | Change values in Pig's pig.properties file. | | pig-log4j | Change values in Pig's log4j.properties file. | | presto-log | Change values in Presto's log.properties file. | | presto-config | Change values in Presto's config.properties file. | | presto-password-authenticator | Change values in Presto's password-authenticator.properties file. | | presto-env | Change values in Presto's presto-env.sh file. | | presto-node | Change values in Presto's node.properties file. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | | presto-connector-hive | Change values in Presto's hive.properties file. | | presto-connector-jmx | Change values in Presto's jmx.properties file. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | | presto-connector-localfile | Change values in Presto's localfile.properties file. | | presto-connector-memory | Change values in Presto's memory.properties file. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | | presto-connector-redis | Change values in Presto's redis.properties file. | | presto-connector-redshift | Change values in Presto's redshift.properties file. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | | presto-connector-tpcds | Change values in Presto's tpcds.properties file. | | ranger-kms-dbks-site | Change values in dbks-site.xml file of Ranger KMS. | | ranger-kms-site | Change values in ranger-kms-site.xml file of Ranger KMS. | | ranger-kms-env | Change values in the Ranger KMS environment. | | ranger-kms-log4j | Change values in kms-log4j.properties file of Ranger KMS. | | ranger-kms-db-ca | Change values for CA file on S3 for MySQL SSL connection with Ranger KMS. | | recordserver-env | Change values in the EMR RecordServer environment. | | recordserver-conf | Change values in EMR RecordServer's erver.properties file. | | recordserver-log4j | Change values in EMR RecordServer's log4j.properties file. | | spark | Amazon EMR-curated settings for Apache Spark. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | | spark-env | Change values in the Spark environment. | | spark-hive-site | Change values in Spark's hive-site.xml file | | spark-log4j | Change values in Spark's log4j.properties file. | | spark-metrics | Change values in Spark's metrics.properties file. | | sqoop-env | Change values in Sqoop's environment. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | | tez-site | Change values in Tez's tez-site.xml file. | | yarn-env | Change values in the YARN environment. | | yarn-site | Change values in YARN's yarn-site.xml file. | | zeppelin-env | Change values in the Zeppelin environment. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | # Amazon EMR release 5.31.0 ## 5.31.0 application versions This release includes the following applications: [https://flink.apache.org/](https://flink.apache.org/), [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [https://hudi.apache.org](https://hudi.apache.org), [http://gethue.com/](http://gethue.com/), [https://jupyterhub.readthedocs.io/en/latest/#](https://jupyterhub.readthedocs.io/en/latest/#), [https://livy.incubator.apache.org/](https://livy.incubator.apache.org/), [https://mxnet.incubator.apache.org/](https://mxnet.incubator.apache.org/), [http://mahout.apache.org/](http://mahout.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://www.tensorflow.org/](https://www.tensorflow.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-5.31.0 | emr-5.30.2 | emr-5.30.1 | emr-5.30.0 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.11.852 | 1.11.759 | 1.11.759 | 1.11.759 | | Python | 2.7, 3.7 | 2.7, 3.7 | 2.7, 3.7 | 2.7, 3.7 | | Scala | 2.11.12 | 2.11.12 | 2.11.12 | 2.11.12 | | AmazonCloudWatchAgent | - | - | - | - | | Delta | - | - | - | - | | Flink | 1.11.0 | 1.10.0 | 1.10.0 | 1.10.0 | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 1.4.13 | 1.4.13 | 1.4.13 | 1.4.13 | | HCatalog | 2.3.7-amzn-1 | 2.3.6-amzn-2 | 2.3.6-amzn-2 | 2.3.6-amzn-2 | | Hadoop | 2.10.0-amzn-0 | 2.8.5-amzn-6.1 | 2.8.5-amzn-6 | 2.8.5-amzn-6 | | Hive | 2.3.7-amzn-1 | 2.3.6-amzn-2 | 2.3.6-amzn-2 | 2.3.6-amzn-2 | | Hudi | 0.6.0-amzn-0 | 0.5.2-incubating | 0.5.2-incubating | 0.5.2-incubating | | Hue | 4.7.1 | 4.6.0 | 4.6.0 | 4.6.0 | | Iceberg | - | - | - | - | | JupyterEnterpriseGateway | - | - | - | - | | JupyterHub | 1.1.0 | 1.1.0 | 1.1.0 | 1.1.0 | | Livy | 0.7.0-incubating | 0.7.0-incubating | 0.7.0-incubating | 0.7.0-incubating | | MXNet | 1.6.0 | 1.5.1 | 1.5.1 | 1.5.1 | | Mahout | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | | Oozie | 5.2.0 | 5.2.0 | 5.2.0 | 5.2.0 | | Phoenix | 4.14.3-HBase-1.4 | 4.14.3-HBase-1.4 | 4.14.3-HBase-1.4 | 4.14.3-HBase-1.4 | | Pig | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | | Presto | 0.238.3-amzn-0 | 0.232 | 0.232 | 0.232 | | Spark | 2.4.6-amzn-0 | 2.4.5-amzn-0.1 | 2.4.5-amzn-0 | 2.4.5-amzn-0 | | Sqoop | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | | TensorFlow | 2.1.0 | 1.14.0 | 1.14.0 | 1.14.0 | | Tez | 0.9.2 | 0.9.2 | 0.9.2 | 0.9.2 | | Trino (PrestoSQL) | - | - | - | - | | Zeppelin | 0.8.2 | 0.8.2 | 0.8.2 | 0.8.2 | | ZooKeeper | 3.4.14 | 3.4.14 | 3.4.14 | 3.4.14 | ## 5.31.0 release notes The following release notes include information for Amazon EMR release 5.31.0. Changes are relative to 5.30.1. Initial release date: Oct 9, 2020 Last updated date: Oct 15, 2020 **Upgrades** + Upgraded Amazon Glue connector to version 1.13.0 + Upgraded Amazon SageMaker Spark SDK to version 1.4.0 + Upgraded Amazon Kinesis connector to version 3.5.9 + Upgraded AWS SDK for Java to version 1.11.852 + Upgraded Bigtop-tomcat to version 8.5.56 + Upgraded EMR FS to version 2.43.0 + Upgraded EMR MetricsAndEventsApiGateway Client to version 1.4.0 + Upgraded EMR S3 Dist CP to version 2.15.0 + Upgraded EMR S3 Select to version 1.6.0 + Upgraded Flink to version 1.11.0 + Upgraded Hadoop to version 2.10.0 + Upgraded Hive to version 2.3.7 + Upgraded Hudi to version 0.6.0 + Upgraded Hue to version 4.7.1 + Upgraded JupyterHub to version 1.1.0 + Upgraded Mxnet to version 1.6.0 + Upgraded OpenCV to version 4.3.0 + Upgraded Presto to version 0.238.3 + Upgraded TensorFlow to version 2.1.0 **Changes, enhancements, and resolved issues** + This is a release to fix issues with Amazon EMR Scaling when it fails to scale up/scale down a cluster successfully or causes application failures. + Fixed an issue where scaling requests failed for a large, highly utilized cluster when Amazon EMR on-cluster daemons were running health checking activities, such as gathering YARN node state and HDFS node state. This was happening because on-cluster daemons were not able to communicate the health status data of a node to internal Amazon EMR components. + Improved EMR on-cluster daemons to correctly track the node states when IP addresses are reused to improve reliability during scaling operations. + [SPARK-29683](https://issues.apache.org/jira/browse/SPARK-29683). Fixed an issue where job failures occurred during cluster scale-down as Spark was assuming all available nodes were deny-listed. + [YARN-9011](https://issues.apache.org/jira/browse/YARN-9011). Fixed an issue where job failures occurred due to a race condition in YARN decommissioning when cluster tried to scale up or down. + Fixed issue with step or job failures during cluster scaling by ensuring that the node states are always consistent between the Amazon EMR on-cluster daemons and YARN/HDFS. + Fixed an issue where cluster operations such as scale down and step submission failed for Amazon EMR clusters enabled with Kerberos authentication. This was because the Amazon EMR on-cluster daemon did not renew the Kerberos ticket, which is required to securely communicate with HDFS/YARN running on the primary node. + Newer Amazon EMR releases fix the issue with a lower "Max open files" limit on older AL2 in Amazon EMR. Amazon EMR releases 5.30.1, 5.30.2, 5.31.1, 5.32.1, 6.0.1, 6.1.1, 6.2.1, 5.33.0, 6.3.0 and later now include a permanent fix with a higher "Max open files" setting. + [Hive column statistics](https://cwiki.apache.org/confluence/display/Hive/StatsDev#StatsDev-ColumnStatistics) are supported for Amazon EMR versions 5.31.0 and later. + Upgraded component versions. + EMRFS S3EC V2 Support in Amazon EMR 5.31.0. In S3 Java SDK releases 1.11.837 and later, encryption client Version 2 (S3EC V2) has been introduced with various security enhancements. For more information, see the following: + S3 blog post: [Updates to the Amazon S3 encryption client](https://aws.amazon.com/blogs/developer/updates-to-the-amazon-s3-encryption-client/). + AWS SDK for Java Developer Guide: [Migrate encryption and decryption clients to V2](https://docs.aws.amazon.com/sdk-for-java/v1/developer-guide/s3-encryption-migration.html#s3-cse-update-code). + EMR Management Guide: [Amazon S3 client-side encryption](http://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-emrfs-encryption-cse.html). Encryption Client V1 is still available in the SDK for backward compatibility. **New features** + **Lower "Max open files" limit on older AL2 [fixed in newer releases].** Amazon EMR releases: emr-5.30.x, emr-5.31.0, emr-5.32.0, emr-6.0.0, emr-6.1.0, and emr-6.2.0 are based on older versions ofAmazon Linux 2 (AL2), which have a lower ulimit setting for "Max open files" when Amazon EMR clusters are created with the default AMI. Amazon EMR releases 5.30.1, 5.30.2, 5.31.1, 5.32.1, 6.0.1, 6.1.1, 6.2.1, 5.33.0, 6.3.0 and later include a permanent fix with a higher "Max open files" setting. Releases with the lower open file limit causes a "Too many open files" error when submitting Spark job. In the impacted releases, the Amazon EMR default AMI has a default ulimit setting of 4096 for "Max open files," which is lower than the 65536 file limit in the latestAmazon Linux 2 AMI. The lower ulimit setting for "Max open files" causes Spark job failure when the Spark driver and executor try to open more than 4096 files. To fix the issue, Amazon EMR has a bootstrap action (BA) script that adjusts the ulimit setting at cluster creation. If you are using an older Amazon EMR version that doesn't have the permanent fix for this issue, the following workaround lets you to explicitly set the instance-controller ulimit to a maximum of 65536 files. **Explicitly set a ulimit from the command line** 1. Edit `/etc/systemd/system/instance-controller.service` to add the following parameters to Service section. `LimitNOFILE=65536` `LimitNPROC=65536` 1. Restart InstanceController `$ sudo systemctl daemon-reload` `$ sudo systemctl restart instance-controller` **Set a ulimit using bootstrap action (BA)** You can also use a bootstrap action (BA) script to configure the instance-controller ulimit to 65536 files at cluster creation. ``` #!/bin/bash for user in hadoop spark hive; do sudo tee /etc/security/limits.d/$user.conf << EOF $user - nofile 65536 $user - nproc 65536 EOF done for proc in instancecontroller logpusher; do sudo mkdir -p /etc/systemd/system/$proc.service.d/ sudo tee /etc/systemd/system/$proc.service.d/override.conf << EOF [Service] LimitNOFILE=65536 LimitNPROC=65536 EOF pid=$(pgrep -f aws157.$proc.Main) sudo prlimit --pid $pid --nofile=65535:65535 --nproc=65535:65535 done sudo systemctl daemon-reload ``` + With Amazon EMR 5.31.0, you can launch a cluster that integrates with Lake Formation. This integration provides fine-grained, column-level data filtering to databases and tables in the AWS Glue Data Catalog. It also enables federated single sign-on to EMR Notebooks or Apache Zeppelin from an enterprise identity system. For more information, see [Integrating Amazon EMR with AWS Lake Formation](https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-lake-formation.html) in the *Amazon EMR Management Guide*. Amazon EMR with Lake Formation is currently available in 16 AWS Regions: US East (Ohio and N. Virginia), US West (N. California and Oregon), Asia Pacific (Mumbai, Seoul, Singapore, Sydney, and Tokyo), Canada (Central), Europe (Frankfurt, Ireland, London, Paris, and Stockholm), South America (São Paulo). **Known issues** + Known issue in clusters with multiple primary nodes and Kerberos authentication If you run clusters with multiple primary nodes and Kerberos authentication in Amazon EMR releases 5.20.0 and later, you may encounter problems with cluster operations such as scale down or step submission, after the cluster has been running for some time. The time period depends on the Kerberos ticket validity period that you defined. The scale-down problem impacts both automatic scale-down and explicit scale down requests that you submitted. Additional cluster operations can also be impacted. Workaround: + SSH as `hadoop` user to the lead primary node of the EMR cluster with multiple primary nodes. + Run the following command to renew Kerberos ticket for `hadoop` user. ``` kinit -kt ``` Typically, the keytab file is located at `/etc/hadoop.keytab` and the principal is in the form of `hadoop/@`. **Note** This workaround will be effective for the time period the Kerberos ticket is valid. This duration is 10 hours by default, but can configured by your Kerberos settings. You must re-run the above command once the Kerberos ticket expires. + When AtRestEncryption or HDFS encryption is enabled on a cluster that uses Amazon EMR 5.31.0 or 5.32.0, Hive queries result in the following runtime exception. ``` TaskAttempt 3 failed, info=[Error: Error while running task ( failure ) : attempt_1604112648850_0001_1_01_000000_3:java.lang.RuntimeException: java.lang.RuntimeException: Hive Runtime Error while closing operators: java.io.IOException: java.util.ServiceConfigurationError: org.apache.hadoop.security.token.TokenIdentifier: Provider org.apache.hadoop.hbase.security.token.AuthenticationTokenIdentifier not found ``` + When you use Spark with Hive partition location formatting to read data in Amazon S3, and you run Spark on Amazon EMR releases 5.30.0 to 5.36.0, and 6.2.0 to 6.9.0, you might encounter an issue that prevents your cluster from reading data correctly. This can happen if your partitions have all of the following characteristics: + Two or more partitions are scanned from the same table. + At least one partition directory path is a prefix of at least one other partition directory path, for example, `s3://bucket/table/p=a` is a prefix of `s3://bucket/table/p=a b`. + The first character that follows the prefix in the other partition directory has a UTF-8 value that’s less than than the `/` character (U\$1002F). For example, the space character (U\$10020) that occurs between a and b in `s3://bucket/table/p=a b` falls into this category. Note that there are 14 other non-control characters: `!"#$%&‘()*+,-`. For more information, see [UTF-8 encoding table and Unicode characters](https://www.utf8-chartable.de/). As a workaround to this issue, set the `spark.sql.sources.fastS3PartitionDiscovery.enabled` configuration to `false` in the `spark-defaults` classification. ## 5.31.0 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | aws-sagemaker-spark-sdk | 1.4.0 | Amazon SageMaker Spark SDK | | emr-ddb | 4.15.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 2.13.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.5.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-s3-dist-cp | 2.15.0 | Distributed copy application optimized for Amazon S3. | | emr-s3-select | 1.6.0 | EMR S3Select Connector | | emrfs | 2.43.0 | Amazon S3 connector for Hadoop ecosystem applications. | | flink-client | 1.11.0 | Apache Flink command line client scripts and applications. | | flink-jobmanager-config | 1.11.0 | Managing resources on EMR nodes for Apache Flink JobManager. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 2.10.0-amzn-0 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 2.10.0-amzn-0 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 2.10.0-amzn-0 | HDFS command-line client and library | | hadoop-hdfs-namenode | 2.10.0-amzn-0 | HDFS service for tracking file names and block locations. | | hadoop-hdfs-journalnode | 2.10.0-amzn-0 | HDFS service for managing the Hadoop filesystem journal on HA clusters. | | hadoop-httpfs-server | 2.10.0-amzn-0 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 2.10.0-amzn-0 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 2.10.0-amzn-0 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 2.10.0-amzn-0 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 2.10.0-amzn-0 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 2.10.0-amzn-0 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 1.4.13 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 1.4.13 | Service for serving one or more HBase regions. | | hbase-client | 1.4.13 | HBase command-line client. | | hbase-rest-server | 1.4.13 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 1.4.13 | Service providing a Thrift endpoint to HBase. | | hcatalog-client | 2.3.7-amzn-1 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 2.3.7-amzn-1 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 2.3.7-amzn-1 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 2.3.7-amzn-1 | Hive command line client. | | hive-hbase | 2.3.7-amzn-1 | Hive-hbase client. | | hive-metastore-server | 2.3.7-amzn-1 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server2 | 2.3.7-amzn-1 | Service for accepting Hive queries as web requests. | | hudi | 0.6.0-amzn-0 | Incremental processing framework to power data pipline at low latency and high efficiency. | | hudi-spark | 0.6.0-amzn-0 | Bundle library for running Spark with Hudi. | | hudi-presto | 0.6.0-amzn-0 | Bundle library for running Presto with Hudi. | | hue-server | 4.7.1 | Web application for analyzing data using Hadoop ecosystem applications | | jupyterhub | 1.1.0 | Multi-user server for Jupyter notebooks | | livy-server | 0.7.0-incubating | REST interface for interacting with Apache Spark | | nginx | 1.12.1 | nginx [engine x] is an HTTP and reverse proxy server | | mahout-client | 0.13.0 | Library for machine learning. | | mxnet | 1.6.0 | A flexible, scalable, and efficient library for deep learning. | | mariadb-server | 5.5.64 | MySQL database server. | | nvidia-cuda | 9.2.88 | Nvidia drivers and Cuda toolkit | | oozie-client | 5.2.0 | Oozie command-line client. | | oozie-server | 5.2.0 | Service for accepting Oozie workflow requests. | | opencv | 4.3.0 | Open Source Computer Vision Library. | | phoenix-library | 4.14.3-HBase-1.4 | The phoenix libraries for server and client | | phoenix-query-server | 4.14.3-HBase-1.4 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.238.3-amzn-0 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.238.3-amzn-0 | Service for executing pieces of a query. | | presto-client | 0.238.3-amzn-0 | Presto command-line client which is installed on an HA cluster's stand-by masters where Presto server is not started. | | pig-client | 0.17.0 | Pig command-line client. | | r | 3.4.3 | The R Project for Statistical Computing | | ranger-kms-server | 1.2.0 | Apache Ranger Key Management System | | spark-client | 2.4.6-amzn-0 | Spark command-line clients. | | spark-history-server | 2.4.6-amzn-0 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 2.4.6-amzn-0 | In-memory execution engine for YARN. | | spark-yarn-slave | 2.4.6-amzn-0 | Apache Spark libraries needed by YARN slaves. | | sqoop-client | 1.4.7 | Apache Sqoop command-line client. | | tensorflow | 2.1.0 | TensorFlow open source software library for high performance numerical computation. | | tez-on-yarn | 0.9.2 | The tez YARN application and libraries. | | webserver | 2.4.25\$1 | Apache HTTP server. | | zeppelin-server | 0.8.2 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.4.14 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.4.14 | ZooKeeper command line client. | ## 5.31.0 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). **emr-5.31.0 classifications** | Classifications | Description | | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | | container-log4j | Change values in Hadoop YARN's container-log4j.properties file. | | core-site | Change values in Hadoop's core-site.xml file. | | emrfs-site | Change EMRFS settings. | | flink-conf | Change flink-conf.yaml settings. | | flink-log4j | Change Flink log4j.properties settings. | | flink-log4j-yarn-session | Change Flink log4j-yarn-session.properties settings. | | flink-log4j-cli | Change Flink log4j-cli.properties settings. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | | hadoop-ssl-server | Change hadoop ssl server configuration | | hadoop-ssl-client | Change hadoop ssl client configuration | | hbase | Amazon EMR-curated settings for Apache HBase. | | hbase-env | Change values in HBase's environment. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbase.properties file. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | | hbase-site | Change values in HBase's hbase-site.xml file. | | hdfs-encryption-zones | Configure HDFS encryption zones. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | | hcatalog-env | Change values in HCatalog's environment. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | | hcatalog-webhcat-log4j2 | Change values in HCatalog WebHCat's log4j2.properties. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | | hive-beeline-log4j2 | Change values in Hive's beeline-log4j2.properties file. | | hive-parquet-logging | Change values in Hive's parquet-logging.properties file. | | hive-env | Change values in the Hive environment. | | hive-exec-log4j2 | Change values in Hive's hive-exec-log4j2.properties file. | | hive-llap-daemon-log4j2 | Change values in Hive's llap-daemon-log4j2.properties file. | | hive-log4j2 | Change values in Hive's hive-log4j2.properties file. | | hive-site | Change values in Hive's hive-site.xml file | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | | hue-ini | Change values in Hue's ini file | | httpfs-env | Change values in the HTTPFS environment. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | | hudi-env | Change values in the Hudi environment. | | jupyter-notebook-conf | Change values in Jupyter Notebook's jupyter\$1notebook\$1config.py file. | | jupyter-hub-conf | Change values in JupyterHubs's jupyterhub\$1config.py file. | | jupyter-s3-conf | Configure Jupyter Notebook S3 persistence. | | jupyter-sparkmagic-conf | Change values in Sparkmagic's config.json file. | | livy-conf | Change values in Livy's livy.conf file. | | livy-env | Change values in the Livy environment. | | livy-log4j | Change Livy log4j.properties settings. | | mapred-env | Change values in the MapReduce application's environment. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | | oozie-env | Change values in Oozie's environment. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | | oozie-site | Change values in Oozie's oozie-site.xml file. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | | pig-env | Change values in the Pig environment. | | pig-properties | Change values in Pig's pig.properties file. | | pig-log4j | Change values in Pig's log4j.properties file. | | presto-log | Change values in Presto's log.properties file. | | presto-config | Change values in Presto's config.properties file. | | presto-password-authenticator | Change values in Presto's password-authenticator.properties file. | | presto-env | Change values in Presto's presto-env.sh file. | | presto-node | Change values in Presto's node.properties file. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | | presto-connector-hive | Change values in Presto's hive.properties file. | | presto-connector-jmx | Change values in Presto's jmx.properties file. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | | presto-connector-localfile | Change values in Presto's localfile.properties file. | | presto-connector-memory | Change values in Presto's memory.properties file. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | | presto-connector-redis | Change values in Presto's redis.properties file. | | presto-connector-redshift | Change values in Presto's redshift.properties file. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | | presto-connector-tpcds | Change values in Presto's tpcds.properties file. | | ranger-kms-dbks-site | Change values in dbks-site.xml file of Ranger KMS. | | ranger-kms-site | Change values in ranger-kms-site.xml file of Ranger KMS. | | ranger-kms-env | Change values in the Ranger KMS environment. | | ranger-kms-log4j | Change values in kms-log4j.properties file of Ranger KMS. | | ranger-kms-db-ca | Change values for CA file on S3 for MySQL SSL connection with Ranger KMS. | | recordserver-env | Change values in the EMR RecordServer environment. | | recordserver-conf | Change values in EMR RecordServer's erver.properties file. | | recordserver-log4j | Change values in EMR RecordServer's log4j.properties file. | | spark | Amazon EMR-curated settings for Apache Spark. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | | spark-env | Change values in the Spark environment. | | spark-hive-site | Change values in Spark's hive-site.xml file | | spark-log4j | Change values in Spark's log4j.properties file. | | spark-metrics | Change values in Spark's metrics.properties file. | | sqoop-env | Change values in Sqoop's environment. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | | tez-site | Change values in Tez's tez-site.xml file. | | yarn-env | Change values in the YARN environment. | | yarn-site | Change values in YARN's yarn-site.xml file. | | zeppelin-env | Change values in the Zeppelin environment. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | # Amazon EMR release 5.30.2 ## 5.30.2 application versions This release includes the following applications: [https://flink.apache.org/](https://flink.apache.org/), [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [https://hudi.apache.org](https://hudi.apache.org), [http://gethue.com/](http://gethue.com/), [https://jupyterhub.readthedocs.io/en/latest/#](https://jupyterhub.readthedocs.io/en/latest/#), [https://livy.incubator.apache.org/](https://livy.incubator.apache.org/), [https://mxnet.incubator.apache.org/](https://mxnet.incubator.apache.org/), [http://mahout.apache.org/](http://mahout.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://www.tensorflow.org/](https://www.tensorflow.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-5.30.2 | emr-5.30.1 | emr-5.30.0 | emr-5.29.0 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.11.759 | 1.11.759 | 1.11.759 | 1.11.682 | | Python | 2.7, 3.7 | 2.7, 3.7 | 2.7, 3.7 | 2.7, 3.6 | | Scala | 2.11.12 | 2.11.12 | 2.11.12 | 2.11.12 | | AmazonCloudWatchAgent | - | - | - | - | | Delta | - | - | - | - | | Flink | 1.10.0 | 1.10.0 | 1.10.0 | 1.9.1 | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 1.4.13 | 1.4.13 | 1.4.13 | 1.4.10 | | HCatalog | 2.3.6-amzn-2 | 2.3.6-amzn-2 | 2.3.6-amzn-2 | 2.3.6-amzn-1 | | Hadoop | 2.8.5-amzn-6.1 | 2.8.5-amzn-6 | 2.8.5-amzn-6 | 2.8.5-amzn-5 | | Hive | 2.3.6-amzn-2 | 2.3.6-amzn-2 | 2.3.6-amzn-2 | 2.3.6-amzn-1 | | Hudi | 0.5.2-incubating | 0.5.2-incubating | 0.5.2-incubating | 0.5.0-incubating | | Hue | 4.6.0 | 4.6.0 | 4.6.0 | 4.4.0 | | Iceberg | - | - | - | - | | JupyterEnterpriseGateway | - | - | - | - | | JupyterHub | 1.1.0 | 1.1.0 | 1.1.0 | 1.0.0 | | Livy | 0.7.0-incubating | 0.7.0-incubating | 0.7.0-incubating | 0.6.0-incubating | | MXNet | 1.5.1 | 1.5.1 | 1.5.1 | 1.5.1 | | Mahout | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | | Oozie | 5.2.0 | 5.2.0 | 5.2.0 | 5.1.0 | | Phoenix | 4.14.3-HBase-1.4 | 4.14.3-HBase-1.4 | 4.14.3-HBase-1.4 | 4.14.3-HBase-1.4 | | Pig | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | | Presto | 0.232 | 0.232 | 0.232 | 0.227 | | Spark | 2.4.5-amzn-0.1 | 2.4.5-amzn-0 | 2.4.5-amzn-0 | 2.4.4 | | Sqoop | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | | TensorFlow | 1.14.0 | 1.14.0 | 1.14.0 | 1.14.0 | | Tez | 0.9.2 | 0.9.2 | 0.9.2 | 0.9.2 | | Trino (PrestoSQL) | - | - | - | - | | Zeppelin | 0.8.2 | 0.8.2 | 0.8.2 | 0.8.2 | | ZooKeeper | 3.4.14 | 3.4.14 | 3.4.14 | 3.4.14 | ## 5.30.2 release notes This is a release to fix issues with Amazon EMR Scaling when it fails to scale up/scale down a cluster successfully or causes application failures. **Changes, Enhancements, and Resolved Issues** + Fixed an issue where scaling requests failed for a large, highly utilized cluster when Amazon EMR on-cluster daemons were running health checking activities, such as gathering YARN node state and HDFS node state. This was happening because on-cluster daemons were not able to communicate the health status data of a node to internal Amazon EMR components. + Improved EMR on-cluster daemons to correctly track the node states when IP addresses are reused to improve reliability during scaling operations. + [SPARK-29683](https://issues.apache.org/jira/browse/SPARK-29683). Fixed an issue where job failures occurred during cluster scale-down as Spark was assuming all available nodes were deny-listed. + [YARN-9011](https://issues.apache.org/jira/browse/YARN-9011). Fixed an issue where job failures occurred due to a race condition in YARN decommissioning when cluster tried to scale up or down. + Fixed issue with step or job failures during cluster scaling by ensuring that the node states are always consistent between the Amazon EMR on-cluster daemons and YARN/HDFS. + Fixed an issue where cluster operations such as scale down and step submission failed for Amazon EMR clusters enabled with Kerberos authentication. This was because the Amazon EMR on-cluster daemon did not renew the Kerberos ticket, which is required to securely communicate with HDFS/YARN running on the primary node. + Newer Amazon EMR releases fix the issue with a lower "Max open files" limit on older AL2 in Amazon EMR. Amazon EMR releases 5.30.1, 5.30.2, 5.31.1, 5.32.1, 6.0.1, 6.1.1, 6.2.1, 5.33.0, 6.3.0 and later now include a permanent fix with a higher "Max open files" setting. + HTTPS is now enabled by default for Amazon Linux repositories. If you are using an Amazon S3 VPCE policy to restrict access to specific buckets, you must add the new Amazon Linux bucket ARN `arn:aws:s3:::amazonlinux-2-repos-$region/*` to your policy (replace `$region` with the region where the endpoint is). For more information, see this topic in the AWS discussion forums. [Announcement: Amazon Linux 2 now supports the ability to use HTTPS while connecting to package repositories ](https://forums.aws.amazon.com/ann.jspa?annID=8528). **Known issues** + When you use Spark with Hive partition location formatting to read data in Amazon S3, and you run Spark on Amazon EMR releases 5.30.0 to 5.36.0, and 6.2.0 to 6.9.0, you might encounter an issue that prevents your cluster from reading data correctly. This can happen if your partitions have all of the following characteristics: + Two or more partitions are scanned from the same table. + At least one partition directory path is a prefix of at least one other partition directory path, for example, `s3://bucket/table/p=a` is a prefix of `s3://bucket/table/p=a b`. + The first character that follows the prefix in the other partition directory has a UTF-8 value that’s less than than the `/` character (U\$1002F). For example, the space character (U\$10020) that occurs between a and b in `s3://bucket/table/p=a b` falls into this category. Note that there are 14 other non-control characters: `!"#$%&‘()*+,-`. For more information, see [UTF-8 encoding table and Unicode characters](https://www.utf8-chartable.de/). As a workaround to this issue, set the `spark.sql.sources.fastS3PartitionDiscovery.enabled` configuration to `false` in the `spark-defaults` classification. ## 5.30.2 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | aws-sagemaker-spark-sdk | 1.3.0 | Amazon SageMaker Spark SDK | | emr-ddb | 4.14.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 2.13.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.5.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-s3-dist-cp | 2.14.0 | Distributed copy application optimized for Amazon S3. | | emr-s3-select | 1.5.0 | EMR S3Select Connector | | emrfs | 2.40.0 | Amazon S3 connector for Hadoop ecosystem applications. | | flink-client | 1.10.0 | Apache Flink command line client scripts and applications. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 2.8.5-amzn-6.1 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 2.8.5-amzn-6.1 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 2.8.5-amzn-6.1 | HDFS command-line client and library | | hadoop-hdfs-namenode | 2.8.5-amzn-6.1 | HDFS service for tracking file names and block locations. | | hadoop-hdfs-journalnode | 2.8.5-amzn-6.1 | HDFS service for managing the Hadoop filesystem journal on HA clusters. | | hadoop-httpfs-server | 2.8.5-amzn-6.1 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 2.8.5-amzn-6.1 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 2.8.5-amzn-6.1 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 2.8.5-amzn-6.1 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 2.8.5-amzn-6.1 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 2.8.5-amzn-6.1 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 1.4.13 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 1.4.13 | Service for serving one or more HBase regions. | | hbase-client | 1.4.13 | HBase command-line client. | | hbase-rest-server | 1.4.13 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 1.4.13 | Service providing a Thrift endpoint to HBase. | | hcatalog-client | 2.3.6-amzn-2 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 2.3.6-amzn-2 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 2.3.6-amzn-2 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 2.3.6-amzn-2 | Hive command line client. | | hive-hbase | 2.3.6-amzn-2 | Hive-hbase client. | | hive-metastore-server | 2.3.6-amzn-2 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server2 | 2.3.6-amzn-2 | Service for accepting Hive queries as web requests. | | hudi | 0.5.2-incubating | Incremental processing framework to power data pipline at low latency and high efficiency. | | hudi-presto | 0.5.2-incubating | Bundle library for running Presto with Hudi. | | hue-server | 4.6.0 | Web application for analyzing data using Hadoop ecosystem applications | | jupyterhub | 1.1.0 | Multi-user server for Jupyter notebooks | | livy-server | 0.7.0-incubating | REST interface for interacting with Apache Spark | | nginx | 1.12.1 | nginx [engine x] is an HTTP and reverse proxy server | | mahout-client | 0.13.0 | Library for machine learning. | | mxnet | 1.5.1 | A flexible, scalable, and efficient library for deep learning. | | mariadb-server | 5.5.64\$1 | MySQL database server. | | nvidia-cuda | 9.2.88 | Nvidia drivers and Cuda toolkit | | oozie-client | 5.2.0 | Oozie command-line client. | | oozie-server | 5.2.0 | Service for accepting Oozie workflow requests. | | opencv | 3.4.0 | Open Source Computer Vision Library. | | phoenix-library | 4.14.3-HBase-1.4 | The phoenix libraries for server and client | | phoenix-query-server | 4.14.3-HBase-1.4 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.232 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.232 | Service for executing pieces of a query. | | presto-client | 0.232 | Presto command-line client which is installed on an HA cluster's stand-by masters where Presto server is not started. | | pig-client | 0.17.0 | Pig command-line client. | | r | 3.4.3 | The R Project for Statistical Computing | | ranger-kms-server | 1.2.0 | Apache Ranger Key Management System | | spark-client | 2.4.5-amzn-0.1 | Spark command-line clients. | | spark-history-server | 2.4.5-amzn-0.1 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 2.4.5-amzn-0.1 | In-memory execution engine for YARN. | | spark-yarn-slave | 2.4.5-amzn-0.1 | Apache Spark libraries needed by YARN slaves. | | sqoop-client | 1.4.7 | Apache Sqoop command-line client. | | tensorflow | 1.14.0 | TensorFlow open source software library for high performance numerical computation. | | tez-on-yarn | 0.9.2 | The tez YARN application and libraries. | | webserver | 2.4.25\$1 | Apache HTTP server. | | zeppelin-server | 0.8.2 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.4.14 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.4.14 | ZooKeeper command line client. | ## 5.30.2 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). **emr-5.30.2 classifications** | Classifications | Description | | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | | container-log4j | Change values in Hadoop YARN's container-log4j.properties file. | | core-site | Change values in Hadoop's core-site.xml file. | | emrfs-site | Change EMRFS settings. | | flink-conf | Change flink-conf.yaml settings. | | flink-log4j | Change Flink log4j.properties settings. | | flink-log4j-yarn-session | Change Flink log4j-yarn-session.properties settings. | | flink-log4j-cli | Change Flink log4j-cli.properties settings. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | | hadoop-ssl-server | Change hadoop ssl server configuration | | hadoop-ssl-client | Change hadoop ssl client configuration | | hbase | Amazon EMR-curated settings for Apache HBase. | | hbase-env | Change values in HBase's environment. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbase.properties file. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | | hbase-site | Change values in HBase's hbase-site.xml file. | | hdfs-encryption-zones | Configure HDFS encryption zones. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | | hcatalog-env | Change values in HCatalog's environment. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | | hcatalog-webhcat-log4j2 | Change values in HCatalog WebHCat's log4j2.properties. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | | hive-beeline-log4j2 | Change values in Hive's beeline-log4j2.properties file. | | hive-parquet-logging | Change values in Hive's parquet-logging.properties file. | | hive-env | Change values in the Hive environment. | | hive-exec-log4j2 | Change values in Hive's hive-exec-log4j2.properties file. | | hive-llap-daemon-log4j2 | Change values in Hive's llap-daemon-log4j2.properties file. | | hive-log4j2 | Change values in Hive's hive-log4j2.properties file. | | hive-site | Change values in Hive's hive-site.xml file | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | | hue-ini | Change values in Hue's ini file | | httpfs-env | Change values in the HTTPFS environment. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | | hudi-env | Change values in the Hudi environment. | | jupyter-notebook-conf | Change values in Jupyter Notebook's jupyter\$1notebook\$1config.py file. | | jupyter-hub-conf | Change values in JupyterHubs's jupyterhub\$1config.py file. | | jupyter-s3-conf | Configure Jupyter Notebook S3 persistence. | | jupyter-sparkmagic-conf | Change values in Sparkmagic's config.json file. | | livy-conf | Change values in Livy's livy.conf file. | | livy-env | Change values in the Livy environment. | | livy-log4j | Change Livy log4j.properties settings. | | mapred-env | Change values in the MapReduce application's environment. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | | oozie-env | Change values in Oozie's environment. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | | oozie-site | Change values in Oozie's oozie-site.xml file. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | | pig-env | Change values in the Pig environment. | | pig-properties | Change values in Pig's pig.properties file. | | pig-log4j | Change values in Pig's log4j.properties file. | | presto-log | Change values in Presto's log.properties file. | | presto-config | Change values in Presto's config.properties file. | | presto-password-authenticator | Change values in Presto's password-authenticator.properties file. | | presto-env | Change values in Presto's presto-env.sh file. | | presto-node | Change values in Presto's node.properties file. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | | presto-connector-hive | Change values in Presto's hive.properties file. | | presto-connector-jmx | Change values in Presto's jmx.properties file. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | | presto-connector-localfile | Change values in Presto's localfile.properties file. | | presto-connector-memory | Change values in Presto's memory.properties file. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | | presto-connector-redis | Change values in Presto's redis.properties file. | | presto-connector-redshift | Change values in Presto's redshift.properties file. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | | presto-connector-tpcds | Change values in Presto's tpcds.properties file. | | ranger-kms-dbks-site | Change values in dbks-site.xml file of Ranger KMS. | | ranger-kms-site | Change values in ranger-kms-site.xml file of Ranger KMS. | | ranger-kms-env | Change values in the Ranger KMS environment. | | ranger-kms-log4j | Change values in kms-log4j.properties file of Ranger KMS. | | ranger-kms-db-ca | Change values for CA file on S3 for MySQL SSL connection with Ranger KMS. | | recordserver-env | Change values in the EMR RecordServer environment. | | recordserver-conf | Change values in EMR RecordServer's erver.properties file. | | recordserver-log4j | Change values in EMR RecordServer's log4j.properties file. | | spark | Amazon EMR-curated settings for Apache Spark. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | | spark-env | Change values in the Spark environment. | | spark-hive-site | Change values in Spark's hive-site.xml file | | spark-log4j | Change values in Spark's log4j.properties file. | | spark-metrics | Change values in Spark's metrics.properties file. | | sqoop-env | Change values in Sqoop's environment. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | | tez-site | Change values in Tez's tez-site.xml file. | | yarn-env | Change values in the YARN environment. | | yarn-site | Change values in YARN's yarn-site.xml file. | | zeppelin-env | Change values in the Zeppelin environment. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | # Amazon EMR release 5.30.1 ## 5.30.1 application versions This release includes the following applications: [https://flink.apache.org/](https://flink.apache.org/), [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [https://hudi.apache.org](https://hudi.apache.org), [http://gethue.com/](http://gethue.com/), [https://jupyterhub.readthedocs.io/en/latest/#](https://jupyterhub.readthedocs.io/en/latest/#), [https://livy.incubator.apache.org/](https://livy.incubator.apache.org/), [https://mxnet.incubator.apache.org/](https://mxnet.incubator.apache.org/), [http://mahout.apache.org/](http://mahout.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://www.tensorflow.org/](https://www.tensorflow.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-5.30.1 | emr-5.30.0 | emr-5.29.0 | emr-5.28.1 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.11.759 | 1.11.759 | 1.11.682 | 1.11.659 | | Python | 2.7, 3.7 | 2.7, 3.7 | 2.7, 3.6 | 2.7, 3.6 | | Scala | 2.11.12 | 2.11.12 | 2.11.12 | 2.11.12 | | AmazonCloudWatchAgent | - | - | - | - | | Delta | - | - | - | - | | Flink | 1.10.0 | 1.10.0 | 1.9.1 | 1.9.0 | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 1.4.13 | 1.4.13 | 1.4.10 | 1.4.10 | | HCatalog | 2.3.6-amzn-2 | 2.3.6-amzn-2 | 2.3.6-amzn-1 | 2.3.6-amzn-0 | | Hadoop | 2.8.5-amzn-6 | 2.8.5-amzn-6 | 2.8.5-amzn-5 | 2.8.5-amzn-5 | | Hive | 2.3.6-amzn-2 | 2.3.6-amzn-2 | 2.3.6-amzn-1 | 2.3.6-amzn-0 | | Hudi | 0.5.2-incubating | 0.5.2-incubating | 0.5.0-incubating | 0.5.0-incubating | | Hue | 4.6.0 | 4.6.0 | 4.4.0 | 4.4.0 | | Iceberg | - | - | - | - | | JupyterEnterpriseGateway | - | - | - | - | | JupyterHub | 1.1.0 | 1.1.0 | 1.0.0 | 1.0.0 | | Livy | 0.7.0-incubating | 0.7.0-incubating | 0.6.0-incubating | 0.6.0-incubating | | MXNet | 1.5.1 | 1.5.1 | 1.5.1 | 1.5.1 | | Mahout | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | | Oozie | 5.2.0 | 5.2.0 | 5.1.0 | 5.1.0 | | Phoenix | 4.14.3-HBase-1.4 | 4.14.3-HBase-1.4 | 4.14.3-HBase-1.4 | 4.14.3-HBase-1.4 | | Pig | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | | Presto | 0.232 | 0.232 | 0.227 | 0.227 | | Spark | 2.4.5-amzn-0 | 2.4.5-amzn-0 | 2.4.4 | 2.4.4 | | Sqoop | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | | TensorFlow | 1.14.0 | 1.14.0 | 1.14.0 | 1.14.0 | | Tez | 0.9.2 | 0.9.2 | 0.9.2 | 0.9.2 | | Trino (PrestoSQL) | - | - | - | - | | Zeppelin | 0.8.2 | 0.8.2 | 0.8.2 | 0.8.2 | | ZooKeeper | 3.4.14 | 3.4.14 | 3.4.14 | 3.4.14 | ## 5.30.1 release notes The following release notes include information for Amazon EMR release 5.30.1. Changes are relative to 5.30.0. Initial release date: June 30, 2020 Last updated date: August 24, 2020 **Changes, enhancements, and resolved issues** + Newer Amazon EMR releases fix the issue with a lower "Max open files" limit on older AL2 in Amazon EMR. Amazon EMR releases 5.30.1, 5.30.2, 5.31.1, 5.32.1, 6.0.1, 6.1.1, 6.2.1, 5.33.0, 6.3.0 and later now include a permanent fix with a higher "Max open files" setting. + Fixed issue where instance controller process spawned infinite number of processes. + Fixed issue where Hue was unable to run an Hive query, showing a "database is locked" message and preventing the execution of queries. + Fixed a Spark issue to enable more tasks to run concurrently on the EMR cluster. + Fixed a Jupyter notebook issue causing a "too many files open error" in the Jupyter server. + Fixed an issue with cluster start times. **New features** + Tez UI and YARN timeline server persistent application interfaces are available with Amazon EMR versions 6.x, and EMR version 5.30.1 and later. One-click link access to persistent application history lets you quickly access job history without setting up a web proxy through an SSH connection. Logs for active and terminated clusters are available for 30 days after the application ends. For more information, see [View Persistent Application User Interfaces](https://docs.aws.amazon.com/emr/latest/ManagementGuide/app-history-spark-UI.html) in the *Amazon EMR Management Guide*. + EMR Notebook execution APIs are available to execute EMR notebooks via a script or command line. The ability to start, stop, list, and describe EMR notebook executions without the AWS console enables you programmatically control an EMR notebook. Using a parameterized notebook cell, you can pass different parameter values to a notebook without having to create a copy of the notebook for each new set of paramter values. See [EMR API Actions.](https://docs.aws.amazon.com/emr/latest/APIReference/API_Operations.html) For sample code, see [Sample commands to execute EMR Notebooks programmatically.](https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-managed-notebooks-headless.html) **Known issues** + **Lower "Max open files" limit on older AL2 [fixed in newer releases].** Amazon EMR releases: emr-5.30.x, emr-5.31.0, emr-5.32.0, emr-6.0.0, emr-6.1.0, and emr-6.2.0 are based on older versions ofAmazon Linux 2 (AL2), which have a lower ulimit setting for "Max open files" when Amazon EMR clusters are created with the default AMI. Amazon EMR releases 5.30.1, 5.30.2, 5.31.1, 5.32.1, 6.0.1, 6.1.1, 6.2.1, 5.33.0, 6.3.0 and later include a permanent fix with a higher "Max open files" setting. Releases with the lower open file limit causes a "Too many open files" error when submitting Spark job. In the impacted releases, the Amazon EMR default AMI has a default ulimit setting of 4096 for "Max open files," which is lower than the 65536 file limit in the latestAmazon Linux 2 AMI. The lower ulimit setting for "Max open files" causes Spark job failure when the Spark driver and executor try to open more than 4096 files. To fix the issue, Amazon EMR has a bootstrap action (BA) script that adjusts the ulimit setting at cluster creation. If you are using an older Amazon EMR version that doesn't have the permanent fix for this issue, the following workaround lets you to explicitly set the instance-controller ulimit to a maximum of 65536 files. **Explicitly set a ulimit from the command line** 1. Edit `/etc/systemd/system/instance-controller.service` to add the following parameters to Service section. `LimitNOFILE=65536` `LimitNPROC=65536` 1. Restart InstanceController `$ sudo systemctl daemon-reload` `$ sudo systemctl restart instance-controller` **Set a ulimit using bootstrap action (BA)** You can also use a bootstrap action (BA) script to configure the instance-controller ulimit to 65536 files at cluster creation. ``` #!/bin/bash for user in hadoop spark hive; do sudo tee /etc/security/limits.d/$user.conf << EOF $user - nofile 65536 $user - nproc 65536 EOF done for proc in instancecontroller logpusher; do sudo mkdir -p /etc/systemd/system/$proc.service.d/ sudo tee /etc/systemd/system/$proc.service.d/override.conf << EOF [Service] LimitNOFILE=65536 LimitNPROC=65536 EOF pid=$(pgrep -f aws157.$proc.Main) sudo prlimit --pid $pid --nofile=65535:65535 --nproc=65535:65535 done sudo systemctl daemon-reload ``` + **EMR Notebooks** The feature that allows you to install kernels and additional Python libraries on the cluster primary node is disabled by default on EMR version 5.30.1. For more information about this feature, see [Installing Kernels and Python Libraries on a Cluster Primary Node](https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-managed-notebooks-installing-libraries-and-kernels.html). To enable the feature, do the following: 1. Make sure that the permissions policy attached to the service role for EMR Notebooks allows the following action: `elasticmapreduce:ListSteps` For more information, see [Service Role for EMR Notebooks](https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-managed-notebooks-service-role.html). 1. Use the AWS CLI to run a step on the cluster that sets up EMR Notebooks as shown in the following example. Replace *us-east-1* with the Region in which your cluster resides. For more information, see [Adding Steps to a Cluster Using the AWS CLI](https://docs.aws.amazon.com/emr/latest/ManagementGuide/add-step-cli.html). ``` aws emr add-steps --cluster-id MyClusterID --steps Type=CUSTOM_JAR,Name=EMRNotebooksSetup,ActionOnFailure=CONTINUE,Jar=s3://us-east-1.elasticmapreduce/libs/script-runner/script-runner.jar,Args=["s3://awssupportdatasvcs.com/bootstrap-actions/EMRNotebooksSetup/emr-notebooks-setup.sh"] ``` + **Managed scaling** Managed scaling operations on 5.30.0 and 5.30.1 clusters without Presto installed may cause application failures or cause a uniform instance group or instance fleet to stay in the `ARRESTED` state, particularly when a scale down operation is followed quickly by a scale up operation. As a workaround, choose Presto as an application to install when you create a cluster with Amazon EMR releases 5.30.0 and 5.30.1, even if your job does not require Presto. + Known issue in clusters with multiple primary nodes and Kerberos authentication If you run clusters with multiple primary nodes and Kerberos authentication in Amazon EMR releases 5.20.0 and later, you may encounter problems with cluster operations such as scale down or step submission, after the cluster has been running for some time. The time period depends on the Kerberos ticket validity period that you defined. The scale-down problem impacts both automatic scale-down and explicit scale down requests that you submitted. Additional cluster operations can also be impacted. Workaround: + SSH as `hadoop` user to the lead primary node of the EMR cluster with multiple primary nodes. + Run the following command to renew Kerberos ticket for `hadoop` user. ``` kinit -kt ``` Typically, the keytab file is located at `/etc/hadoop.keytab` and the principal is in the form of `hadoop/@`. **Note** This workaround will be effective for the time period the Kerberos ticket is valid. This duration is 10 hours by default, but can configured by your Kerberos settings. You must re-run the above command once the Kerberos ticket expires. + When you use Spark with Hive partition location formatting to read data in Amazon S3, and you run Spark on Amazon EMR releases 5.30.0 to 5.36.0, and 6.2.0 to 6.9.0, you might encounter an issue that prevents your cluster from reading data correctly. This can happen if your partitions have all of the following characteristics: + Two or more partitions are scanned from the same table. + At least one partition directory path is a prefix of at least one other partition directory path, for example, `s3://bucket/table/p=a` is a prefix of `s3://bucket/table/p=a b`. + The first character that follows the prefix in the other partition directory has a UTF-8 value that’s less than than the `/` character (U\$1002F). For example, the space character (U\$10020) that occurs between a and b in `s3://bucket/table/p=a b` falls into this category. Note that there are 14 other non-control characters: `!"#$%&‘()*+,-`. For more information, see [UTF-8 encoding table and Unicode characters](https://www.utf8-chartable.de/). As a workaround to this issue, set the `spark.sql.sources.fastS3PartitionDiscovery.enabled` configuration to `false` in the `spark-defaults` classification. ## 5.30.1 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | aws-sagemaker-spark-sdk | 1.3.0 | Amazon SageMaker Spark SDK | | emr-ddb | 4.14.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 2.13.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.5.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-s3-dist-cp | 2.14.0 | Distributed copy application optimized for Amazon S3. | | emr-s3-select | 1.5.0 | EMR S3Select Connector | | emrfs | 2.40.0 | Amazon S3 connector for Hadoop ecosystem applications. | | flink-client | 1.10.0 | Apache Flink command line client scripts and applications. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 2.8.5-amzn-6 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 2.8.5-amzn-6 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 2.8.5-amzn-6 | HDFS command-line client and library | | hadoop-hdfs-namenode | 2.8.5-amzn-6 | HDFS service for tracking file names and block locations. | | hadoop-hdfs-journalnode | 2.8.5-amzn-6 | HDFS service for managing the Hadoop filesystem journal on HA clusters. | | hadoop-httpfs-server | 2.8.5-amzn-6 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 2.8.5-amzn-6 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 2.8.5-amzn-6 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 2.8.5-amzn-6 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 2.8.5-amzn-6 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 2.8.5-amzn-6 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 1.4.13 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 1.4.13 | Service for serving one or more HBase regions. | | hbase-client | 1.4.13 | HBase command-line client. | | hbase-rest-server | 1.4.13 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 1.4.13 | Service providing a Thrift endpoint to HBase. | | hcatalog-client | 2.3.6-amzn-2 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 2.3.6-amzn-2 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 2.3.6-amzn-2 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 2.3.6-amzn-2 | Hive command line client. | | hive-hbase | 2.3.6-amzn-2 | Hive-hbase client. | | hive-metastore-server | 2.3.6-amzn-2 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server2 | 2.3.6-amzn-2 | Service for accepting Hive queries as web requests. | | hudi | 0.5.2-incubating | Incremental processing framework to power data pipline at low latency and high efficiency. | | hudi-presto | 0.5.2-incubating | Bundle library for running Presto with Hudi. | | hue-server | 4.6.0 | Web application for analyzing data using Hadoop ecosystem applications | | jupyterhub | 1.1.0 | Multi-user server for Jupyter notebooks | | livy-server | 0.7.0-incubating | REST interface for interacting with Apache Spark | | nginx | 1.12.1 | nginx [engine x] is an HTTP and reverse proxy server | | mahout-client | 0.13.0 | Library for machine learning. | | mxnet | 1.5.1 | A flexible, scalable, and efficient library for deep learning. | | mariadb-server | 5.5.64 | MySQL database server. | | nvidia-cuda | 9.2.88 | Nvidia drivers and Cuda toolkit | | oozie-client | 5.2.0 | Oozie command-line client. | | oozie-server | 5.2.0 | Service for accepting Oozie workflow requests. | | opencv | 3.4.0 | Open Source Computer Vision Library. | | phoenix-library | 4.14.3-HBase-1.4 | The phoenix libraries for server and client | | phoenix-query-server | 4.14.3-HBase-1.4 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.232 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.232 | Service for executing pieces of a query. | | presto-client | 0.232 | Presto command-line client which is installed on an HA cluster's stand-by masters where Presto server is not started. | | pig-client | 0.17.0 | Pig command-line client. | | r | 3.4.3 | The R Project for Statistical Computing | | ranger-kms-server | 1.2.0 | Apache Ranger Key Management System | | spark-client | 2.4.5-amzn-0 | Spark command-line clients. | | spark-history-server | 2.4.5-amzn-0 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 2.4.5-amzn-0 | In-memory execution engine for YARN. | | spark-yarn-slave | 2.4.5-amzn-0 | Apache Spark libraries needed by YARN slaves. | | sqoop-client | 1.4.7 | Apache Sqoop command-line client. | | tensorflow | 1.14.0 | TensorFlow open source software library for high performance numerical computation. | | tez-on-yarn | 0.9.2 | The tez YARN application and libraries. | | webserver | 2.4.25\$1 | Apache HTTP server. | | zeppelin-server | 0.8.2 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.4.14 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.4.14 | ZooKeeper command line client. | ## 5.30.1 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). **emr-5.30.1 classifications** | Classifications | Description | | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | | container-log4j | Change values in Hadoop YARN's container-log4j.properties file. | | core-site | Change values in Hadoop's core-site.xml file. | | emrfs-site | Change EMRFS settings. | | flink-conf | Change flink-conf.yaml settings. | | flink-log4j | Change Flink log4j.properties settings. | | flink-log4j-yarn-session | Change Flink log4j-yarn-session.properties settings. | | flink-log4j-cli | Change Flink log4j-cli.properties settings. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | | hadoop-ssl-server | Change hadoop ssl server configuration | | hadoop-ssl-client | Change hadoop ssl client configuration | | hbase | Amazon EMR-curated settings for Apache HBase. | | hbase-env | Change values in HBase's environment. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbase.properties file. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | | hbase-site | Change values in HBase's hbase-site.xml file. | | hdfs-encryption-zones | Configure HDFS encryption zones. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | | hcatalog-env | Change values in HCatalog's environment. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | | hcatalog-webhcat-log4j2 | Change values in HCatalog WebHCat's log4j2.properties. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | | hive-beeline-log4j2 | Change values in Hive's beeline-log4j2.properties file. | | hive-parquet-logging | Change values in Hive's parquet-logging.properties file. | | hive-env | Change values in the Hive environment. | | hive-exec-log4j2 | Change values in Hive's hive-exec-log4j2.properties file. | | hive-llap-daemon-log4j2 | Change values in Hive's llap-daemon-log4j2.properties file. | | hive-log4j2 | Change values in Hive's hive-log4j2.properties file. | | hive-site | Change values in Hive's hive-site.xml file | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | | hue-ini | Change values in Hue's ini file | | httpfs-env | Change values in the HTTPFS environment. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | | hudi-env | Change values in the Hudi environment. | | jupyter-notebook-conf | Change values in Jupyter Notebook's jupyter\$1notebook\$1config.py file. | | jupyter-hub-conf | Change values in JupyterHubs's jupyterhub\$1config.py file. | | jupyter-s3-conf | Configure Jupyter Notebook S3 persistence. | | jupyter-sparkmagic-conf | Change values in Sparkmagic's config.json file. | | livy-conf | Change values in Livy's livy.conf file. | | livy-env | Change values in the Livy environment. | | livy-log4j | Change Livy log4j.properties settings. | | mapred-env | Change values in the MapReduce application's environment. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | | oozie-env | Change values in Oozie's environment. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | | oozie-site | Change values in Oozie's oozie-site.xml file. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | | pig-env | Change values in the Pig environment. | | pig-properties | Change values in Pig's pig.properties file. | | pig-log4j | Change values in Pig's log4j.properties file. | | presto-log | Change values in Presto's log.properties file. | | presto-config | Change values in Presto's config.properties file. | | presto-password-authenticator | Change values in Presto's password-authenticator.properties file. | | presto-env | Change values in Presto's presto-env.sh file. | | presto-node | Change values in Presto's node.properties file. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | | presto-connector-hive | Change values in Presto's hive.properties file. | | presto-connector-jmx | Change values in Presto's jmx.properties file. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | | presto-connector-localfile | Change values in Presto's localfile.properties file. | | presto-connector-memory | Change values in Presto's memory.properties file. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | | presto-connector-redis | Change values in Presto's redis.properties file. | | presto-connector-redshift | Change values in Presto's redshift.properties file. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | | presto-connector-tpcds | Change values in Presto's tpcds.properties file. | | ranger-kms-dbks-site | Change values in dbks-site.xml file of Ranger KMS. | | ranger-kms-site | Change values in ranger-kms-site.xml file of Ranger KMS. | | ranger-kms-env | Change values in the Ranger KMS environment. | | ranger-kms-log4j | Change values in kms-log4j.properties file of Ranger KMS. | | ranger-kms-db-ca | Change values for CA file on S3 for MySQL SSL connection with Ranger KMS. | | recordserver-env | Change values in the EMR RecordServer environment. | | recordserver-conf | Change values in EMR RecordServer's erver.properties file. | | recordserver-log4j | Change values in EMR RecordServer's log4j.properties file. | | spark | Amazon EMR-curated settings for Apache Spark. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | | spark-env | Change values in the Spark environment. | | spark-hive-site | Change values in Spark's hive-site.xml file | | spark-log4j | Change values in Spark's log4j.properties file. | | spark-metrics | Change values in Spark's metrics.properties file. | | sqoop-env | Change values in Sqoop's environment. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | | tez-site | Change values in Tez's tez-site.xml file. | | yarn-env | Change values in the YARN environment. | | yarn-site | Change values in YARN's yarn-site.xml file. | | zeppelin-env | Change values in the Zeppelin environment. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | # Amazon EMR release 5.30.0 ## 5.30.0 application versions This release includes the following applications: [https://flink.apache.org/](https://flink.apache.org/), [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [https://hudi.apache.org](https://hudi.apache.org), [http://gethue.com/](http://gethue.com/), [https://jupyterhub.readthedocs.io/en/latest/#](https://jupyterhub.readthedocs.io/en/latest/#), [https://livy.incubator.apache.org/](https://livy.incubator.apache.org/), [https://mxnet.incubator.apache.org/](https://mxnet.incubator.apache.org/), [http://mahout.apache.org/](http://mahout.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://www.tensorflow.org/](https://www.tensorflow.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-5.30.0 | emr-5.29.0 | emr-5.28.1 | emr-5.28.0 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.11.759 | 1.11.682 | 1.11.659 | 1.11.659 | | Python | 2.7, 3.7 | 2.7, 3.6 | 2.7, 3.6 | 2.7, 3.6 | | Scala | 2.11.12 | 2.11.12 | 2.11.12 | 2.11.12 | | AmazonCloudWatchAgent | - | - | - | - | | Delta | - | - | - | - | | Flink | 1.10.0 | 1.9.1 | 1.9.0 | 1.9.0 | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 1.4.13 | 1.4.10 | 1.4.10 | 1.4.10 | | HCatalog | 2.3.6-amzn-2 | 2.3.6-amzn-1 | 2.3.6-amzn-0 | 2.3.6-amzn-0 | | Hadoop | 2.8.5-amzn-6 | 2.8.5-amzn-5 | 2.8.5-amzn-5 | 2.8.5-amzn-5 | | Hive | 2.3.6-amzn-2 | 2.3.6-amzn-1 | 2.3.6-amzn-0 | 2.3.6-amzn-0 | | Hudi | 0.5.2-incubating | 0.5.0-incubating | 0.5.0-incubating | 0.5.0-incubating | | Hue | 4.6.0 | 4.4.0 | 4.4.0 | 4.4.0 | | Iceberg | - | - | - | - | | JupyterEnterpriseGateway | - | - | - | - | | JupyterHub | 1.1.0 | 1.0.0 | 1.0.0 | 1.0.0 | | Livy | 0.7.0-incubating | 0.6.0-incubating | 0.6.0-incubating | 0.6.0-incubating | | MXNet | 1.5.1 | 1.5.1 | 1.5.1 | 1.5.1 | | Mahout | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | | Oozie | 5.2.0 | 5.1.0 | 5.1.0 | 5.1.0 | | Phoenix | 4.14.3-HBase-1.4 | 4.14.3-HBase-1.4 | 4.14.3-HBase-1.4 | 4.14.3-HBase-1.4 | | Pig | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | | Presto | 0.232 | 0.227 | 0.227 | 0.227 | | Spark | 2.4.5-amzn-0 | 2.4.4 | 2.4.4 | 2.4.4 | | Sqoop | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | | TensorFlow | 1.14.0 | 1.14.0 | 1.14.0 | 1.14.0 | | Tez | 0.9.2 | 0.9.2 | 0.9.2 | 0.9.2 | | Trino (PrestoSQL) | - | - | - | - | | Zeppelin | 0.8.2 | 0.8.2 | 0.8.2 | 0.8.2 | | ZooKeeper | 3.4.14 | 3.4.14 | 3.4.14 | 3.4.14 | ## 5.30.0 release notes The following release notes include information for Amazon EMR release 5.30.0. Changes are relative to 5.29.0. Initial release date: May 13, 2020 Last updated date: June 25, 2020 **Upgrades** + Upgraded AWS SDK for Java to version 1.11.759 + Upgraded Amazon SageMaker Spark SDK to version 1.3.0 + Upgraded EMR Record Server to version 1.6.0 + Upgraded Flink to version 1.10.0 + Upgraded Ganglia to version 3.7.2 + Upgraded HBase to version 1.4.13 + Upgraded Hudi to version 0.5.2-incubating + Upgraded Hue to version 4.6.0 + Upgraded JupyterHub to version 1.1.0 + Upgraded Livy to version 0.7.0-incubating + Upgraded Oozie to version 5.2.0 + Upgraded Presto to version 0.232 + Upgraded Spark to version 2.4.5 + Upgraded Connectors and drivers: Amazon Glue Connector 1.12.0; Amazon Kinesis Connector 3.5.0; EMR DynamoDB Connector 4.14.0 **New features** + **EMR Notebooks** – When used with EMR clusters created using 5.30.0, EMR notebook kernels run on cluster. This improves notebook performance and allows you to install and customize kernels. You can also install Python libraries on the cluster primary node. For more information, see [Installing and Using Kernels and Libraries](https://docs.aws.amazon.com//emr/latest/ManagementGuide/emr-managed-notebooks-installing-libraries-and-kernels.html) in the *EMR Management Guide*. + **Managed Scaling** – With Amazon EMR version 5.30.0 and later, you can enable EMR managed scaling to automatically increase or decrease the number of instances or units in your cluster based on workload. Amazon EMR continuously evaluates cluster metrics to make scaling decisions that optimize your clusters for cost and speed. For more information, see [Scaling Cluster Resources](https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-scale-on-demand.html) in the *Amazon EMR Management Guide*. + **Encrypt log files stored in Amazon S3** – With Amazon EMR version 5.30.0 and later, you can encrypt log files stored in Amazon S3 with an AWS KMS customer managed key. For more information, see [Encrypt log files stored in Amazon S3](https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-plan-debugging.html#emr-log-encryption) in the *Amazon EMR Management Guide*. + **Amazon Linux 2 support** – In EMR version 5.30.0 and later, EMR usesAmazon Linux 2 OS. New custom AMIs (Amazon Machine Image) must be based on theAmazon Linux 2 AMI. For more information, see [Using a Custom AMI](https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-custom-ami.html). + **Presto Graceful Auto Scale** – EMR clusters using 5.30.0 can be set with an auto scaling timeout period that gives Presto tasks time to finish running before their node is decommissioned. For more information, see [Using Presto automatic scaling with Graceful Decommission](presto-graceful-autoscale.md). + **Fleet Instance creation with new allocation strategy option** – A new allocation strategy option is available in EMR version 5.12.1 and later. It offers faster cluster provisioning, more accurate spot allocation, and less spot instance interruption. Updates to non-default EMR service roles are required. See [Configure Instance Fleets](https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-instance-fleet.html). + **sudo systemctl stop and sudo systemctl start commands** – In EMR version 5.30.0 and later, which useAmazon Linux 2 OS, EMR uses `sudo systemctl stop` and `sudo systemctl start` commands to restart services. For more information, see [How do I restart a service in Amazon EMR?](https://aws.amazon.com/premiumsupport/knowledge-center/restart-service-emr/). **Changes, enhancements, and resolved issues** + EMR version 5.30.0 doesn't install Ganglia by default. You can explicitly select Ganglia to install when you create a cluster. + Spark performance optimizations. + Presto performance optimizations. + Python 3 is the default for Amazon EMR version 5.30.0 and later. + The default managed security group for service access in private subnets has been updated with new rules. If you use a custom security group for service access, you must include the same rules as the default managed security group. For more information, see [Amazon EMR-Managed Security Group for Service Access (Private Subnets)](https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-man-sec-groups.html#emr-sg-elasticmapreduce-sa-private). If you use a custom service role for Amazon EMR, you must grant permission to `ec2:describeSecurityGroups` so that EMR can validate if the security groups are correctly created. If you use the `EMR_DefaultRole`, this permission is already included in the default managed policy. **Known issues** + **Lower "Max open files" limit on older AL2 [fixed in newer releases].** Amazon EMR releases: emr-5.30.x, emr-5.31.0, emr-5.32.0, emr-6.0.0, emr-6.1.0, and emr-6.2.0 are based on older versions ofAmazon Linux 2 (AL2), which have a lower ulimit setting for "Max open files" when Amazon EMR clusters are created with the default AMI. Amazon EMR releases 5.30.1, 5.30.2, 5.31.1, 5.32.1, 6.0.1, 6.1.1, 6.2.1, 5.33.0, 6.3.0 and later include a permanent fix with a higher "Max open files" setting. Releases with the lower open file limit causes a "Too many open files" error when submitting Spark job. In the impacted releases, the Amazon EMR default AMI has a default ulimit setting of 4096 for "Max open files," which is lower than the 65536 file limit in the latestAmazon Linux 2 AMI. The lower ulimit setting for "Max open files" causes Spark job failure when the Spark driver and executor try to open more than 4096 files. To fix the issue, Amazon EMR has a bootstrap action (BA) script that adjusts the ulimit setting at cluster creation. If you are using an older Amazon EMR version that doesn't have the permanent fix for this issue, the following workaround lets you to explicitly set the instance-controller ulimit to a maximum of 65536 files. **Explicitly set a ulimit from the command line** 1. Edit `/etc/systemd/system/instance-controller.service` to add the following parameters to Service section. `LimitNOFILE=65536` `LimitNPROC=65536` 1. Restart InstanceController `$ sudo systemctl daemon-reload` `$ sudo systemctl restart instance-controller` **Set a ulimit using bootstrap action (BA)** You can also use a bootstrap action (BA) script to configure the instance-controller ulimit to 65536 files at cluster creation. ``` #!/bin/bash for user in hadoop spark hive; do sudo tee /etc/security/limits.d/$user.conf << EOF $user - nofile 65536 $user - nproc 65536 EOF done for proc in instancecontroller logpusher; do sudo mkdir -p /etc/systemd/system/$proc.service.d/ sudo tee /etc/systemd/system/$proc.service.d/override.conf << EOF [Service] LimitNOFILE=65536 LimitNPROC=65536 EOF pid=$(pgrep -f aws157.$proc.Main) sudo prlimit --pid $pid --nofile=65535:65535 --nproc=65535:65535 done sudo systemctl daemon-reload ``` + **Managed scaling** Managed scaling operations on 5.30.0 and 5.30.1 clusters without Presto installed may cause application failures or cause a uniform instance group or instance fleet to stay in the `ARRESTED` state, particularly when a scale down operation is followed quickly by a scale up operation. As a workaround, choose Presto as an application to install when you create a cluster with Amazon EMR releases 5.30.0 and 5.30.1, even if your job does not require Presto. + Known issue in clusters with multiple primary nodes and Kerberos authentication If you run clusters with multiple primary nodes and Kerberos authentication in Amazon EMR releases 5.20.0 and later, you may encounter problems with cluster operations such as scale down or step submission, after the cluster has been running for some time. The time period depends on the Kerberos ticket validity period that you defined. The scale-down problem impacts both automatic scale-down and explicit scale down requests that you submitted. Additional cluster operations can also be impacted. Workaround: + SSH as `hadoop` user to the lead primary node of the EMR cluster with multiple primary nodes. + Run the following command to renew Kerberos ticket for `hadoop` user. ``` kinit -kt ``` Typically, the keytab file is located at `/etc/hadoop.keytab` and the principal is in the form of `hadoop/@`. **Note** This workaround will be effective for the time period the Kerberos ticket is valid. This duration is 10 hours by default, but can configured by your Kerberos settings. You must re-run the above command once the Kerberos ticket expires. + The default database engine for Hue 4.6.0 is SQLite, which causes issues when you try to use Hue with an external database. To fix this, set `engine` in your `hue-ini` configuration classification to `mysql`. This issue has been fixed in Amazon EMR version 5.30.1. + When you use Spark with Hive partition location formatting to read data in Amazon S3, and you run Spark on Amazon EMR releases 5.30.0 to 5.36.0, and 6.2.0 to 6.9.0, you might encounter an issue that prevents your cluster from reading data correctly. This can happen if your partitions have all of the following characteristics: + Two or more partitions are scanned from the same table. + At least one partition directory path is a prefix of at least one other partition directory path, for example, `s3://bucket/table/p=a` is a prefix of `s3://bucket/table/p=a b`. + The first character that follows the prefix in the other partition directory has a UTF-8 value that’s less than than the `/` character (U\$1002F). For example, the space character (U\$10020) that occurs between a and b in `s3://bucket/table/p=a b` falls into this category. Note that there are 14 other non-control characters: `!"#$%&‘()*+,-`. For more information, see [UTF-8 encoding table and Unicode characters](https://www.utf8-chartable.de/). As a workaround to this issue, set the `spark.sql.sources.fastS3PartitionDiscovery.enabled` configuration to `false` in the `spark-defaults` classification. ## 5.30.0 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | aws-sagemaker-spark-sdk | 1.3.0 | Amazon SageMaker Spark SDK | | emr-ddb | 4.14.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 2.13.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.5.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-notebook-env | 1.0.0 | Conda env for emr notebook | | emr-s3-dist-cp | 2.14.0 | Distributed copy application optimized for Amazon S3. | | emr-s3-select | 1.5.0 | EMR S3Select Connector | | emrfs | 2.40.0 | Amazon S3 connector for Hadoop ecosystem applications. | | flink-client | 1.10.0 | Apache Flink command line client scripts and applications. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 2.8.5-amzn-6 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 2.8.5-amzn-6 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 2.8.5-amzn-6 | HDFS command-line client and library | | hadoop-hdfs-namenode | 2.8.5-amzn-6 | HDFS service for tracking file names and block locations. | | hadoop-hdfs-journalnode | 2.8.5-amzn-6 | HDFS service for managing the Hadoop filesystem journal on HA clusters. | | hadoop-httpfs-server | 2.8.5-amzn-6 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 2.8.5-amzn-6 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 2.8.5-amzn-6 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 2.8.5-amzn-6 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 2.8.5-amzn-6 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 2.8.5-amzn-6 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 1.4.13 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 1.4.13 | Service for serving one or more HBase regions. | | hbase-client | 1.4.13 | HBase command-line client. | | hbase-rest-server | 1.4.13 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 1.4.13 | Service providing a Thrift endpoint to HBase. | | hcatalog-client | 2.3.6-amzn-2 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 2.3.6-amzn-2 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 2.3.6-amzn-2 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 2.3.6-amzn-2 | Hive command line client. | | hive-hbase | 2.3.6-amzn-2 | Hive-hbase client. | | hive-metastore-server | 2.3.6-amzn-2 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server2 | 2.3.6-amzn-2 | Service for accepting Hive queries as web requests. | | hudi | 0.5.2-incubating | Incremental processing framework to power data pipline at low latency and high efficiency. | | hudi-presto | 0.5.2-incubating | Bundle library for running Presto with Hudi. | | hue-server | 4.6.0 | Web application for analyzing data using Hadoop ecosystem applications | | jupyterhub | 1.1.0 | Multi-user server for Jupyter notebooks | | livy-server | 0.7.0-incubating | REST interface for interacting with Apache Spark | | nginx | 1.12.1 | nginx [engine x] is an HTTP and reverse proxy server | | mahout-client | 0.13.0 | Library for machine learning. | | mxnet | 1.5.1 | A flexible, scalable, and efficient library for deep learning. | | mariadb-server | 5.5.64 | MySQL database server. | | nvidia-cuda | 9.2.88 | Nvidia drivers and Cuda toolkit | | oozie-client | 5.2.0 | Oozie command-line client. | | oozie-server | 5.2.0 | Service for accepting Oozie workflow requests. | | opencv | 3.4.0 | Open Source Computer Vision Library. | | phoenix-library | 4.14.3-HBase-1.4 | The phoenix libraries for server and client | | phoenix-query-server | 4.14.3-HBase-1.4 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.232 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.232 | Service for executing pieces of a query. | | presto-client | 0.232 | Presto command-line client which is installed on an HA cluster's stand-by masters where Presto server is not started. | | pig-client | 0.17.0 | Pig command-line client. | | r | 3.4.3 | The R Project for Statistical Computing | | ranger-kms-server | 1.2.0 | Apache Ranger Key Management System | | spark-client | 2.4.5-amzn-0 | Spark command-line clients. | | spark-history-server | 2.4.5-amzn-0 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 2.4.5-amzn-0 | In-memory execution engine for YARN. | | spark-yarn-slave | 2.4.5-amzn-0 | Apache Spark libraries needed by YARN slaves. | | sqoop-client | 1.4.7 | Apache Sqoop command-line client. | | tensorflow | 1.14.0 | TensorFlow open source software library for high performance numerical computation. | | tez-on-yarn | 0.9.2 | The tez YARN application and libraries. | | webserver | 2.4.25\$1 | Apache HTTP server. | | zeppelin-server | 0.8.2 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.4.14 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.4.14 | ZooKeeper command line client. | ## 5.30.0 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). **emr-5.30.0 classifications** | Classifications | Description | | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | | container-log4j | Change values in Hadoop YARN's container-log4j.properties file. | | core-site | Change values in Hadoop's core-site.xml file. | | emrfs-site | Change EMRFS settings. | | flink-conf | Change flink-conf.yaml settings. | | flink-log4j | Change Flink log4j.properties settings. | | flink-log4j-yarn-session | Change Flink log4j-yarn-session.properties settings. | | flink-log4j-cli | Change Flink log4j-cli.properties settings. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | | hadoop-ssl-server | Change hadoop ssl server configuration | | hadoop-ssl-client | Change hadoop ssl client configuration | | hbase | Amazon EMR-curated settings for Apache HBase. | | hbase-env | Change values in HBase's environment. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbase.properties file. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | | hbase-site | Change values in HBase's hbase-site.xml file. | | hdfs-encryption-zones | Configure HDFS encryption zones. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | | hcatalog-env | Change values in HCatalog's environment. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | | hcatalog-webhcat-log4j2 | Change values in HCatalog WebHCat's log4j2.properties. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | | hive-beeline-log4j2 | Change values in Hive's beeline-log4j2.properties file. | | hive-parquet-logging | Change values in Hive's parquet-logging.properties file. | | hive-env | Change values in the Hive environment. | | hive-exec-log4j2 | Change values in Hive's hive-exec-log4j2.properties file. | | hive-llap-daemon-log4j2 | Change values in Hive's llap-daemon-log4j2.properties file. | | hive-log4j2 | Change values in Hive's hive-log4j2.properties file. | | hive-site | Change values in Hive's hive-site.xml file | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | | hue-ini | Change values in Hue's ini file | | httpfs-env | Change values in the HTTPFS environment. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | | hudi-env | Change values in the Hudi environment. | | jupyter-notebook-conf | Change values in Jupyter Notebook's jupyter\$1notebook\$1config.py file. | | jupyter-hub-conf | Change values in JupyterHubs's jupyterhub\$1config.py file. | | jupyter-s3-conf | Configure Jupyter Notebook S3 persistence. | | jupyter-sparkmagic-conf | Change values in Sparkmagic's config.json file. | | livy-conf | Change values in Livy's livy.conf file. | | livy-env | Change values in the Livy environment. | | livy-log4j | Change Livy log4j.properties settings. | | mapred-env | Change values in the MapReduce application's environment. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | | oozie-env | Change values in Oozie's environment. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | | oozie-site | Change values in Oozie's oozie-site.xml file. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | | pig-env | Change values in the Pig environment. | | pig-properties | Change values in Pig's pig.properties file. | | pig-log4j | Change values in Pig's log4j.properties file. | | presto-log | Change values in Presto's log.properties file. | | presto-config | Change values in Presto's config.properties file. | | presto-password-authenticator | Change values in Presto's password-authenticator.properties file. | | presto-env | Change values in Presto's presto-env.sh file. | | presto-node | Change values in Presto's node.properties file. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | | presto-connector-hive | Change values in Presto's hive.properties file. | | presto-connector-jmx | Change values in Presto's jmx.properties file. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | | presto-connector-localfile | Change values in Presto's localfile.properties file. | | presto-connector-memory | Change values in Presto's memory.properties file. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | | presto-connector-redis | Change values in Presto's redis.properties file. | | presto-connector-redshift | Change values in Presto's redshift.properties file. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | | presto-connector-tpcds | Change values in Presto's tpcds.properties file. | | ranger-kms-dbks-site | Change values in dbks-site.xml file of Ranger KMS. | | ranger-kms-site | Change values in ranger-kms-site.xml file of Ranger KMS. | | ranger-kms-env | Change values in the Ranger KMS environment. | | ranger-kms-log4j | Change values in kms-log4j.properties file of Ranger KMS. | | ranger-kms-db-ca | Change values for CA file on S3 for MySQL SSL connection with Ranger KMS. | | recordserver-env | Change values in the EMR RecordServer environment. | | recordserver-conf | Change values in EMR RecordServer's erver.properties file. | | recordserver-log4j | Change values in EMR RecordServer's log4j.properties file. | | spark | Amazon EMR-curated settings for Apache Spark. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | | spark-env | Change values in the Spark environment. | | spark-hive-site | Change values in Spark's hive-site.xml file | | spark-log4j | Change values in Spark's log4j.properties file. | | spark-metrics | Change values in Spark's metrics.properties file. | | sqoop-env | Change values in Sqoop's environment. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | | tez-site | Change values in Tez's tez-site.xml file. | | yarn-env | Change values in the YARN environment. | | yarn-site | Change values in YARN's yarn-site.xml file. | | zeppelin-env | Change values in the Zeppelin environment. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | # Amazon EMR release 5.29.0 ## 5.29.0 application versions This release includes the following applications: [https://flink.apache.org/](https://flink.apache.org/), [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [https://hudi.apache.org](https://hudi.apache.org), [http://gethue.com/](http://gethue.com/), [https://jupyterhub.readthedocs.io/en/latest/#](https://jupyterhub.readthedocs.io/en/latest/#), [https://livy.incubator.apache.org/](https://livy.incubator.apache.org/), [https://mxnet.incubator.apache.org/](https://mxnet.incubator.apache.org/), [http://mahout.apache.org/](http://mahout.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://www.tensorflow.org/](https://www.tensorflow.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-5.29.0 | emr-5.28.1 | emr-5.28.0 | emr-5.27.1 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.11.682 | 1.11.659 | 1.11.659 | 1.11.615 | | Python | 2.7, 3.6 | 2.7, 3.6 | 2.7, 3.6 | 2.7, 3.6 | | Scala | 2.11.12 | 2.11.12 | 2.11.12 | 2.11.12 | | AmazonCloudWatchAgent | - | - | - | - | | Delta | - | - | - | - | | Flink | 1.9.1 | 1.9.0 | 1.9.0 | 1.8.1 | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 1.4.10 | 1.4.10 | 1.4.10 | 1.4.10 | | HCatalog | 2.3.6-amzn-1 | 2.3.6-amzn-0 | 2.3.6-amzn-0 | 2.3.5-amzn-1 | | Hadoop | 2.8.5-amzn-5 | 2.8.5-amzn-5 | 2.8.5-amzn-5 | 2.8.5-amzn-4 | | Hive | 2.3.6-amzn-1 | 2.3.6-amzn-0 | 2.3.6-amzn-0 | 2.3.5-amzn-1 | | Hudi | 0.5.0-incubating | 0.5.0-incubating | 0.5.0-incubating | - | | Hue | 4.4.0 | 4.4.0 | 4.4.0 | 4.4.0 | | Iceberg | - | - | - | - | | JupyterEnterpriseGateway | - | - | - | - | | JupyterHub | 1.0.0 | 1.0.0 | 1.0.0 | 1.0.0 | | Livy | 0.6.0-incubating | 0.6.0-incubating | 0.6.0-incubating | 0.6.0-incubating | | MXNet | 1.5.1 | 1.5.1 | 1.5.1 | 1.4.0 | | Mahout | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | | Oozie | 5.1.0 | 5.1.0 | 5.1.0 | 5.1.0 | | Phoenix | 4.14.3-HBase-1.4 | 4.14.3-HBase-1.4 | 4.14.3-HBase-1.4 | 4.14.2-HBase-1.4 | | Pig | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | | Presto | 0.227 | 0.227 | 0.227 | 0.224 | | Spark | 2.4.4 | 2.4.4 | 2.4.4 | 2.4.4 | | Sqoop | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | | TensorFlow | 1.14.0 | 1.14.0 | 1.14.0 | 1.14.0 | | Tez | 0.9.2 | 0.9.2 | 0.9.2 | 0.9.2 | | Trino (PrestoSQL) | - | - | - | - | | Zeppelin | 0.8.2 | 0.8.2 | 0.8.2 | 0.8.1 | | ZooKeeper | 3.4.14 | 3.4.14 | 3.4.14 | 3.4.14 | ## 5.29.0 release notes The following release notes include information for Amazon EMR release 5.29.0. Changes are relative to 5.28.1. Initial release date: Jan 17, 2020 **Upgrades** + Upgraded AWS SDK for Java to version 1.11.682 + Upgraded Hive to version 2.3.6 + Upgraded Flink to version 1.9.1 + Upgraded EmrFS to version 2.38.0 + Upgraded EMR DynamoDB Connector to version 4.13.0 **Changes, enhancements, and resolved issues** + Spark + Spark performance optimizations. + EMRFS + Management Guide updates to emrfs-site.xml default settings for consistent view. **Known issues** + Known issue in clusters with multiple primary nodes and Kerberos authentication If you run clusters with multiple primary nodes and Kerberos authentication in Amazon EMR releases 5.20.0 and later, you may encounter problems with cluster operations such as scale down or step submission, after the cluster has been running for some time. The time period depends on the Kerberos ticket validity period that you defined. The scale-down problem impacts both automatic scale-down and explicit scale down requests that you submitted. Additional cluster operations can also be impacted. Workaround: + SSH as `hadoop` user to the lead primary node of the EMR cluster with multiple primary nodes. + Run the following command to renew Kerberos ticket for `hadoop` user. ``` kinit -kt ``` Typically, the keytab file is located at `/etc/hadoop.keytab` and the principal is in the form of `hadoop/@`. **Note** This workaround will be effective for the time period the Kerberos ticket is valid. This duration is 10 hours by default, but can configured by your Kerberos settings. You must re-run the above command once the Kerberos ticket expires. ## 5.29.0 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | aws-sagemaker-spark-sdk | 1.2.6 | Amazon SageMaker Spark SDK | | emr-ddb | 4.13.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 2.12.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.4.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-s3-dist-cp | 2.13.0 | Distributed copy application optimized for Amazon S3. | | emr-s3-select | 1.4.0 | EMR S3Select Connector | | emrfs | 2.38.0 | Amazon S3 connector for Hadoop ecosystem applications. | | flink-client | 1.9.1 | Apache Flink command line client scripts and applications. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 2.8.5-amzn-5 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 2.8.5-amzn-5 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 2.8.5-amzn-5 | HDFS command-line client and library | | hadoop-hdfs-namenode | 2.8.5-amzn-5 | HDFS service for tracking file names and block locations. | | hadoop-hdfs-journalnode | 2.8.5-amzn-5 | HDFS service for managing the Hadoop filesystem journal on HA clusters. | | hadoop-httpfs-server | 2.8.5-amzn-5 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 2.8.5-amzn-5 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 2.8.5-amzn-5 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 2.8.5-amzn-5 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 2.8.5-amzn-5 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 2.8.5-amzn-5 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 1.4.10 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 1.4.10 | Service for serving one or more HBase regions. | | hbase-client | 1.4.10 | HBase command-line client. | | hbase-rest-server | 1.4.10 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 1.4.10 | Service providing a Thrift endpoint to HBase. | | hcatalog-client | 2.3.6-amzn-1 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 2.3.6-amzn-1 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 2.3.6-amzn-1 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 2.3.6-amzn-1 | Hive command line client. | | hive-hbase | 2.3.6-amzn-1 | Hive-hbase client. | | hive-metastore-server | 2.3.6-amzn-1 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server2 | 2.3.6-amzn-1 | Service for accepting Hive queries as web requests. | | hudi | 0.5.0-incubating | Incremental processing framework to power data pipline at low latency and high efficiency. | | hudi-presto | 0.5.0-incubating | Bundle library for running Presto with Hudi. | | hue-server | 4.4.0 | Web application for analyzing data using Hadoop ecosystem applications | | jupyterhub | 1.0.0 | Multi-user server for Jupyter notebooks | | livy-server | 0.6.0-incubating | REST interface for interacting with Apache Spark | | nginx | 1.12.1 | nginx [engine x] is an HTTP and reverse proxy server | | mahout-client | 0.13.0 | Library for machine learning. | | mxnet | 1.5.1 | A flexible, scalable, and efficient library for deep learning. | | mysql-server | 5.5.54\$1 | MySQL database server. | | nvidia-cuda | 9.2.88 | Nvidia drivers and Cuda toolkit | | oozie-client | 5.1.0 | Oozie command-line client. | | oozie-server | 5.1.0 | Service for accepting Oozie workflow requests. | | opencv | 3.4.0 | Open Source Computer Vision Library. | | phoenix-library | 4.14.3-HBase-1.4 | The phoenix libraries for server and client | | phoenix-query-server | 4.14.3-HBase-1.4 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.227 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.227 | Service for executing pieces of a query. | | presto-client | 0.227 | Presto command-line client which is installed on an HA cluster's stand-by masters where Presto server is not started. | | pig-client | 0.17.0 | Pig command-line client. | | r | 3.4.1 | The R Project for Statistical Computing | | spark-client | 2.4.4 | Spark command-line clients. | | spark-history-server | 2.4.4 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 2.4.4 | In-memory execution engine for YARN. | | spark-yarn-slave | 2.4.4 | Apache Spark libraries needed by YARN slaves. | | sqoop-client | 1.4.7 | Apache Sqoop command-line client. | | tensorflow | 1.14.0 | TensorFlow open source software library for high performance numerical computation. | | tez-on-yarn | 0.9.2 | The tez YARN application and libraries. | | webserver | 2.4.25\$1 | Apache HTTP server. | | zeppelin-server | 0.8.2 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.4.14 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.4.14 | ZooKeeper command line client. | ## 5.29.0 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). **emr-5.29.0 classifications** | Classifications | Description | | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | | container-log4j | Change values in Hadoop YARN's container-log4j.properties file. | | core-site | Change values in Hadoop's core-site.xml file. | | emrfs-site | Change EMRFS settings. | | flink-conf | Change flink-conf.yaml settings. | | flink-log4j | Change Flink log4j.properties settings. | | flink-log4j-yarn-session | Change Flink log4j-yarn-session.properties settings. | | flink-log4j-cli | Change Flink log4j-cli.properties settings. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | | hadoop-ssl-server | Change hadoop ssl server configuration | | hadoop-ssl-client | Change hadoop ssl client configuration | | hbase | Amazon EMR-curated settings for Apache HBase. | | hbase-env | Change values in HBase's environment. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbase.properties file. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | | hbase-site | Change values in HBase's hbase-site.xml file. | | hdfs-encryption-zones | Configure HDFS encryption zones. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | | hcatalog-env | Change values in HCatalog's environment. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | | hcatalog-webhcat-log4j2 | Change values in HCatalog WebHCat's log4j2.properties. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | | hive-beeline-log4j2 | Change values in Hive's beeline-log4j2.properties file. | | hive-parquet-logging | Change values in Hive's parquet-logging.properties file. | | hive-env | Change values in the Hive environment. | | hive-exec-log4j2 | Change values in Hive's hive-exec-log4j2.properties file. | | hive-llap-daemon-log4j2 | Change values in Hive's llap-daemon-log4j2.properties file. | | hive-log4j2 | Change values in Hive's hive-log4j2.properties file. | | hive-site | Change values in Hive's hive-site.xml file | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | | hue-ini | Change values in Hue's ini file | | httpfs-env | Change values in the HTTPFS environment. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | | jupyter-notebook-conf | Change values in Jupyter Notebook's jupyter\$1notebook\$1config.py file. | | jupyter-hub-conf | Change values in JupyterHubs's jupyterhub\$1config.py file. | | jupyter-s3-conf | Configure Jupyter Notebook S3 persistence. | | jupyter-sparkmagic-conf | Change values in Sparkmagic's config.json file. | | livy-conf | Change values in Livy's livy.conf file. | | livy-env | Change values in the Livy environment. | | livy-log4j | Change Livy log4j.properties settings. | | mapred-env | Change values in the MapReduce application's environment. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | | oozie-env | Change values in Oozie's environment. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | | oozie-site | Change values in Oozie's oozie-site.xml file. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | | pig-env | Change values in the Pig environment. | | pig-properties | Change values in Pig's pig.properties file. | | pig-log4j | Change values in Pig's log4j.properties file. | | presto-log | Change values in Presto's log.properties file. | | presto-config | Change values in Presto's config.properties file. | | presto-password-authenticator | Change values in Presto's password-authenticator.properties file. | | presto-env | Change values in Presto's presto-env.sh file. | | presto-node | Change values in Presto's node.properties file. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | | presto-connector-hive | Change values in Presto's hive.properties file. | | presto-connector-jmx | Change values in Presto's jmx.properties file. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | | presto-connector-localfile | Change values in Presto's localfile.properties file. | | presto-connector-memory | Change values in Presto's memory.properties file. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | | presto-connector-redis | Change values in Presto's redis.properties file. | | presto-connector-redshift | Change values in Presto's redshift.properties file. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | | presto-connector-tpcds | Change values in Presto's tpcds.properties file. | | ranger-kms-dbks-site | Change values in dbks-site.xml file of Ranger KMS. | | ranger-kms-site | Change values in ranger-kms-site.xml file of Ranger KMS. | | ranger-kms-env | Change values in the Ranger KMS environment. | | ranger-kms-log4j | Change values in kms-log4j.properties file of Ranger KMS. | | ranger-kms-db-ca | Change values for CA file on S3 for MySQL SSL connection with Ranger KMS. | | recordserver-env | Change values in the EMR RecordServer environment. | | recordserver-conf | Change values in EMR RecordServer's erver.properties file. | | recordserver-log4j | Change values in EMR RecordServer's log4j.properties file. | | spark | Amazon EMR-curated settings for Apache Spark. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | | spark-env | Change values in the Spark environment. | | spark-hive-site | Change values in Spark's hive-site.xml file | | spark-log4j | Change values in Spark's log4j.properties file. | | spark-metrics | Change values in Spark's metrics.properties file. | | sqoop-env | Change values in Sqoop's environment. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | | tez-site | Change values in Tez's tez-site.xml file. | | yarn-env | Change values in the YARN environment. | | yarn-site | Change values in YARN's yarn-site.xml file. | | zeppelin-env | Change values in the Zeppelin environment. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | # Amazon EMR release 5.28.1 ## 5.28.1 application versions This release includes the following applications: [https://flink.apache.org/](https://flink.apache.org/), [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [https://hudi.apache.org](https://hudi.apache.org), [http://gethue.com/](http://gethue.com/), [https://jupyterhub.readthedocs.io/en/latest/#](https://jupyterhub.readthedocs.io/en/latest/#), [https://livy.incubator.apache.org/](https://livy.incubator.apache.org/), [https://mxnet.incubator.apache.org/](https://mxnet.incubator.apache.org/), [http://mahout.apache.org/](http://mahout.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://www.tensorflow.org/](https://www.tensorflow.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-5.28.1 | emr-5.28.0 | emr-5.27.1 | emr-5.27.0 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.11.659 | 1.11.659 | 1.11.615 | 1.11.615 | | Python | 2.7, 3.6 | 2.7, 3.6 | 2.7, 3.6 | 2.7, 3.6 | | Scala | 2.11.12 | 2.11.12 | 2.11.12 | 2.11.12 | | AmazonCloudWatchAgent | - | - | - | - | | Delta | - | - | - | - | | Flink | 1.9.0 | 1.9.0 | 1.8.1 | 1.8.1 | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 1.4.10 | 1.4.10 | 1.4.10 | 1.4.10 | | HCatalog | 2.3.6-amzn-0 | 2.3.6-amzn-0 | 2.3.5-amzn-1 | 2.3.5-amzn-1 | | Hadoop | 2.8.5-amzn-5 | 2.8.5-amzn-5 | 2.8.5-amzn-4 | 2.8.5-amzn-4 | | Hive | 2.3.6-amzn-0 | 2.3.6-amzn-0 | 2.3.5-amzn-1 | 2.3.5-amzn-1 | | Hudi | 0.5.0-incubating | 0.5.0-incubating | - | - | | Hue | 4.4.0 | 4.4.0 | 4.4.0 | 4.4.0 | | Iceberg | - | - | - | - | | JupyterEnterpriseGateway | - | - | - | - | | JupyterHub | 1.0.0 | 1.0.0 | 1.0.0 | 1.0.0 | | Livy | 0.6.0-incubating | 0.6.0-incubating | 0.6.0-incubating | 0.6.0-incubating | | MXNet | 1.5.1 | 1.5.1 | 1.4.0 | 1.4.0 | | Mahout | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | | Oozie | 5.1.0 | 5.1.0 | 5.1.0 | 5.1.0 | | Phoenix | 4.14.3-HBase-1.4 | 4.14.3-HBase-1.4 | 4.14.2-HBase-1.4 | 4.14.2-HBase-1.4 | | Pig | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | | Presto | 0.227 | 0.227 | 0.224 | 0.224 | | Spark | 2.4.4 | 2.4.4 | 2.4.4 | 2.4.4 | | Sqoop | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | | TensorFlow | 1.14.0 | 1.14.0 | 1.14.0 | 1.14.0 | | Tez | 0.9.2 | 0.9.2 | 0.9.2 | 0.9.2 | | Trino (PrestoSQL) | - | - | - | - | | Zeppelin | 0.8.2 | 0.8.2 | 0.8.1 | 0.8.1 | | ZooKeeper | 3.4.14 | 3.4.14 | 3.4.14 | 3.4.14 | ## 5.28.1 release notes The following release notes include information for Amazon EMR release 5.28.1. Changes are relative to 5.28.0. Initial release date: Jan 10, 2020 **Changes, enhancements, and resolved issues** + Spark + Fixed Spark compatibility issues. + CloudWatch Metrics + Fixed Amazon CloudWatch Metrics publishing on an EMR cluster with multiple primary nodes. + Disabled log message + Disabled false log message, "...using old version (<4.5.8) of Apache http client." **Known issues** + Known issue in clusters with multiple primary nodes and Kerberos authentication If you run clusters with multiple primary nodes and Kerberos authentication in Amazon EMR releases 5.20.0 and later, you may encounter problems with cluster operations such as scale down or step submission, after the cluster has been running for some time. The time period depends on the Kerberos ticket validity period that you defined. The scale-down problem impacts both automatic scale-down and explicit scale down requests that you submitted. Additional cluster operations can also be impacted. Workaround: + SSH as `hadoop` user to the lead primary node of the EMR cluster with multiple primary nodes. + Run the following command to renew Kerberos ticket for `hadoop` user. ``` kinit -kt ``` Typically, the keytab file is located at `/etc/hadoop.keytab` and the principal is in the form of `hadoop/@`. **Note** This workaround will be effective for the time period the Kerberos ticket is valid. This duration is 10 hours by default, but can configured by your Kerberos settings. You must re-run the above command once the Kerberos ticket expires. ## 5.28.1 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | aws-sagemaker-spark-sdk | 1.2.6 | Amazon SageMaker Spark SDK | | emr-ddb | 4.12.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 2.11.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.4.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-s3-dist-cp | 2.13.0 | Distributed copy application optimized for Amazon S3. | | emr-s3-select | 1.3.0 | EMR S3Select Connector | | emrfs | 2.37.0 | Amazon S3 connector for Hadoop ecosystem applications. | | flink-client | 1.9.0 | Apache Flink command line client scripts and applications. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 2.8.5-amzn-5 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 2.8.5-amzn-5 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 2.8.5-amzn-5 | HDFS command-line client and library | | hadoop-hdfs-namenode | 2.8.5-amzn-5 | HDFS service for tracking file names and block locations. | | hadoop-hdfs-journalnode | 2.8.5-amzn-5 | HDFS service for managing the Hadoop filesystem journal on HA clusters. | | hadoop-httpfs-server | 2.8.5-amzn-5 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 2.8.5-amzn-5 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 2.8.5-amzn-5 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 2.8.5-amzn-5 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 2.8.5-amzn-5 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 2.8.5-amzn-5 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 1.4.10 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 1.4.10 | Service for serving one or more HBase regions. | | hbase-client | 1.4.10 | HBase command-line client. | | hbase-rest-server | 1.4.10 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 1.4.10 | Service providing a Thrift endpoint to HBase. | | hcatalog-client | 2.3.6-amzn-0 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 2.3.6-amzn-0 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 2.3.6-amzn-0 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 2.3.6-amzn-0 | Hive command line client. | | hive-hbase | 2.3.6-amzn-0 | Hive-hbase client. | | hive-metastore-server | 2.3.6-amzn-0 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server2 | 2.3.6-amzn-0 | Service for accepting Hive queries as web requests. | | hudi | 0.5.0-incubating | Incremental processing framework to power data pipline at low latency and high efficiency. | | hudi-presto | 0.5.0-incubating | Bundle library for running Presto with Hudi. | | hue-server | 4.4.0 | Web application for analyzing data using Hadoop ecosystem applications | | jupyterhub | 1.0.0 | Multi-user server for Jupyter notebooks | | livy-server | 0.6.0-incubating | REST interface for interacting with Apache Spark | | nginx | 1.12.1 | nginx [engine x] is an HTTP and reverse proxy server | | mahout-client | 0.13.0 | Library for machine learning. | | mxnet | 1.5.1 | A flexible, scalable, and efficient library for deep learning. | | mysql-server | 5.5.54\$1 | MySQL database server. | | nvidia-cuda | 9.2.88 | Nvidia drivers and Cuda toolkit | | oozie-client | 5.1.0 | Oozie command-line client. | | oozie-server | 5.1.0 | Service for accepting Oozie workflow requests. | | opencv | 3.4.0 | Open Source Computer Vision Library. | | phoenix-library | 4.14.3-HBase-1.4 | The phoenix libraries for server and client | | phoenix-query-server | 4.14.3-HBase-1.4 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.227 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.227 | Service for executing pieces of a query. | | presto-client | 0.227 | Presto command-line client which is installed on an HA cluster's stand-by masters where Presto server is not started. | | pig-client | 0.17.0 | Pig command-line client. | | r | 3.4.1 | The R Project for Statistical Computing | | spark-client | 2.4.4 | Spark command-line clients. | | spark-history-server | 2.4.4 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 2.4.4 | In-memory execution engine for YARN. | | spark-yarn-slave | 2.4.4 | Apache Spark libraries needed by YARN slaves. | | sqoop-client | 1.4.7 | Apache Sqoop command-line client. | | tensorflow | 1.14.0 | TensorFlow open source software library for high performance numerical computation. | | tez-on-yarn | 0.9.2 | The tez YARN application and libraries. | | webserver | 2.4.25\$1 | Apache HTTP server. | | zeppelin-server | 0.8.2 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.4.14 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.4.14 | ZooKeeper command line client. | ## 5.28.1 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). **emr-5.28.1 classifications** | Classifications | Description | | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | | container-log4j | Change values in Hadoop YARN's container-log4j.properties file. | | core-site | Change values in Hadoop's core-site.xml file. | | emrfs-site | Change EMRFS settings. | | flink-conf | Change flink-conf.yaml settings. | | flink-log4j | Change Flink log4j.properties settings. | | flink-log4j-yarn-session | Change Flink log4j-yarn-session.properties settings. | | flink-log4j-cli | Change Flink log4j-cli.properties settings. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | | hadoop-ssl-server | Change hadoop ssl server configuration | | hadoop-ssl-client | Change hadoop ssl client configuration | | hbase | Amazon EMR-curated settings for Apache HBase. | | hbase-env | Change values in HBase's environment. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbase.properties file. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | | hbase-site | Change values in HBase's hbase-site.xml file. | | hdfs-encryption-zones | Configure HDFS encryption zones. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | | hcatalog-env | Change values in HCatalog's environment. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | | hcatalog-webhcat-log4j2 | Change values in HCatalog WebHCat's log4j2.properties. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | | hive-beeline-log4j2 | Change values in Hive's beeline-log4j2.properties file. | | hive-parquet-logging | Change values in Hive's parquet-logging.properties file. | | hive-env | Change values in the Hive environment. | | hive-exec-log4j2 | Change values in Hive's hive-exec-log4j2.properties file. | | hive-llap-daemon-log4j2 | Change values in Hive's llap-daemon-log4j2.properties file. | | hive-log4j2 | Change values in Hive's hive-log4j2.properties file. | | hive-site | Change values in Hive's hive-site.xml file | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | | hue-ini | Change values in Hue's ini file | | httpfs-env | Change values in the HTTPFS environment. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | | jupyter-notebook-conf | Change values in Jupyter Notebook's jupyter\$1notebook\$1config.py file. | | jupyter-hub-conf | Change values in JupyterHubs's jupyterhub\$1config.py file. | | jupyter-s3-conf | Configure Jupyter Notebook S3 persistence. | | jupyter-sparkmagic-conf | Change values in Sparkmagic's config.json file. | | livy-conf | Change values in Livy's livy.conf file. | | livy-env | Change values in the Livy environment. | | livy-log4j | Change Livy log4j.properties settings. | | mapred-env | Change values in the MapReduce application's environment. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | | oozie-env | Change values in Oozie's environment. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | | oozie-site | Change values in Oozie's oozie-site.xml file. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | | pig-env | Change values in the Pig environment. | | pig-properties | Change values in Pig's pig.properties file. | | pig-log4j | Change values in Pig's log4j.properties file. | | presto-log | Change values in Presto's log.properties file. | | presto-config | Change values in Presto's config.properties file. | | presto-password-authenticator | Change values in Presto's password-authenticator.properties file. | | presto-env | Change values in Presto's presto-env.sh file. | | presto-node | Change values in Presto's node.properties file. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | | presto-connector-hive | Change values in Presto's hive.properties file. | | presto-connector-jmx | Change values in Presto's jmx.properties file. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | | presto-connector-localfile | Change values in Presto's localfile.properties file. | | presto-connector-memory | Change values in Presto's memory.properties file. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | | presto-connector-redis | Change values in Presto's redis.properties file. | | presto-connector-redshift | Change values in Presto's redshift.properties file. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | | presto-connector-tpcds | Change values in Presto's tpcds.properties file. | | ranger-kms-dbks-site | Change values in dbks-site.xml file of Ranger KMS. | | ranger-kms-site | Change values in ranger-kms-site.xml file of Ranger KMS. | | ranger-kms-env | Change values in the Ranger KMS environment. | | ranger-kms-log4j | Change values in kms-log4j.properties file of Ranger KMS. | | ranger-kms-db-ca | Change values for CA file on S3 for MySQL SSL connection with Ranger KMS. | | recordserver-env | Change values in the EMR RecordServer environment. | | recordserver-conf | Change values in EMR RecordServer's erver.properties file. | | recordserver-log4j | Change values in EMR RecordServer's log4j.properties file. | | spark | Amazon EMR-curated settings for Apache Spark. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | | spark-env | Change values in the Spark environment. | | spark-hive-site | Change values in Spark's hive-site.xml file | | spark-log4j | Change values in Spark's log4j.properties file. | | spark-metrics | Change values in Spark's metrics.properties file. | | sqoop-env | Change values in Sqoop's environment. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | | tez-site | Change values in Tez's tez-site.xml file. | | yarn-env | Change values in the YARN environment. | | yarn-site | Change values in YARN's yarn-site.xml file. | | zeppelin-env | Change values in the Zeppelin environment. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | # Amazon EMR release 5.28.0 ## 5.28.0 application versions This release includes the following applications: [https://flink.apache.org/](https://flink.apache.org/), [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [https://hudi.apache.org](https://hudi.apache.org), [http://gethue.com/](http://gethue.com/), [https://jupyterhub.readthedocs.io/en/latest/#](https://jupyterhub.readthedocs.io/en/latest/#), [https://livy.incubator.apache.org/](https://livy.incubator.apache.org/), [https://mxnet.incubator.apache.org/](https://mxnet.incubator.apache.org/), [http://mahout.apache.org/](http://mahout.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://www.tensorflow.org/](https://www.tensorflow.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-5.28.0 | emr-5.27.1 | emr-5.27.0 | emr-5.26.0 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.11.659 | 1.11.615 | 1.11.615 | 1.11.595 | | Python | 2.7, 3.6 | 2.7, 3.6 | 2.7, 3.6 | 2.7, 3.6 | | Scala | 2.11.12 | 2.11.12 | 2.11.12 | 2.11.12 | | AmazonCloudWatchAgent | - | - | - | - | | Delta | - | - | - | - | | Flink | 1.9.0 | 1.8.1 | 1.8.1 | 1.8.0 | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 1.4.10 | 1.4.10 | 1.4.10 | 1.4.10 | | HCatalog | 2.3.6-amzn-0 | 2.3.5-amzn-1 | 2.3.5-amzn-1 | 2.3.5-amzn-0 | | Hadoop | 2.8.5-amzn-5 | 2.8.5-amzn-4 | 2.8.5-amzn-4 | 2.8.5-amzn-4 | | Hive | 2.3.6-amzn-0 | 2.3.5-amzn-1 | 2.3.5-amzn-1 | 2.3.5-amzn-0 | | Hudi | 0.5.0-incubating | - | - | - | | Hue | 4.4.0 | 4.4.0 | 4.4.0 | 4.4.0 | | Iceberg | - | - | - | - | | JupyterEnterpriseGateway | - | - | - | - | | JupyterHub | 1.0.0 | 1.0.0 | 1.0.0 | 0.9.6 | | Livy | 0.6.0-incubating | 0.6.0-incubating | 0.6.0-incubating | 0.6.0-incubating | | MXNet | 1.5.1 | 1.4.0 | 1.4.0 | 1.4.0 | | Mahout | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | | Oozie | 5.1.0 | 5.1.0 | 5.1.0 | 5.1.0 | | Phoenix | 4.14.3-HBase-1.4 | 4.14.2-HBase-1.4 | 4.14.2-HBase-1.4 | 4.14.2-HBase-1.4 | | Pig | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | | Presto | 0.227 | 0.224 | 0.224 | 0.220 | | Spark | 2.4.4 | 2.4.4 | 2.4.4 | 2.4.3 | | Sqoop | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | | TensorFlow | 1.14.0 | 1.14.0 | 1.14.0 | 1.13.1 | | Tez | 0.9.2 | 0.9.2 | 0.9.2 | 0.9.2 | | Trino (PrestoSQL) | - | - | - | - | | Zeppelin | 0.8.2 | 0.8.1 | 0.8.1 | 0.8.1 | | ZooKeeper | 3.4.14 | 3.4.14 | 3.4.14 | 3.4.14 | ## 5.28.0 release notes The following release notes include information for Amazon EMR release 5.28.0. Changes are relative to 5.27.0. Initial release date: Nov 12, 2019 **Upgrades** + Upgraded Flink to version 1.9.0 + Upgraded Hive to version 2.3.6 + Upgraded MXNet to version 1.5.1 + Upgraded Phoenix to version 4.14.3 + Upgraded Presto to version 0.227 + Upgraded Zeppelin to version 0.8.2 **New features** + [Apache Hudi](https://hudi.apache.org/) is now available for Amazon EMR to install when you create a cluster. For more information, see [Hudi](emr-hudi.md). + (Nov 25, 2019) You can now choose to run multiple steps in parallel to improve cluster utilization and save cost. You can also cancel both pending and running steps. For more information, see [Work with Steps Using the AWS CLI and Console](https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-work-with-steps.html). + (Dec 3, 2019) You can now create and run EMR clusters on AWS Outposts. AWS Outposts enables native AWS services, infrastructure, and operating models in on-premises facilities. In AWS Outposts environments, you can use the same AWS APIs, tools, and infrastructure that you use in the AWS cloud. For more information, see [EMR clusters on AWS Outposts](https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-plan-outposts.html). + (Mar 11, 2020) Beginning with Amazon EMR version 5.28.0, you can create and run Amazon EMR clusters on an AWS Local Zones subnet as a logical extension of an AWS Region that supports Local Zones. A Local Zone enables Amazon EMR features and a subset of AWS services, like compute and storage services, to be located closer to users, providing very low latency access to applications running locally. For a list of available Local Zones, see [AWS Local Zones](https://aws.amazon.com/about-aws/global-infrastructure/localzones/). For information about accessing available AWS Local Zones, see [Regions, Availability Zones, and Local Zones](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-regions-availability-zones.html). Local Zones do not currently support Amazon EMR Notebooks and do not support connections directly to Amazon EMR using interface VPC endpoint (AWS PrivateLink). **Changes, enhancements, and resolved issues** + Expanded Application Support for High Availability Clusters + For more information, see [Supported applications in an EMR cluster with Multiple Primary Nodes](https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-plan-ha-applications.html#emr-plan-ha-applications-list) in the *Amazon EMR Management Guide*. + Spark + Performance optimizations + Hive + Performance optimizations + Presto + Performance optimizations **Known issues** + Known issue in clusters with multiple primary nodes and Kerberos authentication If you run clusters with multiple primary nodes and Kerberos authentication in Amazon EMR releases 5.20.0 and later, you may encounter problems with cluster operations such as scale down or step submission, after the cluster has been running for some time. The time period depends on the Kerberos ticket validity period that you defined. The scale-down problem impacts both automatic scale-down and explicit scale down requests that you submitted. Additional cluster operations can also be impacted. Workaround: + SSH as `hadoop` user to the lead primary node of the EMR cluster with multiple primary nodes. + Run the following command to renew Kerberos ticket for `hadoop` user. ``` kinit -kt ``` Typically, the keytab file is located at `/etc/hadoop.keytab` and the principal is in the form of `hadoop/@`. **Note** This workaround will be effective for the time period the Kerberos ticket is valid. This duration is 10 hours by default, but can configured by your Kerberos settings. You must re-run the above command once the Kerberos ticket expires. ## 5.28.0 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | aws-sagemaker-spark-sdk | 1.2.6 | Amazon SageMaker Spark SDK | | emr-ddb | 4.12.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 2.11.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.4.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-s3-dist-cp | 2.13.0 | Distributed copy application optimized for Amazon S3. | | emr-s3-select | 1.3.0 | EMR S3Select Connector | | emrfs | 2.37.0 | Amazon S3 connector for Hadoop ecosystem applications. | | flink-client | 1.9.0 | Apache Flink command line client scripts and applications. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 2.8.5-amzn-5 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 2.8.5-amzn-5 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 2.8.5-amzn-5 | HDFS command-line client and library | | hadoop-hdfs-namenode | 2.8.5-amzn-5 | HDFS service for tracking file names and block locations. | | hadoop-hdfs-journalnode | 2.8.5-amzn-5 | HDFS service for managing the Hadoop filesystem journal on HA clusters. | | hadoop-httpfs-server | 2.8.5-amzn-5 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 2.8.5-amzn-5 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 2.8.5-amzn-5 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 2.8.5-amzn-5 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 2.8.5-amzn-5 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 2.8.5-amzn-5 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 1.4.10 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 1.4.10 | Service for serving one or more HBase regions. | | hbase-client | 1.4.10 | HBase command-line client. | | hbase-rest-server | 1.4.10 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 1.4.10 | Service providing a Thrift endpoint to HBase. | | hcatalog-client | 2.3.6-amzn-0 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 2.3.6-amzn-0 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 2.3.6-amzn-0 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 2.3.6-amzn-0 | Hive command line client. | | hive-hbase | 2.3.6-amzn-0 | Hive-hbase client. | | hive-metastore-server | 2.3.6-amzn-0 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server2 | 2.3.6-amzn-0 | Service for accepting Hive queries as web requests. | | hudi | 0.5.0-incubating | Incremental processing framework to power data pipline at low latency and high efficiency. | | hudi-presto | 0.5.0-incubating | Bundle library for running Presto with Hudi. | | hue-server | 4.4.0 | Web application for analyzing data using Hadoop ecosystem applications | | jupyterhub | 1.0.0 | Multi-user server for Jupyter notebooks | | livy-server | 0.6.0-incubating | REST interface for interacting with Apache Spark | | nginx | 1.12.1 | nginx [engine x] is an HTTP and reverse proxy server | | mahout-client | 0.13.0 | Library for machine learning. | | mxnet | 1.5.1 | A flexible, scalable, and efficient library for deep learning. | | mysql-server | 5.5.54\$1 | MySQL database server. | | nvidia-cuda | 9.2.88 | Nvidia drivers and Cuda toolkit | | oozie-client | 5.1.0 | Oozie command-line client. | | oozie-server | 5.1.0 | Service for accepting Oozie workflow requests. | | opencv | 3.4.0 | Open Source Computer Vision Library. | | phoenix-library | 4.14.3-HBase-1.4 | The phoenix libraries for server and client | | phoenix-query-server | 4.14.3-HBase-1.4 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.227 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.227 | Service for executing pieces of a query. | | presto-client | 0.227 | Presto command-line client which is installed on an HA cluster's stand-by masters where Presto server is not started. | | pig-client | 0.17.0 | Pig command-line client. | | r | 3.4.1 | The R Project for Statistical Computing | | spark-client | 2.4.4 | Spark command-line clients. | | spark-history-server | 2.4.4 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 2.4.4 | In-memory execution engine for YARN. | | spark-yarn-slave | 2.4.4 | Apache Spark libraries needed by YARN slaves. | | sqoop-client | 1.4.7 | Apache Sqoop command-line client. | | tensorflow | 1.14.0 | TensorFlow open source software library for high performance numerical computation. | | tez-on-yarn | 0.9.2 | The tez YARN application and libraries. | | webserver | 2.4.25\$1 | Apache HTTP server. | | zeppelin-server | 0.8.2 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.4.14 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.4.14 | ZooKeeper command line client. | ## 5.28.0 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). **emr-5.28.0 classifications** | Classifications | Description | | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | | container-log4j | Change values in Hadoop YARN's container-log4j.properties file. | | core-site | Change values in Hadoop's core-site.xml file. | | emrfs-site | Change EMRFS settings. | | flink-conf | Change flink-conf.yaml settings. | | flink-log4j | Change Flink log4j.properties settings. | | flink-log4j-yarn-session | Change Flink log4j-yarn-session.properties settings. | | flink-log4j-cli | Change Flink log4j-cli.properties settings. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | | hadoop-ssl-server | Change hadoop ssl server configuration | | hadoop-ssl-client | Change hadoop ssl client configuration | | hbase | Amazon EMR-curated settings for Apache HBase. | | hbase-env | Change values in HBase's environment. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbase.properties file. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | | hbase-site | Change values in HBase's hbase-site.xml file. | | hdfs-encryption-zones | Configure HDFS encryption zones. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | | hcatalog-env | Change values in HCatalog's environment. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | | hcatalog-webhcat-log4j2 | Change values in HCatalog WebHCat's log4j2.properties. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | | hive-beeline-log4j2 | Change values in Hive's beeline-log4j2.properties file. | | hive-parquet-logging | Change values in Hive's parquet-logging.properties file. | | hive-env | Change values in the Hive environment. | | hive-exec-log4j2 | Change values in Hive's hive-exec-log4j2.properties file. | | hive-llap-daemon-log4j2 | Change values in Hive's llap-daemon-log4j2.properties file. | | hive-log4j2 | Change values in Hive's hive-log4j2.properties file. | | hive-site | Change values in Hive's hive-site.xml file | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | | hue-ini | Change values in Hue's ini file | | httpfs-env | Change values in the HTTPFS environment. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | | jupyter-notebook-conf | Change values in Jupyter Notebook's jupyter\$1notebook\$1config.py file. | | jupyter-hub-conf | Change values in JupyterHubs's jupyterhub\$1config.py file. | | jupyter-s3-conf | Configure Jupyter Notebook S3 persistence. | | jupyter-sparkmagic-conf | Change values in Sparkmagic's config.json file. | | livy-conf | Change values in Livy's livy.conf file. | | livy-env | Change values in the Livy environment. | | livy-log4j | Change Livy log4j.properties settings. | | mapred-env | Change values in the MapReduce application's environment. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | | oozie-env | Change values in Oozie's environment. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | | oozie-site | Change values in Oozie's oozie-site.xml file. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | | pig-env | Change values in the Pig environment. | | pig-properties | Change values in Pig's pig.properties file. | | pig-log4j | Change values in Pig's log4j.properties file. | | presto-log | Change values in Presto's log.properties file. | | presto-config | Change values in Presto's config.properties file. | | presto-password-authenticator | Change values in Presto's password-authenticator.properties file. | | presto-env | Change values in Presto's presto-env.sh file. | | presto-node | Change values in Presto's node.properties file. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | | presto-connector-hive | Change values in Presto's hive.properties file. | | presto-connector-jmx | Change values in Presto's jmx.properties file. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | | presto-connector-localfile | Change values in Presto's localfile.properties file. | | presto-connector-memory | Change values in Presto's memory.properties file. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | | presto-connector-redis | Change values in Presto's redis.properties file. | | presto-connector-redshift | Change values in Presto's redshift.properties file. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | | presto-connector-tpcds | Change values in Presto's tpcds.properties file. | | ranger-kms-dbks-site | Change values in dbks-site.xml file of Ranger KMS. | | ranger-kms-site | Change values in ranger-kms-site.xml file of Ranger KMS. | | ranger-kms-env | Change values in the Ranger KMS environment. | | ranger-kms-log4j | Change values in kms-log4j.properties file of Ranger KMS. | | ranger-kms-db-ca | Change values for CA file on S3 for MySQL SSL connection with Ranger KMS. | | recordserver-env | Change values in the EMR RecordServer environment. | | recordserver-conf | Change values in EMR RecordServer's erver.properties file. | | recordserver-log4j | Change values in EMR RecordServer's log4j.properties file. | | spark | Amazon EMR-curated settings for Apache Spark. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | | spark-env | Change values in the Spark environment. | | spark-hive-site | Change values in Spark's hive-site.xml file | | spark-log4j | Change values in Spark's log4j.properties file. | | spark-metrics | Change values in Spark's metrics.properties file. | | sqoop-env | Change values in Sqoop's environment. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | | tez-site | Change values in Tez's tez-site.xml file. | | yarn-env | Change values in the YARN environment. | | yarn-site | Change values in YARN's yarn-site.xml file. | | zeppelin-env | Change values in the Zeppelin environment. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | # Amazon EMR release 5.27.1 ## 5.27.1 application versions This release includes the following applications: [https://flink.apache.org/](https://flink.apache.org/), [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [http://gethue.com/](http://gethue.com/), [https://jupyterhub.readthedocs.io/en/latest/#](https://jupyterhub.readthedocs.io/en/latest/#), [https://livy.incubator.apache.org/](https://livy.incubator.apache.org/), [https://mxnet.incubator.apache.org/](https://mxnet.incubator.apache.org/), [http://mahout.apache.org/](http://mahout.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://www.tensorflow.org/](https://www.tensorflow.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-5.27.1 | emr-5.27.0 | emr-5.26.0 | emr-5.25.0 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.11.615 | 1.11.615 | 1.11.595 | 1.11.566 | | Python | 2.7, 3.6 | 2.7, 3.6 | 2.7, 3.6 | 2.7, 3.6 | | Scala | 2.11.12 | 2.11.12 | 2.11.12 | 2.11.12 | | AmazonCloudWatchAgent | - | - | - | - | | Delta | - | - | - | - | | Flink | 1.8.1 | 1.8.1 | 1.8.0 | 1.8.0 | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 1.4.10 | 1.4.10 | 1.4.10 | 1.4.9 | | HCatalog | 2.3.5-amzn-1 | 2.3.5-amzn-1 | 2.3.5-amzn-0 | 2.3.5-amzn-0 | | Hadoop | 2.8.5-amzn-4 | 2.8.5-amzn-4 | 2.8.5-amzn-4 | 2.8.5-amzn-4 | | Hive | 2.3.5-amzn-1 | 2.3.5-amzn-1 | 2.3.5-amzn-0 | 2.3.5-amzn-0 | | Hudi | - | - | - | - | | Hue | 4.4.0 | 4.4.0 | 4.4.0 | 4.4.0 | | Iceberg | - | - | - | - | | JupyterEnterpriseGateway | - | - | - | - | | JupyterHub | 1.0.0 | 1.0.0 | 0.9.6 | 0.9.6 | | Livy | 0.6.0-incubating | 0.6.0-incubating | 0.6.0-incubating | 0.6.0-incubating | | MXNet | 1.4.0 | 1.4.0 | 1.4.0 | 1.4.0 | | Mahout | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | | Oozie | 5.1.0 | 5.1.0 | 5.1.0 | 5.1.0 | | Phoenix | 4.14.2-HBase-1.4 | 4.14.2-HBase-1.4 | 4.14.2-HBase-1.4 | 4.14.1-HBase-1.4 | | Pig | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | | Presto | 0.224 | 0.224 | 0.220 | 0.220 | | Spark | 2.4.4 | 2.4.4 | 2.4.3 | 2.4.3 | | Sqoop | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | | TensorFlow | 1.14.0 | 1.14.0 | 1.13.1 | 1.13.1 | | Tez | 0.9.2 | 0.9.2 | 0.9.2 | 0.9.2 | | Trino (PrestoSQL) | - | - | - | - | | Zeppelin | 0.8.1 | 0.8.1 | 0.8.1 | 0.8.1 | | ZooKeeper | 3.4.14 | 3.4.14 | 3.4.14 | 3.4.14 | ## 5.27.1 release notes This is a patch release. All applications and components are the same as the previous Amazon EMR release. Instance Metadata Service (IMDS) V2 support status: Amazon EMR 5.23.1, 5.27.1 and 5.32 or later components use IMDSv2 for all IMDS calls. For IMDS calls in your application code, you can use both IMDSv1 and IMDSv2, or configure the IMDS to use only IMDSv2 for added security. For other 5.x EMR releases, disabling IMDSv1 causes cluster startup failure. ## 5.27.1 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | aws-sagemaker-spark-sdk | 1.2.4 | Amazon SageMaker Spark SDK | | emr-ddb | 4.12.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 2.11.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.4.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-s3-dist-cp | 2.13.0 | Distributed copy application optimized for Amazon S3. | | emr-s3-select | 1.3.0 | EMR S3Select Connector | | emrfs | 2.36.0 | Amazon S3 connector for Hadoop ecosystem applications. | | flink-client | 1.8.1 | Apache Flink command line client scripts and applications. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 2.8.5-amzn-4 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 2.8.5-amzn-4 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 2.8.5-amzn-4 | HDFS command-line client and library | | hadoop-hdfs-namenode | 2.8.5-amzn-4 | HDFS service for tracking file names and block locations. | | hadoop-hdfs-journalnode | 2.8.5-amzn-4 | HDFS service for managing the Hadoop filesystem journal on HA clusters. | | hadoop-httpfs-server | 2.8.5-amzn-4 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 2.8.5-amzn-4 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 2.8.5-amzn-4 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 2.8.5-amzn-4 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 2.8.5-amzn-4 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 2.8.5-amzn-4 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 1.4.10 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 1.4.10 | Service for serving one or more HBase regions. | | hbase-client | 1.4.10 | HBase command-line client. | | hbase-rest-server | 1.4.10 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 1.4.10 | Service providing a Thrift endpoint to HBase. | | hcatalog-client | 2.3.5-amzn-1 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 2.3.5-amzn-1 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 2.3.5-amzn-1 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 2.3.5-amzn-1 | Hive command line client. | | hive-hbase | 2.3.5-amzn-1 | Hive-hbase client. | | hive-metastore-server | 2.3.5-amzn-1 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server2 | 2.3.5-amzn-1 | Service for accepting Hive queries as web requests. | | hue-server | 4.4.0 | Web application for analyzing data using Hadoop ecosystem applications | | jupyterhub | 1.0.0 | Multi-user server for Jupyter notebooks | | livy-server | 0.6.0-incubating | REST interface for interacting with Apache Spark | | nginx | 1.12.1 | nginx [engine x] is an HTTP and reverse proxy server | | mahout-client | 0.13.0 | Library for machine learning. | | mxnet | 1.4.0 | A flexible, scalable, and efficient library for deep learning. | | mysql-server | 5.5.54\$1 | MySQL database server. | | nvidia-cuda | 9.2.88 | Nvidia drivers and Cuda toolkit | | oozie-client | 5.1.0 | Oozie command-line client. | | oozie-server | 5.1.0 | Service for accepting Oozie workflow requests. | | opencv | 3.4.0 | Open Source Computer Vision Library. | | phoenix-library | 4.14.2-HBase-1.4 | The phoenix libraries for server and client | | phoenix-query-server | 4.14.2-HBase-1.4 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.224 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.224 | Service for executing pieces of a query. | | pig-client | 0.17.0 | Pig command-line client. | | r | 3.4.1 | The R Project for Statistical Computing | | spark-client | 2.4.4 | Spark command-line clients. | | spark-history-server | 2.4.4 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 2.4.4 | In-memory execution engine for YARN. | | spark-yarn-slave | 2.4.4 | Apache Spark libraries needed by YARN slaves. | | sqoop-client | 1.4.7 | Apache Sqoop command-line client. | | tensorflow | 1.14.0 | TensorFlow open source software library for high performance numerical computation. | | tez-on-yarn | 0.9.2 | The tez YARN application and libraries. | | webserver | 2.4.25\$1 | Apache HTTP server. | | zeppelin-server | 0.8.1 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.4.14 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.4.14 | ZooKeeper command line client. | ## 5.27.1 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). **emr-5.27.1 classifications** | Classifications | Description | | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | | container-log4j | Change values in Hadoop YARN's container-log4j.properties file. | | core-site | Change values in Hadoop's core-site.xml file. | | emrfs-site | Change EMRFS settings. | | flink-conf | Change flink-conf.yaml settings. | | flink-log4j | Change Flink log4j.properties settings. | | flink-log4j-yarn-session | Change Flink log4j-yarn-session.properties settings. | | flink-log4j-cli | Change Flink log4j-cli.properties settings. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | | hadoop-ssl-server | Change hadoop ssl server configuration | | hadoop-ssl-client | Change hadoop ssl client configuration | | hbase | Amazon EMR-curated settings for Apache HBase. | | hbase-env | Change values in HBase's environment. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbase.properties file. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | | hbase-site | Change values in HBase's hbase-site.xml file. | | hdfs-encryption-zones | Configure HDFS encryption zones. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | | hcatalog-env | Change values in HCatalog's environment. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | | hcatalog-webhcat-log4j2 | Change values in HCatalog WebHCat's log4j2.properties. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | | hive-beeline-log4j2 | Change values in Hive's beeline-log4j2.properties file. | | hive-parquet-logging | Change values in Hive's parquet-logging.properties file. | | hive-env | Change values in the Hive environment. | | hive-exec-log4j2 | Change values in Hive's hive-exec-log4j2.properties file. | | hive-llap-daemon-log4j2 | Change values in Hive's llap-daemon-log4j2.properties file. | | hive-log4j2 | Change values in Hive's hive-log4j2.properties file. | | hive-site | Change values in Hive's hive-site.xml file | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | | hue-ini | Change values in Hue's ini file | | httpfs-env | Change values in the HTTPFS environment. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | | jupyter-notebook-conf | Change values in Jupyter Notebook's jupyter\$1notebook\$1config.py file. | | jupyter-hub-conf | Change values in JupyterHubs's jupyterhub\$1config.py file. | | jupyter-s3-conf | Configure Jupyter Notebook S3 persistence. | | jupyter-sparkmagic-conf | Change values in Sparkmagic's config.json file. | | livy-conf | Change values in Livy's livy.conf file. | | livy-env | Change values in the Livy environment. | | livy-log4j | Change Livy log4j.properties settings. | | mapred-env | Change values in the MapReduce application's environment. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | | oozie-env | Change values in Oozie's environment. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | | oozie-site | Change values in Oozie's oozie-site.xml file. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | | pig-env | Change values in the Pig environment. | | pig-properties | Change values in Pig's pig.properties file. | | pig-log4j | Change values in Pig's log4j.properties file. | | presto-log | Change values in Presto's log.properties file. | | presto-config | Change values in Presto's config.properties file. | | presto-password-authenticator | Change values in Presto's password-authenticator.properties file. | | presto-env | Change values in Presto's presto-env.sh file. | | presto-node | Change values in Presto's node.properties file. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | | presto-connector-hive | Change values in Presto's hive.properties file. | | presto-connector-jmx | Change values in Presto's jmx.properties file. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | | presto-connector-localfile | Change values in Presto's localfile.properties file. | | presto-connector-memory | Change values in Presto's memory.properties file. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | | presto-connector-redis | Change values in Presto's redis.properties file. | | presto-connector-redshift | Change values in Presto's redshift.properties file. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | | presto-connector-tpcds | Change values in Presto's tpcds.properties file. | | ranger-kms-dbks-site | Change values in dbks-site.xml file of Ranger KMS. | | ranger-kms-site | Change values in ranger-kms-site.xml file of Ranger KMS. | | ranger-kms-env | Change values in the Ranger KMS environment. | | ranger-kms-log4j | Change values in kms-log4j.properties file of Ranger KMS. | | ranger-kms-db-ca | Change values for CA file on S3 for MySQL SSL connection with Ranger KMS. | | recordserver-env | Change values in the EMR RecordServer environment. | | recordserver-conf | Change values in EMR RecordServer's erver.properties file. | | recordserver-log4j | Change values in EMR RecordServer's log4j.properties file. | | spark | Amazon EMR-curated settings for Apache Spark. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | | spark-env | Change values in the Spark environment. | | spark-hive-site | Change values in Spark's hive-site.xml file | | spark-log4j | Change values in Spark's log4j.properties file. | | spark-metrics | Change values in Spark's metrics.properties file. | | sqoop-env | Change values in Sqoop's environment. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | | tez-site | Change values in Tez's tez-site.xml file. | | yarn-env | Change values in the YARN environment. | | yarn-site | Change values in YARN's yarn-site.xml file. | | zeppelin-env | Change values in the Zeppelin environment. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | # Amazon EMR release 5.27.0 ## 5.27.0 application versions This release includes the following applications: [https://flink.apache.org/](https://flink.apache.org/), [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [http://gethue.com/](http://gethue.com/), [https://jupyterhub.readthedocs.io/en/latest/#](https://jupyterhub.readthedocs.io/en/latest/#), [https://livy.incubator.apache.org/](https://livy.incubator.apache.org/), [https://mxnet.incubator.apache.org/](https://mxnet.incubator.apache.org/), [http://mahout.apache.org/](http://mahout.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://www.tensorflow.org/](https://www.tensorflow.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-5.27.0 | emr-5.26.0 | emr-5.25.0 | emr-5.24.1 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.11.615 | 1.11.595 | 1.11.566 | 1.11.546 | | Python | 2.7, 3.6 | 2.7, 3.6 | 2.7, 3.6 | 2.7, 3.6 | | Scala | 2.11.12 | 2.11.12 | 2.11.12 | 2.11.12 | | AmazonCloudWatchAgent | - | - | - | - | | Delta | - | - | - | - | | Flink | 1.8.1 | 1.8.0 | 1.8.0 | 1.8.0 | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 1.4.10 | 1.4.10 | 1.4.9 | 1.4.9 | | HCatalog | 2.3.5-amzn-1 | 2.3.5-amzn-0 | 2.3.5-amzn-0 | 2.3.4-amzn-2 | | Hadoop | 2.8.5-amzn-4 | 2.8.5-amzn-4 | 2.8.5-amzn-4 | 2.8.5-amzn-4 | | Hive | 2.3.5-amzn-1 | 2.3.5-amzn-0 | 2.3.5-amzn-0 | 2.3.4-amzn-2 | | Hudi | - | - | - | - | | Hue | 4.4.0 | 4.4.0 | 4.4.0 | 4.4.0 | | Iceberg | - | - | - | - | | JupyterEnterpriseGateway | - | - | - | - | | JupyterHub | 1.0.0 | 0.9.6 | 0.9.6 | 0.9.6 | | Livy | 0.6.0-incubating | 0.6.0-incubating | 0.6.0-incubating | 0.6.0-incubating | | MXNet | 1.4.0 | 1.4.0 | 1.4.0 | 1.4.0 | | Mahout | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | | Oozie | 5.1.0 | 5.1.0 | 5.1.0 | 5.1.0 | | Phoenix | 4.14.2-HBase-1.4 | 4.14.2-HBase-1.4 | 4.14.1-HBase-1.4 | 4.14.1-HBase-1.4 | | Pig | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | | Presto | 0.224 | 0.220 | 0.220 | 0.219 | | Spark | 2.4.4 | 2.4.3 | 2.4.3 | 2.4.2 | | Sqoop | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | | TensorFlow | 1.14.0 | 1.13.1 | 1.13.1 | 1.12.0 | | Tez | 0.9.2 | 0.9.2 | 0.9.2 | 0.9.1 | | Trino (PrestoSQL) | - | - | - | - | | Zeppelin | 0.8.1 | 0.8.1 | 0.8.1 | 0.8.1 | | ZooKeeper | 3.4.14 | 3.4.14 | 3.4.14 | 3.4.13 | ## 5.27.0 release notes The following release notes include information for Amazon EMR release 5.27.0. Changes are relative to 5.26.0. Initial release date: Sep 23, 2019 **Upgrades** + AWS SDK for Java 1.11.615 + Flink 1.8.1 + JupyterHub 1.0.0 + Spark 2.4.4 + Tensorflow 1.14.0 + Connectors and drivers: + DynamoDB Connector 4.12.0 **New features** + (Oct 24, 2019) The following New features in EMR notebooks are available with all Amazon EMR releases. + You can now associate Git repositories with EMR notebooks to store your notebooks in a version controlled environment. You can share code with peers and reuse existing Jupyter notebooks through remote Git repositories. For more information, see [Associate Git Repositories with Amazon EMR Notebooks](https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-git-repo.html) in the *Amazon EMR Management Guide*. + The [nbdime utility](https://github.com/jupyter/nbdime) is now available in EMR notebooks to simplify comparing and merging notebooks. + EMR notebooks now support JupyterLab. JupyterLab is a web-based interactive development environment fully compatible with Jupyter notebooks. You can now choose to open your notebook in either JupyterLab or Jupyter notebook editor. + (Oct 30, 2019) With Amazon EMR versions 5.25.0 and later, you can connect to Spark history server UI from the cluster **Summary** page or the **Application history** tab in the console. Instead of setting up a web proxy through an SSH connection, you can quickly access the Spark history server UI to view application metrics and access relevant log files for active and terminated clusters. For more information, see [Off-cluster access to persistent application user interfaces](https://docs.aws.amazon.com/emr/latest/ManagementGuide/app-history-spark-UI.html) in the *Amazon EMR Management Guide*. **Changes, enhancements, and resolved issues** + Amazon EMR cluster with multiple primary nodes + You can install and run Flink on an Amazon EMR cluster with multiple primary nodes. For more information, see [Supported applications and features](https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-plan-ha-applications.html). + You can configure HDFS transparent encryption on an Amazon EMR cluster with multiple primary nodes. For more information, see [HDFS Transparent Encryption on EMR clusters with Multiple Primary Nodes](https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-encryption-tdehdfs.html#emr-hadoop-kms-multi-master). + You can now modify the configuration of applications running on an Amazon EMR cluster with multiple primary nodes. For more information, see [Supplying a Configuration for an Instance Group in a Running Cluster](https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-configure-apps-running-cluster.html). + Amazon EMR-DynamoDB Connector + Amazon EMR-DynamoDB Connector now supports the following DynamoDB data types: boolean, list, map, item, null. For more information, see [Set Up a Hive Table to Run Hive Commands](https://docs.aws.amazon.com/emr/latest/ReleaseGuide/EMR_Interactive_Hive.html). **Known issues** + Known issue in clusters with multiple primary nodes and Kerberos authentication If you run clusters with multiple primary nodes and Kerberos authentication in Amazon EMR releases 5.20.0 and later, you may encounter problems with cluster operations such as scale down or step submission, after the cluster has been running for some time. The time period depends on the Kerberos ticket validity period that you defined. The scale-down problem impacts both automatic scale-down and explicit scale down requests that you submitted. Additional cluster operations can also be impacted. Workaround: + SSH as `hadoop` user to the lead primary node of the EMR cluster with multiple primary nodes. + Run the following command to renew Kerberos ticket for `hadoop` user. ``` kinit -kt ``` Typically, the keytab file is located at `/etc/hadoop.keytab` and the principal is in the form of `hadoop/@`. **Note** This workaround will be effective for the time period the Kerberos ticket is valid. This duration is 10 hours by default, but can configured by your Kerberos settings. You must re-run the above command once the Kerberos ticket expires. ## 5.27.0 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | aws-sagemaker-spark-sdk | 1.2.4 | Amazon SageMaker Spark SDK | | emr-ddb | 4.12.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 2.11.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.4.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-s3-dist-cp | 2.13.0 | Distributed copy application optimized for Amazon S3. | | emr-s3-select | 1.3.0 | EMR S3Select Connector | | emrfs | 2.36.0 | Amazon S3 connector for Hadoop ecosystem applications. | | flink-client | 1.8.1 | Apache Flink command line client scripts and applications. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 2.8.5-amzn-4 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 2.8.5-amzn-4 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 2.8.5-amzn-4 | HDFS command-line client and library | | hadoop-hdfs-namenode | 2.8.5-amzn-4 | HDFS service for tracking file names and block locations. | | hadoop-hdfs-journalnode | 2.8.5-amzn-4 | HDFS service for managing the Hadoop filesystem journal on HA clusters. | | hadoop-httpfs-server | 2.8.5-amzn-4 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 2.8.5-amzn-4 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 2.8.5-amzn-4 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 2.8.5-amzn-4 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 2.8.5-amzn-4 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 2.8.5-amzn-4 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 1.4.10 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 1.4.10 | Service for serving one or more HBase regions. | | hbase-client | 1.4.10 | HBase command-line client. | | hbase-rest-server | 1.4.10 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 1.4.10 | Service providing a Thrift endpoint to HBase. | | hcatalog-client | 2.3.5-amzn-1 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 2.3.5-amzn-1 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 2.3.5-amzn-1 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 2.3.5-amzn-1 | Hive command line client. | | hive-hbase | 2.3.5-amzn-1 | Hive-hbase client. | | hive-metastore-server | 2.3.5-amzn-1 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server2 | 2.3.5-amzn-1 | Service for accepting Hive queries as web requests. | | hue-server | 4.4.0 | Web application for analyzing data using Hadoop ecosystem applications | | jupyterhub | 1.0.0 | Multi-user server for Jupyter notebooks | | livy-server | 0.6.0-incubating | REST interface for interacting with Apache Spark | | nginx | 1.12.1 | nginx [engine x] is an HTTP and reverse proxy server | | mahout-client | 0.13.0 | Library for machine learning. | | mxnet | 1.4.0 | A flexible, scalable, and efficient library for deep learning. | | mysql-server | 5.5.54\$1 | MySQL database server. | | nvidia-cuda | 9.2.88 | Nvidia drivers and Cuda toolkit | | oozie-client | 5.1.0 | Oozie command-line client. | | oozie-server | 5.1.0 | Service for accepting Oozie workflow requests. | | opencv | 3.4.0 | Open Source Computer Vision Library. | | phoenix-library | 4.14.2-HBase-1.4 | The phoenix libraries for server and client | | phoenix-query-server | 4.14.2-HBase-1.4 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.224 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.224 | Service for executing pieces of a query. | | pig-client | 0.17.0 | Pig command-line client. | | r | 3.4.1 | The R Project for Statistical Computing | | spark-client | 2.4.4 | Spark command-line clients. | | spark-history-server | 2.4.4 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 2.4.4 | In-memory execution engine for YARN. | | spark-yarn-slave | 2.4.4 | Apache Spark libraries needed by YARN slaves. | | sqoop-client | 1.4.7 | Apache Sqoop command-line client. | | tensorflow | 1.14.0 | TensorFlow open source software library for high performance numerical computation. | | tez-on-yarn | 0.9.2 | The tez YARN application and libraries. | | webserver | 2.4.25\$1 | Apache HTTP server. | | zeppelin-server | 0.8.1 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.4.14 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.4.14 | ZooKeeper command line client. | ## 5.27.0 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). **emr-5.27.0 classifications** | Classifications | Description | | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | | container-log4j | Change values in Hadoop YARN's container-log4j.properties file. | | core-site | Change values in Hadoop's core-site.xml file. | | emrfs-site | Change EMRFS settings. | | flink-conf | Change flink-conf.yaml settings. | | flink-log4j | Change Flink log4j.properties settings. | | flink-log4j-yarn-session | Change Flink log4j-yarn-session.properties settings. | | flink-log4j-cli | Change Flink log4j-cli.properties settings. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | | hadoop-ssl-server | Change hadoop ssl server configuration | | hadoop-ssl-client | Change hadoop ssl client configuration | | hbase | Amazon EMR-curated settings for Apache HBase. | | hbase-env | Change values in HBase's environment. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbase.properties file. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | | hbase-site | Change values in HBase's hbase-site.xml file. | | hdfs-encryption-zones | Configure HDFS encryption zones. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | | hcatalog-env | Change values in HCatalog's environment. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | | hcatalog-webhcat-log4j2 | Change values in HCatalog WebHCat's log4j2.properties. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | | hive-beeline-log4j2 | Change values in Hive's beeline-log4j2.properties file. | | hive-parquet-logging | Change values in Hive's parquet-logging.properties file. | | hive-env | Change values in the Hive environment. | | hive-exec-log4j2 | Change values in Hive's hive-exec-log4j2.properties file. | | hive-llap-daemon-log4j2 | Change values in Hive's llap-daemon-log4j2.properties file. | | hive-log4j2 | Change values in Hive's hive-log4j2.properties file. | | hive-site | Change values in Hive's hive-site.xml file | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | | hue-ini | Change values in Hue's ini file | | httpfs-env | Change values in the HTTPFS environment. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | | jupyter-notebook-conf | Change values in Jupyter Notebook's jupyter\$1notebook\$1config.py file. | | jupyter-hub-conf | Change values in JupyterHubs's jupyterhub\$1config.py file. | | jupyter-s3-conf | Configure Jupyter Notebook S3 persistence. | | jupyter-sparkmagic-conf | Change values in Sparkmagic's config.json file. | | livy-conf | Change values in Livy's livy.conf file. | | livy-env | Change values in the Livy environment. | | livy-log4j | Change Livy log4j.properties settings. | | mapred-env | Change values in the MapReduce application's environment. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | | oozie-env | Change values in Oozie's environment. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | | oozie-site | Change values in Oozie's oozie-site.xml file. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | | pig-env | Change values in the Pig environment. | | pig-properties | Change values in Pig's pig.properties file. | | pig-log4j | Change values in Pig's log4j.properties file. | | presto-log | Change values in Presto's log.properties file. | | presto-config | Change values in Presto's config.properties file. | | presto-password-authenticator | Change values in Presto's password-authenticator.properties file. | | presto-env | Change values in Presto's presto-env.sh file. | | presto-node | Change values in Presto's node.properties file. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | | presto-connector-hive | Change values in Presto's hive.properties file. | | presto-connector-jmx | Change values in Presto's jmx.properties file. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | | presto-connector-localfile | Change values in Presto's localfile.properties file. | | presto-connector-memory | Change values in Presto's memory.properties file. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | | presto-connector-redis | Change values in Presto's redis.properties file. | | presto-connector-redshift | Change values in Presto's redshift.properties file. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | | presto-connector-tpcds | Change values in Presto's tpcds.properties file. | | ranger-kms-dbks-site | Change values in dbks-site.xml file of Ranger KMS. | | ranger-kms-site | Change values in ranger-kms-site.xml file of Ranger KMS. | | ranger-kms-env | Change values in the Ranger KMS environment. | | ranger-kms-log4j | Change values in kms-log4j.properties file of Ranger KMS. | | ranger-kms-db-ca | Change values for CA file on S3 for MySQL SSL connection with Ranger KMS. | | recordserver-env | Change values in the EMR RecordServer environment. | | recordserver-conf | Change values in EMR RecordServer's erver.properties file. | | recordserver-log4j | Change values in EMR RecordServer's log4j.properties file. | | spark | Amazon EMR-curated settings for Apache Spark. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | | spark-env | Change values in the Spark environment. | | spark-hive-site | Change values in Spark's hive-site.xml file | | spark-log4j | Change values in Spark's log4j.properties file. | | spark-metrics | Change values in Spark's metrics.properties file. | | sqoop-env | Change values in Sqoop's environment. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | | tez-site | Change values in Tez's tez-site.xml file. | | yarn-env | Change values in the YARN environment. | | yarn-site | Change values in YARN's yarn-site.xml file. | | zeppelin-env | Change values in the Zeppelin environment. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | # Amazon EMR release 5.26.0 ## 5.26.0 application versions This release includes the following applications: [https://flink.apache.org/](https://flink.apache.org/), [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [http://gethue.com/](http://gethue.com/), [https://jupyterhub.readthedocs.io/en/latest/#](https://jupyterhub.readthedocs.io/en/latest/#), [https://livy.incubator.apache.org/](https://livy.incubator.apache.org/), [https://mxnet.incubator.apache.org/](https://mxnet.incubator.apache.org/), [http://mahout.apache.org/](http://mahout.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://www.tensorflow.org/](https://www.tensorflow.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-5.26.0 | emr-5.25.0 | emr-5.24.1 | emr-5.24.0 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.11.595 | 1.11.566 | 1.11.546 | 1.11.546 | | Python | 2.7, 3.6 | 2.7, 3.6 | 2.7, 3.6 | 2.7, 3.6 | | Scala | 2.11.12 | 2.11.12 | 2.11.12 | 2.11.12 | | AmazonCloudWatchAgent | - | - | - | - | | Delta | - | - | - | - | | Flink | 1.8.0 | 1.8.0 | 1.8.0 | 1.8.0 | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 1.4.10 | 1.4.9 | 1.4.9 | 1.4.9 | | HCatalog | 2.3.5-amzn-0 | 2.3.5-amzn-0 | 2.3.4-amzn-2 | 2.3.4-amzn-2 | | Hadoop | 2.8.5-amzn-4 | 2.8.5-amzn-4 | 2.8.5-amzn-4 | 2.8.5-amzn-4 | | Hive | 2.3.5-amzn-0 | 2.3.5-amzn-0 | 2.3.4-amzn-2 | 2.3.4-amzn-2 | | Hudi | - | - | - | - | | Hue | 4.4.0 | 4.4.0 | 4.4.0 | 4.4.0 | | Iceberg | - | - | - | - | | JupyterEnterpriseGateway | - | - | - | - | | JupyterHub | 0.9.6 | 0.9.6 | 0.9.6 | 0.9.6 | | Livy | 0.6.0-incubating | 0.6.0-incubating | 0.6.0-incubating | 0.6.0-incubating | | MXNet | 1.4.0 | 1.4.0 | 1.4.0 | 1.4.0 | | Mahout | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | | Oozie | 5.1.0 | 5.1.0 | 5.1.0 | 5.1.0 | | Phoenix | 4.14.2-HBase-1.4 | 4.14.1-HBase-1.4 | 4.14.1-HBase-1.4 | 4.14.1-HBase-1.4 | | Pig | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | | Presto | 0.220 | 0.220 | 0.219 | 0.219 | | Spark | 2.4.3 | 2.4.3 | 2.4.2 | 2.4.2 | | Sqoop | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | | TensorFlow | 1.13.1 | 1.13.1 | 1.12.0 | 1.12.0 | | Tez | 0.9.2 | 0.9.2 | 0.9.1 | 0.9.1 | | Trino (PrestoSQL) | - | - | - | - | | Zeppelin | 0.8.1 | 0.8.1 | 0.8.1 | 0.8.1 | | ZooKeeper | 3.4.14 | 3.4.14 | 3.4.13 | 3.4.13 | ## 5.26.0 release notes The following release notes include information for Amazon EMR release 5.26.0. Changes are relative to 5.25.0. Initial release date: Aug 8, 2019 Last updated date: Aug 19, 2019 **Upgrades** + AWS SDK for Java 1.11.595 + HBase 1.4.10 + Phoenix 4.14.2 + Connectors and drivers: + DynamoDB Connector 4.11.0 + MariaDB Connector 2.4.2 + Amazon Redshift JDBC Driver 1.2.32.1056 **New features** + (Beta) With Amazon EMR 5.26.0, you can launch a cluster that integrates with Lake Formation. This integration provides fine-grained, column-level access to databases and tables in the AWS Glue Data Catalog. It also enables federated single sign-on to EMR Notebooks or Apache Zeppelin from an enterprise identity system. For more information, see [Integrating Amazon EMR with AWS Lake Formation (Beta)](https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-lake-formation.html). + (Aug 19, 2019) Amazon EMR block public access is now available with all Amazon EMR releases that support security groups. Block public access is an account-wide setting applied to each AWS Region. Block public access prevents a cluster from launching when any security group associated with the cluster has a rule that allows inbound traffic from IPv4 0.0.0.0/0 or IPv6 ::/0 (public access) on a port, unless a port is specified as an exception. Port 22 is an exception by default. For more information, see [Using Amazon EMR Block Public Access](https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-block-public-access.html) in the *Amazon EMR Management Guide*. **Changes, enhancements, and resolved issues** + EMR Notebooks + With EMR 5.26.0 and later, EMR Notebooks supports notebook-scoped Python libraries in addition to the default Python libraries. You can install notebook-scoped libraries from within the notebook editor without having to re-create a cluster or re-attach a notebook to a cluster. Notebook-scoped libraries are created in a Python virtual environment, so they apply only to the current notebook session. This allows you to isolate notebook dependencies. For more information, see [Using Notebook Scoped Libraries](https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-managed-notebooks-custom-libraries-limitations.html) in the *Amazon EMR Management Guide*. + EMRFS + You can enable an ETag verification feature (Beta) by setting `fs.s3.consistent.metadata.etag.verification.enabled` to `true`. With this feature, EMRFS uses Amazon S3 ETags to verify that objects being read are the latest available version. This feature is helpful for read-after-update use cases in which files on Amazon S3 are overwritten while retaining the same name. This ETag verification capability currently does not work with S3 Select. For more information, see [Configure Consistent View](https://docs.aws.amazon.com/emr/latest/ManagementGuide/emrfs-configure-consistent-view.html). + Spark + The following optimizations are now enabled by default: dynamic partition pruning, DISTINCT before INTERSECT, improvements in SQL plan statistics inference for JOIN followed by DISTINCT queries, flattening scalar subqueries, optimized join reorder, and bloom filter join. For more information, see [Optimizing Spark Performance](https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-spark-performance.html). + Improved whole stage code generation for Sort Merge Join. + Improved query fragment and subquery reuse. + Improvements to pre-allocate executors on Spark start up. + Bloom filter joins are no longer applied when the smaller side of the join includes a broadcast hint. + Tez + Resolved an issue with Tez. Tez UI now works on an Amazon EMR cluster with multiple primary nodes. **Known issues** + The improved whole stage code generation capabilities for Sort Merge Join can increase memory pressure when enabled. This optimization improves performance, but may result in job retries or failures if the `spark.yarn.executor.memoryOverheadFactor` is not tuned to provide enough memory. To disable this feature, set `spark.sql.sortMergeJoinExec.extendedCodegen.enabled` to false. + Known issue in clusters with multiple primary nodes and Kerberos authentication If you run clusters with multiple primary nodes and Kerberos authentication in Amazon EMR releases 5.20.0 and later, you may encounter problems with cluster operations such as scale down or step submission, after the cluster has been running for some time. The time period depends on the Kerberos ticket validity period that you defined. The scale-down problem impacts both automatic scale-down and explicit scale down requests that you submitted. Additional cluster operations can also be impacted. Workaround: + SSH as `hadoop` user to the lead primary node of the EMR cluster with multiple primary nodes. + Run the following command to renew Kerberos ticket for `hadoop` user. ``` kinit -kt ``` Typically, the keytab file is located at `/etc/hadoop.keytab` and the principal is in the form of `hadoop/@`. **Note** This workaround will be effective for the time period the Kerberos ticket is valid. This duration is 10 hours by default, but can configured by your Kerberos settings. You must re-run the above command once the Kerberos ticket expires. ## 5.26.0 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | aws-sagemaker-spark-sdk | 1.2.4 | Amazon SageMaker Spark SDK | | emr-ddb | 4.11.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 2.10.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.4.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-s3-dist-cp | 2.12.0 | Distributed copy application optimized for Amazon S3. | | emr-s3-select | 1.3.0 | EMR S3Select Connector | | emrfs | 2.35.0 | Amazon S3 connector for Hadoop ecosystem applications. | | flink-client | 1.8.0 | Apache Flink command line client scripts and applications. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 2.8.5-amzn-4 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 2.8.5-amzn-4 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 2.8.5-amzn-4 | HDFS command-line client and library | | hadoop-hdfs-namenode | 2.8.5-amzn-4 | HDFS service for tracking file names and block locations. | | hadoop-hdfs-journalnode | 2.8.5-amzn-4 | HDFS service for managing the Hadoop filesystem journal on HA clusters. | | hadoop-httpfs-server | 2.8.5-amzn-4 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 2.8.5-amzn-4 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 2.8.5-amzn-4 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 2.8.5-amzn-4 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 2.8.5-amzn-4 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 2.8.5-amzn-4 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 1.4.10 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 1.4.10 | Service for serving one or more HBase regions. | | hbase-client | 1.4.10 | HBase command-line client. | | hbase-rest-server | 1.4.10 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 1.4.10 | Service providing a Thrift endpoint to HBase. | | hcatalog-client | 2.3.5-amzn-0 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 2.3.5-amzn-0 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 2.3.5-amzn-0 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 2.3.5-amzn-0 | Hive command line client. | | hive-hbase | 2.3.5-amzn-0 | Hive-hbase client. | | hive-metastore-server | 2.3.5-amzn-0 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server2 | 2.3.5-amzn-0 | Service for accepting Hive queries as web requests. | | hue-server | 4.4.0 | Web application for analyzing data using Hadoop ecosystem applications | | jupyterhub | 0.9.6 | Multi-user server for Jupyter notebooks | | livy-server | 0.6.0-incubating | REST interface for interacting with Apache Spark | | nginx | 1.12.1 | nginx [engine x] is an HTTP and reverse proxy server | | mahout-client | 0.13.0 | Library for machine learning. | | mxnet | 1.4.0 | A flexible, scalable, and efficient library for deep learning. | | mysql-server | 5.5.54\$1 | MySQL database server. | | nvidia-cuda | 9.2.88 | Nvidia drivers and Cuda toolkit | | oozie-client | 5.1.0 | Oozie command-line client. | | oozie-server | 5.1.0 | Service for accepting Oozie workflow requests. | | opencv | 3.4.0 | Open Source Computer Vision Library. | | phoenix-library | 4.14.2-HBase-1.4 | The phoenix libraries for server and client | | phoenix-query-server | 4.14.2-HBase-1.4 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.220 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.220 | Service for executing pieces of a query. | | pig-client | 0.17.0 | Pig command-line client. | | r | 3.4.1 | The R Project for Statistical Computing | | spark-client | 2.4.3 | Spark command-line clients. | | spark-history-server | 2.4.3 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 2.4.3 | In-memory execution engine for YARN. | | spark-yarn-slave | 2.4.3 | Apache Spark libraries needed by YARN slaves. | | sqoop-client | 1.4.7 | Apache Sqoop command-line client. | | tensorflow | 1.13.1 | TensorFlow open source software library for high performance numerical computation. | | tez-on-yarn | 0.9.2 | The tez YARN application and libraries. | | webserver | 2.4.25\$1 | Apache HTTP server. | | zeppelin-server | 0.8.1 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.4.14 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.4.14 | ZooKeeper command line client. | ## 5.26.0 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). **emr-5.26.0 classifications** | Classifications | Description | | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | | container-log4j | Change values in Hadoop YARN's container-log4j.properties file. | | core-site | Change values in Hadoop's core-site.xml file. | | emrfs-site | Change EMRFS settings. | | flink-conf | Change flink-conf.yaml settings. | | flink-log4j | Change Flink log4j.properties settings. | | flink-log4j-yarn-session | Change Flink log4j-yarn-session.properties settings. | | flink-log4j-cli | Change Flink log4j-cli.properties settings. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | | hadoop-ssl-server | Change hadoop ssl server configuration | | hadoop-ssl-client | Change hadoop ssl client configuration | | hbase | Amazon EMR-curated settings for Apache HBase. | | hbase-env | Change values in HBase's environment. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbase.properties file. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | | hbase-site | Change values in HBase's hbase-site.xml file. | | hdfs-encryption-zones | Configure HDFS encryption zones. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | | hcatalog-env | Change values in HCatalog's environment. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | | hcatalog-webhcat-log4j2 | Change values in HCatalog WebHCat's log4j2.properties. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | | hive-beeline-log4j2 | Change values in Hive's beeline-log4j2.properties file. | | hive-parquet-logging | Change values in Hive's parquet-logging.properties file. | | hive-env | Change values in the Hive environment. | | hive-exec-log4j2 | Change values in Hive's hive-exec-log4j2.properties file. | | hive-llap-daemon-log4j2 | Change values in Hive's llap-daemon-log4j2.properties file. | | hive-log4j2 | Change values in Hive's hive-log4j2.properties file. | | hive-site | Change values in Hive's hive-site.xml file | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | | hue-ini | Change values in Hue's ini file | | httpfs-env | Change values in the HTTPFS environment. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | | jupyter-notebook-conf | Change values in Jupyter Notebook's jupyter\$1notebook\$1config.py file. | | jupyter-hub-conf | Change values in JupyterHubs's jupyterhub\$1config.py file. | | jupyter-s3-conf | Configure Jupyter Notebook S3 persistence. | | jupyter-sparkmagic-conf | Change values in Sparkmagic's config.json file. | | livy-conf | Change values in Livy's livy.conf file. | | livy-env | Change values in the Livy environment. | | livy-log4j | Change Livy log4j.properties settings. | | mapred-env | Change values in the MapReduce application's environment. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | | oozie-env | Change values in Oozie's environment. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | | oozie-site | Change values in Oozie's oozie-site.xml file. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | | pig-env | Change values in the Pig environment. | | pig-properties | Change values in Pig's pig.properties file. | | pig-log4j | Change values in Pig's log4j.properties file. | | presto-log | Change values in Presto's log.properties file. | | presto-config | Change values in Presto's config.properties file. | | presto-password-authenticator | Change values in Presto's password-authenticator.properties file. | | presto-env | Change values in Presto's presto-env.sh file. | | presto-node | Change values in Presto's node.properties file. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | | presto-connector-hive | Change values in Presto's hive.properties file. | | presto-connector-jmx | Change values in Presto's jmx.properties file. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | | presto-connector-localfile | Change values in Presto's localfile.properties file. | | presto-connector-memory | Change values in Presto's memory.properties file. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | | presto-connector-redis | Change values in Presto's redis.properties file. | | presto-connector-redshift | Change values in Presto's redshift.properties file. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | | presto-connector-tpcds | Change values in Presto's tpcds.properties file. | | recordserver-env | Change values in the EMR RecordServer environment. | | recordserver-conf | Change values in EMR RecordServer's erver.properties file. | | recordserver-log4j | Change values in EMR RecordServer's log4j.properties file. | | spark | Amazon EMR-curated settings for Apache Spark. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | | spark-env | Change values in the Spark environment. | | spark-hive-site | Change values in Spark's hive-site.xml file | | spark-log4j | Change values in Spark's log4j.properties file. | | spark-metrics | Change values in Spark's metrics.properties file. | | sqoop-env | Change values in Sqoop's environment. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | | tez-site | Change values in Tez's tez-site.xml file. | | yarn-env | Change values in the YARN environment. | | yarn-site | Change values in YARN's yarn-site.xml file. | | zeppelin-env | Change values in the Zeppelin environment. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | # Amazon EMR release 5.25.0 ## 5.25.0 application versions This release includes the following applications: [https://flink.apache.org/](https://flink.apache.org/), [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [http://gethue.com/](http://gethue.com/), [https://jupyterhub.readthedocs.io/en/latest/#](https://jupyterhub.readthedocs.io/en/latest/#), [https://livy.incubator.apache.org/](https://livy.incubator.apache.org/), [https://mxnet.incubator.apache.org/](https://mxnet.incubator.apache.org/), [http://mahout.apache.org/](http://mahout.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://www.tensorflow.org/](https://www.tensorflow.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-5.25.0 | emr-5.24.1 | emr-5.24.0 | emr-5.23.1 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.11.566 | 1.11.546 | 1.11.546 | 1.11.519 | | Python | 2.7, 3.6 | 2.7, 3.6 | 2.7, 3.6 | 2.7, 3.6 | | Scala | 2.11.12 | 2.11.12 | 2.11.12 | 2.11.12 | | AmazonCloudWatchAgent | - | - | - | - | | Delta | - | - | - | - | | Flink | 1.8.0 | 1.8.0 | 1.8.0 | 1.7.1 | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 1.4.9 | 1.4.9 | 1.4.9 | 1.4.9 | | HCatalog | 2.3.5-amzn-0 | 2.3.4-amzn-2 | 2.3.4-amzn-2 | 2.3.4-amzn-1 | | Hadoop | 2.8.5-amzn-4 | 2.8.5-amzn-4 | 2.8.5-amzn-4 | 2.8.5-amzn-3 | | Hive | 2.3.5-amzn-0 | 2.3.4-amzn-2 | 2.3.4-amzn-2 | 2.3.4-amzn-1 | | Hudi | - | - | - | - | | Hue | 4.4.0 | 4.4.0 | 4.4.0 | 4.3.0 | | Iceberg | - | - | - | - | | JupyterEnterpriseGateway | - | - | - | - | | JupyterHub | 0.9.6 | 0.9.6 | 0.9.6 | 0.9.4 | | Livy | 0.6.0-incubating | 0.6.0-incubating | 0.6.0-incubating | 0.5.0-incubating | | MXNet | 1.4.0 | 1.4.0 | 1.4.0 | 1.3.1 | | Mahout | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | | Oozie | 5.1.0 | 5.1.0 | 5.1.0 | 5.1.0 | | Phoenix | 4.14.1-HBase-1.4 | 4.14.1-HBase-1.4 | 4.14.1-HBase-1.4 | 4.14.1-HBase-1.4 | | Pig | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | | Presto | 0.220 | 0.219 | 0.219 | 0.215 | | Spark | 2.4.3 | 2.4.2 | 2.4.2 | 2.4.0 | | Sqoop | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | | TensorFlow | 1.13.1 | 1.12.0 | 1.12.0 | 1.12.0 | | Tez | 0.9.2 | 0.9.1 | 0.9.1 | 0.9.1 | | Trino (PrestoSQL) | - | - | - | - | | Zeppelin | 0.8.1 | 0.8.1 | 0.8.1 | 0.8.1 | | ZooKeeper | 3.4.14 | 3.4.13 | 3.4.13 | 3.4.13 | ## 5.25.0 release notes The following release notes include information for Amazon EMR release 5.25.0. Changes are relative to 5.24.1. Initial release date: July 17, 2019 Last updated date: Oct 30, 2019 **Amazon EMR 5.25.0** **Upgrades** + AWS SDK for Java 1.11.566 + Hive 2.3.5 + Presto 0.220 + Spark 2.4.3 + TensorFlow 1.13.1 + Tez 0.9.2 + Zookeeper 3.4.14 **New features** + (Oct 30, 2019) Beginning with Amazon EMR version 5.25.0, you can connect to Spark history server UI from the cluster **Summary** page or the **Application history** tab in the console. Instead of setting up a web proxy through an SSH connection, you can quickly access the Spark history server UI to view application metrics and access relevant log files for active and terminated clusters. For more information, see [Off-cluster access to persistent application user interfaces](https://docs.aws.amazon.com/emr/latest/ManagementGuide/app-history-spark-UI.html) in the *Amazon EMR Management Guide*. **Changes, enhancements, and resolved issues** + Spark + Improved the performance of some joins by using Bloom filters to pre-filter inputs. The optimization is disabled by default and can be enabled by setting the Spark configuration parameter `spark.sql.bloomFilterJoin.enabled` to `true`. + Improved the performance of grouping by string type columns. + Improved the default Spark executor memory and cores configuration of R4 instance types for clusters without HBase installed. + Resolved a previous issue with the dynamic partition pruning feature where the pruned table has to be on the left side of the join. + Improved DISTINCT before INTERSECT optimization to apply to additional cases involving aliases. + Improved SQL plan statistics inference for JOIN followed by DISTINCT queries. This improvement is disabled by default and can be enabled by setting the Spark configuration parameter `spark.sql.statsImprovements.enabled` to `true`. This optimization is required by the Distinct before Intersect feature and will be enabled automatically when `spark.sql.optimizer.distinctBeforeIntersect.enabled` is set to `true`. + Optimized join order based on table size and filters. This optimization is disabled by default and can be enabled by setting the Spark configuration parameter `spark.sql.optimizer.sizeBasedJoinReorder.enabled` to `true`. For more information, see [Optimizing Spark Performance](https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-spark-performance.html). + EMRFS + The EMRFS setting, `fs.s3.buckets.create.enabled`, is now disabled by default. With testing, we found that disabling this setting improves performance and prevents unintentional creation of S3 buckets. If your application relies on this functionality, you can enable it by setting the property `fs.s3.buckets.create.enabled` to `true` in the `emrfs-site` configuration classification. For information, see [Supplying a Configuration when Creating a Cluster](https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-configure-apps-create-cluster.html). + Local Disk Encryption and S3 Encryption Improvements in Security Configurations (August 5, 2019) + Separated Amazon S3 encryption settings from local disk encryption settings in security configuration setup. + Added an option to enable EBS encryption with release 5.24.0 and later. Selecting this option encrypts the root device volume in addition to storage volumes. Previous versions required using a custom AMI to encrypt the root device volume. + For more information, see [Encryption Options](https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-data-encryption-options.html) in the *Amazon EMR Management Guide*. **Known issues** + Known issue in clusters with multiple primary nodes and Kerberos authentication If you run clusters with multiple primary nodes and Kerberos authentication in Amazon EMR releases 5.20.0 and later, you may encounter problems with cluster operations such as scale down or step submission, after the cluster has been running for some time. The time period depends on the Kerberos ticket validity period that you defined. The scale-down problem impacts both automatic scale-down and explicit scale down requests that you submitted. Additional cluster operations can also be impacted. Workaround: + SSH as `hadoop` user to the lead primary node of the EMR cluster with multiple primary nodes. + Run the following command to renew Kerberos ticket for `hadoop` user. ``` kinit -kt ``` Typically, the keytab file is located at `/etc/hadoop.keytab` and the principal is in the form of `hadoop/@`. **Note** This workaround will be effective for the time period the Kerberos ticket is valid. This duration is 10 hours by default, but can configured by your Kerberos settings. You must re-run the above command once the Kerberos ticket expires. ## 5.25.0 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | aws-sagemaker-spark-sdk | 1.2.4 | Amazon SageMaker Spark SDK | | emr-ddb | 4.10.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 2.9.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.4.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-s3-dist-cp | 2.11.0 | Distributed copy application optimized for Amazon S3. | | emr-s3-select | 1.3.0 | EMR S3Select Connector | | emrfs | 2.34.0 | Amazon S3 connector for Hadoop ecosystem applications. | | flink-client | 1.8.0 | Apache Flink command line client scripts and applications. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 2.8.5-amzn-4 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 2.8.5-amzn-4 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 2.8.5-amzn-4 | HDFS command-line client and library | | hadoop-hdfs-namenode | 2.8.5-amzn-4 | HDFS service for tracking file names and block locations. | | hadoop-hdfs-journalnode | 2.8.5-amzn-4 | HDFS service for managing the Hadoop filesystem journal on HA clusters. | | hadoop-httpfs-server | 2.8.5-amzn-4 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 2.8.5-amzn-4 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 2.8.5-amzn-4 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 2.8.5-amzn-4 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 2.8.5-amzn-4 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 2.8.5-amzn-4 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 1.4.9 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 1.4.9 | Service for serving one or more HBase regions. | | hbase-client | 1.4.9 | HBase command-line client. | | hbase-rest-server | 1.4.9 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 1.4.9 | Service providing a Thrift endpoint to HBase. | | hcatalog-client | 2.3.5-amzn-0 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 2.3.5-amzn-0 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 2.3.5-amzn-0 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 2.3.5-amzn-0 | Hive command line client. | | hive-hbase | 2.3.5-amzn-0 | Hive-hbase client. | | hive-metastore-server | 2.3.5-amzn-0 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server2 | 2.3.5-amzn-0 | Service for accepting Hive queries as web requests. | | hue-server | 4.4.0 | Web application for analyzing data using Hadoop ecosystem applications | | jupyterhub | 0.9.6 | Multi-user server for Jupyter notebooks | | livy-server | 0.6.0-incubating | REST interface for interacting with Apache Spark | | nginx | 1.12.1 | nginx [engine x] is an HTTP and reverse proxy server | | mahout-client | 0.13.0 | Library for machine learning. | | mxnet | 1.4.0 | A flexible, scalable, and efficient library for deep learning. | | mysql-server | 5.5.54\$1 | MySQL database server. | | nvidia-cuda | 9.2.88 | Nvidia drivers and Cuda toolkit | | oozie-client | 5.1.0 | Oozie command-line client. | | oozie-server | 5.1.0 | Service for accepting Oozie workflow requests. | | opencv | 3.4.0 | Open Source Computer Vision Library. | | phoenix-library | 4.14.1-HBase-1.4 | The phoenix libraries for server and client | | phoenix-query-server | 4.14.1-HBase-1.4 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.220 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.220 | Service for executing pieces of a query. | | pig-client | 0.17.0 | Pig command-line client. | | r | 3.4.1 | The R Project for Statistical Computing | | spark-client | 2.4.3 | Spark command-line clients. | | spark-history-server | 2.4.3 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 2.4.3 | In-memory execution engine for YARN. | | spark-yarn-slave | 2.4.3 | Apache Spark libraries needed by YARN slaves. | | sqoop-client | 1.4.7 | Apache Sqoop command-line client. | | tensorflow | 1.13.1 | TensorFlow open source software library for high performance numerical computation. | | tez-on-yarn | 0.9.2 | The tez YARN application and libraries. | | webserver | 2.4.25\$1 | Apache HTTP server. | | zeppelin-server | 0.8.1 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.4.14 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.4.14 | ZooKeeper command line client. | ## 5.25.0 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). **emr-5.25.0 classifications** | Classifications | Description | | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | | container-log4j | Change values in Hadoop YARN's container-log4j.properties file. | | core-site | Change values in Hadoop's core-site.xml file. | | emrfs-site | Change EMRFS settings. | | flink-conf | Change flink-conf.yaml settings. | | flink-log4j | Change Flink log4j.properties settings. | | flink-log4j-yarn-session | Change Flink log4j-yarn-session.properties settings. | | flink-log4j-cli | Change Flink log4j-cli.properties settings. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | | hadoop-ssl-server | Change hadoop ssl server configuration | | hadoop-ssl-client | Change hadoop ssl client configuration | | hbase | Amazon EMR-curated settings for Apache HBase. | | hbase-env | Change values in HBase's environment. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbase.properties file. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | | hbase-site | Change values in HBase's hbase-site.xml file. | | hdfs-encryption-zones | Configure HDFS encryption zones. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | | hcatalog-env | Change values in HCatalog's environment. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | | hcatalog-webhcat-log4j2 | Change values in HCatalog WebHCat's log4j2.properties. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | | hive-beeline-log4j2 | Change values in Hive's beeline-log4j2.properties file. | | hive-parquet-logging | Change values in Hive's parquet-logging.properties file. | | hive-env | Change values in the Hive environment. | | hive-exec-log4j2 | Change values in Hive's hive-exec-log4j2.properties file. | | hive-llap-daemon-log4j2 | Change values in Hive's llap-daemon-log4j2.properties file. | | hive-log4j2 | Change values in Hive's hive-log4j2.properties file. | | hive-site | Change values in Hive's hive-site.xml file | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | | hue-ini | Change values in Hue's ini file | | httpfs-env | Change values in the HTTPFS environment. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | | jupyter-notebook-conf | Change values in Jupyter Notebook's jupyter\$1notebook\$1config.py file. | | jupyter-hub-conf | Change values in JupyterHubs's jupyterhub\$1config.py file. | | jupyter-s3-conf | Configure Jupyter Notebook S3 persistence. | | jupyter-sparkmagic-conf | Change values in Sparkmagic's config.json file. | | livy-conf | Change values in Livy's livy.conf file. | | livy-env | Change values in the Livy environment. | | livy-log4j | Change Livy log4j.properties settings. | | mapred-env | Change values in the MapReduce application's environment. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | | oozie-env | Change values in Oozie's environment. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | | oozie-site | Change values in Oozie's oozie-site.xml file. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | | pig-env | Change values in the Pig environment. | | pig-properties | Change values in Pig's pig.properties file. | | pig-log4j | Change values in Pig's log4j.properties file. | | presto-log | Change values in Presto's log.properties file. | | presto-config | Change values in Presto's config.properties file. | | presto-password-authenticator | Change values in Presto's password-authenticator.properties file. | | presto-env | Change values in Presto's presto-env.sh file. | | presto-node | Change values in Presto's node.properties file. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | | presto-connector-hive | Change values in Presto's hive.properties file. | | presto-connector-jmx | Change values in Presto's jmx.properties file. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | | presto-connector-localfile | Change values in Presto's localfile.properties file. | | presto-connector-memory | Change values in Presto's memory.properties file. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | | presto-connector-redis | Change values in Presto's redis.properties file. | | presto-connector-redshift | Change values in Presto's redshift.properties file. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | | presto-connector-tpcds | Change values in Presto's tpcds.properties file. | | recordserver-env | Change values in the EMR RecordServer environment. | | recordserver-conf | Change values in EMR RecordServer's erver.properties file. | | recordserver-log4j | Change values in EMR RecordServer's log4j.properties file. | | spark | Amazon EMR-curated settings for Apache Spark. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | | spark-env | Change values in the Spark environment. | | spark-hive-site | Change values in Spark's hive-site.xml file | | spark-log4j | Change values in Spark's log4j.properties file. | | spark-metrics | Change values in Spark's metrics.properties file. | | sqoop-env | Change values in Sqoop's environment. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | | tez-site | Change values in Tez's tez-site.xml file. | | yarn-env | Change values in the YARN environment. | | yarn-site | Change values in YARN's yarn-site.xml file. | | zeppelin-env | Change values in the Zeppelin environment. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | # Amazon EMR release 5.24.1 ## 5.24.1 application versions This release includes the following applications: [https://flink.apache.org/](https://flink.apache.org/), [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [http://gethue.com/](http://gethue.com/), [https://jupyterhub.readthedocs.io/en/latest/#](https://jupyterhub.readthedocs.io/en/latest/#), [https://livy.incubator.apache.org/](https://livy.incubator.apache.org/), [https://mxnet.incubator.apache.org/](https://mxnet.incubator.apache.org/), [http://mahout.apache.org/](http://mahout.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://www.tensorflow.org/](https://www.tensorflow.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-5.24.1 | emr-5.24.0 | emr-5.23.1 | emr-5.23.0 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.11.546 | 1.11.546 | 1.11.519 | 1.11.519 | | Python | 2.7, 3.6 | 2.7, 3.6 | 2.7, 3.6 | 2.7, 3.6 | | Scala | 2.11.12 | 2.11.12 | 2.11.12 | 2.11.12 | | AmazonCloudWatchAgent | - | - | - | - | | Delta | - | - | - | - | | Flink | 1.8.0 | 1.8.0 | 1.7.1 | 1.7.1 | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 1.4.9 | 1.4.9 | 1.4.9 | 1.4.9 | | HCatalog | 2.3.4-amzn-2 | 2.3.4-amzn-2 | 2.3.4-amzn-1 | 2.3.4-amzn-1 | | Hadoop | 2.8.5-amzn-4 | 2.8.5-amzn-4 | 2.8.5-amzn-3 | 2.8.5-amzn-3 | | Hive | 2.3.4-amzn-2 | 2.3.4-amzn-2 | 2.3.4-amzn-1 | 2.3.4-amzn-1 | | Hudi | - | - | - | - | | Hue | 4.4.0 | 4.4.0 | 4.3.0 | 4.3.0 | | Iceberg | - | - | - | - | | JupyterEnterpriseGateway | - | - | - | - | | JupyterHub | 0.9.6 | 0.9.6 | 0.9.4 | 0.9.4 | | Livy | 0.6.0-incubating | 0.6.0-incubating | 0.5.0-incubating | 0.5.0-incubating | | MXNet | 1.4.0 | 1.4.0 | 1.3.1 | 1.3.1 | | Mahout | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | | Oozie | 5.1.0 | 5.1.0 | 5.1.0 | 5.1.0 | | Phoenix | 4.14.1-HBase-1.4 | 4.14.1-HBase-1.4 | 4.14.1-HBase-1.4 | 4.14.1-HBase-1.4 | | Pig | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | | Presto | 0.219 | 0.219 | 0.215 | 0.215 | | Spark | 2.4.2 | 2.4.2 | 2.4.0 | 2.4.0 | | Sqoop | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | | TensorFlow | 1.12.0 | 1.12.0 | 1.12.0 | 1.12.0 | | Tez | 0.9.1 | 0.9.1 | 0.9.1 | 0.9.1 | | Trino (PrestoSQL) | - | - | - | - | | Zeppelin | 0.8.1 | 0.8.1 | 0.8.1 | 0.8.1 | | ZooKeeper | 3.4.13 | 3.4.13 | 3.4.13 | 3.4.13 | ## 5.24.1 release notes The following release notes include information for Amazon EMR release 5.24.1. Changes are relative to 5.24.0. Initial release date: June 26, 2019 **Changes, enhancements, and resolved issues** + Updated the default Amazon Linux AMI for Amazon EMR to include important Linux kernel security updates, including the TCP SACK Denial of Service Issue ([AWS-2019-005](https://aws.amazon.com/security/security-bulletins/AWS-2019-005/)). **Known issues** + Known issue in clusters with multiple primary nodes and Kerberos authentication If you run clusters with multiple primary nodes and Kerberos authentication in Amazon EMR releases 5.20.0 and later, you may encounter problems with cluster operations such as scale down or step submission, after the cluster has been running for some time. The time period depends on the Kerberos ticket validity period that you defined. The scale-down problem impacts both automatic scale-down and explicit scale down requests that you submitted. Additional cluster operations can also be impacted. Workaround: + SSH as `hadoop` user to the lead primary node of the EMR cluster with multiple primary nodes. + Run the following command to renew Kerberos ticket for `hadoop` user. ``` kinit -kt ``` Typically, the keytab file is located at `/etc/hadoop.keytab` and the principal is in the form of `hadoop/@`. **Note** This workaround will be effective for the time period the Kerberos ticket is valid. This duration is 10 hours by default, but can configured by your Kerberos settings. You must re-run the above command once the Kerberos ticket expires. ## 5.24.1 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | aws-sagemaker-spark-sdk | 1.2.1 | Amazon SageMaker Spark SDK | | emr-ddb | 4.9.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 2.8.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.4.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-s3-dist-cp | 2.11.0 | Distributed copy application optimized for Amazon S3. | | emr-s3-select | 1.3.0 | EMR S3Select Connector | | emrfs | 2.33.0 | Amazon S3 connector for Hadoop ecosystem applications. | | flink-client | 1.8.0 | Apache Flink command line client scripts and applications. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 2.8.5-amzn-4 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 2.8.5-amzn-4 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 2.8.5-amzn-4 | HDFS command-line client and library | | hadoop-hdfs-namenode | 2.8.5-amzn-4 | HDFS service for tracking file names and block locations. | | hadoop-hdfs-journalnode | 2.8.5-amzn-4 | HDFS service for managing the Hadoop filesystem journal on HA clusters. | | hadoop-httpfs-server | 2.8.5-amzn-4 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 2.8.5-amzn-4 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 2.8.5-amzn-4 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 2.8.5-amzn-4 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 2.8.5-amzn-4 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 2.8.5-amzn-4 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 1.4.9 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 1.4.9 | Service for serving one or more HBase regions. | | hbase-client | 1.4.9 | HBase command-line client. | | hbase-rest-server | 1.4.9 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 1.4.9 | Service providing a Thrift endpoint to HBase. | | hcatalog-client | 2.3.4-amzn-2 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 2.3.4-amzn-2 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 2.3.4-amzn-2 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 2.3.4-amzn-2 | Hive command line client. | | hive-hbase | 2.3.4-amzn-2 | Hive-hbase client. | | hive-metastore-server | 2.3.4-amzn-2 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server2 | 2.3.4-amzn-2 | Service for accepting Hive queries as web requests. | | hue-server | 4.4.0 | Web application for analyzing data using Hadoop ecosystem applications | | jupyterhub | 0.9.6 | Multi-user server for Jupyter notebooks | | livy-server | 0.6.0-incubating | REST interface for interacting with Apache Spark | | nginx | 1.12.1 | nginx [engine x] is an HTTP and reverse proxy server | | mahout-client | 0.13.0 | Library for machine learning. | | mxnet | 1.4.0 | A flexible, scalable, and efficient library for deep learning. | | mysql-server | 5.5.54\$1 | MySQL database server. | | nvidia-cuda | 9.2.88 | Nvidia drivers and Cuda toolkit | | oozie-client | 5.1.0 | Oozie command-line client. | | oozie-server | 5.1.0 | Service for accepting Oozie workflow requests. | | opencv | 3.4.0 | Open Source Computer Vision Library. | | phoenix-library | 4.14.1-HBase-1.4 | The phoenix libraries for server and client | | phoenix-query-server | 4.14.1-HBase-1.4 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.219 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.219 | Service for executing pieces of a query. | | pig-client | 0.17.0 | Pig command-line client. | | r | 3.4.1 | The R Project for Statistical Computing | | spark-client | 2.4.2 | Spark command-line clients. | | spark-history-server | 2.4.2 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 2.4.2 | In-memory execution engine for YARN. | | spark-yarn-slave | 2.4.2 | Apache Spark libraries needed by YARN slaves. | | sqoop-client | 1.4.7 | Apache Sqoop command-line client. | | tensorflow | 1.12.0 | TensorFlow open source software library for high performance numerical computation. | | tez-on-yarn | 0.9.1 | The tez YARN application and libraries. | | webserver | 2.4.25\$1 | Apache HTTP server. | | zeppelin-server | 0.8.1 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.4.13 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.4.13 | ZooKeeper command line client. | ## 5.24.1 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). **emr-5.24.1 classifications** | Classifications | Description | | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | | container-log4j | Change values in Hadoop YARN's container-log4j.properties file. | | core-site | Change values in Hadoop's core-site.xml file. | | emrfs-site | Change EMRFS settings. | | flink-conf | Change flink-conf.yaml settings. | | flink-log4j | Change Flink log4j.properties settings. | | flink-log4j-yarn-session | Change Flink log4j-yarn-session.properties settings. | | flink-log4j-cli | Change Flink log4j-cli.properties settings. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | | hadoop-ssl-server | Change hadoop ssl server configuration | | hadoop-ssl-client | Change hadoop ssl client configuration | | hbase | Amazon EMR-curated settings for Apache HBase. | | hbase-env | Change values in HBase's environment. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbase.properties file. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | | hbase-site | Change values in HBase's hbase-site.xml file. | | hdfs-encryption-zones | Configure HDFS encryption zones. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | | hcatalog-env | Change values in HCatalog's environment. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | | hcatalog-webhcat-log4j2 | Change values in HCatalog WebHCat's log4j2.properties. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | | hive-beeline-log4j2 | Change values in Hive's beeline-log4j2.properties file. | | hive-parquet-logging | Change values in Hive's parquet-logging.properties file. | | hive-env | Change values in the Hive environment. | | hive-exec-log4j2 | Change values in Hive's hive-exec-log4j2.properties file. | | hive-llap-daemon-log4j2 | Change values in Hive's llap-daemon-log4j2.properties file. | | hive-log4j2 | Change values in Hive's hive-log4j2.properties file. | | hive-site | Change values in Hive's hive-site.xml file | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | | hue-ini | Change values in Hue's ini file | | httpfs-env | Change values in the HTTPFS environment. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | | jupyter-notebook-conf | Change values in Jupyter Notebook's jupyter\$1notebook\$1config.py file. | | jupyter-hub-conf | Change values in JupyterHubs's jupyterhub\$1config.py file. | | jupyter-s3-conf | Configure Jupyter Notebook S3 persistence. | | jupyter-sparkmagic-conf | Change values in Sparkmagic's config.json file. | | livy-conf | Change values in Livy's livy.conf file. | | livy-env | Change values in the Livy environment. | | livy-log4j | Change Livy log4j.properties settings. | | mapred-env | Change values in the MapReduce application's environment. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | | oozie-env | Change values in Oozie's environment. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | | oozie-site | Change values in Oozie's oozie-site.xml file. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | | pig-env | Change values in the Pig environment. | | pig-properties | Change values in Pig's pig.properties file. | | pig-log4j | Change values in Pig's log4j.properties file. | | presto-log | Change values in Presto's log.properties file. | | presto-config | Change values in Presto's config.properties file. | | presto-password-authenticator | Change values in Presto's password-authenticator.properties file. | | presto-env | Change values in Presto's presto-env.sh file. | | presto-node | Change values in Presto's node.properties file. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | | presto-connector-hive | Change values in Presto's hive.properties file. | | presto-connector-jmx | Change values in Presto's jmx.properties file. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | | presto-connector-localfile | Change values in Presto's localfile.properties file. | | presto-connector-memory | Change values in Presto's memory.properties file. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | | presto-connector-redis | Change values in Presto's redis.properties file. | | presto-connector-redshift | Change values in Presto's redshift.properties file. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | | presto-connector-tpcds | Change values in Presto's tpcds.properties file. | | spark | Amazon EMR-curated settings for Apache Spark. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | | spark-env | Change values in the Spark environment. | | spark-hive-site | Change values in Spark's hive-site.xml file | | spark-log4j | Change values in Spark's log4j.properties file. | | spark-metrics | Change values in Spark's metrics.properties file. | | sqoop-env | Change values in Sqoop's environment. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | | tez-site | Change values in Tez's tez-site.xml file. | | yarn-env | Change values in the YARN environment. | | yarn-site | Change values in YARN's yarn-site.xml file. | | zeppelin-env | Change values in the Zeppelin environment. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | # Amazon EMR release 5.24.0 ## 5.24.0 application versions This release includes the following applications: [https://flink.apache.org/](https://flink.apache.org/), [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [http://gethue.com/](http://gethue.com/), [https://jupyterhub.readthedocs.io/en/latest/#](https://jupyterhub.readthedocs.io/en/latest/#), [https://livy.incubator.apache.org/](https://livy.incubator.apache.org/), [https://mxnet.incubator.apache.org/](https://mxnet.incubator.apache.org/), [http://mahout.apache.org/](http://mahout.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://www.tensorflow.org/](https://www.tensorflow.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-5.24.0 | emr-5.23.1 | emr-5.23.0 | emr-5.22.0 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.11.546 | 1.11.519 | 1.11.519 | 1.11.510 | | Python | 2.7, 3.6 | 2.7, 3.6 | 2.7, 3.6 | 2.7, 3.6 | | Scala | 2.11.12 | 2.11.12 | 2.11.12 | 2.11.12 | | AmazonCloudWatchAgent | - | - | - | - | | Delta | - | - | - | - | | Flink | 1.8.0 | 1.7.1 | 1.7.1 | 1.7.1 | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 1.4.9 | 1.4.9 | 1.4.9 | 1.4.9 | | HCatalog | 2.3.4-amzn-2 | 2.3.4-amzn-1 | 2.3.4-amzn-1 | 2.3.4-amzn-1 | | Hadoop | 2.8.5-amzn-4 | 2.8.5-amzn-3 | 2.8.5-amzn-3 | 2.8.5-amzn-2 | | Hive | 2.3.4-amzn-2 | 2.3.4-amzn-1 | 2.3.4-amzn-1 | 2.3.4-amzn-1 | | Hudi | - | - | - | - | | Hue | 4.4.0 | 4.3.0 | 4.3.0 | 4.3.0 | | Iceberg | - | - | - | - | | JupyterEnterpriseGateway | - | - | - | - | | JupyterHub | 0.9.6 | 0.9.4 | 0.9.4 | 0.9.4 | | Livy | 0.6.0-incubating | 0.5.0-incubating | 0.5.0-incubating | 0.5.0-incubating | | MXNet | 1.4.0 | 1.3.1 | 1.3.1 | 1.3.1 | | Mahout | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | | Oozie | 5.1.0 | 5.1.0 | 5.1.0 | 5.1.0 | | Phoenix | 4.14.1-HBase-1.4 | 4.14.1-HBase-1.4 | 4.14.1-HBase-1.4 | 4.14.1-HBase-1.4 | | Pig | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | | Presto | 0.219 | 0.215 | 0.215 | 0.215 | | Spark | 2.4.2 | 2.4.0 | 2.4.0 | 2.4.0 | | Sqoop | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | | TensorFlow | 1.12.0 | 1.12.0 | 1.12.0 | 1.12.0 | | Tez | 0.9.1 | 0.9.1 | 0.9.1 | 0.9.1 | | Trino (PrestoSQL) | - | - | - | - | | Zeppelin | 0.8.1 | 0.8.1 | 0.8.1 | 0.8.1 | | ZooKeeper | 3.4.13 | 3.4.13 | 3.4.13 | 3.4.13 | ## 5.24.0 release notes The following release notes include information for Amazon EMR release 5.24.0. Changes are relative to 5.23.0. Initial release date: June 11, 2019 Last updated date: August 5, 2019 **Upgrades** + Flink 1.8.0 + Hue 4.4.0 + JupyterHub 0.9.6 + Livy 0.6.0 + MxNet 1.4.0 + Presto 0.219 + Spark 2.4.2 + AWS SDK for Java 1.11.546 + Connectors and drivers: + DynamoDB Connector 4.9.0 + MariaDB Connector 2.4.1 + Amazon Redshift JDBC Driver 1.2.27.1051 **Changes, enhancements, and resolved issues** + Spark + Added optimization to dynamically prune partitions. The optimization is disabled by default. To enable it, set the Spark configuration parameter `spark.sql.dynamicPartitionPruning.enabled` to `true`. + Improved performance of `INTERSECT` queries. This optimization is disabled by default. To enable it, set the Spark configuration parameter `spark.sql.optimizer.distinctBeforeIntersect.enabled` to `true`. + Added optimization to flatten scalar subqueries with aggregates that use the same relation. The optimization is disabled by default. To enable it, set the Spark configuration parameter `spark.sql.optimizer.flattenScalarSubqueriesWithAggregates.enabled` to `true`. + Improved whole stage code generation. For more information, see [Optimizing Spark Performance](https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-spark-performance.html). + Local Disk Encryption and S3 Encryption Improvements in Security Configurations (August 5, 2019) + Separated Amazon S3 encryption settings from local disk encryption settings in security configuration setup. + Added an option to enable EBS encryption. Selecting this option encrypts the root device volume in addition to storage volumes. Previous versions required using a custom AMI to encrypt the root device volume. + For more information, see [Encryption Options](https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-data-encryption-options.html) in the *Amazon EMR Management Guide*. **Known issues** + Known issue in clusters with multiple primary nodes and Kerberos authentication If you run clusters with multiple primary nodes and Kerberos authentication in Amazon EMR releases 5.20.0 and later, you may encounter problems with cluster operations such as scale down or step submission, after the cluster has been running for some time. The time period depends on the Kerberos ticket validity period that you defined. The scale-down problem impacts both automatic scale-down and explicit scale down requests that you submitted. Additional cluster operations can also be impacted. Workaround: + SSH as `hadoop` user to the lead primary node of the EMR cluster with multiple primary nodes. + Run the following command to renew Kerberos ticket for `hadoop` user. ``` kinit -kt ``` Typically, the keytab file is located at `/etc/hadoop.keytab` and the principal is in the form of `hadoop/@`. **Note** This workaround will be effective for the time period the Kerberos ticket is valid. This duration is 10 hours by default, but can configured by your Kerberos settings. You must re-run the above command once the Kerberos ticket expires. ## 5.24.0 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | aws-sagemaker-spark-sdk | 1.2.1 | Amazon SageMaker Spark SDK | | emr-ddb | 4.9.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 2.8.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.4.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-s3-dist-cp | 2.11.0 | Distributed copy application optimized for Amazon S3. | | emr-s3-select | 1.3.0 | EMR S3Select Connector | | emrfs | 2.33.0 | Amazon S3 connector for Hadoop ecosystem applications. | | flink-client | 1.8.0 | Apache Flink command line client scripts and applications. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 2.8.5-amzn-4 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 2.8.5-amzn-4 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 2.8.5-amzn-4 | HDFS command-line client and library | | hadoop-hdfs-namenode | 2.8.5-amzn-4 | HDFS service for tracking file names and block locations. | | hadoop-hdfs-journalnode | 2.8.5-amzn-4 | HDFS service for managing the Hadoop filesystem journal on HA clusters. | | hadoop-httpfs-server | 2.8.5-amzn-4 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 2.8.5-amzn-4 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 2.8.5-amzn-4 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 2.8.5-amzn-4 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 2.8.5-amzn-4 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 2.8.5-amzn-4 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 1.4.9 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 1.4.9 | Service for serving one or more HBase regions. | | hbase-client | 1.4.9 | HBase command-line client. | | hbase-rest-server | 1.4.9 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 1.4.9 | Service providing a Thrift endpoint to HBase. | | hcatalog-client | 2.3.4-amzn-2 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 2.3.4-amzn-2 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 2.3.4-amzn-2 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 2.3.4-amzn-2 | Hive command line client. | | hive-hbase | 2.3.4-amzn-2 | Hive-hbase client. | | hive-metastore-server | 2.3.4-amzn-2 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server2 | 2.3.4-amzn-2 | Service for accepting Hive queries as web requests. | | hue-server | 4.4.0 | Web application for analyzing data using Hadoop ecosystem applications | | jupyterhub | 0.9.6 | Multi-user server for Jupyter notebooks | | livy-server | 0.6.0-incubating | REST interface for interacting with Apache Spark | | nginx | 1.12.1 | nginx [engine x] is an HTTP and reverse proxy server | | mahout-client | 0.13.0 | Library for machine learning. | | mxnet | 1.4.0 | A flexible, scalable, and efficient library for deep learning. | | mysql-server | 5.5.54\$1 | MySQL database server. | | nvidia-cuda | 9.2.88 | Nvidia drivers and Cuda toolkit | | oozie-client | 5.1.0 | Oozie command-line client. | | oozie-server | 5.1.0 | Service for accepting Oozie workflow requests. | | opencv | 3.4.0 | Open Source Computer Vision Library. | | phoenix-library | 4.14.1-HBase-1.4 | The phoenix libraries for server and client | | phoenix-query-server | 4.14.1-HBase-1.4 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.219 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.219 | Service for executing pieces of a query. | | pig-client | 0.17.0 | Pig command-line client. | | r | 3.4.1 | The R Project for Statistical Computing | | spark-client | 2.4.2 | Spark command-line clients. | | spark-history-server | 2.4.2 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 2.4.2 | In-memory execution engine for YARN. | | spark-yarn-slave | 2.4.2 | Apache Spark libraries needed by YARN slaves. | | sqoop-client | 1.4.7 | Apache Sqoop command-line client. | | tensorflow | 1.12.0 | TensorFlow open source software library for high performance numerical computation. | | tez-on-yarn | 0.9.1 | The tez YARN application and libraries. | | webserver | 2.4.25\$1 | Apache HTTP server. | | zeppelin-server | 0.8.1 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.4.13 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.4.13 | ZooKeeper command line client. | ## 5.24.0 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). **emr-5.24.0 classifications** | Classifications | Description | | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | | container-log4j | Change values in Hadoop YARN's container-log4j.properties file. | | core-site | Change values in Hadoop's core-site.xml file. | | emrfs-site | Change EMRFS settings. | | flink-conf | Change flink-conf.yaml settings. | | flink-log4j | Change Flink log4j.properties settings. | | flink-log4j-yarn-session | Change Flink log4j-yarn-session.properties settings. | | flink-log4j-cli | Change Flink log4j-cli.properties settings. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | | hadoop-ssl-server | Change hadoop ssl server configuration | | hadoop-ssl-client | Change hadoop ssl client configuration | | hbase | Amazon EMR-curated settings for Apache HBase. | | hbase-env | Change values in HBase's environment. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbase.properties file. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | | hbase-site | Change values in HBase's hbase-site.xml file. | | hdfs-encryption-zones | Configure HDFS encryption zones. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | | hcatalog-env | Change values in HCatalog's environment. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | | hcatalog-webhcat-log4j2 | Change values in HCatalog WebHCat's log4j2.properties. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | | hive-beeline-log4j2 | Change values in Hive's beeline-log4j2.properties file. | | hive-parquet-logging | Change values in Hive's parquet-logging.properties file. | | hive-env | Change values in the Hive environment. | | hive-exec-log4j2 | Change values in Hive's hive-exec-log4j2.properties file. | | hive-llap-daemon-log4j2 | Change values in Hive's llap-daemon-log4j2.properties file. | | hive-log4j2 | Change values in Hive's hive-log4j2.properties file. | | hive-site | Change values in Hive's hive-site.xml file | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | | hue-ini | Change values in Hue's ini file | | httpfs-env | Change values in the HTTPFS environment. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | | jupyter-notebook-conf | Change values in Jupyter Notebook's jupyter\$1notebook\$1config.py file. | | jupyter-hub-conf | Change values in JupyterHubs's jupyterhub\$1config.py file. | | jupyter-s3-conf | Configure Jupyter Notebook S3 persistence. | | jupyter-sparkmagic-conf | Change values in Sparkmagic's config.json file. | | livy-conf | Change values in Livy's livy.conf file. | | livy-env | Change values in the Livy environment. | | livy-log4j | Change Livy log4j.properties settings. | | mapred-env | Change values in the MapReduce application's environment. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | | oozie-env | Change values in Oozie's environment. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | | oozie-site | Change values in Oozie's oozie-site.xml file. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | | pig-env | Change values in the Pig environment. | | pig-properties | Change values in Pig's pig.properties file. | | pig-log4j | Change values in Pig's log4j.properties file. | | presto-log | Change values in Presto's log.properties file. | | presto-config | Change values in Presto's config.properties file. | | presto-password-authenticator | Change values in Presto's password-authenticator.properties file. | | presto-env | Change values in Presto's presto-env.sh file. | | presto-node | Change values in Presto's node.properties file. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | | presto-connector-hive | Change values in Presto's hive.properties file. | | presto-connector-jmx | Change values in Presto's jmx.properties file. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | | presto-connector-localfile | Change values in Presto's localfile.properties file. | | presto-connector-memory | Change values in Presto's memory.properties file. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | | presto-connector-redis | Change values in Presto's redis.properties file. | | presto-connector-redshift | Change values in Presto's redshift.properties file. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | | presto-connector-tpcds | Change values in Presto's tpcds.properties file. | | spark | Amazon EMR-curated settings for Apache Spark. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | | spark-env | Change values in the Spark environment. | | spark-hive-site | Change values in Spark's hive-site.xml file | | spark-log4j | Change values in Spark's log4j.properties file. | | spark-metrics | Change values in Spark's metrics.properties file. | | sqoop-env | Change values in Sqoop's environment. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | | tez-site | Change values in Tez's tez-site.xml file. | | yarn-env | Change values in the YARN environment. | | yarn-site | Change values in YARN's yarn-site.xml file. | | zeppelin-env | Change values in the Zeppelin environment. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | # Amazon EMR release 5.23.1 ## 5.23.1 application versions This release includes the following applications: [https://flink.apache.org/](https://flink.apache.org/), [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [http://gethue.com/](http://gethue.com/), [https://jupyterhub.readthedocs.io/en/latest/#](https://jupyterhub.readthedocs.io/en/latest/#), [https://livy.incubator.apache.org/](https://livy.incubator.apache.org/), [https://mxnet.incubator.apache.org/](https://mxnet.incubator.apache.org/), [http://mahout.apache.org/](http://mahout.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://www.tensorflow.org/](https://www.tensorflow.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-5.23.1 | emr-5.23.0 | emr-5.22.0 | emr-5.21.2 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.11.519 | 1.11.519 | 1.11.510 | 1.11.479 | | Python | 2.7, 3.6 | 2.7, 3.6 | 2.7, 3.6 | 2.7, 3.6 | | Scala | 2.11.12 | 2.11.12 | 2.11.12 | 2.11.12 | | AmazonCloudWatchAgent | - | - | - | - | | Delta | - | - | - | - | | Flink | 1.7.1 | 1.7.1 | 1.7.1 | 1.7.0 | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 1.4.9 | 1.4.9 | 1.4.9 | 1.4.8 | | HCatalog | 2.3.4-amzn-1 | 2.3.4-amzn-1 | 2.3.4-amzn-1 | 2.3.4-amzn-0 | | Hadoop | 2.8.5-amzn-3 | 2.8.5-amzn-3 | 2.8.5-amzn-2 | 2.8.5-amzn-1 | | Hive | 2.3.4-amzn-1 | 2.3.4-amzn-1 | 2.3.4-amzn-1 | 2.3.4-amzn-0 | | Hudi | - | - | - | - | | Hue | 4.3.0 | 4.3.0 | 4.3.0 | 4.3.0 | | Iceberg | - | - | - | - | | JupyterEnterpriseGateway | - | - | - | - | | JupyterHub | 0.9.4 | 0.9.4 | 0.9.4 | 0.9.4 | | Livy | 0.5.0-incubating | 0.5.0-incubating | 0.5.0-incubating | 0.5.0-incubating | | MXNet | 1.3.1 | 1.3.1 | 1.3.1 | 1.3.1 | | Mahout | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | | Oozie | 5.1.0 | 5.1.0 | 5.1.0 | 5.0.0 | | Phoenix | 4.14.1-HBase-1.4 | 4.14.1-HBase-1.4 | 4.14.1-HBase-1.4 | 4.14.0-HBase-1.4 | | Pig | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | | Presto | 0.215 | 0.215 | 0.215 | 0.215 | | Spark | 2.4.0 | 2.4.0 | 2.4.0 | 2.4.0 | | Sqoop | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | | TensorFlow | 1.12.0 | 1.12.0 | 1.12.0 | 1.12.0 | | Tez | 0.9.1 | 0.9.1 | 0.9.1 | 0.9.1 | | Trino (PrestoSQL) | - | - | - | - | | Zeppelin | 0.8.1 | 0.8.1 | 0.8.1 | 0.8.0 | | ZooKeeper | 3.4.13 | 3.4.13 | 3.4.13 | 3.4.13 | ## 5.23.1 release notes This is a patch release. All applications and components are the same as the previous Amazon EMR release. Instance Metadata Service (IMDS) V2 support status: Amazon EMR 5.23.1, 5.27.1 and 5.32 or later components use IMDSv2 for all IMDS calls. For IMDS calls in your application code, you can use both IMDSv1 and IMDSv2, or configure the IMDS to use only IMDSv2 for added security. For other 5.x EMR releases, disabling IMDSv1 causes cluster startup failure. ## 5.23.1 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | aws-sagemaker-spark-sdk | 1.2.1 | Amazon SageMaker Spark SDK | | emr-ddb | 4.8.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 2.7.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.4.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-s3-dist-cp | 2.11.0 | Distributed copy application optimized for Amazon S3. | | emr-s3-select | 1.2.0 | EMR S3Select Connector | | emrfs | 2.32.0 | Amazon S3 connector for Hadoop ecosystem applications. | | flink-client | 1.7.1 | Apache Flink command line client scripts and applications. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 2.8.5-amzn-3 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 2.8.5-amzn-3 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 2.8.5-amzn-3 | HDFS command-line client and library | | hadoop-hdfs-namenode | 2.8.5-amzn-3 | HDFS service for tracking file names and block locations. | | hadoop-hdfs-journalnode | 2.8.5-amzn-3 | HDFS service for managing the Hadoop filesystem journal on HA clusters. | | hadoop-httpfs-server | 2.8.5-amzn-3 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 2.8.5-amzn-3 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 2.8.5-amzn-3 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 2.8.5-amzn-3 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 2.8.5-amzn-3 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 2.8.5-amzn-3 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 1.4.9 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 1.4.9 | Service for serving one or more HBase regions. | | hbase-client | 1.4.9 | HBase command-line client. | | hbase-rest-server | 1.4.9 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 1.4.9 | Service providing a Thrift endpoint to HBase. | | hcatalog-client | 2.3.4-amzn-1 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 2.3.4-amzn-1 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 2.3.4-amzn-1 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 2.3.4-amzn-1 | Hive command line client. | | hive-hbase | 2.3.4-amzn-1 | Hive-hbase client. | | hive-metastore-server | 2.3.4-amzn-1 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server2 | 2.3.4-amzn-1 | Service for accepting Hive queries as web requests. | | hue-server | 4.3.0 | Web application for analyzing data using Hadoop ecosystem applications | | jupyterhub | 0.9.4 | Multi-user server for Jupyter notebooks | | livy-server | 0.5.0-incubating | REST interface for interacting with Apache Spark | | nginx | 1.12.1 | nginx [engine x] is an HTTP and reverse proxy server | | mahout-client | 0.13.0 | Library for machine learning. | | mxnet | 1.3.1 | A flexible, scalable, and efficient library for deep learning. | | mysql-server | 5.5.54\$1 | MySQL database server. | | nvidia-cuda | 9.2.88 | Nvidia drivers and Cuda toolkit | | oozie-client | 5.1.0 | Oozie command-line client. | | oozie-server | 5.1.0 | Service for accepting Oozie workflow requests. | | opencv | 3.4.0 | Open Source Computer Vision Library. | | phoenix-library | 4.14.1-HBase-1.4 | The phoenix libraries for server and client | | phoenix-query-server | 4.14.1-HBase-1.4 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.215 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.215 | Service for executing pieces of a query. | | pig-client | 0.17.0 | Pig command-line client. | | r | 3.4.1 | The R Project for Statistical Computing | | spark-client | 2.4.0 | Spark command-line clients. | | spark-history-server | 2.4.0 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 2.4.0 | In-memory execution engine for YARN. | | spark-yarn-slave | 2.4.0 | Apache Spark libraries needed by YARN slaves. | | sqoop-client | 1.4.7 | Apache Sqoop command-line client. | | tensorflow | 1.12.0 | TensorFlow open source software library for high performance numerical computation. | | tez-on-yarn | 0.9.1 | The tez YARN application and libraries. | | webserver | 2.4.25\$1 | Apache HTTP server. | | zeppelin-server | 0.8.1 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.4.13 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.4.13 | ZooKeeper command line client. | ## 5.23.1 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). **emr-5.23.1 classifications** | Classifications | Description | | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | | container-log4j | Change values in Hadoop YARN's container-log4j.properties file. | | core-site | Change values in Hadoop's core-site.xml file. | | emrfs-site | Change EMRFS settings. | | flink-conf | Change flink-conf.yaml settings. | | flink-log4j | Change Flink log4j.properties settings. | | flink-log4j-yarn-session | Change Flink log4j-yarn-session.properties settings. | | flink-log4j-cli | Change Flink log4j-cli.properties settings. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | | hadoop-ssl-server | Change hadoop ssl server configuration | | hadoop-ssl-client | Change hadoop ssl client configuration | | hbase | Amazon EMR-curated settings for Apache HBase. | | hbase-env | Change values in HBase's environment. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbase.properties file. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | | hbase-site | Change values in HBase's hbase-site.xml file. | | hdfs-encryption-zones | Configure HDFS encryption zones. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | | hcatalog-env | Change values in HCatalog's environment. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | | hcatalog-webhcat-log4j2 | Change values in HCatalog WebHCat's log4j2.properties. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | | hive-beeline-log4j2 | Change values in Hive's beeline-log4j2.properties file. | | hive-parquet-logging | Change values in Hive's parquet-logging.properties file. | | hive-env | Change values in the Hive environment. | | hive-exec-log4j2 | Change values in Hive's hive-exec-log4j2.properties file. | | hive-llap-daemon-log4j2 | Change values in Hive's llap-daemon-log4j2.properties file. | | hive-log4j2 | Change values in Hive's hive-log4j2.properties file. | | hive-site | Change values in Hive's hive-site.xml file | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | | hue-ini | Change values in Hue's ini file | | httpfs-env | Change values in the HTTPFS environment. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | | jupyter-notebook-conf | Change values in Jupyter Notebook's jupyter\$1notebook\$1config.py file. | | jupyter-hub-conf | Change values in JupyterHubs's jupyterhub\$1config.py file. | | jupyter-s3-conf | Configure Jupyter Notebook S3 persistence. | | jupyter-sparkmagic-conf | Change values in Sparkmagic's config.json file. | | livy-conf | Change values in Livy's livy.conf file. | | livy-env | Change values in the Livy environment. | | livy-log4j | Change Livy log4j.properties settings. | | mapred-env | Change values in the MapReduce application's environment. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | | oozie-env | Change values in Oozie's environment. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | | oozie-site | Change values in Oozie's oozie-site.xml file. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | | pig-env | Change values in the Pig environment. | | pig-properties | Change values in Pig's pig.properties file. | | pig-log4j | Change values in Pig's log4j.properties file. | | presto-log | Change values in Presto's log.properties file. | | presto-config | Change values in Presto's config.properties file. | | presto-password-authenticator | Change values in Presto's password-authenticator.properties file. | | presto-env | Change values in Presto's presto-env.sh file. | | presto-node | Change values in Presto's node.properties file. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | | presto-connector-hive | Change values in Presto's hive.properties file. | | presto-connector-jmx | Change values in Presto's jmx.properties file. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | | presto-connector-localfile | Change values in Presto's localfile.properties file. | | presto-connector-memory | Change values in Presto's memory.properties file. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | | presto-connector-redis | Change values in Presto's redis.properties file. | | presto-connector-redshift | Change values in Presto's redshift.properties file. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | | presto-connector-tpcds | Change values in Presto's tpcds.properties file. | | spark | Amazon EMR-curated settings for Apache Spark. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | | spark-env | Change values in the Spark environment. | | spark-hive-site | Change values in Spark's hive-site.xml file | | spark-log4j | Change values in Spark's log4j.properties file. | | spark-metrics | Change values in Spark's metrics.properties file. | | sqoop-env | Change values in Sqoop's environment. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | | tez-site | Change values in Tez's tez-site.xml file. | | yarn-env | Change values in the YARN environment. | | yarn-site | Change values in YARN's yarn-site.xml file. | | zeppelin-env | Change values in the Zeppelin environment. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | # Amazon EMR release 5.23.0 ## 5.23.0 application versions This release includes the following applications: [https://flink.apache.org/](https://flink.apache.org/), [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [http://gethue.com/](http://gethue.com/), [https://jupyterhub.readthedocs.io/en/latest/#](https://jupyterhub.readthedocs.io/en/latest/#), [https://livy.incubator.apache.org/](https://livy.incubator.apache.org/), [https://mxnet.incubator.apache.org/](https://mxnet.incubator.apache.org/), [http://mahout.apache.org/](http://mahout.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://www.tensorflow.org/](https://www.tensorflow.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-5.23.0 | emr-5.22.0 | emr-5.21.2 | emr-5.21.1 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.11.519 | 1.11.510 | 1.11.479 | 1.11.479 | | Python | 2.7, 3.6 | 2.7, 3.6 | 2.7, 3.6 | 2.7, 3.6 | | Scala | 2.11.12 | 2.11.12 | 2.11.12 | 2.11.12 | | AmazonCloudWatchAgent | - | - | - | - | | Delta | - | - | - | - | | Flink | 1.7.1 | 1.7.1 | 1.7.0 | 1.7.0 | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 1.4.9 | 1.4.9 | 1.4.8 | 1.4.8 | | HCatalog | 2.3.4-amzn-1 | 2.3.4-amzn-1 | 2.3.4-amzn-0 | 2.3.4-amzn-0 | | Hadoop | 2.8.5-amzn-3 | 2.8.5-amzn-2 | 2.8.5-amzn-1 | 2.8.5-amzn-1 | | Hive | 2.3.4-amzn-1 | 2.3.4-amzn-1 | 2.3.4-amzn-0 | 2.3.4-amzn-0 | | Hudi | - | - | - | - | | Hue | 4.3.0 | 4.3.0 | 4.3.0 | 4.3.0 | | Iceberg | - | - | - | - | | JupyterEnterpriseGateway | - | - | - | - | | JupyterHub | 0.9.4 | 0.9.4 | 0.9.4 | 0.9.4 | | Livy | 0.5.0-incubating | 0.5.0-incubating | 0.5.0-incubating | 0.5.0-incubating | | MXNet | 1.3.1 | 1.3.1 | 1.3.1 | 1.3.1 | | Mahout | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | | Oozie | 5.1.0 | 5.1.0 | 5.0.0 | 5.0.0 | | Phoenix | 4.14.1-HBase-1.4 | 4.14.1-HBase-1.4 | 4.14.0-HBase-1.4 | 4.14.0-HBase-1.4 | | Pig | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | | Presto | 0.215 | 0.215 | 0.215 | 0.215 | | Spark | 2.4.0 | 2.4.0 | 2.4.0 | 2.4.0 | | Sqoop | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | | TensorFlow | 1.12.0 | 1.12.0 | 1.12.0 | 1.12.0 | | Tez | 0.9.1 | 0.9.1 | 0.9.1 | 0.9.1 | | Trino (PrestoSQL) | - | - | - | - | | Zeppelin | 0.8.1 | 0.8.1 | 0.8.0 | 0.8.0 | | ZooKeeper | 3.4.13 | 3.4.13 | 3.4.13 | 3.4.13 | ## 5.23.0 release notes The following release notes include information for Amazon EMR release 5.23.0. Changes are relative to 5.22.0. Initial release date: April 01, 2019 Last updated date: April 30, 2019 **Upgrades** + AWS SDK for Java 1.11.519 **New features** + (April 30, 2019) With Amazon EMR 5.23.0 and later, you can launch a cluster with three primary nodes to support high availability of applications like YARN Resource Manager, HDFS NameNode, Spark, Hive, and Ganglia. The primary node is no longer a potential single point of failure with this feature. If one of the primary nodes fails, Amazon EMR automatically fails over to a standby primary node and replaces the failed primary node with a new one with the same configuration and bootstrap actions. For more information, see [Plan and Configure Primary Nodes](https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-plan-ha.html). **Known issues** + Tez UI (Fixed in Amazon EMR release 5.26.0) Tez UI does not work on an EMR cluster with multiple primary nodes. + Hue (Fixed in Amazon EMR release 5.24.0) + Hue running on Amazon EMR does not support Solr. Beginning with Amazon EMR release 5.20.0, a misconfiguration issue causes Solr to be enabled and a harmless error message to appear similar to the following: `Solr server could not be contacted properly: HTTPConnectionPool('host=ip-xx-xx-xx-xx.ec2.internal', port=1978): Max retries exceeded with url: /solr/admin/info/system?user.name=hue&doAs=administrator&wt=json (Caused by NewConnectionError(': Failed to establish a new connection: [Errno 111] Connection refused',))` **To prevent the Solr error message from appearing:** 1. Connect to the primary node command line using SSH. 1. Use a text editor to open the `hue.ini` file. For example: `sudo vim /etc/hue/conf/hue.ini` 1. Search for the term `appblacklist` and modify the line to the following: ``` appblacklist = search ``` 1. Save your changes and restart Hue as shown in the following example: ``` sudo stop hue; sudo start hue ``` + Known issue in clusters with multiple primary nodes and Kerberos authentication If you run clusters with multiple primary nodes and Kerberos authentication in Amazon EMR releases 5.20.0 and later, you may encounter problems with cluster operations such as scale down or step submission, after the cluster has been running for some time. The time period depends on the Kerberos ticket validity period that you defined. The scale-down problem impacts both automatic scale-down and explicit scale down requests that you submitted. Additional cluster operations can also be impacted. Workaround: + SSH as `hadoop` user to the lead primary node of the EMR cluster with multiple primary nodes. + Run the following command to renew Kerberos ticket for `hadoop` user. ``` kinit -kt ``` Typically, the keytab file is located at `/etc/hadoop.keytab` and the principal is in the form of `hadoop/@`. **Note** This workaround will be effective for the time period the Kerberos ticket is valid. This duration is 10 hours by default, but can configured by your Kerberos settings. You must re-run the above command once the Kerberos ticket expires. ## 5.23.0 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | aws-sagemaker-spark-sdk | 1.2.1 | Amazon SageMaker Spark SDK | | emr-ddb | 4.8.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 2.7.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.4.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-s3-dist-cp | 2.11.0 | Distributed copy application optimized for Amazon S3. | | emr-s3-select | 1.2.0 | EMR S3Select Connector | | emrfs | 2.32.0 | Amazon S3 connector for Hadoop ecosystem applications. | | flink-client | 1.7.1 | Apache Flink command line client scripts and applications. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 2.8.5-amzn-3 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 2.8.5-amzn-3 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 2.8.5-amzn-3 | HDFS command-line client and library | | hadoop-hdfs-namenode | 2.8.5-amzn-3 | HDFS service for tracking file names and block locations. | | hadoop-hdfs-journalnode | 2.8.5-amzn-3 | HDFS service for managing the Hadoop filesystem journal on HA clusters. | | hadoop-httpfs-server | 2.8.5-amzn-3 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 2.8.5-amzn-3 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 2.8.5-amzn-3 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 2.8.5-amzn-3 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 2.8.5-amzn-3 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 2.8.5-amzn-3 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 1.4.9 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 1.4.9 | Service for serving one or more HBase regions. | | hbase-client | 1.4.9 | HBase command-line client. | | hbase-rest-server | 1.4.9 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 1.4.9 | Service providing a Thrift endpoint to HBase. | | hcatalog-client | 2.3.4-amzn-1 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 2.3.4-amzn-1 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 2.3.4-amzn-1 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 2.3.4-amzn-1 | Hive command line client. | | hive-hbase | 2.3.4-amzn-1 | Hive-hbase client. | | hive-metastore-server | 2.3.4-amzn-1 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server2 | 2.3.4-amzn-1 | Service for accepting Hive queries as web requests. | | hue-server | 4.3.0 | Web application for analyzing data using Hadoop ecosystem applications | | jupyterhub | 0.9.4 | Multi-user server for Jupyter notebooks | | livy-server | 0.5.0-incubating | REST interface for interacting with Apache Spark | | nginx | 1.12.1 | nginx [engine x] is an HTTP and reverse proxy server | | mahout-client | 0.13.0 | Library for machine learning. | | mxnet | 1.3.1 | A flexible, scalable, and efficient library for deep learning. | | mysql-server | 5.5.54\$1 | MySQL database server. | | nvidia-cuda | 9.2.88 | Nvidia drivers and Cuda toolkit | | oozie-client | 5.1.0 | Oozie command-line client. | | oozie-server | 5.1.0 | Service for accepting Oozie workflow requests. | | opencv | 3.4.0 | Open Source Computer Vision Library. | | phoenix-library | 4.14.1-HBase-1.4 | The phoenix libraries for server and client | | phoenix-query-server | 4.14.1-HBase-1.4 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.215 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.215 | Service for executing pieces of a query. | | pig-client | 0.17.0 | Pig command-line client. | | r | 3.4.1 | The R Project for Statistical Computing | | spark-client | 2.4.0 | Spark command-line clients. | | spark-history-server | 2.4.0 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 2.4.0 | In-memory execution engine for YARN. | | spark-yarn-slave | 2.4.0 | Apache Spark libraries needed by YARN slaves. | | sqoop-client | 1.4.7 | Apache Sqoop command-line client. | | tensorflow | 1.12.0 | TensorFlow open source software library for high performance numerical computation. | | tez-on-yarn | 0.9.1 | The tez YARN application and libraries. | | webserver | 2.4.25\$1 | Apache HTTP server. | | zeppelin-server | 0.8.1 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.4.13 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.4.13 | ZooKeeper command line client. | ## 5.23.0 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). **emr-5.23.0 classifications** | Classifications | Description | | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | | container-log4j | Change values in Hadoop YARN's container-log4j.properties file. | | core-site | Change values in Hadoop's core-site.xml file. | | emrfs-site | Change EMRFS settings. | | flink-conf | Change flink-conf.yaml settings. | | flink-log4j | Change Flink log4j.properties settings. | | flink-log4j-yarn-session | Change Flink log4j-yarn-session.properties settings. | | flink-log4j-cli | Change Flink log4j-cli.properties settings. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | | hadoop-ssl-server | Change hadoop ssl server configuration | | hadoop-ssl-client | Change hadoop ssl client configuration | | hbase | Amazon EMR-curated settings for Apache HBase. | | hbase-env | Change values in HBase's environment. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbase.properties file. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | | hbase-site | Change values in HBase's hbase-site.xml file. | | hdfs-encryption-zones | Configure HDFS encryption zones. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | | hcatalog-env | Change values in HCatalog's environment. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | | hcatalog-webhcat-log4j2 | Change values in HCatalog WebHCat's log4j2.properties. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | | hive-beeline-log4j2 | Change values in Hive's beeline-log4j2.properties file. | | hive-parquet-logging | Change values in Hive's parquet-logging.properties file. | | hive-env | Change values in the Hive environment. | | hive-exec-log4j2 | Change values in Hive's hive-exec-log4j2.properties file. | | hive-llap-daemon-log4j2 | Change values in Hive's llap-daemon-log4j2.properties file. | | hive-log4j2 | Change values in Hive's hive-log4j2.properties file. | | hive-site | Change values in Hive's hive-site.xml file | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | | hue-ini | Change values in Hue's ini file | | httpfs-env | Change values in the HTTPFS environment. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | | jupyter-notebook-conf | Change values in Jupyter Notebook's jupyter\$1notebook\$1config.py file. | | jupyter-hub-conf | Change values in JupyterHubs's jupyterhub\$1config.py file. | | jupyter-s3-conf | Configure Jupyter Notebook S3 persistence. | | jupyter-sparkmagic-conf | Change values in Sparkmagic's config.json file. | | livy-conf | Change values in Livy's livy.conf file. | | livy-env | Change values in the Livy environment. | | livy-log4j | Change Livy log4j.properties settings. | | mapred-env | Change values in the MapReduce application's environment. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | | oozie-env | Change values in Oozie's environment. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | | oozie-site | Change values in Oozie's oozie-site.xml file. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | | pig-env | Change values in the Pig environment. | | pig-properties | Change values in Pig's pig.properties file. | | pig-log4j | Change values in Pig's log4j.properties file. | | presto-log | Change values in Presto's log.properties file. | | presto-config | Change values in Presto's config.properties file. | | presto-password-authenticator | Change values in Presto's password-authenticator.properties file. | | presto-env | Change values in Presto's presto-env.sh file. | | presto-node | Change values in Presto's node.properties file. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | | presto-connector-hive | Change values in Presto's hive.properties file. | | presto-connector-jmx | Change values in Presto's jmx.properties file. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | | presto-connector-localfile | Change values in Presto's localfile.properties file. | | presto-connector-memory | Change values in Presto's memory.properties file. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | | presto-connector-redis | Change values in Presto's redis.properties file. | | presto-connector-redshift | Change values in Presto's redshift.properties file. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | | presto-connector-tpcds | Change values in Presto's tpcds.properties file. | | spark | Amazon EMR-curated settings for Apache Spark. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | | spark-env | Change values in the Spark environment. | | spark-hive-site | Change values in Spark's hive-site.xml file | | spark-log4j | Change values in Spark's log4j.properties file. | | spark-metrics | Change values in Spark's metrics.properties file. | | sqoop-env | Change values in Sqoop's environment. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | | tez-site | Change values in Tez's tez-site.xml file. | | yarn-env | Change values in the YARN environment. | | yarn-site | Change values in YARN's yarn-site.xml file. | | zeppelin-env | Change values in the Zeppelin environment. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | # Amazon EMR release 5.22.0 ## 5.22.0 application versions This release includes the following applications: [https://flink.apache.org/](https://flink.apache.org/), [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [http://gethue.com/](http://gethue.com/), [https://jupyterhub.readthedocs.io/en/latest/#](https://jupyterhub.readthedocs.io/en/latest/#), [https://livy.incubator.apache.org/](https://livy.incubator.apache.org/), [https://mxnet.incubator.apache.org/](https://mxnet.incubator.apache.org/), [http://mahout.apache.org/](http://mahout.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://www.tensorflow.org/](https://www.tensorflow.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-5.22.0 | emr-5.21.2 | emr-5.21.1 | emr-5.21.0 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.11.510 | 1.11.479 | 1.11.479 | 1.11.479 | | Python | 2.7, 3.6 | 2.7, 3.6 | 2.7, 3.6 | 2.7, 3.6 | | Scala | 2.11.12 | 2.11.12 | 2.11.12 | 2.11.12 | | AmazonCloudWatchAgent | - | - | - | - | | Delta | - | - | - | - | | Flink | 1.7.1 | 1.7.0 | 1.7.0 | 1.7.0 | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 1.4.9 | 1.4.8 | 1.4.8 | 1.4.8 | | HCatalog | 2.3.4-amzn-1 | 2.3.4-amzn-0 | 2.3.4-amzn-0 | 2.3.4-amzn-0 | | Hadoop | 2.8.5-amzn-2 | 2.8.5-amzn-1 | 2.8.5-amzn-1 | 2.8.5-amzn-1 | | Hive | 2.3.4-amzn-1 | 2.3.4-amzn-0 | 2.3.4-amzn-0 | 2.3.4-amzn-0 | | Hudi | - | - | - | - | | Hue | 4.3.0 | 4.3.0 | 4.3.0 | 4.3.0 | | Iceberg | - | - | - | - | | JupyterEnterpriseGateway | - | - | - | - | | JupyterHub | 0.9.4 | 0.9.4 | 0.9.4 | 0.9.4 | | Livy | 0.5.0-incubating | 0.5.0-incubating | 0.5.0-incubating | 0.5.0-incubating | | MXNet | 1.3.1 | 1.3.1 | 1.3.1 | 1.3.1 | | Mahout | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | | Oozie | 5.1.0 | 5.0.0 | 5.0.0 | 5.0.0 | | Phoenix | 4.14.1-HBase-1.4 | 4.14.0-HBase-1.4 | 4.14.0-HBase-1.4 | 4.14.0-HBase-1.4 | | Pig | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | | Presto | 0.215 | 0.215 | 0.215 | 0.215 | | Spark | 2.4.0 | 2.4.0 | 2.4.0 | 2.4.0 | | Sqoop | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | | TensorFlow | 1.12.0 | 1.12.0 | 1.12.0 | 1.12.0 | | Tez | 0.9.1 | 0.9.1 | 0.9.1 | 0.9.1 | | Trino (PrestoSQL) | - | - | - | - | | Zeppelin | 0.8.1 | 0.8.0 | 0.8.0 | 0.8.0 | | ZooKeeper | 3.4.13 | 3.4.13 | 3.4.13 | 3.4.13 | ## 5.22.0 release notes The following release notes include information for Amazon EMR release 5.22.0. Changes are relative to 5.21.0. **Important** Beginning with Amazon EMR release 5.22.0, Amazon EMR uses AWS Signature Version 4 exclusively to authenticate requests to Amazon S3. Earlier Amazon EMR releases use AWS Signature Version 2 in some cases, unless the release notes indicate that Signature Version 4 is used exclusively. For more information, see [Authenticating Requests (AWS Signature Version 4)](https://docs.aws.amazon.com/AmazonS3/latest/API/sig-v4-authenticating-requests.html) and [Authenticating Requests (AWS Signature Version 2)](https://docs.aws.amazon.com/AmazonS3/latest/API/auth-request-sig-v2.html) in the *Amazon Simple Storage Service Developer Guide*. Initial release date: March 20, 2019 **Upgrades** + Flink 1.7.1 + HBase 1.4.9 + Oozie 5.1.0 + Phoenix 4.14.1 + Zeppelin 0.8.1 + Connectors and drivers: + DynamoDB Connector 4.8.0 + MariaDB Connector 2.2.6 + Amazon Redshift JDBC Driver 1.2.20.1043 **New features** + Modified the default EBS configuration for EC2 instance types with EBS-only storage. When you create a cluster using Amazon EMR release 5.22.0 and later, the default amount of EBS storage increases based on the size of the instance. In addition, we split increased storage across multiple volumes, giving increased IOPS performance. If you want to use a different EBS instance storage configuration, you can specify it when you create an EMR cluster or add nodes to an existing cluster. For more information about the amount of storage and number of volumes allocated by default for each instance type, see [Default EBS Storage for Instances](https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-plan-storage.html#emr-plan-storage-ebs-storage-default) in the *Amazon EMR Management Guide*. **Changes, enhancements, and resolved issues** + Spark + Introduced a new configuration property for Spark on YARN, `spark.yarn.executor.memoryOverheadFactor`. The value of this property is a scale factor that sets the value of memory overhead to a percentage of executor memory, with a minimum of 384 MB. If memory overhead is set explicitly using `spark.yarn.executor.memoryOverhead`, this property has no effect. The default value is `0.1875`, representing 18.75%. This default for Amazon EMR leaves more space in YARN containers for executor memory overhead than the 10% default set internally by Spark. The Amazon EMR default of 18.75% empirically showed fewer memory-related failures in TPC-DS benchmarks. + Backported [SPARK-26316](https://issues.apache.org/jira/browse/SPARK-26316) to improve performance. + In Amazon EMR version 5.19.0, 5.20.0, and 5.21.0, YARN node labels are stored in an HDFS directory. In some situations, this leads to core node startup delays and then causes cluster time-out and launch failure. Beginning with Amazon EMR 5.22.0, this issue is resolved. YARN node labels are stored on the local disk of each cluster node, avoiding dependencies on HDFS. **Known issues** + Hue (Fixed in Amazon EMR release 5.24.0) + Hue running on Amazon EMR does not support Solr. Beginning with Amazon EMR release 5.20.0, a misconfiguration issue causes Solr to be enabled and a harmless error message to appear similar to the following: `Solr server could not be contacted properly: HTTPConnectionPool('host=ip-xx-xx-xx-xx.ec2.internal', port=1978): Max retries exceeded with url: /solr/admin/info/system?user.name=hue&doAs=administrator&wt=json (Caused by NewConnectionError(': Failed to establish a new connection: [Errno 111] Connection refused',))` **To prevent the Solr error message from appearing:** 1. Connect to the primary node command line using SSH. 1. Use a text editor to open the `hue.ini` file. For example: `sudo vim /etc/hue/conf/hue.ini` 1. Search for the term `appblacklist` and modify the line to the following: ``` appblacklist = search ``` 1. Save your changes and restart Hue as shown in the following example: ``` sudo stop hue; sudo start hue ``` + Known issue in clusters with multiple primary nodes and Kerberos authentication If you run clusters with multiple primary nodes and Kerberos authentication in Amazon EMR releases 5.20.0 and later, you may encounter problems with cluster operations such as scale down or step submission, after the cluster has been running for some time. The time period depends on the Kerberos ticket validity period that you defined. The scale-down problem impacts both automatic scale-down and explicit scale down requests that you submitted. Additional cluster operations can also be impacted. Workaround: + SSH as `hadoop` user to the lead primary node of the EMR cluster with multiple primary nodes. + Run the following command to renew Kerberos ticket for `hadoop` user. ``` kinit -kt ``` Typically, the keytab file is located at `/etc/hadoop.keytab` and the principal is in the form of `hadoop/@`. **Note** This workaround will be effective for the time period the Kerberos ticket is valid. This duration is 10 hours by default, but can configured by your Kerberos settings. You must re-run the above command once the Kerberos ticket expires. ## 5.22.0 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | aws-sagemaker-spark-sdk | 1.2.1 | Amazon SageMaker Spark SDK | | emr-ddb | 4.8.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 2.6.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.4.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-s3-dist-cp | 2.11.0 | Distributed copy application optimized for Amazon S3. | | emr-s3-select | 1.2.0 | EMR S3Select Connector | | emrfs | 2.31.0 | Amazon S3 connector for Hadoop ecosystem applications. | | flink-client | 1.7.1 | Apache Flink command line client scripts and applications. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 2.8.5-amzn-2 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 2.8.5-amzn-2 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 2.8.5-amzn-2 | HDFS command-line client and library | | hadoop-hdfs-namenode | 2.8.5-amzn-2 | HDFS service for tracking file names and block locations. | | hadoop-httpfs-server | 2.8.5-amzn-2 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 2.8.5-amzn-2 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 2.8.5-amzn-2 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 2.8.5-amzn-2 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 2.8.5-amzn-2 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 2.8.5-amzn-2 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 1.4.9 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 1.4.9 | Service for serving one or more HBase regions. | | hbase-client | 1.4.9 | HBase command-line client. | | hbase-rest-server | 1.4.9 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 1.4.9 | Service providing a Thrift endpoint to HBase. | | hcatalog-client | 2.3.4-amzn-1 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 2.3.4-amzn-1 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 2.3.4-amzn-1 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 2.3.4-amzn-1 | Hive command line client. | | hive-hbase | 2.3.4-amzn-1 | Hive-hbase client. | | hive-metastore-server | 2.3.4-amzn-1 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server2 | 2.3.4-amzn-1 | Service for accepting Hive queries as web requests. | | hue-server | 4.3.0 | Web application for analyzing data using Hadoop ecosystem applications | | jupyterhub | 0.9.4 | Multi-user server for Jupyter notebooks | | livy-server | 0.5.0-incubating | REST interface for interacting with Apache Spark | | nginx | 1.12.1 | nginx [engine x] is an HTTP and reverse proxy server | | mahout-client | 0.13.0 | Library for machine learning. | | mxnet | 1.3.1 | A flexible, scalable, and efficient library for deep learning. | | mysql-server | 5.5.54\$1 | MySQL database server. | | nvidia-cuda | 9.2.88 | Nvidia drivers and Cuda toolkit | | oozie-client | 5.1.0 | Oozie command-line client. | | oozie-server | 5.1.0 | Service for accepting Oozie workflow requests. | | opencv | 3.4.0 | Open Source Computer Vision Library. | | phoenix-library | 4.14.1-HBase-1.4 | The phoenix libraries for server and client | | phoenix-query-server | 4.14.1-HBase-1.4 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.215 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.215 | Service for executing pieces of a query. | | pig-client | 0.17.0 | Pig command-line client. | | r | 3.4.1 | The R Project for Statistical Computing | | spark-client | 2.4.0 | Spark command-line clients. | | spark-history-server | 2.4.0 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 2.4.0 | In-memory execution engine for YARN. | | spark-yarn-slave | 2.4.0 | Apache Spark libraries needed by YARN slaves. | | sqoop-client | 1.4.7 | Apache Sqoop command-line client. | | tensorflow | 1.12.0 | TensorFlow open source software library for high performance numerical computation. | | tez-on-yarn | 0.9.1 | The tez YARN application and libraries. | | webserver | 2.4.25\$1 | Apache HTTP server. | | zeppelin-server | 0.8.1 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.4.13 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.4.13 | ZooKeeper command line client. | ## 5.22.0 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). **emr-5.22.0 classifications** | Classifications | Description | | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | | container-log4j | Change values in Hadoop YARN's container-log4j.properties file. | | core-site | Change values in Hadoop's core-site.xml file. | | emrfs-site | Change EMRFS settings. | | flink-conf | Change flink-conf.yaml settings. | | flink-log4j | Change Flink log4j.properties settings. | | flink-log4j-yarn-session | Change Flink log4j-yarn-session.properties settings. | | flink-log4j-cli | Change Flink log4j-cli.properties settings. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | | hadoop-ssl-server | Change hadoop ssl server configuration | | hadoop-ssl-client | Change hadoop ssl client configuration | | hbase | Amazon EMR-curated settings for Apache HBase. | | hbase-env | Change values in HBase's environment. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbase.properties file. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | | hbase-site | Change values in HBase's hbase-site.xml file. | | hdfs-encryption-zones | Configure HDFS encryption zones. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | | hcatalog-env | Change values in HCatalog's environment. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | | hcatalog-webhcat-log4j2 | Change values in HCatalog WebHCat's log4j2.properties. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | | hive-beeline-log4j2 | Change values in Hive's beeline-log4j2.properties file. | | hive-parquet-logging | Change values in Hive's parquet-logging.properties file. | | hive-env | Change values in the Hive environment. | | hive-exec-log4j2 | Change values in Hive's hive-exec-log4j2.properties file. | | hive-llap-daemon-log4j2 | Change values in Hive's llap-daemon-log4j2.properties file. | | hive-log4j2 | Change values in Hive's hive-log4j2.properties file. | | hive-site | Change values in Hive's hive-site.xml file | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | | hue-ini | Change values in Hue's ini file | | httpfs-env | Change values in the HTTPFS environment. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | | jupyter-notebook-conf | Change values in Jupyter Notebook's jupyter\$1notebook\$1config.py file. | | jupyter-hub-conf | Change values in JupyterHubs's jupyterhub\$1config.py file. | | jupyter-s3-conf | Configure Jupyter Notebook S3 persistence. | | jupyter-sparkmagic-conf | Change values in Sparkmagic's config.json file. | | livy-conf | Change values in Livy's livy.conf file. | | livy-env | Change values in the Livy environment. | | livy-log4j | Change Livy log4j.properties settings. | | mapred-env | Change values in the MapReduce application's environment. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | | oozie-env | Change values in Oozie's environment. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | | oozie-site | Change values in Oozie's oozie-site.xml file. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | | pig-env | Change values in the Pig environment. | | pig-properties | Change values in Pig's pig.properties file. | | pig-log4j | Change values in Pig's log4j.properties file. | | presto-log | Change values in Presto's log.properties file. | | presto-config | Change values in Presto's config.properties file. | | presto-password-authenticator | Change values in Presto's password-authenticator.properties file. | | presto-env | Change values in Presto's presto-env.sh file. | | presto-node | Change values in Presto's node.properties file. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | | presto-connector-hive | Change values in Presto's hive.properties file. | | presto-connector-jmx | Change values in Presto's jmx.properties file. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | | presto-connector-localfile | Change values in Presto's localfile.properties file. | | presto-connector-memory | Change values in Presto's memory.properties file. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | | presto-connector-redis | Change values in Presto's redis.properties file. | | presto-connector-redshift | Change values in Presto's redshift.properties file. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | | presto-connector-tpcds | Change values in Presto's tpcds.properties file. | | spark | Amazon EMR-curated settings for Apache Spark. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | | spark-env | Change values in the Spark environment. | | spark-hive-site | Change values in Spark's hive-site.xml file | | spark-log4j | Change values in Spark's log4j.properties file. | | spark-metrics | Change values in Spark's metrics.properties file. | | sqoop-env | Change values in Sqoop's environment. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | | tez-site | Change values in Tez's tez-site.xml file. | | yarn-env | Change values in the YARN environment. | | yarn-site | Change values in YARN's yarn-site.xml file. | | zeppelin-env | Change values in the Zeppelin environment. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | # Amazon EMR release 5.21.2 ## 5.21.2 application versions This release includes the following applications: [https://flink.apache.org/](https://flink.apache.org/), [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [http://gethue.com/](http://gethue.com/), [https://jupyterhub.readthedocs.io/en/latest/#](https://jupyterhub.readthedocs.io/en/latest/#), [https://livy.incubator.apache.org/](https://livy.incubator.apache.org/), [https://mxnet.incubator.apache.org/](https://mxnet.incubator.apache.org/), [http://mahout.apache.org/](http://mahout.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://www.tensorflow.org/](https://www.tensorflow.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-5.21.2 | emr-5.21.1 | emr-5.21.0 | emr-5.20.1 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.11.479 | 1.11.479 | 1.11.479 | 1.11.461 | | Python | 2.7, 3.6 | 2.7, 3.6 | 2.7, 3.6 | 2.7, 3.6 | | Scala | 2.11.12 | 2.11.12 | 2.11.12 | 2.11.12 | | AmazonCloudWatchAgent | - | - | - | - | | Delta | - | - | - | - | | Flink | 1.7.0 | 1.7.0 | 1.7.0 | 1.6.2 | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 1.4.8 | 1.4.8 | 1.4.8 | 1.4.8 | | HCatalog | 2.3.4-amzn-0 | 2.3.4-amzn-0 | 2.3.4-amzn-0 | 2.3.4-amzn-0 | | Hadoop | 2.8.5-amzn-1 | 2.8.5-amzn-1 | 2.8.5-amzn-1 | 2.8.5-amzn-1 | | Hive | 2.3.4-amzn-0 | 2.3.4-amzn-0 | 2.3.4-amzn-0 | 2.3.4-amzn-0 | | Hudi | - | - | - | - | | Hue | 4.3.0 | 4.3.0 | 4.3.0 | 4.3.0 | | Iceberg | - | - | - | - | | JupyterEnterpriseGateway | - | - | - | - | | JupyterHub | 0.9.4 | 0.9.4 | 0.9.4 | 0.9.4 | | Livy | 0.5.0-incubating | 0.5.0-incubating | 0.5.0-incubating | 0.5.0-incubating | | MXNet | 1.3.1 | 1.3.1 | 1.3.1 | 1.3.1 | | Mahout | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | | Oozie | 5.0.0 | 5.0.0 | 5.0.0 | 5.0.0 | | Phoenix | 4.14.0-HBase-1.4 | 4.14.0-HBase-1.4 | 4.14.0-HBase-1.4 | 4.14.0-HBase-1.4 | | Pig | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | | Presto | 0.215 | 0.215 | 0.215 | 0.214 | | Spark | 2.4.0 | 2.4.0 | 2.4.0 | 2.4.0 | | Sqoop | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | | TensorFlow | 1.12.0 | 1.12.0 | 1.12.0 | 1.12.0 | | Tez | 0.9.1 | 0.9.1 | 0.9.1 | 0.9.1 | | Trino (PrestoSQL) | - | - | - | - | | Zeppelin | 0.8.0 | 0.8.0 | 0.8.0 | 0.8.0 | | ZooKeeper | 3.4.13 | 3.4.13 | 3.4.13 | 3.4.13 | ## 5.21.2 release notes This is a patch release to add AWS Signature Version 4 authentication for requests to Amazon S3. All applications and components are the same as the previous Amazon EMR release. **Important** In this release version, Amazon EMR uses AWS Signature Version 4 exclusively to authenticate requests to Amazon S3. For more information, see [Whats New](https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-whatsnew.html). ## 5.21.2 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | aws-sagemaker-spark-sdk | 1.2.1 | Amazon SageMaker Spark SDK | | emr-ddb | 4.7.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 2.5.1 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.4.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-s3-dist-cp | 2.11.0 | Distributed copy application optimized for Amazon S3. | | emr-s3-select | 1.2.0 | EMR S3Select Connector | | emrfs | 2.30.0 | Amazon S3 connector for Hadoop ecosystem applications. | | flink-client | 1.7.0 | Apache Flink command line client scripts and applications. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 2.8.5-amzn-1 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 2.8.5-amzn-1 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 2.8.5-amzn-1 | HDFS command-line client and library | | hadoop-hdfs-namenode | 2.8.5-amzn-1 | HDFS service for tracking file names and block locations. | | hadoop-httpfs-server | 2.8.5-amzn-1 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 2.8.5-amzn-1 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 2.8.5-amzn-1 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 2.8.5-amzn-1 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 2.8.5-amzn-1 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 2.8.5-amzn-1 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 1.4.8 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 1.4.8 | Service for serving one or more HBase regions. | | hbase-client | 1.4.8 | HBase command-line client. | | hbase-rest-server | 1.4.8 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 1.4.8 | Service providing a Thrift endpoint to HBase. | | hcatalog-client | 2.3.4-amzn-0 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 2.3.4-amzn-0 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 2.3.4-amzn-0 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 2.3.4-amzn-0 | Hive command line client. | | hive-hbase | 2.3.4-amzn-0 | Hive-hbase client. | | hive-metastore-server | 2.3.4-amzn-0 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server2 | 2.3.4-amzn-0 | Service for accepting Hive queries as web requests. | | hue-server | 4.3.0 | Web application for analyzing data using Hadoop ecosystem applications | | jupyterhub | 0.9.4 | Multi-user server for Jupyter notebooks | | livy-server | 0.5.0-incubating | REST interface for interacting with Apache Spark | | nginx | 1.12.1 | nginx [engine x] is an HTTP and reverse proxy server | | mahout-client | 0.13.0 | Library for machine learning. | | mxnet | 1.3.1 | A flexible, scalable, and efficient library for deep learning. | | mysql-server | 5.5.54\$1 | MySQL database server. | | nvidia-cuda | 9.2.88 | Nvidia drivers and Cuda toolkit | | oozie-client | 5.0.0 | Oozie command-line client. | | oozie-server | 5.0.0 | Service for accepting Oozie workflow requests. | | opencv | 3.4.0 | Open Source Computer Vision Library. | | phoenix-library | 4.14.0-HBase-1.4 | The phoenix libraries for server and client | | phoenix-query-server | 4.14.0-HBase-1.4 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.215 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.215 | Service for executing pieces of a query. | | pig-client | 0.17.0 | Pig command-line client. | | r | 3.4.1 | The R Project for Statistical Computing | | spark-client | 2.4.0 | Spark command-line clients. | | spark-history-server | 2.4.0 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 2.4.0 | In-memory execution engine for YARN. | | spark-yarn-slave | 2.4.0 | Apache Spark libraries needed by YARN slaves. | | sqoop-client | 1.4.7 | Apache Sqoop command-line client. | | tensorflow | 1.12.0 | TensorFlow open source software library for high performance numerical computation. | | tez-on-yarn | 0.9.1 | The tez YARN application and libraries. | | webserver | 2.4.25\$1 | Apache HTTP server. | | zeppelin-server | 0.8.0 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.4.13 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.4.13 | ZooKeeper command line client. | ## 5.21.2 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). **emr-5.21.2 classifications** | Classifications | Description | | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | | container-log4j | Change values in Hadoop YARN's container-log4j.properties file. | | core-site | Change values in Hadoop's core-site.xml file. | | emrfs-site | Change EMRFS settings. | | flink-conf | Change flink-conf.yaml settings. | | flink-log4j | Change Flink log4j.properties settings. | | flink-log4j-yarn-session | Change Flink log4j-yarn-session.properties settings. | | flink-log4j-cli | Change Flink log4j-cli.properties settings. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | | hadoop-ssl-server | Change hadoop ssl server configuration | | hadoop-ssl-client | Change hadoop ssl client configuration | | hbase | Amazon EMR-curated settings for Apache HBase. | | hbase-env | Change values in HBase's environment. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbase.properties file. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | | hbase-site | Change values in HBase's hbase-site.xml file. | | hdfs-encryption-zones | Configure HDFS encryption zones. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | | hcatalog-env | Change values in HCatalog's environment. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | | hcatalog-webhcat-log4j2 | Change values in HCatalog WebHCat's log4j2.properties. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | | hive-beeline-log4j2 | Change values in Hive's beeline-log4j2.properties file. | | hive-parquet-logging | Change values in Hive's parquet-logging.properties file. | | hive-env | Change values in the Hive environment. | | hive-exec-log4j2 | Change values in Hive's hive-exec-log4j2.properties file. | | hive-llap-daemon-log4j2 | Change values in Hive's llap-daemon-log4j2.properties file. | | hive-log4j2 | Change values in Hive's hive-log4j2.properties file. | | hive-site | Change values in Hive's hive-site.xml file | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | | hue-ini | Change values in Hue's ini file | | httpfs-env | Change values in the HTTPFS environment. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | | jupyter-notebook-conf | Change values in Jupyter Notebook's jupyter\$1notebook\$1config.py file. | | jupyter-hub-conf | Change values in JupyterHubs's jupyterhub\$1config.py file. | | jupyter-s3-conf | Configure Jupyter Notebook S3 persistence. | | jupyter-sparkmagic-conf | Change values in Sparkmagic's config.json file. | | livy-conf | Change values in Livy's livy.conf file. | | livy-env | Change values in the Livy environment. | | livy-log4j | Change Livy log4j.properties settings. | | mapred-env | Change values in the MapReduce application's environment. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | | oozie-env | Change values in Oozie's environment. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | | oozie-site | Change values in Oozie's oozie-site.xml file. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | | pig-env | Change values in the Pig environment. | | pig-properties | Change values in Pig's pig.properties file. | | pig-log4j | Change values in Pig's log4j.properties file. | | presto-log | Change values in Presto's log.properties file. | | presto-config | Change values in Presto's config.properties file. | | presto-password-authenticator | Change values in Presto's password-authenticator.properties file. | | presto-env | Change values in Presto's presto-env.sh file. | | presto-node | Change values in Presto's node.properties file. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | | presto-connector-hive | Change values in Presto's hive.properties file. | | presto-connector-jmx | Change values in Presto's jmx.properties file. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | | presto-connector-localfile | Change values in Presto's localfile.properties file. | | presto-connector-memory | Change values in Presto's memory.properties file. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | | presto-connector-redis | Change values in Presto's redis.properties file. | | presto-connector-redshift | Change values in Presto's redshift.properties file. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | | presto-connector-tpcds | Change values in Presto's tpcds.properties file. | | spark | Amazon EMR-curated settings for Apache Spark. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | | spark-env | Change values in the Spark environment. | | spark-hive-site | Change values in Spark's hive-site.xml file | | spark-log4j | Change values in Spark's log4j.properties file. | | spark-metrics | Change values in Spark's metrics.properties file. | | sqoop-env | Change values in Sqoop's environment. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | | tez-site | Change values in Tez's tez-site.xml file. | | yarn-env | Change values in the YARN environment. | | yarn-site | Change values in YARN's yarn-site.xml file. | | zeppelin-env | Change values in the Zeppelin environment. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | # Amazon EMR release 5.21.1 ## 5.21.1 application versions This release includes the following applications: [https://flink.apache.org/](https://flink.apache.org/), [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [http://gethue.com/](http://gethue.com/), [https://jupyterhub.readthedocs.io/en/latest/#](https://jupyterhub.readthedocs.io/en/latest/#), [https://livy.incubator.apache.org/](https://livy.incubator.apache.org/), [https://mxnet.incubator.apache.org/](https://mxnet.incubator.apache.org/), [http://mahout.apache.org/](http://mahout.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://www.tensorflow.org/](https://www.tensorflow.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-5.21.1 | emr-5.21.0 | emr-5.20.1 | emr-5.20.0 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.11.479 | 1.11.479 | 1.11.461 | 1.11.461 | | Python | 2.7, 3.6 | 2.7, 3.6 | 2.7, 3.6 | 2.7, 3.6 | | Scala | 2.11.12 | 2.11.12 | 2.11.12 | 2.11.12 | | AmazonCloudWatchAgent | - | - | - | - | | Delta | - | - | - | - | | Flink | 1.7.0 | 1.7.0 | 1.6.2 | 1.6.2 | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 1.4.8 | 1.4.8 | 1.4.8 | 1.4.8 | | HCatalog | 2.3.4-amzn-0 | 2.3.4-amzn-0 | 2.3.4-amzn-0 | 2.3.4-amzn-0 | | Hadoop | 2.8.5-amzn-1 | 2.8.5-amzn-1 | 2.8.5-amzn-1 | 2.8.5-amzn-1 | | Hive | 2.3.4-amzn-0 | 2.3.4-amzn-0 | 2.3.4-amzn-0 | 2.3.4-amzn-0 | | Hudi | - | - | - | - | | Hue | 4.3.0 | 4.3.0 | 4.3.0 | 4.3.0 | | Iceberg | - | - | - | - | | JupyterEnterpriseGateway | - | - | - | - | | JupyterHub | 0.9.4 | 0.9.4 | 0.9.4 | 0.9.4 | | Livy | 0.5.0-incubating | 0.5.0-incubating | 0.5.0-incubating | 0.5.0-incubating | | MXNet | 1.3.1 | 1.3.1 | 1.3.1 | 1.3.1 | | Mahout | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | | Oozie | 5.0.0 | 5.0.0 | 5.0.0 | 5.0.0 | | Phoenix | 4.14.0-HBase-1.4 | 4.14.0-HBase-1.4 | 4.14.0-HBase-1.4 | 4.14.0-HBase-1.4 | | Pig | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | | Presto | 0.215 | 0.215 | 0.214 | 0.214 | | Spark | 2.4.0 | 2.4.0 | 2.4.0 | 2.4.0 | | Sqoop | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | | TensorFlow | 1.12.0 | 1.12.0 | 1.12.0 | 1.12.0 | | Tez | 0.9.1 | 0.9.1 | 0.9.1 | 0.9.1 | | Trino (PrestoSQL) | - | - | - | - | | Zeppelin | 0.8.0 | 0.8.0 | 0.8.0 | 0.8.0 | | ZooKeeper | 3.4.13 | 3.4.13 | 3.4.13 | 3.4.13 | ## 5.21.1 release notes The following release notes include information for Amazon EMR release 5.21.1. Changes are relative to 5.21.0. Initial release date: July 18, 2019 **Changes, enhancements, and resolved issues** + Updated the default Amazon Linux AMI for Amazon EMR to include important Linux kernel security updates, including the TCP SACK Denial of Service Issue ([AWS-2019-005](https://aws.amazon.com/security/security-bulletins/AWS-2019-005/)). **Known issues** + Known issue in clusters with multiple primary nodes and Kerberos authentication If you run clusters with multiple primary nodes and Kerberos authentication in Amazon EMR releases 5.20.0 and later, you may encounter problems with cluster operations such as scale down or step submission, after the cluster has been running for some time. The time period depends on the Kerberos ticket validity period that you defined. The scale-down problem impacts both automatic scale-down and explicit scale down requests that you submitted. Additional cluster operations can also be impacted. Workaround: + SSH as `hadoop` user to the lead primary node of the EMR cluster with multiple primary nodes. + Run the following command to renew Kerberos ticket for `hadoop` user. ``` kinit -kt ``` Typically, the keytab file is located at `/etc/hadoop.keytab` and the principal is in the form of `hadoop/@`. **Note** This workaround will be effective for the time period the Kerberos ticket is valid. This duration is 10 hours by default, but can configured by your Kerberos settings. You must re-run the above command once the Kerberos ticket expires. ## 5.21.1 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | aws-sagemaker-spark-sdk | 1.2.1 | Amazon SageMaker Spark SDK | | emr-ddb | 4.7.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 2.5.1 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.4.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-s3-dist-cp | 2.11.0 | Distributed copy application optimized for Amazon S3. | | emr-s3-select | 1.2.0 | EMR S3Select Connector | | emrfs | 2.30.0 | Amazon S3 connector for Hadoop ecosystem applications. | | flink-client | 1.7.0 | Apache Flink command line client scripts and applications. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 2.8.5-amzn-1 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 2.8.5-amzn-1 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 2.8.5-amzn-1 | HDFS command-line client and library | | hadoop-hdfs-namenode | 2.8.5-amzn-1 | HDFS service for tracking file names and block locations. | | hadoop-httpfs-server | 2.8.5-amzn-1 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 2.8.5-amzn-1 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 2.8.5-amzn-1 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 2.8.5-amzn-1 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 2.8.5-amzn-1 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 2.8.5-amzn-1 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 1.4.8 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 1.4.8 | Service for serving one or more HBase regions. | | hbase-client | 1.4.8 | HBase command-line client. | | hbase-rest-server | 1.4.8 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 1.4.8 | Service providing a Thrift endpoint to HBase. | | hcatalog-client | 2.3.4-amzn-0 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 2.3.4-amzn-0 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 2.3.4-amzn-0 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 2.3.4-amzn-0 | Hive command line client. | | hive-hbase | 2.3.4-amzn-0 | Hive-hbase client. | | hive-metastore-server | 2.3.4-amzn-0 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server2 | 2.3.4-amzn-0 | Service for accepting Hive queries as web requests. | | hue-server | 4.3.0 | Web application for analyzing data using Hadoop ecosystem applications | | jupyterhub | 0.9.4 | Multi-user server for Jupyter notebooks | | livy-server | 0.5.0-incubating | REST interface for interacting with Apache Spark | | nginx | 1.12.1 | nginx [engine x] is an HTTP and reverse proxy server | | mahout-client | 0.13.0 | Library for machine learning. | | mxnet | 1.3.1 | A flexible, scalable, and efficient library for deep learning. | | mysql-server | 5.5.54\$1 | MySQL database server. | | nvidia-cuda | 9.2.88 | Nvidia drivers and Cuda toolkit | | oozie-client | 5.0.0 | Oozie command-line client. | | oozie-server | 5.0.0 | Service for accepting Oozie workflow requests. | | opencv | 3.4.0 | Open Source Computer Vision Library. | | phoenix-library | 4.14.0-HBase-1.4 | The phoenix libraries for server and client | | phoenix-query-server | 4.14.0-HBase-1.4 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.215 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.215 | Service for executing pieces of a query. | | pig-client | 0.17.0 | Pig command-line client. | | r | 3.4.1 | The R Project for Statistical Computing | | spark-client | 2.4.0 | Spark command-line clients. | | spark-history-server | 2.4.0 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 2.4.0 | In-memory execution engine for YARN. | | spark-yarn-slave | 2.4.0 | Apache Spark libraries needed by YARN slaves. | | sqoop-client | 1.4.7 | Apache Sqoop command-line client. | | tensorflow | 1.12.0 | TensorFlow open source software library for high performance numerical computation. | | tez-on-yarn | 0.9.1 | The tez YARN application and libraries. | | webserver | 2.4.25\$1 | Apache HTTP server. | | zeppelin-server | 0.8.0 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.4.13 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.4.13 | ZooKeeper command line client. | ## 5.21.1 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). **emr-5.21.1 classifications** | Classifications | Description | | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | | container-log4j | Change values in Hadoop YARN's container-log4j.properties file. | | core-site | Change values in Hadoop's core-site.xml file. | | emrfs-site | Change EMRFS settings. | | flink-conf | Change flink-conf.yaml settings. | | flink-log4j | Change Flink log4j.properties settings. | | flink-log4j-yarn-session | Change Flink log4j-yarn-session.properties settings. | | flink-log4j-cli | Change Flink log4j-cli.properties settings. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | | hadoop-ssl-server | Change hadoop ssl server configuration | | hadoop-ssl-client | Change hadoop ssl client configuration | | hbase | Amazon EMR-curated settings for Apache HBase. | | hbase-env | Change values in HBase's environment. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbase.properties file. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | | hbase-site | Change values in HBase's hbase-site.xml file. | | hdfs-encryption-zones | Configure HDFS encryption zones. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | | hcatalog-env | Change values in HCatalog's environment. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | | hcatalog-webhcat-log4j2 | Change values in HCatalog WebHCat's log4j2.properties. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | | hive-beeline-log4j2 | Change values in Hive's beeline-log4j2.properties file. | | hive-parquet-logging | Change values in Hive's parquet-logging.properties file. | | hive-env | Change values in the Hive environment. | | hive-exec-log4j2 | Change values in Hive's hive-exec-log4j2.properties file. | | hive-llap-daemon-log4j2 | Change values in Hive's llap-daemon-log4j2.properties file. | | hive-log4j2 | Change values in Hive's hive-log4j2.properties file. | | hive-site | Change values in Hive's hive-site.xml file | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | | hue-ini | Change values in Hue's ini file | | httpfs-env | Change values in the HTTPFS environment. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | | jupyter-notebook-conf | Change values in Jupyter Notebook's jupyter\$1notebook\$1config.py file. | | jupyter-hub-conf | Change values in JupyterHubs's jupyterhub\$1config.py file. | | jupyter-s3-conf | Configure Jupyter Notebook S3 persistence. | | jupyter-sparkmagic-conf | Change values in Sparkmagic's config.json file. | | livy-conf | Change values in Livy's livy.conf file. | | livy-env | Change values in the Livy environment. | | livy-log4j | Change Livy log4j.properties settings. | | mapred-env | Change values in the MapReduce application's environment. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | | oozie-env | Change values in Oozie's environment. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | | oozie-site | Change values in Oozie's oozie-site.xml file. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | | pig-env | Change values in the Pig environment. | | pig-properties | Change values in Pig's pig.properties file. | | pig-log4j | Change values in Pig's log4j.properties file. | | presto-log | Change values in Presto's log.properties file. | | presto-config | Change values in Presto's config.properties file. | | presto-password-authenticator | Change values in Presto's password-authenticator.properties file. | | presto-env | Change values in Presto's presto-env.sh file. | | presto-node | Change values in Presto's node.properties file. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | | presto-connector-hive | Change values in Presto's hive.properties file. | | presto-connector-jmx | Change values in Presto's jmx.properties file. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | | presto-connector-localfile | Change values in Presto's localfile.properties file. | | presto-connector-memory | Change values in Presto's memory.properties file. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | | presto-connector-redis | Change values in Presto's redis.properties file. | | presto-connector-redshift | Change values in Presto's redshift.properties file. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | | presto-connector-tpcds | Change values in Presto's tpcds.properties file. | | spark | Amazon EMR-curated settings for Apache Spark. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | | spark-env | Change values in the Spark environment. | | spark-hive-site | Change values in Spark's hive-site.xml file | | spark-log4j | Change values in Spark's log4j.properties file. | | spark-metrics | Change values in Spark's metrics.properties file. | | sqoop-env | Change values in Sqoop's environment. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | | tez-site | Change values in Tez's tez-site.xml file. | | yarn-env | Change values in the YARN environment. | | yarn-site | Change values in YARN's yarn-site.xml file. | | zeppelin-env | Change values in the Zeppelin environment. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | # Amazon EMR release 5.21.0 ## 5.21.0 application versions This release includes the following applications: [https://flink.apache.org/](https://flink.apache.org/), [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [http://gethue.com/](http://gethue.com/), [https://jupyterhub.readthedocs.io/en/latest/#](https://jupyterhub.readthedocs.io/en/latest/#), [https://livy.incubator.apache.org/](https://livy.incubator.apache.org/), [https://mxnet.incubator.apache.org/](https://mxnet.incubator.apache.org/), [http://mahout.apache.org/](http://mahout.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://www.tensorflow.org/](https://www.tensorflow.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-5.21.0 | emr-5.20.1 | emr-5.20.0 | emr-5.19.1 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.11.479 | 1.11.461 | 1.11.461 | 1.11.433 | | Python | 2.7, 3.6 | 2.7, 3.6 | 2.7, 3.6 | 2.7, 3.4 | | Scala | 2.11.12 | 2.11.12 | 2.11.12 | 2.11.8 | | AmazonCloudWatchAgent | - | - | - | - | | Delta | - | - | - | - | | Flink | 1.7.0 | 1.6.2 | 1.6.2 | 1.6.1 | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 1.4.8 | 1.4.8 | 1.4.8 | 1.4.7 | | HCatalog | 2.3.4-amzn-0 | 2.3.4-amzn-0 | 2.3.4-amzn-0 | 2.3.3-amzn-2 | | Hadoop | 2.8.5-amzn-1 | 2.8.5-amzn-1 | 2.8.5-amzn-1 | 2.8.5-amzn-0 | | Hive | 2.3.4-amzn-0 | 2.3.4-amzn-0 | 2.3.4-amzn-0 | 2.3.3-amzn-1 | | Hudi | - | - | - | - | | Hue | 4.3.0 | 4.3.0 | 4.3.0 | 4.2.0 | | Iceberg | - | - | - | - | | JupyterEnterpriseGateway | - | - | - | - | | JupyterHub | 0.9.4 | 0.9.4 | 0.9.4 | 0.9.4 | | Livy | 0.5.0-incubating | 0.5.0-incubating | 0.5.0-incubating | 0.5.0-incubating | | MXNet | 1.3.1 | 1.3.1 | 1.3.1 | 1.3.0 | | Mahout | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | | Oozie | 5.0.0 | 5.0.0 | 5.0.0 | 5.0.0 | | Phoenix | 4.14.0-HBase-1.4 | 4.14.0-HBase-1.4 | 4.14.0-HBase-1.4 | 4.14.0-HBase-1.4 | | Pig | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | | Presto | 0.215 | 0.214 | 0.214 | 0.212 | | Spark | 2.4.0 | 2.4.0 | 2.4.0 | 2.3.2 | | Sqoop | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | | TensorFlow | 1.12.0 | 1.12.0 | 1.12.0 | 1.11.0 | | Tez | 0.9.1 | 0.9.1 | 0.9.1 | 0.8.4 | | Trino (PrestoSQL) | - | - | - | - | | Zeppelin | 0.8.0 | 0.8.0 | 0.8.0 | 0.8.0 | | ZooKeeper | 3.4.13 | 3.4.13 | 3.4.13 | 3.4.13 | ## 5.21.0 release notes The following release notes include information for Amazon EMR release 5.21.0. Changes are relative to 5.20.0. Initial release date: February 18, 2019 Last updated date: April 3, 2019 **Upgrades** + Flink 1.7.0 + Presto 0.215 + AWS SDK for Java 1.11.479 **New features** + (April 3, 2019) With Amazon EMR version 5.21.0 and later, you can override cluster configurations and specify additional configuration classifications for each instance group in a running cluster. You do this by using the Amazon EMR console, the AWS Command Line Interface (AWS CLI), or the AWS SDK. For more information, see [Supplying a Configuration for an Instance Group in a Running Cluster](https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-configure-apps-running-cluster.html). **Changes, enhancements, and resolved issues** + Zeppelin + Backported [ZEPPELIN-3878](https://issues.apache.org/jira/browse/ZEPPELIN-3878). **Known issues** + Hue (Fixed in Amazon EMR release 5.24.0) + Hue running on Amazon EMR does not support Solr. Beginning with Amazon EMR release 5.20.0, a misconfiguration issue causes Solr to be enabled and a harmless error message to appear similar to the following: `Solr server could not be contacted properly: HTTPConnectionPool('host=ip-xx-xx-xx-xx.ec2.internal', port=1978): Max retries exceeded with url: /solr/admin/info/system?user.name=hue&doAs=administrator&wt=json (Caused by NewConnectionError(': Failed to establish a new connection: [Errno 111] Connection refused',))` **To prevent the Solr error message from appearing:** 1. Connect to the primary node command line using SSH. 1. Use a text editor to open the `hue.ini` file. For example: `sudo vim /etc/hue/conf/hue.ini` 1. Search for the term `appblacklist` and modify the line to the following: ``` appblacklist = search ``` 1. Save your changes and restart Hue as shown in the following example: ``` sudo stop hue; sudo start hue ``` + Tez + This issue was fixed in Amazon EMR 5.22.0. When you connect to the Tez UI at http://*MasterDNS*:8080/tez-ui through an SSH connection to the cluster primary node, the error "Adapter operation failed - Timeline server (ATS) is out of reach. Either it is down, or CORS is not enabled" appears, or tasks unexpectedly show N/A. This is caused by the Tez UI making requests to the YARN Timeline Server using `localhost` rather than the host name of the primary node. As a workaround, a script is available to run as a bootstrap action or step. The script updates the host name in the Tez `configs.env` file. For more information and the location of the script, see the [Bootstrap Instructions](http://awssupportdatasvcs.com/bootstrap-actions/fix_tez_ui_0-9-1/). + In Amazon EMR version 5.19.0, 5.20.0, and 5.21.0, YARN node labels are stored in an HDFS directory. In some situations, this leads to core node startup delays and then causes cluster time-out and launch failure. Beginning with Amazon EMR 5.22.0, this issue is resolved. YARN node labels are stored on the local disk of each cluster node, avoiding dependencies on HDFS. + Known issue in clusters with multiple primary nodes and Kerberos authentication If you run clusters with multiple primary nodes and Kerberos authentication in Amazon EMR releases 5.20.0 and later, you may encounter problems with cluster operations such as scale down or step submission, after the cluster has been running for some time. The time period depends on the Kerberos ticket validity period that you defined. The scale-down problem impacts both automatic scale-down and explicit scale down requests that you submitted. Additional cluster operations can also be impacted. Workaround: + SSH as `hadoop` user to the lead primary node of the EMR cluster with multiple primary nodes. + Run the following command to renew Kerberos ticket for `hadoop` user. ``` kinit -kt ``` Typically, the keytab file is located at `/etc/hadoop.keytab` and the principal is in the form of `hadoop/@`. **Note** This workaround will be effective for the time period the Kerberos ticket is valid. This duration is 10 hours by default, but can configured by your Kerberos settings. You must re-run the above command once the Kerberos ticket expires. ## 5.21.0 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | aws-sagemaker-spark-sdk | 1.2.1 | Amazon SageMaker Spark SDK | | emr-ddb | 4.7.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 2.5.1 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.4.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-s3-dist-cp | 2.11.0 | Distributed copy application optimized for Amazon S3. | | emr-s3-select | 1.2.0 | EMR S3Select Connector | | emrfs | 2.30.0 | Amazon S3 connector for Hadoop ecosystem applications. | | flink-client | 1.7.0 | Apache Flink command line client scripts and applications. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 2.8.5-amzn-1 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 2.8.5-amzn-1 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 2.8.5-amzn-1 | HDFS command-line client and library | | hadoop-hdfs-namenode | 2.8.5-amzn-1 | HDFS service for tracking file names and block locations. | | hadoop-httpfs-server | 2.8.5-amzn-1 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 2.8.5-amzn-1 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 2.8.5-amzn-1 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 2.8.5-amzn-1 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 2.8.5-amzn-1 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 2.8.5-amzn-1 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 1.4.8 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 1.4.8 | Service for serving one or more HBase regions. | | hbase-client | 1.4.8 | HBase command-line client. | | hbase-rest-server | 1.4.8 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 1.4.8 | Service providing a Thrift endpoint to HBase. | | hcatalog-client | 2.3.4-amzn-0 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 2.3.4-amzn-0 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 2.3.4-amzn-0 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 2.3.4-amzn-0 | Hive command line client. | | hive-hbase | 2.3.4-amzn-0 | Hive-hbase client. | | hive-metastore-server | 2.3.4-amzn-0 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server2 | 2.3.4-amzn-0 | Service for accepting Hive queries as web requests. | | hue-server | 4.3.0 | Web application for analyzing data using Hadoop ecosystem applications | | jupyterhub | 0.9.4 | Multi-user server for Jupyter notebooks | | livy-server | 0.5.0-incubating | REST interface for interacting with Apache Spark | | nginx | 1.12.1 | nginx [engine x] is an HTTP and reverse proxy server | | mahout-client | 0.13.0 | Library for machine learning. | | mxnet | 1.3.1 | A flexible, scalable, and efficient library for deep learning. | | mysql-server | 5.5.54\$1 | MySQL database server. | | nvidia-cuda | 9.2.88 | Nvidia drivers and Cuda toolkit | | oozie-client | 5.0.0 | Oozie command-line client. | | oozie-server | 5.0.0 | Service for accepting Oozie workflow requests. | | opencv | 3.4.0 | Open Source Computer Vision Library. | | phoenix-library | 4.14.0-HBase-1.4 | The phoenix libraries for server and client | | phoenix-query-server | 4.14.0-HBase-1.4 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.215 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.215 | Service for executing pieces of a query. | | pig-client | 0.17.0 | Pig command-line client. | | r | 3.4.1 | The R Project for Statistical Computing | | spark-client | 2.4.0 | Spark command-line clients. | | spark-history-server | 2.4.0 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 2.4.0 | In-memory execution engine for YARN. | | spark-yarn-slave | 2.4.0 | Apache Spark libraries needed by YARN slaves. | | sqoop-client | 1.4.7 | Apache Sqoop command-line client. | | tensorflow | 1.12.0 | TensorFlow open source software library for high performance numerical computation. | | tez-on-yarn | 0.9.1 | The tez YARN application and libraries. | | webserver | 2.4.25\$1 | Apache HTTP server. | | zeppelin-server | 0.8.0 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.4.13 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.4.13 | ZooKeeper command line client. | ## 5.21.0 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). **emr-5.21.0 classifications** | Classifications | Description | | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | | container-log4j | Change values in Hadoop YARN's container-log4j.properties file. | | core-site | Change values in Hadoop's core-site.xml file. | | emrfs-site | Change EMRFS settings. | | flink-conf | Change flink-conf.yaml settings. | | flink-log4j | Change Flink log4j.properties settings. | | flink-log4j-yarn-session | Change Flink log4j-yarn-session.properties settings. | | flink-log4j-cli | Change Flink log4j-cli.properties settings. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | | hadoop-ssl-server | Change hadoop ssl server configuration | | hadoop-ssl-client | Change hadoop ssl client configuration | | hbase | Amazon EMR-curated settings for Apache HBase. | | hbase-env | Change values in HBase's environment. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbase.properties file. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | | hbase-site | Change values in HBase's hbase-site.xml file. | | hdfs-encryption-zones | Configure HDFS encryption zones. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | | hcatalog-env | Change values in HCatalog's environment. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | | hcatalog-webhcat-log4j2 | Change values in HCatalog WebHCat's log4j2.properties. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | | hive-beeline-log4j2 | Change values in Hive's beeline-log4j2.properties file. | | hive-parquet-logging | Change values in Hive's parquet-logging.properties file. | | hive-env | Change values in the Hive environment. | | hive-exec-log4j2 | Change values in Hive's hive-exec-log4j2.properties file. | | hive-llap-daemon-log4j2 | Change values in Hive's llap-daemon-log4j2.properties file. | | hive-log4j2 | Change values in Hive's hive-log4j2.properties file. | | hive-site | Change values in Hive's hive-site.xml file | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | | hue-ini | Change values in Hue's ini file | | httpfs-env | Change values in the HTTPFS environment. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | | jupyter-notebook-conf | Change values in Jupyter Notebook's jupyter\$1notebook\$1config.py file. | | jupyter-hub-conf | Change values in JupyterHubs's jupyterhub\$1config.py file. | | jupyter-s3-conf | Configure Jupyter Notebook S3 persistence. | | jupyter-sparkmagic-conf | Change values in Sparkmagic's config.json file. | | livy-conf | Change values in Livy's livy.conf file. | | livy-env | Change values in the Livy environment. | | livy-log4j | Change Livy log4j.properties settings. | | mapred-env | Change values in the MapReduce application's environment. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | | oozie-env | Change values in Oozie's environment. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | | oozie-site | Change values in Oozie's oozie-site.xml file. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | | pig-env | Change values in the Pig environment. | | pig-properties | Change values in Pig's pig.properties file. | | pig-log4j | Change values in Pig's log4j.properties file. | | presto-log | Change values in Presto's log.properties file. | | presto-config | Change values in Presto's config.properties file. | | presto-password-authenticator | Change values in Presto's password-authenticator.properties file. | | presto-env | Change values in Presto's presto-env.sh file. | | presto-node | Change values in Presto's node.properties file. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | | presto-connector-hive | Change values in Presto's hive.properties file. | | presto-connector-jmx | Change values in Presto's jmx.properties file. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | | presto-connector-localfile | Change values in Presto's localfile.properties file. | | presto-connector-memory | Change values in Presto's memory.properties file. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | | presto-connector-redis | Change values in Presto's redis.properties file. | | presto-connector-redshift | Change values in Presto's redshift.properties file. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | | presto-connector-tpcds | Change values in Presto's tpcds.properties file. | | spark | Amazon EMR-curated settings for Apache Spark. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | | spark-env | Change values in the Spark environment. | | spark-hive-site | Change values in Spark's hive-site.xml file | | spark-log4j | Change values in Spark's log4j.properties file. | | spark-metrics | Change values in Spark's metrics.properties file. | | sqoop-env | Change values in Sqoop's environment. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | | tez-site | Change values in Tez's tez-site.xml file. | | yarn-env | Change values in the YARN environment. | | yarn-site | Change values in YARN's yarn-site.xml file. | | zeppelin-env | Change values in the Zeppelin environment. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | # Amazon EMR release 5.20.1 ## 5.20.1 application versions This release includes the following applications: [https://flink.apache.org/](https://flink.apache.org/), [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [http://gethue.com/](http://gethue.com/), [https://jupyterhub.readthedocs.io/en/latest/#](https://jupyterhub.readthedocs.io/en/latest/#), [https://livy.incubator.apache.org/](https://livy.incubator.apache.org/), [https://mxnet.incubator.apache.org/](https://mxnet.incubator.apache.org/), [http://mahout.apache.org/](http://mahout.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://www.tensorflow.org/](https://www.tensorflow.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-5.20.1 | emr-5.20.0 | emr-5.19.1 | emr-5.19.0 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.11.461 | 1.11.461 | 1.11.433 | 1.11.433 | | Python | 2.7, 3.6 | 2.7, 3.6 | 2.7, 3.4 | 2.7, 3.4 | | Scala | 2.11.12 | 2.11.12 | 2.11.8 | 2.11.8 | | AmazonCloudWatchAgent | - | - | - | - | | Delta | - | - | - | - | | Flink | 1.6.2 | 1.6.2 | 1.6.1 | 1.6.1 | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 1.4.8 | 1.4.8 | 1.4.7 | 1.4.7 | | HCatalog | 2.3.4-amzn-0 | 2.3.4-amzn-0 | 2.3.3-amzn-2 | 2.3.3-amzn-2 | | Hadoop | 2.8.5-amzn-1 | 2.8.5-amzn-1 | 2.8.5-amzn-0 | 2.8.5-amzn-0 | | Hive | 2.3.4-amzn-0 | 2.3.4-amzn-0 | 2.3.3-amzn-1 | 2.3.3-amzn-1 | | Hudi | - | - | - | - | | Hue | 4.3.0 | 4.3.0 | 4.2.0 | 4.2.0 | | Iceberg | - | - | - | - | | JupyterEnterpriseGateway | - | - | - | - | | JupyterHub | 0.9.4 | 0.9.4 | 0.9.4 | 0.9.4 | | Livy | 0.5.0-incubating | 0.5.0-incubating | 0.5.0-incubating | 0.5.0-incubating | | MXNet | 1.3.1 | 1.3.1 | 1.3.0 | 1.3.0 | | Mahout | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | | Oozie | 5.0.0 | 5.0.0 | 5.0.0 | 5.0.0 | | Phoenix | 4.14.0-HBase-1.4 | 4.14.0-HBase-1.4 | 4.14.0-HBase-1.4 | 4.14.0-HBase-1.4 | | Pig | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | | Presto | 0.214 | 0.214 | 0.212 | 0.212 | | Spark | 2.4.0 | 2.4.0 | 2.3.2 | 2.3.2 | | Sqoop | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | | TensorFlow | 1.12.0 | 1.12.0 | 1.11.0 | 1.11.0 | | Tez | 0.9.1 | 0.9.1 | 0.8.4 | 0.8.4 | | Trino (PrestoSQL) | - | - | - | - | | Zeppelin | 0.8.0 | 0.8.0 | 0.8.0 | 0.8.0 | | ZooKeeper | 3.4.13 | 3.4.13 | 3.4.13 | 3.4.13 | ## 5.20.1 release notes This is a patch release to add AWS Signature Version 4 authentication for requests to Amazon S3. All applications and components are the same as the previous Amazon EMR release. **Important** In this release version, Amazon EMR uses AWS Signature Version 4 exclusively to authenticate requests to Amazon S3. For more information, see [Whats New](https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-whatsnew.html). ## 5.20.1 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | aws-sagemaker-spark-sdk | 1.2.1 | Amazon SageMaker Spark SDK | | emr-ddb | 4.7.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 2.5.1 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.4.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-s3-dist-cp | 2.10.0 | Distributed copy application optimized for Amazon S3. | | emr-s3-select | 1.2.0 | EMR S3Select Connector | | emrfs | 2.29.0 | Amazon S3 connector for Hadoop ecosystem applications. | | flink-client | 1.6.2 | Apache Flink command line client scripts and applications. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 2.8.5-amzn-1 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 2.8.5-amzn-1 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 2.8.5-amzn-1 | HDFS command-line client and library | | hadoop-hdfs-namenode | 2.8.5-amzn-1 | HDFS service for tracking file names and block locations. | | hadoop-httpfs-server | 2.8.5-amzn-1 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 2.8.5-amzn-1 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 2.8.5-amzn-1 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 2.8.5-amzn-1 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 2.8.5-amzn-1 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 2.8.5-amzn-1 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 1.4.8 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 1.4.8 | Service for serving one or more HBase regions. | | hbase-client | 1.4.8 | HBase command-line client. | | hbase-rest-server | 1.4.8 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 1.4.8 | Service providing a Thrift endpoint to HBase. | | hcatalog-client | 2.3.4-amzn-0 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 2.3.4-amzn-0 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 2.3.4-amzn-0 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 2.3.4-amzn-0 | Hive command line client. | | hive-hbase | 2.3.4-amzn-0 | Hive-hbase client. | | hive-metastore-server | 2.3.4-amzn-0 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server2 | 2.3.4-amzn-0 | Service for accepting Hive queries as web requests. | | hue-server | 4.3.0 | Web application for analyzing data using Hadoop ecosystem applications | | jupyterhub | 0.9.4 | Multi-user server for Jupyter notebooks | | livy-server | 0.5.0-incubating | REST interface for interacting with Apache Spark | | nginx | 1.12.1 | nginx [engine x] is an HTTP and reverse proxy server | | mahout-client | 0.13.0 | Library for machine learning. | | mxnet | 1.3.1 | A flexible, scalable, and efficient library for deep learning. | | mysql-server | 5.5.54\$1 | MySQL database server. | | nvidia-cuda | 9.2.88 | Nvidia drivers and Cuda toolkit | | oozie-client | 5.0.0 | Oozie command-line client. | | oozie-server | 5.0.0 | Service for accepting Oozie workflow requests. | | opencv | 3.4.0 | Open Source Computer Vision Library. | | phoenix-library | 4.14.0-HBase-1.4 | The phoenix libraries for server and client | | phoenix-query-server | 4.14.0-HBase-1.4 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.214 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.214 | Service for executing pieces of a query. | | pig-client | 0.17.0 | Pig command-line client. | | r | 3.4.1 | The R Project for Statistical Computing | | spark-client | 2.4.0 | Spark command-line clients. | | spark-history-server | 2.4.0 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 2.4.0 | In-memory execution engine for YARN. | | spark-yarn-slave | 2.4.0 | Apache Spark libraries needed by YARN slaves. | | sqoop-client | 1.4.7 | Apache Sqoop command-line client. | | tensorflow | 1.12.0 | TensorFlow open source software library for high performance numerical computation. | | tez-on-yarn | 0.9.1 | The tez YARN application and libraries. | | webserver | 2.4.25\$1 | Apache HTTP server. | | zeppelin-server | 0.8.0 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.4.13 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.4.13 | ZooKeeper command line client. | ## 5.20.1 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). **emr-5.20.1 classifications** | Classifications | Description | | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | | container-log4j | Change values in Hadoop YARN's container-log4j.properties file. | | core-site | Change values in Hadoop's core-site.xml file. | | emrfs-site | Change EMRFS settings. | | flink-conf | Change flink-conf.yaml settings. | | flink-log4j | Change Flink log4j.properties settings. | | flink-log4j-yarn-session | Change Flink log4j-yarn-session.properties settings. | | flink-log4j-cli | Change Flink log4j-cli.properties settings. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | | hadoop-ssl-server | Change hadoop ssl server configuration | | hadoop-ssl-client | Change hadoop ssl client configuration | | hbase | Amazon EMR-curated settings for Apache HBase. | | hbase-env | Change values in HBase's environment. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbase.properties file. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | | hbase-site | Change values in HBase's hbase-site.xml file. | | hdfs-encryption-zones | Configure HDFS encryption zones. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | | hcatalog-env | Change values in HCatalog's environment. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | | hcatalog-webhcat-log4j2 | Change values in HCatalog WebHCat's log4j2.properties. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | | hive-beeline-log4j2 | Change values in Hive's beeline-log4j2.properties file. | | hive-parquet-logging | Change values in Hive's parquet-logging.properties file. | | hive-env | Change values in the Hive environment. | | hive-exec-log4j2 | Change values in Hive's hive-exec-log4j2.properties file. | | hive-llap-daemon-log4j2 | Change values in Hive's llap-daemon-log4j2.properties file. | | hive-log4j2 | Change values in Hive's hive-log4j2.properties file. | | hive-site | Change values in Hive's hive-site.xml file | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | | hue-ini | Change values in Hue's ini file | | httpfs-env | Change values in the HTTPFS environment. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | | jupyter-notebook-conf | Change values in Jupyter Notebook's jupyter\$1notebook\$1config.py file. | | jupyter-hub-conf | Change values in JupyterHubs's jupyterhub\$1config.py file. | | jupyter-s3-conf | Configure Jupyter Notebook S3 persistence. | | jupyter-sparkmagic-conf | Change values in Sparkmagic's config.json file. | | livy-conf | Change values in Livy's livy.conf file. | | livy-env | Change values in the Livy environment. | | livy-log4j | Change Livy log4j.properties settings. | | mapred-env | Change values in the MapReduce application's environment. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | | oozie-env | Change values in Oozie's environment. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | | oozie-site | Change values in Oozie's oozie-site.xml file. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | | pig-env | Change values in the Pig environment. | | pig-properties | Change values in Pig's pig.properties file. | | pig-log4j | Change values in Pig's log4j.properties file. | | presto-log | Change values in Presto's log.properties file. | | presto-config | Change values in Presto's config.properties file. | | presto-password-authenticator | Change values in Presto's password-authenticator.properties file. | | presto-env | Change values in Presto's presto-env.sh file. | | presto-node | Change values in Presto's node.properties file. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | | presto-connector-hive | Change values in Presto's hive.properties file. | | presto-connector-jmx | Change values in Presto's jmx.properties file. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | | presto-connector-localfile | Change values in Presto's localfile.properties file. | | presto-connector-memory | Change values in Presto's memory.properties file. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | | presto-connector-redis | Change values in Presto's redis.properties file. | | presto-connector-redshift | Change values in Presto's redshift.properties file. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | | presto-connector-tpcds | Change values in Presto's tpcds.properties file. | | spark | Amazon EMR-curated settings for Apache Spark. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | | spark-env | Change values in the Spark environment. | | spark-hive-site | Change values in Spark's hive-site.xml file | | spark-log4j | Change values in Spark's log4j.properties file. | | spark-metrics | Change values in Spark's metrics.properties file. | | sqoop-env | Change values in Sqoop's environment. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | | tez-site | Change values in Tez's tez-site.xml file. | | yarn-env | Change values in the YARN environment. | | yarn-site | Change values in YARN's yarn-site.xml file. | | zeppelin-env | Change values in the Zeppelin environment. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | # Amazon EMR release 5.20.0 ## 5.20.0 application versions This release includes the following applications: [https://flink.apache.org/](https://flink.apache.org/), [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [http://gethue.com/](http://gethue.com/), [https://jupyterhub.readthedocs.io/en/latest/#](https://jupyterhub.readthedocs.io/en/latest/#), [https://livy.incubator.apache.org/](https://livy.incubator.apache.org/), [https://mxnet.incubator.apache.org/](https://mxnet.incubator.apache.org/), [http://mahout.apache.org/](http://mahout.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://www.tensorflow.org/](https://www.tensorflow.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-5.20.0 | emr-5.19.1 | emr-5.19.0 | emr-5.18.1 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.11.461 | 1.11.433 | 1.11.433 | 1.11.393 | | Python | 2.7, 3.6 | 2.7, 3.4 | 2.7, 3.4 | 2.7, 3.4 | | Scala | 2.11.12 | 2.11.8 | 2.11.8 | 2.11.8 | | AmazonCloudWatchAgent | - | - | - | - | | Delta | - | - | - | - | | Flink | 1.6.2 | 1.6.1 | 1.6.1 | 1.6.0 | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 1.4.8 | 1.4.7 | 1.4.7 | 1.4.7 | | HCatalog | 2.3.4-amzn-0 | 2.3.3-amzn-2 | 2.3.3-amzn-2 | 2.3.3-amzn-2 | | Hadoop | 2.8.5-amzn-1 | 2.8.5-amzn-0 | 2.8.5-amzn-0 | 2.8.4-amzn-1 | | Hive | 2.3.4-amzn-0 | 2.3.3-amzn-1 | 2.3.3-amzn-1 | 2.3.3-amzn-1 | | Hudi | - | - | - | - | | Hue | 4.3.0 | 4.2.0 | 4.2.0 | 4.2.0 | | Iceberg | - | - | - | - | | JupyterEnterpriseGateway | - | - | - | - | | JupyterHub | 0.9.4 | 0.9.4 | 0.9.4 | 0.8.1 | | Livy | 0.5.0-incubating | 0.5.0-incubating | 0.5.0-incubating | 0.5.0-incubating | | MXNet | 1.3.1 | 1.3.0 | 1.3.0 | 1.2.0 | | Mahout | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | | Oozie | 5.0.0 | 5.0.0 | 5.0.0 | 5.0.0 | | Phoenix | 4.14.0-HBase-1.4 | 4.14.0-HBase-1.4 | 4.14.0-HBase-1.4 | 4.14.0-HBase-1.4 | | Pig | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | | Presto | 0.214 | 0.212 | 0.212 | 0.210 | | Spark | 2.4.0 | 2.3.2 | 2.3.2 | 2.3.2 | | Sqoop | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | | TensorFlow | 1.12.0 | 1.11.0 | 1.11.0 | 1.9.0 | | Tez | 0.9.1 | 0.8.4 | 0.8.4 | 0.8.4 | | Trino (PrestoSQL) | - | - | - | - | | Zeppelin | 0.8.0 | 0.8.0 | 0.8.0 | 0.8.0 | | ZooKeeper | 3.4.13 | 3.4.13 | 3.4.13 | 3.4.12 | ## 5.20.0 release notes The following release notes include information for Amazon EMR release 5.20.0. Changes are relative to 5.19.0. Initial release date: December 18, 2018 Last updated date: January 22, 2019 **Upgrades** + Flink 1.6.2 + HBase 1.4.8 + Hive 2.3.4 + Hue 4.3.0 + MXNet 1.3.1 + Presto 0.214 + Spark 2.4.0 + TensorFlow 1.12.0 + Tez 0.9.1 + AWS SDK for Java 1.11.461 **New features** + (January 22, 2019) Kerberos in Amazon EMR has been improved to support authenticating principals from an external KDC. This centralizes principal management because multiple clusters can share a single, external KDC. In addition, the external KDC can have a cross-realm trust with an Active Directory domain. This allows all clusters to authenticate principals from Active Directory. For more information, see [Use Kerberos Authentication](https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-kerberos.html) in the *Amazon EMR Management Guide*. **Changes, enhancements, and resolved issues** + Default Amazon Linux AMI for Amazon EMR + Python3 package was upgraded from python 3.4 to 3.6. + The EMRFS S3-optimized committer + The EMRFS S3-optimized committer is now enabled by default, which improves write performance. For more information, see [Use the EMRFS S3-optimized committer](emr-spark-s3-optimized-committer.md). + Hive + Backported [HIVE-16686](https://issues.apache.org/jira/browse/HIVE-16686). + Glue with Spark and Hive + In EMR 5.20.0 or later, parallel partition pruning is enabled automatically for Spark and Hive when AWS Glue Data Catalog is used as the metastore. This change significantly reduces query planning time by executing multiple requests in parallel to retrieve partitions. The total number of segments that can be executed concurrently range between 1 and 10. The default value is 5, which is a recommended setting. You can change it by specifying the property `aws.glue.partition.num.segments` in `hive-site` configuration classification. If throttling occurs, you can turn off the feature by changing the value to 1. For more information, see [AWS Glue Segment Structure](https://docs.aws.amazon.com/glue/latest/dg/aws-glue-api-catalog-partitions.html#aws-glue-api-catalog-partitions-Segment). **Known issues** + Hue (Fixed in Amazon EMR release 5.24.0) + Hue running on Amazon EMR does not support Solr. Beginning with Amazon EMR release 5.20.0, a misconfiguration issue causes Solr to be enabled and a harmless error message to appear similar to the following: `Solr server could not be contacted properly: HTTPConnectionPool('host=ip-xx-xx-xx-xx.ec2.internal', port=1978): Max retries exceeded with url: /solr/admin/info/system?user.name=hue&doAs=administrator&wt=json (Caused by NewConnectionError(': Failed to establish a new connection: [Errno 111] Connection refused',))` **To prevent the Solr error message from appearing:** 1. Connect to the primary node command line using SSH. 1. Use a text editor to open the `hue.ini` file. For example: `sudo vim /etc/hue/conf/hue.ini` 1. Search for the term `appblacklist` and modify the line to the following: ``` appblacklist = search ``` 1. Save your changes and restart Hue as shown in the following example: ``` sudo stop hue; sudo start hue ``` + Tez + This issue was fixed in Amazon EMR 5.22.0. When you connect to the Tez UI at http://*MasterDNS*:8080/tez-ui through an SSH connection to the cluster primary node, the error "Adapter operation failed - Timeline server (ATS) is out of reach. Either it is down, or CORS is not enabled" appears, or tasks unexpectedly show N/A. This is caused by the Tez UI making requests to the YARN Timeline Server using `localhost` rather than the host name of the primary node. As a workaround, a script is available to run as a bootstrap action or step. The script updates the host name in the Tez `configs.env` file. For more information and the location of the script, see the [Bootstrap Instructions](http://awssupportdatasvcs.com/bootstrap-actions/fix_tez_ui_0-9-1/). + In Amazon EMR version 5.19.0, 5.20.0, and 5.21.0, YARN node labels are stored in an HDFS directory. In some situations, this leads to core node startup delays and then causes cluster time-out and launch failure. Beginning with Amazon EMR 5.22.0, this issue is resolved. YARN node labels are stored on the local disk of each cluster node, avoiding dependencies on HDFS. + Known issue in clusters with multiple primary nodes and Kerberos authentication If you run clusters with multiple primary nodes and Kerberos authentication in Amazon EMR releases 5.20.0 and later, you may encounter problems with cluster operations such as scale down or step submission, after the cluster has been running for some time. The time period depends on the Kerberos ticket validity period that you defined. The scale-down problem impacts both automatic scale-down and explicit scale down requests that you submitted. Additional cluster operations can also be impacted. Workaround: + SSH as `hadoop` user to the lead primary node of the EMR cluster with multiple primary nodes. + Run the following command to renew Kerberos ticket for `hadoop` user. ``` kinit -kt ``` Typically, the keytab file is located at `/etc/hadoop.keytab` and the principal is in the form of `hadoop/@`. **Note** This workaround will be effective for the time period the Kerberos ticket is valid. This duration is 10 hours by default, but can configured by your Kerberos settings. You must re-run the above command once the Kerberos ticket expires. ## 5.20.0 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | aws-sagemaker-spark-sdk | 1.2.1 | Amazon SageMaker Spark SDK | | emr-ddb | 4.7.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 2.5.1 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.4.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-s3-dist-cp | 2.10.0 | Distributed copy application optimized for Amazon S3. | | emr-s3-select | 1.2.0 | EMR S3Select Connector | | emrfs | 2.29.0 | Amazon S3 connector for Hadoop ecosystem applications. | | flink-client | 1.6.2 | Apache Flink command line client scripts and applications. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 2.8.5-amzn-1 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 2.8.5-amzn-1 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 2.8.5-amzn-1 | HDFS command-line client and library | | hadoop-hdfs-namenode | 2.8.5-amzn-1 | HDFS service for tracking file names and block locations. | | hadoop-httpfs-server | 2.8.5-amzn-1 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 2.8.5-amzn-1 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 2.8.5-amzn-1 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 2.8.5-amzn-1 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 2.8.5-amzn-1 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 2.8.5-amzn-1 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 1.4.8 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 1.4.8 | Service for serving one or more HBase regions. | | hbase-client | 1.4.8 | HBase command-line client. | | hbase-rest-server | 1.4.8 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 1.4.8 | Service providing a Thrift endpoint to HBase. | | hcatalog-client | 2.3.4-amzn-0 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 2.3.4-amzn-0 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 2.3.4-amzn-0 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 2.3.4-amzn-0 | Hive command line client. | | hive-hbase | 2.3.4-amzn-0 | Hive-hbase client. | | hive-metastore-server | 2.3.4-amzn-0 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server2 | 2.3.4-amzn-0 | Service for accepting Hive queries as web requests. | | hue-server | 4.3.0 | Web application for analyzing data using Hadoop ecosystem applications | | jupyterhub | 0.9.4 | Multi-user server for Jupyter notebooks | | livy-server | 0.5.0-incubating | REST interface for interacting with Apache Spark | | nginx | 1.12.1 | nginx [engine x] is an HTTP and reverse proxy server | | mahout-client | 0.13.0 | Library for machine learning. | | mxnet | 1.3.1 | A flexible, scalable, and efficient library for deep learning. | | mysql-server | 5.5.54\$1 | MySQL database server. | | nvidia-cuda | 9.2.88 | Nvidia drivers and Cuda toolkit | | oozie-client | 5.0.0 | Oozie command-line client. | | oozie-server | 5.0.0 | Service for accepting Oozie workflow requests. | | opencv | 3.4.0 | Open Source Computer Vision Library. | | phoenix-library | 4.14.0-HBase-1.4 | The phoenix libraries for server and client | | phoenix-query-server | 4.14.0-HBase-1.4 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.214 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.214 | Service for executing pieces of a query. | | pig-client | 0.17.0 | Pig command-line client. | | r | 3.4.1 | The R Project for Statistical Computing | | spark-client | 2.4.0 | Spark command-line clients. | | spark-history-server | 2.4.0 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 2.4.0 | In-memory execution engine for YARN. | | spark-yarn-slave | 2.4.0 | Apache Spark libraries needed by YARN slaves. | | sqoop-client | 1.4.7 | Apache Sqoop command-line client. | | tensorflow | 1.12.0 | TensorFlow open source software library for high performance numerical computation. | | tez-on-yarn | 0.9.1 | The tez YARN application and libraries. | | webserver | 2.4.25\$1 | Apache HTTP server. | | zeppelin-server | 0.8.0 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.4.13 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.4.13 | ZooKeeper command line client. | ## 5.20.0 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). **emr-5.20.0 classifications** | Classifications | Description | | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | | container-log4j | Change values in Hadoop YARN's container-log4j.properties file. | | core-site | Change values in Hadoop's core-site.xml file. | | emrfs-site | Change EMRFS settings. | | flink-conf | Change flink-conf.yaml settings. | | flink-log4j | Change Flink log4j.properties settings. | | flink-log4j-yarn-session | Change Flink log4j-yarn-session.properties settings. | | flink-log4j-cli | Change Flink log4j-cli.properties settings. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | | hadoop-ssl-server | Change hadoop ssl server configuration | | hadoop-ssl-client | Change hadoop ssl client configuration | | hbase | Amazon EMR-curated settings for Apache HBase. | | hbase-env | Change values in HBase's environment. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbase.properties file. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | | hbase-site | Change values in HBase's hbase-site.xml file. | | hdfs-encryption-zones | Configure HDFS encryption zones. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | | hcatalog-env | Change values in HCatalog's environment. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | | hcatalog-webhcat-log4j2 | Change values in HCatalog WebHCat's log4j2.properties. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | | hive-beeline-log4j2 | Change values in Hive's beeline-log4j2.properties file. | | hive-parquet-logging | Change values in Hive's parquet-logging.properties file. | | hive-env | Change values in the Hive environment. | | hive-exec-log4j2 | Change values in Hive's hive-exec-log4j2.properties file. | | hive-llap-daemon-log4j2 | Change values in Hive's llap-daemon-log4j2.properties file. | | hive-log4j2 | Change values in Hive's hive-log4j2.properties file. | | hive-site | Change values in Hive's hive-site.xml file | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | | hue-ini | Change values in Hue's ini file | | httpfs-env | Change values in the HTTPFS environment. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | | jupyter-notebook-conf | Change values in Jupyter Notebook's jupyter\$1notebook\$1config.py file. | | jupyter-hub-conf | Change values in JupyterHubs's jupyterhub\$1config.py file. | | jupyter-s3-conf | Configure Jupyter Notebook S3 persistence. | | jupyter-sparkmagic-conf | Change values in Sparkmagic's config.json file. | | livy-conf | Change values in Livy's livy.conf file. | | livy-env | Change values in the Livy environment. | | livy-log4j | Change Livy log4j.properties settings. | | mapred-env | Change values in the MapReduce application's environment. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | | oozie-env | Change values in Oozie's environment. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | | oozie-site | Change values in Oozie's oozie-site.xml file. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | | pig-env | Change values in the Pig environment. | | pig-properties | Change values in Pig's pig.properties file. | | pig-log4j | Change values in Pig's log4j.properties file. | | presto-log | Change values in Presto's log.properties file. | | presto-config | Change values in Presto's config.properties file. | | presto-password-authenticator | Change values in Presto's password-authenticator.properties file. | | presto-env | Change values in Presto's presto-env.sh file. | | presto-node | Change values in Presto's node.properties file. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | | presto-connector-hive | Change values in Presto's hive.properties file. | | presto-connector-jmx | Change values in Presto's jmx.properties file. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | | presto-connector-localfile | Change values in Presto's localfile.properties file. | | presto-connector-memory | Change values in Presto's memory.properties file. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | | presto-connector-redis | Change values in Presto's redis.properties file. | | presto-connector-redshift | Change values in Presto's redshift.properties file. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | | presto-connector-tpcds | Change values in Presto's tpcds.properties file. | | spark | Amazon EMR-curated settings for Apache Spark. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | | spark-env | Change values in the Spark environment. | | spark-hive-site | Change values in Spark's hive-site.xml file | | spark-log4j | Change values in Spark's log4j.properties file. | | spark-metrics | Change values in Spark's metrics.properties file. | | sqoop-env | Change values in Sqoop's environment. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | | tez-site | Change values in Tez's tez-site.xml file. | | yarn-env | Change values in the YARN environment. | | yarn-site | Change values in YARN's yarn-site.xml file. | | zeppelin-env | Change values in the Zeppelin environment. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | # Amazon EMR release 5.19.1 ## 5.19.1 application versions This release includes the following applications: [https://flink.apache.org/](https://flink.apache.org/), [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [http://gethue.com/](http://gethue.com/), [https://jupyterhub.readthedocs.io/en/latest/#](https://jupyterhub.readthedocs.io/en/latest/#), [https://livy.incubator.apache.org/](https://livy.incubator.apache.org/), [https://mxnet.incubator.apache.org/](https://mxnet.incubator.apache.org/), [http://mahout.apache.org/](http://mahout.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://www.tensorflow.org/](https://www.tensorflow.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-5.19.1 | emr-5.19.0 | emr-5.18.1 | emr-5.18.0 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.11.433 | 1.11.433 | 1.11.393 | 1.11.393 | | Python | 2.7, 3.4 | 2.7, 3.4 | 2.7, 3.4 | 2.7, 3.4 | | Scala | 2.11.8 | 2.11.8 | 2.11.8 | 2.11.8 | | AmazonCloudWatchAgent | - | - | - | - | | Delta | - | - | - | - | | Flink | 1.6.1 | 1.6.1 | 1.6.0 | 1.6.0 | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | | HCatalog | 2.3.3-amzn-2 | 2.3.3-amzn-2 | 2.3.3-amzn-2 | 2.3.3-amzn-2 | | Hadoop | 2.8.5-amzn-0 | 2.8.5-amzn-0 | 2.8.4-amzn-1 | 2.8.4-amzn-1 | | Hive | 2.3.3-amzn-1 | 2.3.3-amzn-1 | 2.3.3-amzn-1 | 2.3.3-amzn-1 | | Hudi | - | - | - | - | | Hue | 4.2.0 | 4.2.0 | 4.2.0 | 4.2.0 | | Iceberg | - | - | - | - | | JupyterEnterpriseGateway | - | - | - | - | | JupyterHub | 0.9.4 | 0.9.4 | 0.8.1 | 0.8.1 | | Livy | 0.5.0-incubating | 0.5.0-incubating | 0.5.0-incubating | 0.5.0-incubating | | MXNet | 1.3.0 | 1.3.0 | 1.2.0 | 1.2.0 | | Mahout | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | | Oozie | 5.0.0 | 5.0.0 | 5.0.0 | 5.0.0 | | Phoenix | 4.14.0-HBase-1.4 | 4.14.0-HBase-1.4 | 4.14.0-HBase-1.4 | 4.14.0-HBase-1.4 | | Pig | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | | Presto | 0.212 | 0.212 | 0.210 | 0.210 | | Spark | 2.3.2 | 2.3.2 | 2.3.2 | 2.3.2 | | Sqoop | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | | TensorFlow | 1.11.0 | 1.11.0 | 1.9.0 | 1.9.0 | | Tez | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | | Trino (PrestoSQL) | - | - | - | - | | Zeppelin | 0.8.0 | 0.8.0 | 0.8.0 | 0.8.0 | | ZooKeeper | 3.4.13 | 3.4.13 | 3.4.12 | 3.4.12 | ## 5.19.1 release notes This is a patch release to add AWS Signature Version 4 authentication for requests to Amazon S3. All applications and components are the same as the previous Amazon EMR release. **Important** In this release version, Amazon EMR uses AWS Signature Version 4 exclusively to authenticate requests to Amazon S3. For more information, see [Whats New](https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-whatsnew.html). ## 5.19.1 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | aws-sagemaker-spark-sdk | 1.2.0 | Amazon SageMaker Spark SDK | | emr-ddb | 4.7.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 2.5.1 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.4.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-s3-dist-cp | 2.10.0 | Distributed copy application optimized for Amazon S3. | | emr-s3-select | 1.1.0 | EMR S3Select Connector | | emrfs | 2.28.0 | Amazon S3 connector for Hadoop ecosystem applications. | | flink-client | 1.6.1 | Apache Flink command line client scripts and applications. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 2.8.5-amzn-0 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 2.8.5-amzn-0 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 2.8.5-amzn-0 | HDFS command-line client and library | | hadoop-hdfs-namenode | 2.8.5-amzn-0 | HDFS service for tracking file names and block locations. | | hadoop-httpfs-server | 2.8.5-amzn-0 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 2.8.5-amzn-0 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 2.8.5-amzn-0 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 2.8.5-amzn-0 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 2.8.5-amzn-0 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 2.8.5-amzn-0 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 1.4.7 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 1.4.7 | Service for serving one or more HBase regions. | | hbase-client | 1.4.7 | HBase command-line client. | | hbase-rest-server | 1.4.7 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 1.4.7 | Service providing a Thrift endpoint to HBase. | | hcatalog-client | 2.3.3-amzn-2 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 2.3.3-amzn-2 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 2.3.3-amzn-2 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 2.3.3-amzn-2 | Hive command line client. | | hive-hbase | 2.3.3-amzn-2 | Hive-hbase client. | | hive-metastore-server | 2.3.3-amzn-2 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server2 | 2.3.3-amzn-2 | Service for accepting Hive queries as web requests. | | hue-server | 4.2.0 | Web application for analyzing data using Hadoop ecosystem applications | | jupyterhub | 0.9.4 | Multi-user server for Jupyter notebooks | | livy-server | 0.5.0-incubating | REST interface for interacting with Apache Spark | | nginx | 1.12.1 | nginx [engine x] is an HTTP and reverse proxy server | | mahout-client | 0.13.0 | Library for machine learning. | | mxnet | 1.3.0 | A flexible, scalable, and efficient library for deep learning. | | mysql-server | 5.5.54\$1 | MySQL database server. | | nvidia-cuda | 9.2.88 | Nvidia drivers and Cuda toolkit | | oozie-client | 5.0.0 | Oozie command-line client. | | oozie-server | 5.0.0 | Service for accepting Oozie workflow requests. | | opencv | 3.4.0 | Open Source Computer Vision Library. | | phoenix-library | 4.14.0-HBase-1.4 | The phoenix libraries for server and client | | phoenix-query-server | 4.14.0-HBase-1.4 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.212 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.212 | Service for executing pieces of a query. | | pig-client | 0.17.0 | Pig command-line client. | | r | 3.4.1 | The R Project for Statistical Computing | | spark-client | 2.3.2 | Spark command-line clients. | | spark-history-server | 2.3.2 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 2.3.2 | In-memory execution engine for YARN. | | spark-yarn-slave | 2.3.2 | Apache Spark libraries needed by YARN slaves. | | sqoop-client | 1.4.7 | Apache Sqoop command-line client. | | tensorflow | 1.11.0 | TensorFlow open source software library for high performance numerical computation. | | tez-on-yarn | 0.8.4 | The tez YARN application and libraries. | | webserver | 2.4.25\$1 | Apache HTTP server. | | zeppelin-server | 0.8.0 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.4.13 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.4.13 | ZooKeeper command line client. | ## 5.19.1 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). **emr-5.19.1 classifications** | Classifications | Description | | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | | container-log4j | Change values in Hadoop YARN's container-log4j.properties file. | | core-site | Change values in Hadoop's core-site.xml file. | | emrfs-site | Change EMRFS settings. | | flink-conf | Change flink-conf.yaml settings. | | flink-log4j | Change Flink log4j.properties settings. | | flink-log4j-yarn-session | Change Flink log4j-yarn-session.properties settings. | | flink-log4j-cli | Change Flink log4j-cli.properties settings. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | | hadoop-ssl-server | Change hadoop ssl server configuration | | hadoop-ssl-client | Change hadoop ssl client configuration | | hbase | Amazon EMR-curated settings for Apache HBase. | | hbase-env | Change values in HBase's environment. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbase.properties file. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | | hbase-site | Change values in HBase's hbase-site.xml file. | | hdfs-encryption-zones | Configure HDFS encryption zones. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | | hcatalog-env | Change values in HCatalog's environment. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | | hcatalog-webhcat-log4j2 | Change values in HCatalog WebHCat's log4j2.properties. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | | hive-beeline-log4j2 | Change values in Hive's beeline-log4j2.properties file. | | hive-parquet-logging | Change values in Hive's parquet-logging.properties file. | | hive-env | Change values in the Hive environment. | | hive-exec-log4j2 | Change values in Hive's hive-exec-log4j2.properties file. | | hive-llap-daemon-log4j2 | Change values in Hive's llap-daemon-log4j2.properties file. | | hive-log4j2 | Change values in Hive's hive-log4j2.properties file. | | hive-site | Change values in Hive's hive-site.xml file | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | | hue-ini | Change values in Hue's ini file | | httpfs-env | Change values in the HTTPFS environment. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | | jupyter-notebook-conf | Change values in Jupyter Notebook's jupyter\$1notebook\$1config.py file. | | jupyter-hub-conf | Change values in JupyterHubs's jupyterhub\$1config.py file. | | jupyter-s3-conf | Configure Jupyter Notebook S3 persistence. | | jupyter-sparkmagic-conf | Change values in Sparkmagic's config.json file. | | livy-conf | Change values in Livy's livy.conf file. | | livy-env | Change values in the Livy environment. | | livy-log4j | Change Livy log4j.properties settings. | | mapred-env | Change values in the MapReduce application's environment. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | | oozie-env | Change values in Oozie's environment. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | | oozie-site | Change values in Oozie's oozie-site.xml file. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | | pig-env | Change values in the Pig environment. | | pig-properties | Change values in Pig's pig.properties file. | | pig-log4j | Change values in Pig's log4j.properties file. | | presto-log | Change values in Presto's log.properties file. | | presto-config | Change values in Presto's config.properties file. | | presto-password-authenticator | Change values in Presto's password-authenticator.properties file. | | presto-env | Change values in Presto's presto-env.sh file. | | presto-node | Change values in Presto's node.properties file. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | | presto-connector-hive | Change values in Presto's hive.properties file. | | presto-connector-jmx | Change values in Presto's jmx.properties file. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | | presto-connector-localfile | Change values in Presto's localfile.properties file. | | presto-connector-memory | Change values in Presto's memory.properties file. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | | presto-connector-redis | Change values in Presto's redis.properties file. | | presto-connector-redshift | Change values in Presto's redshift.properties file. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | | presto-connector-tpcds | Change values in Presto's tpcds.properties file. | | spark | Amazon EMR-curated settings for Apache Spark. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | | spark-env | Change values in the Spark environment. | | spark-hive-site | Change values in Spark's hive-site.xml file | | spark-log4j | Change values in Spark's log4j.properties file. | | spark-metrics | Change values in Spark's metrics.properties file. | | sqoop-env | Change values in Sqoop's environment. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | | tez-site | Change values in Tez's tez-site.xml file. | | yarn-env | Change values in the YARN environment. | | yarn-site | Change values in YARN's yarn-site.xml file. | | zeppelin-env | Change values in the Zeppelin environment. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | # Amazon EMR release 5.19.0 ## 5.19.0 application versions This release includes the following applications: [https://flink.apache.org/](https://flink.apache.org/), [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [http://gethue.com/](http://gethue.com/), [https://jupyterhub.readthedocs.io/en/latest/#](https://jupyterhub.readthedocs.io/en/latest/#), [https://livy.incubator.apache.org/](https://livy.incubator.apache.org/), [https://mxnet.incubator.apache.org/](https://mxnet.incubator.apache.org/), [http://mahout.apache.org/](http://mahout.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://www.tensorflow.org/](https://www.tensorflow.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-5.19.0 | emr-5.18.1 | emr-5.18.0 | emr-5.17.2 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.11.433 | 1.11.393 | 1.11.393 | 1.11.336 | | Python | 2.7, 3.4 | 2.7, 3.4 | 2.7, 3.4 | 2.7, 3.4 | | Scala | 2.11.8 | 2.11.8 | 2.11.8 | 2.11.8 | | AmazonCloudWatchAgent | - | - | - | - | | Delta | - | - | - | - | | Flink | 1.6.1 | 1.6.0 | 1.6.0 | 1.5.2 | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.6 | | HCatalog | 2.3.3-amzn-2 | 2.3.3-amzn-2 | 2.3.3-amzn-2 | 2.3.3-amzn-1 | | Hadoop | 2.8.5-amzn-0 | 2.8.4-amzn-1 | 2.8.4-amzn-1 | 2.8.4-amzn-1 | | Hive | 2.3.3-amzn-1 | 2.3.3-amzn-1 | 2.3.3-amzn-1 | 2.3.3-amzn-1 | | Hudi | - | - | - | - | | Hue | 4.2.0 | 4.2.0 | 4.2.0 | 4.2.0 | | Iceberg | - | - | - | - | | JupyterEnterpriseGateway | - | - | - | - | | JupyterHub | 0.9.4 | 0.8.1 | 0.8.1 | 0.8.1 | | Livy | 0.5.0-incubating | 0.5.0-incubating | 0.5.0-incubating | 0.5.0-incubating | | MXNet | 1.3.0 | 1.2.0 | 1.2.0 | 1.2.0 | | Mahout | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | | Oozie | 5.0.0 | 5.0.0 | 5.0.0 | 5.0.0 | | Phoenix | 4.14.0-HBase-1.4 | 4.14.0-HBase-1.4 | 4.14.0-HBase-1.4 | 4.14.0-HBase-1.4 | | Pig | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | | Presto | 0.212 | 0.210 | 0.210 | 0.206 | | Spark | 2.3.2 | 2.3.2 | 2.3.2 | 2.3.1 | | Sqoop | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | | TensorFlow | 1.11.0 | 1.9.0 | 1.9.0 | 1.9.0 | | Tez | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | | Trino (PrestoSQL) | - | - | - | - | | Zeppelin | 0.8.0 | 0.8.0 | 0.8.0 | 0.7.3 | | ZooKeeper | 3.4.13 | 3.4.12 | 3.4.12 | 3.4.12 | ## 5.19.0 release notes The following release notes include information for Amazon EMR release 5.19.0. Changes are relative to 5.18.0. Initial release date: November 7, 2018 Last updated date: November 19, 2018 **Upgrades** + Hadoop 2.8.5 + Flink 1.6.1 + JupyterHub 0.9.4 + MXNet 1.3.0 + Presto 0.212 + TensorFlow 1.11.0 + Zookeeper 3.4.13 + AWS SDK for Java 1.11.433 **New features** + (Nov. 19, 2018) EMR Notebooks is a managed environment based on Jupyter Notebook. It supports Spark magic kernels for PySpark, Spark SQL, Spark R, and Scala. EMR Notebooks can be used with clusters created using Amazon EMR release 5.18.0 and later. For more information, see [Using EMR Notebooks](https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-managed-notebooks.html) in the *Amazon EMR Management Guide*. + The EMRFS S3-optimized committer is available when writing Parquet files using Spark and EMRFS. This committer improves write performance. For more information, see [Use the EMRFS S3-optimized committer](emr-spark-s3-optimized-committer.md). **Changes, enhancements, and resolved issues** + YARN + Modified the logic that limits the application master process to running on core nodes. This functionality now uses the YARN node labels feature and properties in the `yarn-site` and `capacity-scheduler` configuration classifications. For information, see [https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-plan-instances-guidelines.html#emr-plan-spot-YARN.](https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-plan-instances-guidelines.html#emr-plan-spot-YARN.) + Default Amazon Linux AMI for Amazon EMR + `ruby18`, `php56`, and `gcc48` are no longer installed by default. These can be installed if desired using `yum`. + The aws-sdk ruby gem is no longer installed by default. It can be installed using `gem install aws-sdk`, if desired. Specific components can also be installed. For example, `gem install aws-sdk-s3`. **Known issues** + **EMR Notebooks**—In some circumstances, with multiple notebook editors open, the notebook editor may appear unable to connect to the cluster. If this happens, clear browser cookies and then reopen notebook editors. + **CloudWatch ContainerPending Metric and Automatic Scaling**—(Fixed in 5.20.0)Amazon EMR may emit a negative value for `ContainerPending`. If `ContainerPending` is used in an automatic scaling rule, automatic scaling does not behave as expected. Avoid using `ContainerPending` with automatic scaling. + In Amazon EMR version 5.19.0, 5.20.0, and 5.21.0, YARN node labels are stored in an HDFS directory. In some situations, this leads to core node startup delays and then causes cluster time-out and launch failure. Beginning with Amazon EMR 5.22.0, this issue is resolved. YARN node labels are stored on the local disk of each cluster node, avoiding dependencies on HDFS. ## 5.19.0 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | aws-sagemaker-spark-sdk | 1.2.0 | Amazon SageMaker Spark SDK | | emr-ddb | 4.7.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 2.5.1 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.4.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-s3-dist-cp | 2.10.0 | Distributed copy application optimized for Amazon S3. | | emr-s3-select | 1.1.0 | EMR S3Select Connector | | emrfs | 2.28.0 | Amazon S3 connector for Hadoop ecosystem applications. | | flink-client | 1.6.1 | Apache Flink command line client scripts and applications. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 2.8.5-amzn-0 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 2.8.5-amzn-0 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 2.8.5-amzn-0 | HDFS command-line client and library | | hadoop-hdfs-namenode | 2.8.5-amzn-0 | HDFS service for tracking file names and block locations. | | hadoop-httpfs-server | 2.8.5-amzn-0 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 2.8.5-amzn-0 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 2.8.5-amzn-0 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 2.8.5-amzn-0 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 2.8.5-amzn-0 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 2.8.5-amzn-0 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 1.4.7 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 1.4.7 | Service for serving one or more HBase regions. | | hbase-client | 1.4.7 | HBase command-line client. | | hbase-rest-server | 1.4.7 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 1.4.7 | Service providing a Thrift endpoint to HBase. | | hcatalog-client | 2.3.3-amzn-2 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 2.3.3-amzn-2 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 2.3.3-amzn-2 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 2.3.3-amzn-2 | Hive command line client. | | hive-hbase | 2.3.3-amzn-2 | Hive-hbase client. | | hive-metastore-server | 2.3.3-amzn-2 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server2 | 2.3.3-amzn-2 | Service for accepting Hive queries as web requests. | | hue-server | 4.2.0 | Web application for analyzing data using Hadoop ecosystem applications | | jupyterhub | 0.9.4 | Multi-user server for Jupyter notebooks | | livy-server | 0.5.0-incubating | REST interface for interacting with Apache Spark | | nginx | 1.12.1 | nginx [engine x] is an HTTP and reverse proxy server | | mahout-client | 0.13.0 | Library for machine learning. | | mxnet | 1.3.0 | A flexible, scalable, and efficient library for deep learning. | | mysql-server | 5.5.54\$1 | MySQL database server. | | nvidia-cuda | 9.2.88 | Nvidia drivers and Cuda toolkit | | oozie-client | 5.0.0 | Oozie command-line client. | | oozie-server | 5.0.0 | Service for accepting Oozie workflow requests. | | opencv | 3.4.0 | Open Source Computer Vision Library. | | phoenix-library | 4.14.0-HBase-1.4 | The phoenix libraries for server and client | | phoenix-query-server | 4.14.0-HBase-1.4 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.212 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.212 | Service for executing pieces of a query. | | pig-client | 0.17.0 | Pig command-line client. | | r | 3.4.1 | The R Project for Statistical Computing | | spark-client | 2.3.2 | Spark command-line clients. | | spark-history-server | 2.3.2 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 2.3.2 | In-memory execution engine for YARN. | | spark-yarn-slave | 2.3.2 | Apache Spark libraries needed by YARN slaves. | | sqoop-client | 1.4.7 | Apache Sqoop command-line client. | | tensorflow | 1.11.0 | TensorFlow open source software library for high performance numerical computation. | | tez-on-yarn | 0.8.4 | The tez YARN application and libraries. | | webserver | 2.4.25\$1 | Apache HTTP server. | | zeppelin-server | 0.8.0 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.4.13 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.4.13 | ZooKeeper command line client. | ## 5.19.0 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). **emr-5.19.0 classifications** | Classifications | Description | | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | | container-log4j | Change values in Hadoop YARN's container-log4j.properties file. | | core-site | Change values in Hadoop's core-site.xml file. | | emrfs-site | Change EMRFS settings. | | flink-conf | Change flink-conf.yaml settings. | | flink-log4j | Change Flink log4j.properties settings. | | flink-log4j-yarn-session | Change Flink log4j-yarn-session.properties settings. | | flink-log4j-cli | Change Flink log4j-cli.properties settings. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | | hadoop-ssl-server | Change hadoop ssl server configuration | | hadoop-ssl-client | Change hadoop ssl client configuration | | hbase | Amazon EMR-curated settings for Apache HBase. | | hbase-env | Change values in HBase's environment. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbase.properties file. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | | hbase-site | Change values in HBase's hbase-site.xml file. | | hdfs-encryption-zones | Configure HDFS encryption zones. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | | hcatalog-env | Change values in HCatalog's environment. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | | hcatalog-webhcat-log4j2 | Change values in HCatalog WebHCat's log4j2.properties. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | | hive-beeline-log4j2 | Change values in Hive's beeline-log4j2.properties file. | | hive-parquet-logging | Change values in Hive's parquet-logging.properties file. | | hive-env | Change values in the Hive environment. | | hive-exec-log4j2 | Change values in Hive's hive-exec-log4j2.properties file. | | hive-llap-daemon-log4j2 | Change values in Hive's llap-daemon-log4j2.properties file. | | hive-log4j2 | Change values in Hive's hive-log4j2.properties file. | | hive-site | Change values in Hive's hive-site.xml file | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | | hue-ini | Change values in Hue's ini file | | httpfs-env | Change values in the HTTPFS environment. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | | jupyter-notebook-conf | Change values in Jupyter Notebook's jupyter\$1notebook\$1config.py file. | | jupyter-hub-conf | Change values in JupyterHubs's jupyterhub\$1config.py file. | | jupyter-s3-conf | Configure Jupyter Notebook S3 persistence. | | jupyter-sparkmagic-conf | Change values in Sparkmagic's config.json file. | | livy-conf | Change values in Livy's livy.conf file. | | livy-env | Change values in the Livy environment. | | livy-log4j | Change Livy log4j.properties settings. | | mapred-env | Change values in the MapReduce application's environment. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | | oozie-env | Change values in Oozie's environment. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | | oozie-site | Change values in Oozie's oozie-site.xml file. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | | pig-env | Change values in the Pig environment. | | pig-properties | Change values in Pig's pig.properties file. | | pig-log4j | Change values in Pig's log4j.properties file. | | presto-log | Change values in Presto's log.properties file. | | presto-config | Change values in Presto's config.properties file. | | presto-password-authenticator | Change values in Presto's password-authenticator.properties file. | | presto-env | Change values in Presto's presto-env.sh file. | | presto-node | Change values in Presto's node.properties file. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | | presto-connector-hive | Change values in Presto's hive.properties file. | | presto-connector-jmx | Change values in Presto's jmx.properties file. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | | presto-connector-localfile | Change values in Presto's localfile.properties file. | | presto-connector-memory | Change values in Presto's memory.properties file. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | | presto-connector-redis | Change values in Presto's redis.properties file. | | presto-connector-redshift | Change values in Presto's redshift.properties file. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | | presto-connector-tpcds | Change values in Presto's tpcds.properties file. | | spark | Amazon EMR-curated settings for Apache Spark. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | | spark-env | Change values in the Spark environment. | | spark-hive-site | Change values in Spark's hive-site.xml file | | spark-log4j | Change values in Spark's log4j.properties file. | | spark-metrics | Change values in Spark's metrics.properties file. | | sqoop-env | Change values in Sqoop's environment. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | | tez-site | Change values in Tez's tez-site.xml file. | | yarn-env | Change values in the YARN environment. | | yarn-site | Change values in YARN's yarn-site.xml file. | | zeppelin-env | Change values in the Zeppelin environment. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | # Amazon EMR release 5.18.1 ## 5.18.1 application versions This release includes the following applications: [https://flink.apache.org/](https://flink.apache.org/), [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [http://gethue.com/](http://gethue.com/), [https://jupyterhub.readthedocs.io/en/latest/#](https://jupyterhub.readthedocs.io/en/latest/#), [https://livy.incubator.apache.org/](https://livy.incubator.apache.org/), [https://mxnet.incubator.apache.org/](https://mxnet.incubator.apache.org/), [http://mahout.apache.org/](http://mahout.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://www.tensorflow.org/](https://www.tensorflow.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-5.18.1 | emr-5.18.0 | emr-5.17.2 | emr-5.17.1 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.11.393 | 1.11.393 | 1.11.336 | 1.11.336 | | Python | 2.7, 3.4 | 2.7, 3.4 | 2.7, 3.4 | 2.7, 3.4 | | Scala | 2.11.8 | 2.11.8 | 2.11.8 | 2.11.8 | | AmazonCloudWatchAgent | - | - | - | - | | Delta | - | - | - | - | | Flink | 1.6.0 | 1.6.0 | 1.5.2 | 1.5.2 | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 1.4.7 | 1.4.7 | 1.4.6 | 1.4.6 | | HCatalog | 2.3.3-amzn-2 | 2.3.3-amzn-2 | 2.3.3-amzn-1 | 2.3.3-amzn-1 | | Hadoop | 2.8.4-amzn-1 | 2.8.4-amzn-1 | 2.8.4-amzn-1 | 2.8.4-amzn-1 | | Hive | 2.3.3-amzn-1 | 2.3.3-amzn-1 | 2.3.3-amzn-1 | 2.3.3-amzn-1 | | Hudi | - | - | - | - | | Hue | 4.2.0 | 4.2.0 | 4.2.0 | 4.2.0 | | Iceberg | - | - | - | - | | JupyterEnterpriseGateway | - | - | - | - | | JupyterHub | 0.8.1 | 0.8.1 | 0.8.1 | 0.8.1 | | Livy | 0.5.0-incubating | 0.5.0-incubating | 0.5.0-incubating | 0.5.0-incubating | | MXNet | 1.2.0 | 1.2.0 | 1.2.0 | 1.2.0 | | Mahout | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | | Oozie | 5.0.0 | 5.0.0 | 5.0.0 | 5.0.0 | | Phoenix | 4.14.0-HBase-1.4 | 4.14.0-HBase-1.4 | 4.14.0-HBase-1.4 | 4.14.0-HBase-1.4 | | Pig | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | | Presto | 0.210 | 0.210 | 0.206 | 0.206 | | Spark | 2.3.2 | 2.3.2 | 2.3.1 | 2.3.1 | | Sqoop | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | | TensorFlow | 1.9.0 | 1.9.0 | 1.9.0 | 1.9.0 | | Tez | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | | Trino (PrestoSQL) | - | - | - | - | | Zeppelin | 0.8.0 | 0.8.0 | 0.7.3 | 0.7.3 | | ZooKeeper | 3.4.12 | 3.4.12 | 3.4.12 | 3.4.12 | ## 5.18.1 release notes This is a patch release to add AWS Signature Version 4 authentication for requests to Amazon S3. All applications and components are the same as the previous Amazon EMR release. **Important** In this release version, Amazon EMR uses AWS Signature Version 4 exclusively to authenticate requests to Amazon S3. For more information, see [Whats New](https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-whatsnew.html). ## 5.18.1 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | aws-sagemaker-spark-sdk | 1.1.3 | Amazon SageMaker Spark SDK | | emr-ddb | 4.6.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 2.5.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.4.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-s3-dist-cp | 2.10.0 | Distributed copy application optimized for Amazon S3. | | emr-s3-select | 1.1.0 | EMR S3Select Connector | | emrfs | 2.27.0 | Amazon S3 connector for Hadoop ecosystem applications. | | flink-client | 1.6.0 | Apache Flink command line client scripts and applications. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 2.8.4-amzn-1 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 2.8.4-amzn-1 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 2.8.4-amzn-1 | HDFS command-line client and library | | hadoop-hdfs-namenode | 2.8.4-amzn-1 | HDFS service for tracking file names and block locations. | | hadoop-httpfs-server | 2.8.4-amzn-1 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 2.8.4-amzn-1 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 2.8.4-amzn-1 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 2.8.4-amzn-1 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 2.8.4-amzn-1 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 2.8.4-amzn-1 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 1.4.7 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 1.4.7 | Service for serving one or more HBase regions. | | hbase-client | 1.4.7 | HBase command-line client. | | hbase-rest-server | 1.4.7 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 1.4.7 | Service providing a Thrift endpoint to HBase. | | hcatalog-client | 2.3.3-amzn-2 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 2.3.3-amzn-2 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 2.3.3-amzn-2 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 2.3.3-amzn-2 | Hive command line client. | | hive-hbase | 2.3.3-amzn-2 | Hive-hbase client. | | hive-metastore-server | 2.3.3-amzn-2 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server2 | 2.3.3-amzn-2 | Service for accepting Hive queries as web requests. | | hue-server | 4.2.0 | Web application for analyzing data using Hadoop ecosystem applications | | jupyterhub | 0.8.1 | Multi-user server for Jupyter notebooks | | livy-server | 0.5.0-incubating | REST interface for interacting with Apache Spark | | nginx | 1.12.1 | nginx [engine x] is an HTTP and reverse proxy server | | mahout-client | 0.13.0 | Library for machine learning. | | mxnet | 1.2.0 | A flexible, scalable, and efficient library for deep learning. | | mysql-server | 5.5.54\$1 | MySQL database server. | | nvidia-cuda | 9.2.88 | Nvidia drivers and Cuda toolkit | | oozie-client | 5.0.0 | Oozie command-line client. | | oozie-server | 5.0.0 | Service for accepting Oozie workflow requests. | | opencv | 3.4.0 | Open Source Computer Vision Library. | | phoenix-library | 4.14.0-HBase-1.4 | The phoenix libraries for server and client | | phoenix-query-server | 4.14.0-HBase-1.4 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.210 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.210 | Service for executing pieces of a query. | | pig-client | 0.17.0 | Pig command-line client. | | r | 3.4.1 | The R Project for Statistical Computing | | spark-client | 2.3.2 | Spark command-line clients. | | spark-history-server | 2.3.2 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 2.3.2 | In-memory execution engine for YARN. | | spark-yarn-slave | 2.3.2 | Apache Spark libraries needed by YARN slaves. | | sqoop-client | 1.4.7 | Apache Sqoop command-line client. | | tensorflow | 1.9.0 | TensorFlow open source software library for high performance numerical computation. | | tez-on-yarn | 0.8.4 | The tez YARN application and libraries. | | webserver | 2.4.25\$1 | Apache HTTP server. | | zeppelin-server | 0.8.0 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.4.12 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.4.12 | ZooKeeper command line client. | ## 5.18.1 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). **emr-5.18.1 classifications** | Classifications | Description | | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | | container-log4j | Change values in Hadoop YARN's container-log4j.properties file. | | core-site | Change values in Hadoop's core-site.xml file. | | emrfs-site | Change EMRFS settings. | | flink-conf | Change flink-conf.yaml settings. | | flink-log4j | Change Flink log4j.properties settings. | | flink-log4j-yarn-session | Change Flink log4j-yarn-session.properties settings. | | flink-log4j-cli | Change Flink log4j-cli.properties settings. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | | hadoop-ssl-server | Change hadoop ssl server configuration | | hadoop-ssl-client | Change hadoop ssl client configuration | | hbase | Amazon EMR-curated settings for Apache HBase. | | hbase-env | Change values in HBase's environment. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbase.properties file. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | | hbase-site | Change values in HBase's hbase-site.xml file. | | hdfs-encryption-zones | Configure HDFS encryption zones. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | | hcatalog-env | Change values in HCatalog's environment. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | | hcatalog-webhcat-log4j2 | Change values in HCatalog WebHCat's log4j2.properties. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | | hive-beeline-log4j2 | Change values in Hive's beeline-log4j2.properties file. | | hive-parquet-logging | Change values in Hive's parquet-logging.properties file. | | hive-env | Change values in the Hive environment. | | hive-exec-log4j2 | Change values in Hive's hive-exec-log4j2.properties file. | | hive-llap-daemon-log4j2 | Change values in Hive's llap-daemon-log4j2.properties file. | | hive-log4j2 | Change values in Hive's hive-log4j2.properties file. | | hive-site | Change values in Hive's hive-site.xml file | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | | hue-ini | Change values in Hue's ini file | | httpfs-env | Change values in the HTTPFS environment. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | | jupyter-notebook-conf | Change values in Jupyter Notebook's jupyter\$1notebook\$1config.py file. | | jupyter-hub-conf | Change values in JupyterHubs's jupyterhub\$1config.py file. | | jupyter-s3-conf | Configure Jupyter Notebook S3 persistence. | | jupyter-sparkmagic-conf | Change values in Sparkmagic's config.json file. | | livy-conf | Change values in Livy's livy.conf file. | | livy-env | Change values in the Livy environment. | | livy-log4j | Change Livy log4j.properties settings. | | mapred-env | Change values in the MapReduce application's environment. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | | oozie-env | Change values in Oozie's environment. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | | oozie-site | Change values in Oozie's oozie-site.xml file. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | | pig-env | Change values in the Pig environment. | | pig-properties | Change values in Pig's pig.properties file. | | pig-log4j | Change values in Pig's log4j.properties file. | | presto-log | Change values in Presto's log.properties file. | | presto-config | Change values in Presto's config.properties file. | | presto-password-authenticator | Change values in Presto's password-authenticator.properties file. | | presto-env | Change values in Presto's presto-env.sh file. | | presto-node | Change values in Presto's node.properties file. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | | presto-connector-hive | Change values in Presto's hive.properties file. | | presto-connector-jmx | Change values in Presto's jmx.properties file. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | | presto-connector-localfile | Change values in Presto's localfile.properties file. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | | presto-connector-redis | Change values in Presto's redis.properties file. | | presto-connector-redshift | Change values in Presto's redshift.properties file. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | | spark | Amazon EMR-curated settings for Apache Spark. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | | spark-env | Change values in the Spark environment. | | spark-hive-site | Change values in Spark's hive-site.xml file | | spark-log4j | Change values in Spark's log4j.properties file. | | spark-metrics | Change values in Spark's metrics.properties file. | | sqoop-env | Change values in Sqoop's environment. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | | tez-site | Change values in Tez's tez-site.xml file. | | yarn-env | Change values in the YARN environment. | | yarn-site | Change values in YARN's yarn-site.xml file. | | zeppelin-env | Change values in the Zeppelin environment. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | # Amazon EMR release 5.18.0 ## 5.18.0 application versions This release includes the following applications: [https://flink.apache.org/](https://flink.apache.org/), [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [http://gethue.com/](http://gethue.com/), [https://jupyterhub.readthedocs.io/en/latest/#](https://jupyterhub.readthedocs.io/en/latest/#), [https://livy.incubator.apache.org/](https://livy.incubator.apache.org/), [https://mxnet.incubator.apache.org/](https://mxnet.incubator.apache.org/), [http://mahout.apache.org/](http://mahout.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://www.tensorflow.org/](https://www.tensorflow.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-5.18.0 | emr-5.17.2 | emr-5.17.1 | emr-5.17.0 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.11.393 | 1.11.336 | 1.11.336 | 1.11.336 | | Python | 2.7, 3.4 | 2.7, 3.4 | 2.7, 3.4 | 2.7, 3.4 | | Scala | 2.11.8 | 2.11.8 | 2.11.8 | 2.11.8 | | AmazonCloudWatchAgent | - | - | - | - | | Delta | - | - | - | - | | Flink | 1.6.0 | 1.5.2 | 1.5.2 | 1.5.2 | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 1.4.7 | 1.4.6 | 1.4.6 | 1.4.6 | | HCatalog | 2.3.3-amzn-2 | 2.3.3-amzn-1 | 2.3.3-amzn-1 | 2.3.3-amzn-1 | | Hadoop | 2.8.4-amzn-1 | 2.8.4-amzn-1 | 2.8.4-amzn-1 | 2.8.4-amzn-1 | | Hive | 2.3.3-amzn-1 | 2.3.3-amzn-1 | 2.3.3-amzn-1 | 2.3.3-amzn-1 | | Hudi | - | - | - | - | | Hue | 4.2.0 | 4.2.0 | 4.2.0 | 4.2.0 | | Iceberg | - | - | - | - | | JupyterEnterpriseGateway | - | - | - | - | | JupyterHub | 0.8.1 | 0.8.1 | 0.8.1 | 0.8.1 | | Livy | 0.5.0-incubating | 0.5.0-incubating | 0.5.0-incubating | 0.5.0-incubating | | MXNet | 1.2.0 | 1.2.0 | 1.2.0 | 1.2.0 | | Mahout | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | | Oozie | 5.0.0 | 5.0.0 | 5.0.0 | 5.0.0 | | Phoenix | 4.14.0-HBase-1.4 | 4.14.0-HBase-1.4 | 4.14.0-HBase-1.4 | 4.14.0-HBase-1.4 | | Pig | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | | Presto | 0.210 | 0.206 | 0.206 | 0.206 | | Spark | 2.3.2 | 2.3.1 | 2.3.1 | 2.3.1 | | Sqoop | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | | TensorFlow | 1.9.0 | 1.9.0 | 1.9.0 | 1.9.0 | | Tez | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | | Trino (PrestoSQL) | - | - | - | - | | Zeppelin | 0.8.0 | 0.7.3 | 0.7.3 | 0.7.3 | | ZooKeeper | 3.4.12 | 3.4.12 | 3.4.12 | 3.4.12 | ## 5.18.0 release notes The following release notes include information for Amazon EMR release 5.18.0. Changes are relative to 5.17.0. Initial release date: October 24, 2018 **Upgrades** + Flink 1.6.0 + HBase 1.4.7 + Presto 0.210 + Spark 2.3.2 + Zeppelin 0.8.0 **New features** + Beginning with Amazon EMR 5.18.0, you can use the Amazon EMR artifact repository to build your job code against the exact versions of libraries and dependencies that are available with specific Amazon EMR releases. For more information, see [Checking dependencies using the Amazon EMR artifact repository](emr-artifact-repository.md). **Changes, enhancements, and resolved issues** + Hive + Added support for S3 Select. For more information, see [Using S3 Select with Hive to improve performance](emr-hive-s3select.md). + Presto + Added support for [S3 Select](https://aws.amazon.com/blogs/aws/s3-glacier-select/) Pushdown. For more information, see [Using S3 Select Pushdown with Presto to improve performance](emr-presto-s3select.md). + Spark + The default log4j configuration for Spark has been changed to roll container logs hourly for Spark streaming jobs. This helps prevent the deletion of logs for long-running Spark streaming jobs. ## 5.18.0 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | aws-sagemaker-spark-sdk | 1.1.3 | Amazon SageMaker Spark SDK | | emr-ddb | 4.6.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 2.5.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.4.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-s3-dist-cp | 2.10.0 | Distributed copy application optimized for Amazon S3. | | emr-s3-select | 1.1.0 | EMR S3Select Connector | | emrfs | 2.27.0 | Amazon S3 connector for Hadoop ecosystem applications. | | flink-client | 1.6.0 | Apache Flink command line client scripts and applications. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 2.8.4-amzn-1 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 2.8.4-amzn-1 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 2.8.4-amzn-1 | HDFS command-line client and library | | hadoop-hdfs-namenode | 2.8.4-amzn-1 | HDFS service for tracking file names and block locations. | | hadoop-httpfs-server | 2.8.4-amzn-1 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 2.8.4-amzn-1 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 2.8.4-amzn-1 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 2.8.4-amzn-1 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 2.8.4-amzn-1 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 2.8.4-amzn-1 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 1.4.7 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 1.4.7 | Service for serving one or more HBase regions. | | hbase-client | 1.4.7 | HBase command-line client. | | hbase-rest-server | 1.4.7 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 1.4.7 | Service providing a Thrift endpoint to HBase. | | hcatalog-client | 2.3.3-amzn-2 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 2.3.3-amzn-2 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 2.3.3-amzn-2 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 2.3.3-amzn-2 | Hive command line client. | | hive-hbase | 2.3.3-amzn-2 | Hive-hbase client. | | hive-metastore-server | 2.3.3-amzn-2 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server2 | 2.3.3-amzn-2 | Service for accepting Hive queries as web requests. | | hue-server | 4.2.0 | Web application for analyzing data using Hadoop ecosystem applications | | jupyterhub | 0.8.1 | Multi-user server for Jupyter notebooks | | livy-server | 0.5.0-incubating | REST interface for interacting with Apache Spark | | nginx | 1.12.1 | nginx [engine x] is an HTTP and reverse proxy server | | mahout-client | 0.13.0 | Library for machine learning. | | mxnet | 1.2.0 | A flexible, scalable, and efficient library for deep learning. | | mysql-server | 5.5.54\$1 | MySQL database server. | | nvidia-cuda | 9.2.88 | Nvidia drivers and Cuda toolkit | | oozie-client | 5.0.0 | Oozie command-line client. | | oozie-server | 5.0.0 | Service for accepting Oozie workflow requests. | | opencv | 3.4.0 | Open Source Computer Vision Library. | | phoenix-library | 4.14.0-HBase-1.4 | The phoenix libraries for server and client | | phoenix-query-server | 4.14.0-HBase-1.4 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.210 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.210 | Service for executing pieces of a query. | | pig-client | 0.17.0 | Pig command-line client. | | r | 3.4.1 | The R Project for Statistical Computing | | spark-client | 2.3.2 | Spark command-line clients. | | spark-history-server | 2.3.2 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 2.3.2 | In-memory execution engine for YARN. | | spark-yarn-slave | 2.3.2 | Apache Spark libraries needed by YARN slaves. | | sqoop-client | 1.4.7 | Apache Sqoop command-line client. | | tensorflow | 1.9.0 | TensorFlow open source software library for high performance numerical computation. | | tez-on-yarn | 0.8.4 | The tez YARN application and libraries. | | webserver | 2.4.25\$1 | Apache HTTP server. | | zeppelin-server | 0.8.0 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.4.12 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.4.12 | ZooKeeper command line client. | ## 5.18.0 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). **emr-5.18.0 classifications** | Classifications | Description | | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | | container-log4j | Change values in Hadoop YARN's container-log4j.properties file. | | core-site | Change values in Hadoop's core-site.xml file. | | emrfs-site | Change EMRFS settings. | | flink-conf | Change flink-conf.yaml settings. | | flink-log4j | Change Flink log4j.properties settings. | | flink-log4j-yarn-session | Change Flink log4j-yarn-session.properties settings. | | flink-log4j-cli | Change Flink log4j-cli.properties settings. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | | hadoop-ssl-server | Change hadoop ssl server configuration | | hadoop-ssl-client | Change hadoop ssl client configuration | | hbase | Amazon EMR-curated settings for Apache HBase. | | hbase-env | Change values in HBase's environment. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbase.properties file. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | | hbase-site | Change values in HBase's hbase-site.xml file. | | hdfs-encryption-zones | Configure HDFS encryption zones. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | | hcatalog-env | Change values in HCatalog's environment. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | | hcatalog-webhcat-log4j2 | Change values in HCatalog WebHCat's log4j2.properties. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | | hive-beeline-log4j2 | Change values in Hive's beeline-log4j2.properties file. | | hive-parquet-logging | Change values in Hive's parquet-logging.properties file. | | hive-env | Change values in the Hive environment. | | hive-exec-log4j2 | Change values in Hive's hive-exec-log4j2.properties file. | | hive-llap-daemon-log4j2 | Change values in Hive's llap-daemon-log4j2.properties file. | | hive-log4j2 | Change values in Hive's hive-log4j2.properties file. | | hive-site | Change values in Hive's hive-site.xml file | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | | hue-ini | Change values in Hue's ini file | | httpfs-env | Change values in the HTTPFS environment. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | | jupyter-notebook-conf | Change values in Jupyter Notebook's jupyter\$1notebook\$1config.py file. | | jupyter-hub-conf | Change values in JupyterHubs's jupyterhub\$1config.py file. | | jupyter-s3-conf | Configure Jupyter Notebook S3 persistence. | | jupyter-sparkmagic-conf | Change values in Sparkmagic's config.json file. | | livy-conf | Change values in Livy's livy.conf file. | | livy-env | Change values in the Livy environment. | | livy-log4j | Change Livy log4j.properties settings. | | mapred-env | Change values in the MapReduce application's environment. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | | oozie-env | Change values in Oozie's environment. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | | oozie-site | Change values in Oozie's oozie-site.xml file. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | | pig-env | Change values in the Pig environment. | | pig-properties | Change values in Pig's pig.properties file. | | pig-log4j | Change values in Pig's log4j.properties file. | | presto-log | Change values in Presto's log.properties file. | | presto-config | Change values in Presto's config.properties file. | | presto-password-authenticator | Change values in Presto's password-authenticator.properties file. | | presto-env | Change values in Presto's presto-env.sh file. | | presto-node | Change values in Presto's node.properties file. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | | presto-connector-hive | Change values in Presto's hive.properties file. | | presto-connector-jmx | Change values in Presto's jmx.properties file. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | | presto-connector-localfile | Change values in Presto's localfile.properties file. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | | presto-connector-redis | Change values in Presto's redis.properties file. | | presto-connector-redshift | Change values in Presto's redshift.properties file. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | | spark | Amazon EMR-curated settings for Apache Spark. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | | spark-env | Change values in the Spark environment. | | spark-hive-site | Change values in Spark's hive-site.xml file | | spark-log4j | Change values in Spark's log4j.properties file. | | spark-metrics | Change values in Spark's metrics.properties file. | | sqoop-env | Change values in Sqoop's environment. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | | tez-site | Change values in Tez's tez-site.xml file. | | yarn-env | Change values in the YARN environment. | | yarn-site | Change values in YARN's yarn-site.xml file. | | zeppelin-env | Change values in the Zeppelin environment. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | # Amazon EMR release 5.17.2 ## 5.17.2 application versions This release includes the following applications: [https://flink.apache.org/](https://flink.apache.org/), [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [http://gethue.com/](http://gethue.com/), [https://jupyterhub.readthedocs.io/en/latest/#](https://jupyterhub.readthedocs.io/en/latest/#), [https://livy.incubator.apache.org/](https://livy.incubator.apache.org/), [https://mxnet.incubator.apache.org/](https://mxnet.incubator.apache.org/), [http://mahout.apache.org/](http://mahout.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://www.tensorflow.org/](https://www.tensorflow.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-5.17.2 | emr-5.17.1 | emr-5.17.0 | emr-5.16.1 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.11.336 | 1.11.336 | 1.11.336 | 1.11.336 | | Python | 2.7, 3.4 | 2.7, 3.4 | 2.7, 3.4 | 2.7, 3.4 | | Scala | 2.11.8 | 2.11.8 | 2.11.8 | 2.11.8 | | AmazonCloudWatchAgent | - | - | - | - | | Delta | - | - | - | - | | Flink | 1.5.2 | 1.5.2 | 1.5.2 | 1.5.0 | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 1.4.6 | 1.4.6 | 1.4.6 | 1.4.4 | | HCatalog | 2.3.3-amzn-1 | 2.3.3-amzn-1 | 2.3.3-amzn-1 | 2.3.3-amzn-1 | | Hadoop | 2.8.4-amzn-1 | 2.8.4-amzn-1 | 2.8.4-amzn-1 | 2.8.4-amzn-0 | | Hive | 2.3.3-amzn-1 | 2.3.3-amzn-1 | 2.3.3-amzn-1 | 2.3.3-amzn-1 | | Hudi | - | - | - | - | | Hue | 4.2.0 | 4.2.0 | 4.2.0 | 4.2.0 | | Iceberg | - | - | - | - | | JupyterEnterpriseGateway | - | - | - | - | | JupyterHub | 0.8.1 | 0.8.1 | 0.8.1 | 0.8.1 | | Livy | 0.5.0-incubating | 0.5.0-incubating | 0.5.0-incubating | 0.5.0-incubating | | MXNet | 1.2.0 | 1.2.0 | 1.2.0 | 1.2.0 | | Mahout | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | | Oozie | 5.0.0 | 5.0.0 | 5.0.0 | 5.0.0 | | Phoenix | 4.14.0-HBase-1.4 | 4.14.0-HBase-1.4 | 4.14.0-HBase-1.4 | 4.14.0-HBase-1.4 | | Pig | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | | Presto | 0.206 | 0.206 | 0.206 | 0.203 | | Spark | 2.3.1 | 2.3.1 | 2.3.1 | 2.3.1 | | Sqoop | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | | TensorFlow | 1.9.0 | 1.9.0 | 1.9.0 | - | | Tez | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | | Trino (PrestoSQL) | - | - | - | - | | Zeppelin | 0.7.3 | 0.7.3 | 0.7.3 | 0.7.3 | | ZooKeeper | 3.4.12 | 3.4.12 | 3.4.12 | 3.4.12 | ## 5.17.2 release notes This is a patch release to add AWS Signature Version 4 authentication for requests to Amazon S3. All applications and components are the same as the previous Amazon EMR release. **Important** In this release version, Amazon EMR uses AWS Signature Version 4 exclusively to authenticate requests to Amazon S3. For more information, see [Whats New](https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-whatsnew.html). ## 5.17.2 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | aws-sagemaker-spark-sdk | 1.1.3 | Amazon SageMaker Spark SDK | | emr-ddb | 4.6.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 2.5.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.4.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-s3-dist-cp | 2.10.0 | Distributed copy application optimized for Amazon S3. | | emr-s3-select | 1.0.0 | EMR S3Select Connector | | emrfs | 2.26.0 | Amazon S3 connector for Hadoop ecosystem applications. | | flink-client | 1.5.2 | Apache Flink command line client scripts and applications. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 2.8.4-amzn-1 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 2.8.4-amzn-1 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 2.8.4-amzn-1 | HDFS command-line client and library | | hadoop-hdfs-namenode | 2.8.4-amzn-1 | HDFS service for tracking file names and block locations. | | hadoop-httpfs-server | 2.8.4-amzn-1 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 2.8.4-amzn-1 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 2.8.4-amzn-1 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 2.8.4-amzn-1 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 2.8.4-amzn-1 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 2.8.4-amzn-1 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 1.4.6 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 1.4.6 | Service for serving one or more HBase regions. | | hbase-client | 1.4.6 | HBase command-line client. | | hbase-rest-server | 1.4.6 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 1.4.6 | Service providing a Thrift endpoint to HBase. | | hcatalog-client | 2.3.3-amzn-1 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 2.3.3-amzn-1 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 2.3.3-amzn-1 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 2.3.3-amzn-1 | Hive command line client. | | hive-hbase | 2.3.3-amzn-1 | Hive-hbase client. | | hive-metastore-server | 2.3.3-amzn-1 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server2 | 2.3.3-amzn-1 | Service for accepting Hive queries as web requests. | | hue-server | 4.2.0 | Web application for analyzing data using Hadoop ecosystem applications | | jupyterhub | 0.8.1 | Multi-user server for Jupyter notebooks | | livy-server | 0.5.0-incubating | REST interface for interacting with Apache Spark | | mahout-client | 0.13.0 | Library for machine learning. | | mxnet | 1.2.0 | A flexible, scalable, and efficient library for deep learning. | | mysql-server | 5.5.54\$1 | MySQL database server. | | nvidia-cuda | 9.2.88 | Nvidia drivers and Cuda toolkit | | oozie-client | 5.0.0 | Oozie command-line client. | | oozie-server | 5.0.0 | Service for accepting Oozie workflow requests. | | opencv | 3.4.0 | Open Source Computer Vision Library. | | phoenix-library | 4.14.0-HBase-1.4 | The phoenix libraries for server and client | | phoenix-query-server | 4.14.0-HBase-1.4 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.206 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.206 | Service for executing pieces of a query. | | pig-client | 0.17.0 | Pig command-line client. | | r | 3.4.1 | The R Project for Statistical Computing | | spark-client | 2.3.1 | Spark command-line clients. | | spark-history-server | 2.3.1 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 2.3.1 | In-memory execution engine for YARN. | | spark-yarn-slave | 2.3.1 | Apache Spark libraries needed by YARN slaves. | | sqoop-client | 1.4.7 | Apache Sqoop command-line client. | | tensorflow | 1.9.0 | TensorFlow open source software library for high performance numerical computation. | | tez-on-yarn | 0.8.4 | The tez YARN application and libraries. | | webserver | 2.4.25\$1 | Apache HTTP server. | | zeppelin-server | 0.7.3 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.4.12 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.4.12 | ZooKeeper command line client. | ## 5.17.2 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). **emr-5.17.2 classifications** | Classifications | Description | | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | | container-log4j | Change values in Hadoop YARN's container-log4j.properties file. | | core-site | Change values in Hadoop's core-site.xml file. | | emrfs-site | Change EMRFS settings. | | flink-conf | Change flink-conf.yaml settings. | | flink-log4j | Change Flink log4j.properties settings. | | flink-log4j-yarn-session | Change Flink log4j-yarn-session.properties settings. | | flink-log4j-cli | Change Flink log4j-cli.properties settings. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | | hadoop-ssl-server | Change hadoop ssl server configuration | | hadoop-ssl-client | Change hadoop ssl client configuration | | hbase | Amazon EMR-curated settings for Apache HBase. | | hbase-env | Change values in HBase's environment. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbase.properties file. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | | hbase-site | Change values in HBase's hbase-site.xml file. | | hdfs-encryption-zones | Configure HDFS encryption zones. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | | hcatalog-env | Change values in HCatalog's environment. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | | hcatalog-webhcat-log4j2 | Change values in HCatalog WebHCat's log4j2.properties. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | | hive-beeline-log4j2 | Change values in Hive's beeline-log4j2.properties file. | | hive-parquet-logging | Change values in Hive's parquet-logging.properties file. | | hive-env | Change values in the Hive environment. | | hive-exec-log4j2 | Change values in Hive's hive-exec-log4j2.properties file. | | hive-llap-daemon-log4j2 | Change values in Hive's llap-daemon-log4j2.properties file. | | hive-log4j2 | Change values in Hive's hive-log4j2.properties file. | | hive-site | Change values in Hive's hive-site.xml file | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | | hue-ini | Change values in Hue's ini file | | httpfs-env | Change values in the HTTPFS environment. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | | jupyter-notebook-conf | Change values in Jupyter Notebook's jupyter\$1notebook\$1config.py file. | | jupyter-hub-conf | Change values in JupyterHubs's jupyterhub\$1config.py file. | | jupyter-s3-conf | Configure Jupyter Notebook S3 persistence. | | jupyter-sparkmagic-conf | Change values in Sparkmagic's config.json file. | | livy-conf | Change values in Livy's livy.conf file. | | livy-env | Change values in the Livy environment. | | livy-log4j | Change Livy log4j.properties settings. | | mapred-env | Change values in the MapReduce application's environment. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | | oozie-env | Change values in Oozie's environment. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | | oozie-site | Change values in Oozie's oozie-site.xml file. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | | pig-env | Change values in the Pig environment. | | pig-properties | Change values in Pig's pig.properties file. | | pig-log4j | Change values in Pig's log4j.properties file. | | presto-log | Change values in Presto's log.properties file. | | presto-config | Change values in Presto's config.properties file. | | presto-password-authenticator | Change values in Presto's password-authenticator.properties file. | | presto-env | Change values in Presto's presto-env.sh file. | | presto-node | Change values in Presto's node.properties file. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | | presto-connector-hive | Change values in Presto's hive.properties file. | | presto-connector-jmx | Change values in Presto's jmx.properties file. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | | presto-connector-localfile | Change values in Presto's localfile.properties file. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | | presto-connector-redis | Change values in Presto's redis.properties file. | | presto-connector-redshift | Change values in Presto's redshift.properties file. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | | spark | Amazon EMR-curated settings for Apache Spark. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | | spark-env | Change values in the Spark environment. | | spark-hive-site | Change values in Spark's hive-site.xml file | | spark-log4j | Change values in Spark's log4j.properties file. | | spark-metrics | Change values in Spark's metrics.properties file. | | sqoop-env | Change values in Sqoop's environment. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | | tez-site | Change values in Tez's tez-site.xml file. | | yarn-env | Change values in the YARN environment. | | yarn-site | Change values in YARN's yarn-site.xml file. | | zeppelin-env | Change values in the Zeppelin environment. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | # Amazon EMR release 5.17.1 ## 5.17.1 application versions This release includes the following applications: [https://flink.apache.org/](https://flink.apache.org/), [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [http://gethue.com/](http://gethue.com/), [https://jupyterhub.readthedocs.io/en/latest/#](https://jupyterhub.readthedocs.io/en/latest/#), [https://livy.incubator.apache.org/](https://livy.incubator.apache.org/), [https://mxnet.incubator.apache.org/](https://mxnet.incubator.apache.org/), [http://mahout.apache.org/](http://mahout.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://www.tensorflow.org/](https://www.tensorflow.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-5.17.1 | emr-5.17.0 | emr-5.16.1 | emr-5.16.0 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.11.336 | 1.11.336 | 1.11.336 | 1.11.336 | | Python | 2.7, 3.4 | 2.7, 3.4 | 2.7, 3.4 | 2.7, 3.4 | | Scala | 2.11.8 | 2.11.8 | 2.11.8 | 2.11.8 | | AmazonCloudWatchAgent | - | - | - | - | | Delta | - | - | - | - | | Flink | 1.5.2 | 1.5.2 | 1.5.0 | 1.5.0 | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 1.4.6 | 1.4.6 | 1.4.4 | 1.4.4 | | HCatalog | 2.3.3-amzn-1 | 2.3.3-amzn-1 | 2.3.3-amzn-1 | 2.3.3-amzn-1 | | Hadoop | 2.8.4-amzn-1 | 2.8.4-amzn-1 | 2.8.4-amzn-0 | 2.8.4-amzn-0 | | Hive | 2.3.3-amzn-1 | 2.3.3-amzn-1 | 2.3.3-amzn-1 | 2.3.3-amzn-1 | | Hudi | - | - | - | - | | Hue | 4.2.0 | 4.2.0 | 4.2.0 | 4.2.0 | | Iceberg | - | - | - | - | | JupyterEnterpriseGateway | - | - | - | - | | JupyterHub | 0.8.1 | 0.8.1 | 0.8.1 | 0.8.1 | | Livy | 0.5.0-incubating | 0.5.0-incubating | 0.5.0-incubating | 0.5.0-incubating | | MXNet | 1.2.0 | 1.2.0 | 1.2.0 | 1.2.0 | | Mahout | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | | Oozie | 5.0.0 | 5.0.0 | 5.0.0 | 5.0.0 | | Phoenix | 4.14.0-HBase-1.4 | 4.14.0-HBase-1.4 | 4.14.0-HBase-1.4 | 4.14.0-HBase-1.4 | | Pig | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | | Presto | 0.206 | 0.206 | 0.203 | 0.203 | | Spark | 2.3.1 | 2.3.1 | 2.3.1 | 2.3.1 | | Sqoop | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | | TensorFlow | 1.9.0 | 1.9.0 | - | - | | Tez | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | | Trino (PrestoSQL) | - | - | - | - | | Zeppelin | 0.7.3 | 0.7.3 | 0.7.3 | 0.7.3 | | ZooKeeper | 3.4.12 | 3.4.12 | 3.4.12 | 3.4.12 | ## 5.17.1 release notes The following release notes include information for Amazon EMR release 5.17.1. Changes are relative to 5.17.0. Initial release date: July 18, 2019 **Changes, enhancements, and resolved issues** + Updated the default Amazon Linux AMI for Amazon EMR to include important Linux kernel security updates, including the TCP SACK Denial of Service Issue ([AWS-2019-005](https://aws.amazon.com/security/security-bulletins/AWS-2019-005/)). ## 5.17.1 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | aws-sagemaker-spark-sdk | 1.1.3 | Amazon SageMaker Spark SDK | | emr-ddb | 4.6.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 2.5.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.4.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-s3-dist-cp | 2.10.0 | Distributed copy application optimized for Amazon S3. | | emr-s3-select | 1.0.0 | EMR S3Select Connector | | emrfs | 2.26.0 | Amazon S3 connector for Hadoop ecosystem applications. | | flink-client | 1.5.2 | Apache Flink command line client scripts and applications. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 2.8.4-amzn-1 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 2.8.4-amzn-1 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 2.8.4-amzn-1 | HDFS command-line client and library | | hadoop-hdfs-namenode | 2.8.4-amzn-1 | HDFS service for tracking file names and block locations. | | hadoop-httpfs-server | 2.8.4-amzn-1 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 2.8.4-amzn-1 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 2.8.4-amzn-1 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 2.8.4-amzn-1 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 2.8.4-amzn-1 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 2.8.4-amzn-1 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 1.4.6 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 1.4.6 | Service for serving one or more HBase regions. | | hbase-client | 1.4.6 | HBase command-line client. | | hbase-rest-server | 1.4.6 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 1.4.6 | Service providing a Thrift endpoint to HBase. | | hcatalog-client | 2.3.3-amzn-1 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 2.3.3-amzn-1 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 2.3.3-amzn-1 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 2.3.3-amzn-1 | Hive command line client. | | hive-hbase | 2.3.3-amzn-1 | Hive-hbase client. | | hive-metastore-server | 2.3.3-amzn-1 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server2 | 2.3.3-amzn-1 | Service for accepting Hive queries as web requests. | | hue-server | 4.2.0 | Web application for analyzing data using Hadoop ecosystem applications | | jupyterhub | 0.8.1 | Multi-user server for Jupyter notebooks | | livy-server | 0.5.0-incubating | REST interface for interacting with Apache Spark | | mahout-client | 0.13.0 | Library for machine learning. | | mxnet | 1.2.0 | A flexible, scalable, and efficient library for deep learning. | | mysql-server | 5.5.54\$1 | MySQL database server. | | nvidia-cuda | 9.2.88 | Nvidia drivers and Cuda toolkit | | oozie-client | 5.0.0 | Oozie command-line client. | | oozie-server | 5.0.0 | Service for accepting Oozie workflow requests. | | opencv | 3.4.0 | Open Source Computer Vision Library. | | phoenix-library | 4.14.0-HBase-1.4 | The phoenix libraries for server and client | | phoenix-query-server | 4.14.0-HBase-1.4 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.206 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.206 | Service for executing pieces of a query. | | pig-client | 0.17.0 | Pig command-line client. | | r | 3.4.1 | The R Project for Statistical Computing | | spark-client | 2.3.1 | Spark command-line clients. | | spark-history-server | 2.3.1 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 2.3.1 | In-memory execution engine for YARN. | | spark-yarn-slave | 2.3.1 | Apache Spark libraries needed by YARN slaves. | | sqoop-client | 1.4.7 | Apache Sqoop command-line client. | | tensorflow | 1.9.0 | TensorFlow open source software library for high performance numerical computation. | | tez-on-yarn | 0.8.4 | The tez YARN application and libraries. | | webserver | 2.4.25\$1 | Apache HTTP server. | | zeppelin-server | 0.7.3 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.4.12 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.4.12 | ZooKeeper command line client. | ## 5.17.1 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). **emr-5.17.1 classifications** | Classifications | Description | | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | | container-log4j | Change values in Hadoop YARN's container-log4j.properties file. | | core-site | Change values in Hadoop's core-site.xml file. | | emrfs-site | Change EMRFS settings. | | flink-conf | Change flink-conf.yaml settings. | | flink-log4j | Change Flink log4j.properties settings. | | flink-log4j-yarn-session | Change Flink log4j-yarn-session.properties settings. | | flink-log4j-cli | Change Flink log4j-cli.properties settings. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | | hadoop-ssl-server | Change hadoop ssl server configuration | | hadoop-ssl-client | Change hadoop ssl client configuration | | hbase | Amazon EMR-curated settings for Apache HBase. | | hbase-env | Change values in HBase's environment. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbase.properties file. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | | hbase-site | Change values in HBase's hbase-site.xml file. | | hdfs-encryption-zones | Configure HDFS encryption zones. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | | hcatalog-env | Change values in HCatalog's environment. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | | hcatalog-webhcat-log4j2 | Change values in HCatalog WebHCat's log4j2.properties. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | | hive-beeline-log4j2 | Change values in Hive's beeline-log4j2.properties file. | | hive-parquet-logging | Change values in Hive's parquet-logging.properties file. | | hive-env | Change values in the Hive environment. | | hive-exec-log4j2 | Change values in Hive's hive-exec-log4j2.properties file. | | hive-llap-daemon-log4j2 | Change values in Hive's llap-daemon-log4j2.properties file. | | hive-log4j2 | Change values in Hive's hive-log4j2.properties file. | | hive-site | Change values in Hive's hive-site.xml file | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | | hue-ini | Change values in Hue's ini file | | httpfs-env | Change values in the HTTPFS environment. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | | jupyter-notebook-conf | Change values in Jupyter Notebook's jupyter\$1notebook\$1config.py file. | | jupyter-hub-conf | Change values in JupyterHubs's jupyterhub\$1config.py file. | | jupyter-s3-conf | Configure Jupyter Notebook S3 persistence. | | jupyter-sparkmagic-conf | Change values in Sparkmagic's config.json file. | | livy-conf | Change values in Livy's livy.conf file. | | livy-env | Change values in the Livy environment. | | livy-log4j | Change Livy log4j.properties settings. | | mapred-env | Change values in the MapReduce application's environment. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | | oozie-env | Change values in Oozie's environment. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | | oozie-site | Change values in Oozie's oozie-site.xml file. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | | pig-env | Change values in the Pig environment. | | pig-properties | Change values in Pig's pig.properties file. | | pig-log4j | Change values in Pig's log4j.properties file. | | presto-log | Change values in Presto's log.properties file. | | presto-config | Change values in Presto's config.properties file. | | presto-password-authenticator | Change values in Presto's password-authenticator.properties file. | | presto-env | Change values in Presto's presto-env.sh file. | | presto-node | Change values in Presto's node.properties file. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | | presto-connector-hive | Change values in Presto's hive.properties file. | | presto-connector-jmx | Change values in Presto's jmx.properties file. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | | presto-connector-localfile | Change values in Presto's localfile.properties file. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | | presto-connector-redis | Change values in Presto's redis.properties file. | | presto-connector-redshift | Change values in Presto's redshift.properties file. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | | spark | Amazon EMR-curated settings for Apache Spark. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | | spark-env | Change values in the Spark environment. | | spark-hive-site | Change values in Spark's hive-site.xml file | | spark-log4j | Change values in Spark's log4j.properties file. | | spark-metrics | Change values in Spark's metrics.properties file. | | sqoop-env | Change values in Sqoop's environment. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | | tez-site | Change values in Tez's tez-site.xml file. | | yarn-env | Change values in the YARN environment. | | yarn-site | Change values in YARN's yarn-site.xml file. | | zeppelin-env | Change values in the Zeppelin environment. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | # Amazon EMR release 5.17.0 ## 5.17.0 application versions This release includes the following applications: [https://flink.apache.org/](https://flink.apache.org/), [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [http://gethue.com/](http://gethue.com/), [https://jupyterhub.readthedocs.io/en/latest/#](https://jupyterhub.readthedocs.io/en/latest/#), [https://livy.incubator.apache.org/](https://livy.incubator.apache.org/), [https://mxnet.incubator.apache.org/](https://mxnet.incubator.apache.org/), [http://mahout.apache.org/](http://mahout.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://www.tensorflow.org/](https://www.tensorflow.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-5.17.0 | emr-5.16.1 | emr-5.16.0 | emr-5.15.1 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.11.336 | 1.11.336 | 1.11.336 | 1.11.333 | | Python | 2.7, 3.4 | 2.7, 3.4 | 2.7, 3.4 | 2.7, 3.4 | | Scala | 2.11.8 | 2.11.8 | 2.11.8 | 2.11.8 | | AmazonCloudWatchAgent | - | - | - | - | | Delta | - | - | - | - | | Flink | 1.5.2 | 1.5.0 | 1.5.0 | 1.4.2 | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 1.4.6 | 1.4.4 | 1.4.4 | 1.4.4 | | HCatalog | 2.3.3-amzn-1 | 2.3.3-amzn-1 | 2.3.3-amzn-1 | 2.3.3-amzn-0 | | Hadoop | 2.8.4-amzn-1 | 2.8.4-amzn-0 | 2.8.4-amzn-0 | 2.8.3-amzn-0 | | Hive | 2.3.3-amzn-1 | 2.3.3-amzn-1 | 2.3.3-amzn-1 | 2.3.3-amzn-0 | | Hudi | - | - | - | - | | Hue | 4.2.0 | 4.2.0 | 4.2.0 | 4.2.0 | | Iceberg | - | - | - | - | | JupyterEnterpriseGateway | - | - | - | - | | JupyterHub | 0.8.1 | 0.8.1 | 0.8.1 | 0.8.1 | | Livy | 0.5.0-incubating | 0.5.0-incubating | 0.5.0-incubating | 0.4.0-incubating | | MXNet | 1.2.0 | 1.2.0 | 1.2.0 | 1.1.0 | | Mahout | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | | Oozie | 5.0.0 | 5.0.0 | 5.0.0 | 5.0.0 | | Phoenix | 4.14.0-HBase-1.4 | 4.14.0-HBase-1.4 | 4.14.0-HBase-1.4 | 4.13.0-HBase-1.4 | | Pig | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | | Presto | 0.206 | 0.203 | 0.203 | 0.194 | | Spark | 2.3.1 | 2.3.1 | 2.3.1 | 2.3.0 | | Sqoop | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | | TensorFlow | 1.9.0 | - | - | - | | Tez | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | | Trino (PrestoSQL) | - | - | - | - | | Zeppelin | 0.7.3 | 0.7.3 | 0.7.3 | 0.7.3 | | ZooKeeper | 3.4.12 | 3.4.12 | 3.4.12 | 3.4.12 | ## 5.17.0 release notes The following release notes include information for Amazon EMR release 5.17.0. Changes are relative to 5.16.0. Initial release date: August 30, 2018 **Upgrades** + Flink 1.5.2 + HBase 1.4.6 + Presto 0.206 **New features** + Added support for Tensorflow. For more information, see [TensorFlow](emr-tensorflow.md). **Changes, enhancements, and resolved issues** + JupyterHub + Added support for notebook persistence in Amazon S3. For more information, see [Configuring persistence for notebooks in Amazon S3](emr-jupyterhub-s3.md). + Spark + Added support for [S3 Select](https://aws.amazon.com/blogs/aws/s3-glacier-select/). For more information, see [Use S3 Select with Spark to improve query performance](emr-spark-s3select.md). + Resolved the issues with the Cloudwatch metrics and the automatic scaling feature in Amazon EMR version 5.14.0, 5.15.0, or 5.16.0. **Known issues** + When you create a kerberized cluster with Livy installed, Livy fails with an error that simple authentication is not enabled. Rebooting the Livy server resolves the issue. As a workaround, add a step during cluster creation that runs `sudo restart livy-server` on the primary node. + If you use a custom Amazon Linux AMI based on an Amazon Linux AMI with a creation date of 2018-08-11, the Oozie server fails to start. If you use Oozie, create a custom AMI based on an Amazon Linux AMI ID with a different creation date. You can use the following AWS CLI command to return a list of Image IDs for all HVM Amazon Linux AMIs with a 2018.03 version, along with the release date, so that you can choose an appropriate Amazon Linux AMI as your base. Replace MyRegion with your Region identifier, such as us-west-2. ``` aws ec2 --region MyRegion describe-images --owner amazon --query 'Images[?Name!=`null`]|[?starts_with(Name, `amzn-ami-hvm-2018.03`) == `true`].[CreationDate,ImageId,Name]' --output text | sort -rk1 ``` ## 5.17.0 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | aws-sagemaker-spark-sdk | 1.1.3 | Amazon SageMaker Spark SDK | | emr-ddb | 4.6.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 2.5.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.4.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-s3-dist-cp | 2.10.0 | Distributed copy application optimized for Amazon S3. | | emr-s3-select | 1.0.0 | EMR S3Select Connector | | emrfs | 2.26.0 | Amazon S3 connector for Hadoop ecosystem applications. | | flink-client | 1.5.2 | Apache Flink command line client scripts and applications. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 2.8.4-amzn-1 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 2.8.4-amzn-1 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 2.8.4-amzn-1 | HDFS command-line client and library | | hadoop-hdfs-namenode | 2.8.4-amzn-1 | HDFS service for tracking file names and block locations. | | hadoop-httpfs-server | 2.8.4-amzn-1 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 2.8.4-amzn-1 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 2.8.4-amzn-1 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 2.8.4-amzn-1 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 2.8.4-amzn-1 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 2.8.4-amzn-1 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 1.4.6 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 1.4.6 | Service for serving one or more HBase regions. | | hbase-client | 1.4.6 | HBase command-line client. | | hbase-rest-server | 1.4.6 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 1.4.6 | Service providing a Thrift endpoint to HBase. | | hcatalog-client | 2.3.3-amzn-1 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 2.3.3-amzn-1 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 2.3.3-amzn-1 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 2.3.3-amzn-1 | Hive command line client. | | hive-hbase | 2.3.3-amzn-1 | Hive-hbase client. | | hive-metastore-server | 2.3.3-amzn-1 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server2 | 2.3.3-amzn-1 | Service for accepting Hive queries as web requests. | | hue-server | 4.2.0 | Web application for analyzing data using Hadoop ecosystem applications | | jupyterhub | 0.8.1 | Multi-user server for Jupyter notebooks | | livy-server | 0.5.0-incubating | REST interface for interacting with Apache Spark | | mahout-client | 0.13.0 | Library for machine learning. | | mxnet | 1.2.0 | A flexible, scalable, and efficient library for deep learning. | | mysql-server | 5.5.54\$1 | MySQL database server. | | nvidia-cuda | 9.2.88 | Nvidia drivers and Cuda toolkit | | oozie-client | 5.0.0 | Oozie command-line client. | | oozie-server | 5.0.0 | Service for accepting Oozie workflow requests. | | opencv | 3.4.0 | Open Source Computer Vision Library. | | phoenix-library | 4.14.0-HBase-1.4 | The phoenix libraries for server and client | | phoenix-query-server | 4.14.0-HBase-1.4 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.206 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.206 | Service for executing pieces of a query. | | pig-client | 0.17.0 | Pig command-line client. | | r | 3.4.1 | The R Project for Statistical Computing | | spark-client | 2.3.1 | Spark command-line clients. | | spark-history-server | 2.3.1 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 2.3.1 | In-memory execution engine for YARN. | | spark-yarn-slave | 2.3.1 | Apache Spark libraries needed by YARN slaves. | | sqoop-client | 1.4.7 | Apache Sqoop command-line client. | | tensorflow | 1.9.0 | TensorFlow open source software library for high performance numerical computation. | | tez-on-yarn | 0.8.4 | The tez YARN application and libraries. | | webserver | 2.4.25\$1 | Apache HTTP server. | | zeppelin-server | 0.7.3 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.4.12 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.4.12 | ZooKeeper command line client. | ## 5.17.0 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). **emr-5.17.0 classifications** | Classifications | Description | | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | | container-log4j | Change values in Hadoop YARN's container-log4j.properties file. | | core-site | Change values in Hadoop's core-site.xml file. | | emrfs-site | Change EMRFS settings. | | flink-conf | Change flink-conf.yaml settings. | | flink-log4j | Change Flink log4j.properties settings. | | flink-log4j-yarn-session | Change Flink log4j-yarn-session.properties settings. | | flink-log4j-cli | Change Flink log4j-cli.properties settings. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | | hadoop-ssl-server | Change hadoop ssl server configuration | | hadoop-ssl-client | Change hadoop ssl client configuration | | hbase | Amazon EMR-curated settings for Apache HBase. | | hbase-env | Change values in HBase's environment. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbase.properties file. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | | hbase-site | Change values in HBase's hbase-site.xml file. | | hdfs-encryption-zones | Configure HDFS encryption zones. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | | hcatalog-env | Change values in HCatalog's environment. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | | hcatalog-webhcat-log4j2 | Change values in HCatalog WebHCat's log4j2.properties. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | | hive-beeline-log4j2 | Change values in Hive's beeline-log4j2.properties file. | | hive-parquet-logging | Change values in Hive's parquet-logging.properties file. | | hive-env | Change values in the Hive environment. | | hive-exec-log4j2 | Change values in Hive's hive-exec-log4j2.properties file. | | hive-llap-daemon-log4j2 | Change values in Hive's llap-daemon-log4j2.properties file. | | hive-log4j2 | Change values in Hive's hive-log4j2.properties file. | | hive-site | Change values in Hive's hive-site.xml file | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | | hue-ini | Change values in Hue's ini file | | httpfs-env | Change values in the HTTPFS environment. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | | jupyter-notebook-conf | Change values in Jupyter Notebook's jupyter\$1notebook\$1config.py file. | | jupyter-hub-conf | Change values in JupyterHubs's jupyterhub\$1config.py file. | | jupyter-s3-conf | Configure Jupyter Notebook S3 persistence. | | jupyter-sparkmagic-conf | Change values in Sparkmagic's config.json file. | | livy-conf | Change values in Livy's livy.conf file. | | livy-env | Change values in the Livy environment. | | livy-log4j | Change Livy log4j.properties settings. | | mapred-env | Change values in the MapReduce application's environment. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | | oozie-env | Change values in Oozie's environment. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | | oozie-site | Change values in Oozie's oozie-site.xml file. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | | pig-env | Change values in the Pig environment. | | pig-properties | Change values in Pig's pig.properties file. | | pig-log4j | Change values in Pig's log4j.properties file. | | presto-log | Change values in Presto's log.properties file. | | presto-config | Change values in Presto's config.properties file. | | presto-password-authenticator | Change values in Presto's password-authenticator.properties file. | | presto-env | Change values in Presto's presto-env.sh file. | | presto-node | Change values in Presto's node.properties file. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | | presto-connector-hive | Change values in Presto's hive.properties file. | | presto-connector-jmx | Change values in Presto's jmx.properties file. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | | presto-connector-localfile | Change values in Presto's localfile.properties file. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | | presto-connector-redis | Change values in Presto's redis.properties file. | | presto-connector-redshift | Change values in Presto's redshift.properties file. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | | spark | Amazon EMR-curated settings for Apache Spark. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | | spark-env | Change values in the Spark environment. | | spark-hive-site | Change values in Spark's hive-site.xml file | | spark-log4j | Change values in Spark's log4j.properties file. | | spark-metrics | Change values in Spark's metrics.properties file. | | sqoop-env | Change values in Sqoop's environment. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | | tez-site | Change values in Tez's tez-site.xml file. | | yarn-env | Change values in the YARN environment. | | yarn-site | Change values in YARN's yarn-site.xml file. | | zeppelin-env | Change values in the Zeppelin environment. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | # Amazon EMR release 5.16.1 ## 5.16.1 application versions This release includes the following applications: [https://flink.apache.org/](https://flink.apache.org/), [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [http://gethue.com/](http://gethue.com/), [https://jupyterhub.readthedocs.io/en/latest/#](https://jupyterhub.readthedocs.io/en/latest/#), [https://livy.incubator.apache.org/](https://livy.incubator.apache.org/), [https://mxnet.incubator.apache.org/](https://mxnet.incubator.apache.org/), [http://mahout.apache.org/](http://mahout.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-5.16.1 | emr-5.16.0 | emr-5.15.1 | emr-5.15.0 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.11.336 | 1.11.336 | 1.11.333 | 1.11.333 | | Python | 2.7, 3.4 | 2.7, 3.4 | 2.7, 3.4 | 2.7, 3.4 | | Scala | 2.11.8 | 2.11.8 | 2.11.8 | 2.11.8 | | AmazonCloudWatchAgent | - | - | - | - | | Delta | - | - | - | - | | Flink | 1.5.0 | 1.5.0 | 1.4.2 | 1.4.2 | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 1.4.4 | 1.4.4 | 1.4.4 | 1.4.4 | | HCatalog | 2.3.3-amzn-1 | 2.3.3-amzn-1 | 2.3.3-amzn-0 | 2.3.3-amzn-0 | | Hadoop | 2.8.4-amzn-0 | 2.8.4-amzn-0 | 2.8.3-amzn-0 | 2.8.3-amzn-0 | | Hive | 2.3.3-amzn-1 | 2.3.3-amzn-1 | 2.3.3-amzn-0 | 2.3.3-amzn-0 | | Hudi | - | - | - | - | | Hue | 4.2.0 | 4.2.0 | 4.2.0 | 4.2.0 | | Iceberg | - | - | - | - | | JupyterEnterpriseGateway | - | - | - | - | | JupyterHub | 0.8.1 | 0.8.1 | 0.8.1 | 0.8.1 | | Livy | 0.5.0-incubating | 0.5.0-incubating | 0.4.0-incubating | 0.4.0-incubating | | MXNet | 1.2.0 | 1.2.0 | 1.1.0 | 1.1.0 | | Mahout | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | | Oozie | 5.0.0 | 5.0.0 | 5.0.0 | 5.0.0 | | Phoenix | 4.14.0-HBase-1.4 | 4.14.0-HBase-1.4 | 4.13.0-HBase-1.4 | 4.13.0-HBase-1.4 | | Pig | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | | Presto | 0.203 | 0.203 | 0.194 | 0.194 | | Spark | 2.3.1 | 2.3.1 | 2.3.0 | 2.3.0 | | Sqoop | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | | TensorFlow | - | - | - | - | | Tez | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | | Trino (PrestoSQL) | - | - | - | - | | Zeppelin | 0.7.3 | 0.7.3 | 0.7.3 | 0.7.3 | | ZooKeeper | 3.4.12 | 3.4.12 | 3.4.12 | 3.4.12 | ## 5.16.1 release notes This is a patch release to add AWS Signature Version 4 authentication for requests to Amazon S3. All applications and components are the same as the previous Amazon EMR release. **Important** In this release version, Amazon EMR uses AWS Signature Version 4 exclusively to authenticate requests to Amazon S3. For more information, see [Whats New](https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-whatsnew.html). ## 5.16.1 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | aws-sagemaker-spark-sdk | 1.1.0 | Amazon SageMaker Spark SDK | | emr-ddb | 4.6.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 2.4.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.4.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-s3-dist-cp | 2.10.0 | Distributed copy application optimized for Amazon S3. | | emrfs | 2.25.0 | Amazon S3 connector for Hadoop ecosystem applications. | | flink-client | 1.5.0 | Apache Flink command line client scripts and applications. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 2.8.4-amzn-0 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 2.8.4-amzn-0 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 2.8.4-amzn-0 | HDFS command-line client and library | | hadoop-hdfs-namenode | 2.8.4-amzn-0 | HDFS service for tracking file names and block locations. | | hadoop-httpfs-server | 2.8.4-amzn-0 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 2.8.4-amzn-0 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 2.8.4-amzn-0 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 2.8.4-amzn-0 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 2.8.4-amzn-0 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 2.8.4-amzn-0 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 1.4.4 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 1.4.4 | Service for serving one or more HBase regions. | | hbase-client | 1.4.4 | HBase command-line client. | | hbase-rest-server | 1.4.4 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 1.4.4 | Service providing a Thrift endpoint to HBase. | | hcatalog-client | 2.3.3-amzn-1 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 2.3.3-amzn-1 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 2.3.3-amzn-1 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 2.3.3-amzn-1 | Hive command line client. | | hive-hbase | 2.3.3-amzn-1 | Hive-hbase client. | | hive-metastore-server | 2.3.3-amzn-1 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server2 | 2.3.3-amzn-1 | Service for accepting Hive queries as web requests. | | hue-server | 4.2.0 | Web application for analyzing data using Hadoop ecosystem applications | | jupyterhub | 0.8.1 | Multi-user server for Jupyter notebooks | | livy-server | 0.5.0-incubating | REST interface for interacting with Apache Spark | | mahout-client | 0.13.0 | Library for machine learning. | | mxnet | 1.2.0 | A flexible, scalable, and efficient library for deep learning. | | mysql-server | 5.5.54\$1 | MySQL database server. | | nvidia-cuda | 9.2.88 | Nvidia drivers and Cuda toolkit | | oozie-client | 5.0.0 | Oozie command-line client. | | oozie-server | 5.0.0 | Service for accepting Oozie workflow requests. | | opencv | 3.4.0 | Open Source Computer Vision Library. | | phoenix-library | 4.14.0-HBase-1.4 | The phoenix libraries for server and client | | phoenix-query-server | 4.14.0-HBase-1.4 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.203 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.203 | Service for executing pieces of a query. | | pig-client | 0.17.0 | Pig command-line client. | | r | 3.4.1 | The R Project for Statistical Computing | | spark-client | 2.3.1 | Spark command-line clients. | | spark-history-server | 2.3.1 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 2.3.1 | In-memory execution engine for YARN. | | spark-yarn-slave | 2.3.1 | Apache Spark libraries needed by YARN slaves. | | sqoop-client | 1.4.7 | Apache Sqoop command-line client. | | tez-on-yarn | 0.8.4 | The tez YARN application and libraries. | | webserver | 2.4.25\$1 | Apache HTTP server. | | zeppelin-server | 0.7.3 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.4.12 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.4.12 | ZooKeeper command line client. | ## 5.16.1 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). **emr-5.16.1 classifications** | Classifications | Description | | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | | container-log4j | Change values in Hadoop YARN's container-log4j.properties file. | | core-site | Change values in Hadoop's core-site.xml file. | | emrfs-site | Change EMRFS settings. | | flink-conf | Change flink-conf.yaml settings. | | flink-log4j | Change Flink log4j.properties settings. | | flink-log4j-yarn-session | Change Flink log4j-yarn-session.properties settings. | | flink-log4j-cli | Change Flink log4j-cli.properties settings. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | | hadoop-ssl-server | Change hadoop ssl server configuration | | hadoop-ssl-client | Change hadoop ssl client configuration | | hbase | Amazon EMR-curated settings for Apache HBase. | | hbase-env | Change values in HBase's environment. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbase.properties file. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | | hbase-site | Change values in HBase's hbase-site.xml file. | | hdfs-encryption-zones | Configure HDFS encryption zones. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | | hcatalog-env | Change values in HCatalog's environment. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | | hcatalog-webhcat-log4j2 | Change values in HCatalog WebHCat's log4j2.properties. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | | hive-beeline-log4j2 | Change values in Hive's beeline-log4j2.properties file. | | hive-parquet-logging | Change values in Hive's parquet-logging.properties file. | | hive-env | Change values in the Hive environment. | | hive-exec-log4j2 | Change values in Hive's hive-exec-log4j2.properties file. | | hive-llap-daemon-log4j2 | Change values in Hive's llap-daemon-log4j2.properties file. | | hive-log4j2 | Change values in Hive's hive-log4j2.properties file. | | hive-site | Change values in Hive's hive-site.xml file | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | | hue-ini | Change values in Hue's ini file | | httpfs-env | Change values in the HTTPFS environment. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | | jupyter-notebook-conf | Change values in Jupyter Notebook's jupyter\$1notebook\$1config.py file. | | jupyter-hub-conf | Change values in JupyterHubs's jupyterhub\$1config.py file. | | jupyter-sparkmagic-conf | Change values in Sparkmagic's config.json file. | | livy-conf | Change values in Livy's livy.conf file. | | livy-env | Change values in the Livy environment. | | livy-log4j | Change Livy log4j.properties settings. | | mapred-env | Change values in the MapReduce application's environment. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | | oozie-env | Change values in Oozie's environment. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | | oozie-site | Change values in Oozie's oozie-site.xml file. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | | pig-env | Change values in the Pig environment. | | pig-properties | Change values in Pig's pig.properties file. | | pig-log4j | Change values in Pig's log4j.properties file. | | presto-log | Change values in Presto's log.properties file. | | presto-config | Change values in Presto's config.properties file. | | presto-password-authenticator | Change values in Presto's password-authenticator.properties file. | | presto-env | Change values in Presto's presto-env.sh file. | | presto-node | Change values in Presto's node.properties file. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | | presto-connector-hive | Change values in Presto's hive.properties file. | | presto-connector-jmx | Change values in Presto's jmx.properties file. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | | presto-connector-localfile | Change values in Presto's localfile.properties file. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | | presto-connector-redis | Change values in Presto's redis.properties file. | | presto-connector-redshift | Change values in Presto's redshift.properties file. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | | spark | Amazon EMR-curated settings for Apache Spark. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | | spark-env | Change values in the Spark environment. | | spark-hive-site | Change values in Spark's hive-site.xml file | | spark-log4j | Change values in Spark's log4j.properties file. | | spark-metrics | Change values in Spark's metrics.properties file. | | sqoop-env | Change values in Sqoop's environment. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | | tez-site | Change values in Tez's tez-site.xml file. | | yarn-env | Change values in the YARN environment. | | yarn-site | Change values in YARN's yarn-site.xml file. | | zeppelin-env | Change values in the Zeppelin environment. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | # Amazon EMR release 5.16.0 ## 5.16.0 application versions This release includes the following applications: [https://flink.apache.org/](https://flink.apache.org/), [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [http://gethue.com/](http://gethue.com/), [https://jupyterhub.readthedocs.io/en/latest/#](https://jupyterhub.readthedocs.io/en/latest/#), [https://livy.incubator.apache.org/](https://livy.incubator.apache.org/), [https://mxnet.incubator.apache.org/](https://mxnet.incubator.apache.org/), [http://mahout.apache.org/](http://mahout.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-5.16.0 | emr-5.15.1 | emr-5.15.0 | emr-5.14.2 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.11.336 | 1.11.333 | 1.11.333 | 1.11.297 | | Python | 2.7, 3.4 | 2.7, 3.4 | 2.7, 3.4 | 2.7, 3.4 | | Scala | 2.11.8 | 2.11.8 | 2.11.8 | 2.11.8 | | AmazonCloudWatchAgent | - | - | - | - | | Delta | - | - | - | - | | Flink | 1.5.0 | 1.4.2 | 1.4.2 | 1.4.2 | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 1.4.4 | 1.4.4 | 1.4.4 | 1.4.2 | | HCatalog | 2.3.3-amzn-1 | 2.3.3-amzn-0 | 2.3.3-amzn-0 | 2.3.2-amzn-2 | | Hadoop | 2.8.4-amzn-0 | 2.8.3-amzn-0 | 2.8.3-amzn-0 | 2.8.3-amzn-0 | | Hive | 2.3.3-amzn-1 | 2.3.3-amzn-0 | 2.3.3-amzn-0 | 2.3.2-amzn-2 | | Hudi | - | - | - | - | | Hue | 4.2.0 | 4.2.0 | 4.2.0 | 4.1.0 | | Iceberg | - | - | - | - | | JupyterEnterpriseGateway | - | - | - | - | | JupyterHub | 0.8.1 | 0.8.1 | 0.8.1 | 0.8.1 | | Livy | 0.5.0-incubating | 0.4.0-incubating | 0.4.0-incubating | 0.4.0-incubating | | MXNet | 1.2.0 | 1.1.0 | 1.1.0 | 1.1.0 | | Mahout | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | | Oozie | 5.0.0 | 5.0.0 | 5.0.0 | 4.3.0 | | Phoenix | 4.14.0-HBase-1.4 | 4.13.0-HBase-1.4 | 4.13.0-HBase-1.4 | 4.13.0-HBase-1.4 | | Pig | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | | Presto | 0.203 | 0.194 | 0.194 | 0.194 | | Spark | 2.3.1 | 2.3.0 | 2.3.0 | 2.3.0 | | Sqoop | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | | TensorFlow | - | - | - | - | | Tez | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | | Trino (PrestoSQL) | - | - | - | - | | Zeppelin | 0.7.3 | 0.7.3 | 0.7.3 | 0.7.3 | | ZooKeeper | 3.4.12 | 3.4.12 | 3.4.12 | 3.4.10 | ## 5.16.0 release notes The following release notes include information for Amazon EMR release 5.16.0. Changes are relative to 5.15.0. Initial release date: July 19, 2018 **Upgrades** + Hadoop 2.8.4 + Flink 1.5.0 + Livy 0.5.0 + MXNet 1.2.0 + Phoenix 4.14.0 + Presto 0.203 + Spark 2.3.1 + AWS SDK for Java 1.11.336 + CUDA 9.2 + Redshift JDBC Driver 1.2.15.1025 **Changes, enhancements, and resolved issues** + HBase + Backported [HBASE-20723](https://issues.apache.org/jira/browse/HBASE-20723) + Presto + Configuration changes to support LDAP authentication. For more information, see [Using LDAP authentication for Presto on Amazon EMR](emr-presto-ldap.md). + Spark + Apache Spark version 2.3.1, available beginning with Amazon EMR release 5.16.0, addresses [CVE-2018-8024](https://nvd.nist.gov/vuln/detail/CVE-2018-8024) and [CVE-2018-1334](https://nvd.nist.gov/vuln/detail/CVE-2018-1334). We recommend that you migrate earlier versions of Spark to Spark version 2.3.1 or later. **Known issues** + This release version does not support the c1.medium or m1.small instance types. Clusters using either of these instance types fail to start. As a workaround, specify a different instance type or use a different release version. + When you create a kerberized cluster with Livy installed, Livy fails with an error that simple authentication is not enabled. Rebooting the Livy server resolves the issue. As a workaround, add a step during cluster creation that runs `sudo restart livy-server` on the primary node. + After the primary node reboots or the instance controller restarts, the CloudWatch metrics will not be collected and the automatic scaling feature will not be available in Amazon EMR version 5.14.0, 5.15.0, or 5.16.0. This issue is fixed in Amazon EMR 5.17.0. ## 5.16.0 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | aws-sagemaker-spark-sdk | 1.1.0 | Amazon SageMaker Spark SDK | | emr-ddb | 4.6.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 2.4.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.4.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-s3-dist-cp | 2.10.0 | Distributed copy application optimized for Amazon S3. | | emrfs | 2.25.0 | Amazon S3 connector for Hadoop ecosystem applications. | | flink-client | 1.5.0 | Apache Flink command line client scripts and applications. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 2.8.4-amzn-0 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 2.8.4-amzn-0 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 2.8.4-amzn-0 | HDFS command-line client and library | | hadoop-hdfs-namenode | 2.8.4-amzn-0 | HDFS service for tracking file names and block locations. | | hadoop-httpfs-server | 2.8.4-amzn-0 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 2.8.4-amzn-0 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 2.8.4-amzn-0 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 2.8.4-amzn-0 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 2.8.4-amzn-0 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 2.8.4-amzn-0 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 1.4.4 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 1.4.4 | Service for serving one or more HBase regions. | | hbase-client | 1.4.4 | HBase command-line client. | | hbase-rest-server | 1.4.4 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 1.4.4 | Service providing a Thrift endpoint to HBase. | | hcatalog-client | 2.3.3-amzn-1 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 2.3.3-amzn-1 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 2.3.3-amzn-1 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 2.3.3-amzn-1 | Hive command line client. | | hive-hbase | 2.3.3-amzn-1 | Hive-hbase client. | | hive-metastore-server | 2.3.3-amzn-1 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server2 | 2.3.3-amzn-1 | Service for accepting Hive queries as web requests. | | hue-server | 4.2.0 | Web application for analyzing data using Hadoop ecosystem applications | | jupyterhub | 0.8.1 | Multi-user server for Jupyter notebooks | | livy-server | 0.5.0-incubating | REST interface for interacting with Apache Spark | | mahout-client | 0.13.0 | Library for machine learning. | | mxnet | 1.2.0 | A flexible, scalable, and efficient library for deep learning. | | mysql-server | 5.5.54\$1 | MySQL database server. | | nvidia-cuda | 9.2.88 | Nvidia drivers and Cuda toolkit | | oozie-client | 5.0.0 | Oozie command-line client. | | oozie-server | 5.0.0 | Service for accepting Oozie workflow requests. | | opencv | 3.4.0 | Open Source Computer Vision Library. | | phoenix-library | 4.14.0-HBase-1.4 | The phoenix libraries for server and client | | phoenix-query-server | 4.14.0-HBase-1.4 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.203 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.203 | Service for executing pieces of a query. | | pig-client | 0.17.0 | Pig command-line client. | | r | 3.4.1 | The R Project for Statistical Computing | | spark-client | 2.3.1 | Spark command-line clients. | | spark-history-server | 2.3.1 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 2.3.1 | In-memory execution engine for YARN. | | spark-yarn-slave | 2.3.1 | Apache Spark libraries needed by YARN slaves. | | sqoop-client | 1.4.7 | Apache Sqoop command-line client. | | tez-on-yarn | 0.8.4 | The tez YARN application and libraries. | | webserver | 2.4.25\$1 | Apache HTTP server. | | zeppelin-server | 0.7.3 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.4.12 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.4.12 | ZooKeeper command line client. | ## 5.16.0 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). **emr-5.16.0 classifications** | Classifications | Description | | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | | container-log4j | Change values in Hadoop YARN's container-log4j.properties file. | | core-site | Change values in Hadoop's core-site.xml file. | | emrfs-site | Change EMRFS settings. | | flink-conf | Change flink-conf.yaml settings. | | flink-log4j | Change Flink log4j.properties settings. | | flink-log4j-yarn-session | Change Flink log4j-yarn-session.properties settings. | | flink-log4j-cli | Change Flink log4j-cli.properties settings. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | | hadoop-ssl-server | Change hadoop ssl server configuration | | hadoop-ssl-client | Change hadoop ssl client configuration | | hbase | Amazon EMR-curated settings for Apache HBase. | | hbase-env | Change values in HBase's environment. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbase.properties file. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | | hbase-site | Change values in HBase's hbase-site.xml file. | | hdfs-encryption-zones | Configure HDFS encryption zones. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | | hcatalog-env | Change values in HCatalog's environment. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | | hcatalog-webhcat-log4j2 | Change values in HCatalog WebHCat's log4j2.properties. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | | hive-beeline-log4j2 | Change values in Hive's beeline-log4j2.properties file. | | hive-parquet-logging | Change values in Hive's parquet-logging.properties file. | | hive-env | Change values in the Hive environment. | | hive-exec-log4j2 | Change values in Hive's hive-exec-log4j2.properties file. | | hive-llap-daemon-log4j2 | Change values in Hive's llap-daemon-log4j2.properties file. | | hive-log4j2 | Change values in Hive's hive-log4j2.properties file. | | hive-site | Change values in Hive's hive-site.xml file | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | | hue-ini | Change values in Hue's ini file | | httpfs-env | Change values in the HTTPFS environment. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | | jupyter-notebook-conf | Change values in Jupyter Notebook's jupyter\$1notebook\$1config.py file. | | jupyter-hub-conf | Change values in JupyterHubs's jupyterhub\$1config.py file. | | jupyter-sparkmagic-conf | Change values in Sparkmagic's config.json file. | | livy-conf | Change values in Livy's livy.conf file. | | livy-env | Change values in the Livy environment. | | livy-log4j | Change Livy log4j.properties settings. | | mapred-env | Change values in the MapReduce application's environment. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | | oozie-env | Change values in Oozie's environment. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | | oozie-site | Change values in Oozie's oozie-site.xml file. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | | pig-env | Change values in the Pig environment. | | pig-properties | Change values in Pig's pig.properties file. | | pig-log4j | Change values in Pig's log4j.properties file. | | presto-log | Change values in Presto's log.properties file. | | presto-config | Change values in Presto's config.properties file. | | presto-password-authenticator | Change values in Presto's password-authenticator.properties file. | | presto-env | Change values in Presto's presto-env.sh file. | | presto-node | Change values in Presto's node.properties file. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | | presto-connector-hive | Change values in Presto's hive.properties file. | | presto-connector-jmx | Change values in Presto's jmx.properties file. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | | presto-connector-localfile | Change values in Presto's localfile.properties file. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | | presto-connector-redis | Change values in Presto's redis.properties file. | | presto-connector-redshift | Change values in Presto's redshift.properties file. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | | spark | Amazon EMR-curated settings for Apache Spark. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | | spark-env | Change values in the Spark environment. | | spark-hive-site | Change values in Spark's hive-site.xml file | | spark-log4j | Change values in Spark's log4j.properties file. | | spark-metrics | Change values in Spark's metrics.properties file. | | sqoop-env | Change values in Sqoop's environment. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | | tez-site | Change values in Tez's tez-site.xml file. | | yarn-env | Change values in the YARN environment. | | yarn-site | Change values in YARN's yarn-site.xml file. | | zeppelin-env | Change values in the Zeppelin environment. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | # Amazon EMR release 5.15.1 ## 5.15.1 application versions This release includes the following applications: [https://flink.apache.org/](https://flink.apache.org/), [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [http://gethue.com/](http://gethue.com/), [https://jupyterhub.readthedocs.io/en/latest/#](https://jupyterhub.readthedocs.io/en/latest/#), [https://livy.incubator.apache.org/](https://livy.incubator.apache.org/), [https://mxnet.incubator.apache.org/](https://mxnet.incubator.apache.org/), [http://mahout.apache.org/](http://mahout.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-5.15.1 | emr-5.15.0 | emr-5.14.2 | emr-5.14.1 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.11.333 | 1.11.333 | 1.11.297 | 1.11.297 | | Python | 2.7, 3.4 | 2.7, 3.4 | 2.7, 3.4 | 2.7, 3.4 | | Scala | 2.11.8 | 2.11.8 | 2.11.8 | 2.11.8 | | AmazonCloudWatchAgent | - | - | - | - | | Delta | - | - | - | - | | Flink | 1.4.2 | 1.4.2 | 1.4.2 | 1.4.2 | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 1.4.4 | 1.4.4 | 1.4.2 | 1.4.2 | | HCatalog | 2.3.3-amzn-0 | 2.3.3-amzn-0 | 2.3.2-amzn-2 | 2.3.2-amzn-2 | | Hadoop | 2.8.3-amzn-0 | 2.8.3-amzn-0 | 2.8.3-amzn-0 | 2.8.3-amzn-0 | | Hive | 2.3.3-amzn-0 | 2.3.3-amzn-0 | 2.3.2-amzn-2 | 2.3.2-amzn-2 | | Hudi | - | - | - | - | | Hue | 4.2.0 | 4.2.0 | 4.1.0 | 4.1.0 | | Iceberg | - | - | - | - | | JupyterEnterpriseGateway | - | - | - | - | | JupyterHub | 0.8.1 | 0.8.1 | 0.8.1 | 0.8.1 | | Livy | 0.4.0-incubating | 0.4.0-incubating | 0.4.0-incubating | 0.4.0-incubating | | MXNet | 1.1.0 | 1.1.0 | 1.1.0 | 1.1.0 | | Mahout | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | | Oozie | 5.0.0 | 5.0.0 | 4.3.0 | 4.3.0 | | Phoenix | 4.13.0-HBase-1.4 | 4.13.0-HBase-1.4 | 4.13.0-HBase-1.4 | 4.13.0-HBase-1.4 | | Pig | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | | Presto | 0.194 | 0.194 | 0.194 | 0.194 | | Spark | 2.3.0 | 2.3.0 | 2.3.0 | 2.3.0 | | Sqoop | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | | TensorFlow | - | - | - | - | | Tez | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | | Trino (PrestoSQL) | - | - | - | - | | Zeppelin | 0.7.3 | 0.7.3 | 0.7.3 | 0.7.3 | | ZooKeeper | 3.4.12 | 3.4.12 | 3.4.10 | 3.4.10 | ## 5.15.1 release notes This is a patch release to add AWS Signature Version 4 authentication for requests to Amazon S3. All applications and components are the same as the previous Amazon EMR release. **Important** In this release version, Amazon EMR uses AWS Signature Version 4 exclusively to authenticate requests to Amazon S3. For more information, see [Whats New](https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-whatsnew.html). ## 5.15.1 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | aws-sagemaker-spark-sdk | 1.0.1 | Amazon SageMaker Spark SDK | | emr-ddb | 4.5.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 2.4.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.4.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-s3-dist-cp | 2.10.0 | Distributed copy application optimized for Amazon S3. | | emrfs | 2.24.0 | Amazon S3 connector for Hadoop ecosystem applications. | | flink-client | 1.4.2 | Apache Flink command line client scripts and applications. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 2.8.3-amzn-1 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 2.8.3-amzn-1 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 2.8.3-amzn-1 | HDFS command-line client and library | | hadoop-hdfs-namenode | 2.8.3-amzn-1 | HDFS service for tracking file names and block locations. | | hadoop-httpfs-server | 2.8.3-amzn-1 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 2.8.3-amzn-1 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 2.8.3-amzn-1 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 2.8.3-amzn-1 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 2.8.3-amzn-1 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 2.8.3-amzn-1 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 1.4.4 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 1.4.4 | Service for serving one or more HBase regions. | | hbase-client | 1.4.4 | HBase command-line client. | | hbase-rest-server | 1.4.4 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 1.4.4 | Service providing a Thrift endpoint to HBase. | | hcatalog-client | 2.3.3-amzn-0 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 2.3.3-amzn-0 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 2.3.3-amzn-0 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 2.3.3-amzn-0 | Hive command line client. | | hive-hbase | 2.3.3-amzn-0 | Hive-hbase client. | | hive-metastore-server | 2.3.3-amzn-0 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server2 | 2.3.3-amzn-0 | Service for accepting Hive queries as web requests. | | hue-server | 4.2.0 | Web application for analyzing data using Hadoop ecosystem applications | | jupyterhub | 0.8.1 | Multi-user server for Jupyter notebooks | | livy-server | 0.4.0-incubating | REST interface for interacting with Apache Spark | | mahout-client | 0.13.0 | Library for machine learning. | | mxnet | 1.1.0 | A flexible, scalable, and efficient library for deep learning. | | mysql-server | 5.5.54\$1 | MySQL database server. | | nvidia-cuda | 9.1.85 | Nvidia drivers and Cuda toolkit | | oozie-client | 5.0.0 | Oozie command-line client. | | oozie-server | 5.0.0 | Service for accepting Oozie workflow requests. | | opencv | 3.4.0 | Open Source Computer Vision Library. | | phoenix-library | 4.13.0-HBase-1.4 | The phoenix libraries for server and client | | phoenix-query-server | 4.13.0-HBase-1.4 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.194 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.194 | Service for executing pieces of a query. | | pig-client | 0.17.0 | Pig command-line client. | | r | 3.4.1 | The R Project for Statistical Computing | | spark-client | 2.3.0 | Spark command-line clients. | | spark-history-server | 2.3.0 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 2.3.0 | In-memory execution engine for YARN. | | spark-yarn-slave | 2.3.0 | Apache Spark libraries needed by YARN slaves. | | sqoop-client | 1.4.7 | Apache Sqoop command-line client. | | tez-on-yarn | 0.8.4 | The tez YARN application and libraries. | | webserver | 2.4.25\$1 | Apache HTTP server. | | zeppelin-server | 0.7.3 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.4.12 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.4.12 | ZooKeeper command line client. | ## 5.15.1 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). **emr-5.15.1 classifications** | Classifications | Description | | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | | container-log4j | Change values in Hadoop YARN's container-log4j.properties file. | | core-site | Change values in Hadoop's core-site.xml file. | | emrfs-site | Change EMRFS settings. | | flink-conf | Change flink-conf.yaml settings. | | flink-log4j | Change Flink log4j.properties settings. | | flink-log4j-yarn-session | Change Flink log4j-yarn-session.properties settings. | | flink-log4j-cli | Change Flink log4j-cli.properties settings. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | | hadoop-ssl-server | Change hadoop ssl server configuration | | hadoop-ssl-client | Change hadoop ssl client configuration | | hbase | Amazon EMR-curated settings for Apache HBase. | | hbase-env | Change values in HBase's environment. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbase.properties file. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | | hbase-site | Change values in HBase's hbase-site.xml file. | | hdfs-encryption-zones | Configure HDFS encryption zones. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | | hcatalog-env | Change values in HCatalog's environment. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | | hcatalog-webhcat-log4j2 | Change values in HCatalog WebHCat's log4j2.properties. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | | hive-beeline-log4j2 | Change values in Hive's beeline-log4j2.properties file. | | hive-parquet-logging | Change values in Hive's parquet-logging.properties file. | | hive-env | Change values in the Hive environment. | | hive-exec-log4j2 | Change values in Hive's hive-exec-log4j2.properties file. | | hive-llap-daemon-log4j2 | Change values in Hive's llap-daemon-log4j2.properties file. | | hive-log4j2 | Change values in Hive's hive-log4j2.properties file. | | hive-site | Change values in Hive's hive-site.xml file | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | | hue-ini | Change values in Hue's ini file | | httpfs-env | Change values in the HTTPFS environment. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | | jupyter-notebook-conf | Change values in Jupyter Notebook's jupyter\$1notebook\$1config.py file. | | jupyter-hub-conf | Change values in JupyterHubs's jupyterhub\$1config.py file. | | jupyter-sparkmagic-conf | Change values in Sparkmagic's config.json file. | | livy-conf | Change values in Livy's livy.conf file. | | livy-env | Change values in the Livy environment. | | livy-log4j | Change Livy log4j.properties settings. | | mapred-env | Change values in the MapReduce application's environment. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | | oozie-env | Change values in Oozie's environment. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | | oozie-site | Change values in Oozie's oozie-site.xml file. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | | pig-env | Change values in the Pig environment. | | pig-properties | Change values in Pig's pig.properties file. | | pig-log4j | Change values in Pig's log4j.properties file. | | presto-log | Change values in Presto's log.properties file. | | presto-config | Change values in Presto's config.properties file. | | presto-env | Change values in Presto's presto-env.sh file. | | presto-node | Change values in Presto's node.properties file. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | | presto-connector-hive | Change values in Presto's hive.properties file. | | presto-connector-jmx | Change values in Presto's jmx.properties file. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | | presto-connector-localfile | Change values in Presto's localfile.properties file. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | | presto-connector-redis | Change values in Presto's redis.properties file. | | presto-connector-redshift | Change values in Presto's redshift.properties file. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | | spark | Amazon EMR-curated settings for Apache Spark. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | | spark-env | Change values in the Spark environment. | | spark-hive-site | Change values in Spark's hive-site.xml file | | spark-log4j | Change values in Spark's log4j.properties file. | | spark-metrics | Change values in Spark's metrics.properties file. | | sqoop-env | Change values in Sqoop's environment. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | | tez-site | Change values in Tez's tez-site.xml file. | | yarn-env | Change values in the YARN environment. | | yarn-site | Change values in YARN's yarn-site.xml file. | | zeppelin-env | Change values in the Zeppelin environment. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | # Amazon EMR release 5.15.0 ## 5.15.0 application versions This release includes the following applications: [https://flink.apache.org/](https://flink.apache.org/), [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [http://gethue.com/](http://gethue.com/), [https://jupyterhub.readthedocs.io/en/latest/#](https://jupyterhub.readthedocs.io/en/latest/#), [https://livy.incubator.apache.org/](https://livy.incubator.apache.org/), [https://mxnet.incubator.apache.org/](https://mxnet.incubator.apache.org/), [http://mahout.apache.org/](http://mahout.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-5.15.0 | emr-5.14.2 | emr-5.14.1 | emr-5.14.0 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.11.333 | 1.11.297 | 1.11.297 | 1.11.297 | | Python | 2.7, 3.4 | 2.7, 3.4 | 2.7, 3.4 | 2.7, 3.4 | | Scala | 2.11.8 | 2.11.8 | 2.11.8 | 2.11.8 | | AmazonCloudWatchAgent | - | - | - | - | | Delta | - | - | - | - | | Flink | 1.4.2 | 1.4.2 | 1.4.2 | 1.4.2 | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 1.4.4 | 1.4.2 | 1.4.2 | 1.4.2 | | HCatalog | 2.3.3-amzn-0 | 2.3.2-amzn-2 | 2.3.2-amzn-2 | 2.3.2-amzn-2 | | Hadoop | 2.8.3-amzn-0 | 2.8.3-amzn-0 | 2.8.3-amzn-0 | 2.8.3-amzn-0 | | Hive | 2.3.3-amzn-0 | 2.3.2-amzn-2 | 2.3.2-amzn-2 | 2.3.2-amzn-2 | | Hudi | - | - | - | - | | Hue | 4.2.0 | 4.1.0 | 4.1.0 | 4.1.0 | | Iceberg | - | - | - | - | | JupyterEnterpriseGateway | - | - | - | - | | JupyterHub | 0.8.1 | 0.8.1 | 0.8.1 | 0.8.1 | | Livy | 0.4.0-incubating | 0.4.0-incubating | 0.4.0-incubating | 0.4.0-incubating | | MXNet | 1.1.0 | 1.1.0 | 1.1.0 | 1.1.0 | | Mahout | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | | Oozie | 5.0.0 | 4.3.0 | 4.3.0 | 4.3.0 | | Phoenix | 4.13.0-HBase-1.4 | 4.13.0-HBase-1.4 | 4.13.0-HBase-1.4 | 4.13.0-HBase-1.4 | | Pig | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | | Presto | 0.194 | 0.194 | 0.194 | 0.194 | | Spark | 2.3.0 | 2.3.0 | 2.3.0 | 2.3.0 | | Sqoop | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.7 | | TensorFlow | - | - | - | - | | Tez | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | | Trino (PrestoSQL) | - | - | - | - | | Zeppelin | 0.7.3 | 0.7.3 | 0.7.3 | 0.7.3 | | ZooKeeper | 3.4.12 | 3.4.10 | 3.4.10 | 3.4.10 | ## 5.15.0 release notes The following release notes include information for Amazon EMR release 5.15.0. Changes are relative to 5.14.0. Initial release date: June 21, 2018 **Upgrades** + Upgraded HBase to 1.4.4 + Upgraded Hive to 2.3.3 + Upgraded Hue to 4.2.0 + Upgraded Oozie to 5.0.0 + Upgraded Zookeeper to 3.4.12 + Upgraded AWS SDK to 1.11.333 **Changes, enhancements, and resolved issues** + Hive + Backported [HIVE-18069](https://issues.apache.org/jira/browse/HIVE-18069) + Hue + Updated Hue to correctly authenticate with Livy when Kerberos is enabled. Livy is now supported when using Kerberos with Amazon EMR. + JupyterHub + Updated JupyterHub so that Amazon EMR installs LDAP client libraries by default. + Fixed an error in the script that generates self-signed certificates. **Known issues** + This release version does not support the c1.medium or m1.small instance types. Clusters using either of these instance types fail to start. As a workaround, specify a different instance type or use a different release version. + After the primary node reboots or the instance controller restarts, the CloudWatch metrics will not be collected and the automatic scaling feature will not be available in Amazon EMR version 5.14.0, 5.15.0, or 5.16.0. This issue is fixed in Amazon EMR 5.17.0. ## 5.15.0 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | aws-sagemaker-spark-sdk | 1.0.1 | Amazon SageMaker Spark SDK | | emr-ddb | 4.5.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 2.4.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.4.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-s3-dist-cp | 2.10.0 | Distributed copy application optimized for Amazon S3. | | emrfs | 2.24.0 | Amazon S3 connector for Hadoop ecosystem applications. | | flink-client | 1.4.2 | Apache Flink command line client scripts and applications. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 2.8.3-amzn-1 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 2.8.3-amzn-1 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 2.8.3-amzn-1 | HDFS command-line client and library | | hadoop-hdfs-namenode | 2.8.3-amzn-1 | HDFS service for tracking file names and block locations. | | hadoop-httpfs-server | 2.8.3-amzn-1 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 2.8.3-amzn-1 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 2.8.3-amzn-1 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 2.8.3-amzn-1 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 2.8.3-amzn-1 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 2.8.3-amzn-1 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 1.4.4 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 1.4.4 | Service for serving one or more HBase regions. | | hbase-client | 1.4.4 | HBase command-line client. | | hbase-rest-server | 1.4.4 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 1.4.4 | Service providing a Thrift endpoint to HBase. | | hcatalog-client | 2.3.3-amzn-0 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 2.3.3-amzn-0 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 2.3.3-amzn-0 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 2.3.3-amzn-0 | Hive command line client. | | hive-hbase | 2.3.3-amzn-0 | Hive-hbase client. | | hive-metastore-server | 2.3.3-amzn-0 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server2 | 2.3.3-amzn-0 | Service for accepting Hive queries as web requests. | | hue-server | 4.2.0 | Web application for analyzing data using Hadoop ecosystem applications | | jupyterhub | 0.8.1 | Multi-user server for Jupyter notebooks | | livy-server | 0.4.0-incubating | REST interface for interacting with Apache Spark | | mahout-client | 0.13.0 | Library for machine learning. | | mxnet | 1.1.0 | A flexible, scalable, and efficient library for deep learning. | | mysql-server | 5.5.54\$1 | MySQL database server. | | nvidia-cuda | 9.1.85 | Nvidia drivers and Cuda toolkit | | oozie-client | 5.0.0 | Oozie command-line client. | | oozie-server | 5.0.0 | Service for accepting Oozie workflow requests. | | opencv | 3.4.0 | Open Source Computer Vision Library. | | phoenix-library | 4.13.0-HBase-1.4 | The phoenix libraries for server and client | | phoenix-query-server | 4.13.0-HBase-1.4 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.194 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.194 | Service for executing pieces of a query. | | pig-client | 0.17.0 | Pig command-line client. | | r | 3.4.1 | The R Project for Statistical Computing | | spark-client | 2.3.0 | Spark command-line clients. | | spark-history-server | 2.3.0 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 2.3.0 | In-memory execution engine for YARN. | | spark-yarn-slave | 2.3.0 | Apache Spark libraries needed by YARN slaves. | | sqoop-client | 1.4.7 | Apache Sqoop command-line client. | | tez-on-yarn | 0.8.4 | The tez YARN application and libraries. | | webserver | 2.4.25\$1 | Apache HTTP server. | | zeppelin-server | 0.7.3 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.4.12 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.4.12 | ZooKeeper command line client. | ## 5.15.0 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). **emr-5.15.0 classifications** | Classifications | Description | | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | | container-log4j | Change values in Hadoop YARN's container-log4j.properties file. | | core-site | Change values in Hadoop's core-site.xml file. | | emrfs-site | Change EMRFS settings. | | flink-conf | Change flink-conf.yaml settings. | | flink-log4j | Change Flink log4j.properties settings. | | flink-log4j-yarn-session | Change Flink log4j-yarn-session.properties settings. | | flink-log4j-cli | Change Flink log4j-cli.properties settings. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | | hadoop-ssl-server | Change hadoop ssl server configuration | | hadoop-ssl-client | Change hadoop ssl client configuration | | hbase | Amazon EMR-curated settings for Apache HBase. | | hbase-env | Change values in HBase's environment. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbase.properties file. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | | hbase-site | Change values in HBase's hbase-site.xml file. | | hdfs-encryption-zones | Configure HDFS encryption zones. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | | hcatalog-env | Change values in HCatalog's environment. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | | hcatalog-webhcat-log4j2 | Change values in HCatalog WebHCat's log4j2.properties. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | | hive-beeline-log4j2 | Change values in Hive's beeline-log4j2.properties file. | | hive-parquet-logging | Change values in Hive's parquet-logging.properties file. | | hive-env | Change values in the Hive environment. | | hive-exec-log4j2 | Change values in Hive's hive-exec-log4j2.properties file. | | hive-llap-daemon-log4j2 | Change values in Hive's llap-daemon-log4j2.properties file. | | hive-log4j2 | Change values in Hive's hive-log4j2.properties file. | | hive-site | Change values in Hive's hive-site.xml file | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | | hue-ini | Change values in Hue's ini file | | httpfs-env | Change values in the HTTPFS environment. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | | jupyter-notebook-conf | Change values in Jupyter Notebook's jupyter\$1notebook\$1config.py file. | | jupyter-hub-conf | Change values in JupyterHubs's jupyterhub\$1config.py file. | | jupyter-sparkmagic-conf | Change values in Sparkmagic's config.json file. | | livy-conf | Change values in Livy's livy.conf file. | | livy-env | Change values in the Livy environment. | | livy-log4j | Change Livy log4j.properties settings. | | mapred-env | Change values in the MapReduce application's environment. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | | oozie-env | Change values in Oozie's environment. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | | oozie-site | Change values in Oozie's oozie-site.xml file. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | | pig-env | Change values in the Pig environment. | | pig-properties | Change values in Pig's pig.properties file. | | pig-log4j | Change values in Pig's log4j.properties file. | | presto-log | Change values in Presto's log.properties file. | | presto-config | Change values in Presto's config.properties file. | | presto-env | Change values in Presto's presto-env.sh file. | | presto-node | Change values in Presto's node.properties file. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | | presto-connector-hive | Change values in Presto's hive.properties file. | | presto-connector-jmx | Change values in Presto's jmx.properties file. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | | presto-connector-localfile | Change values in Presto's localfile.properties file. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | | presto-connector-redis | Change values in Presto's redis.properties file. | | presto-connector-redshift | Change values in Presto's redshift.properties file. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | | spark | Amazon EMR-curated settings for Apache Spark. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | | spark-env | Change values in the Spark environment. | | spark-hive-site | Change values in Spark's hive-site.xml file | | spark-log4j | Change values in Spark's log4j.properties file. | | spark-metrics | Change values in Spark's metrics.properties file. | | sqoop-env | Change values in Sqoop's environment. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | | tez-site | Change values in Tez's tez-site.xml file. | | yarn-env | Change values in the YARN environment. | | yarn-site | Change values in YARN's yarn-site.xml file. | | zeppelin-env | Change values in the Zeppelin environment. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | # Amazon EMR release 5.14.2 ## 5.14.2 application versions This release includes the following applications: [https://flink.apache.org/](https://flink.apache.org/), [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [http://gethue.com/](http://gethue.com/), [https://jupyterhub.readthedocs.io/en/latest/#](https://jupyterhub.readthedocs.io/en/latest/#), [https://livy.incubator.apache.org/](https://livy.incubator.apache.org/), [https://mxnet.incubator.apache.org/](https://mxnet.incubator.apache.org/), [http://mahout.apache.org/](http://mahout.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-5.14.2 | emr-5.14.1 | emr-5.14.0 | emr-5.13.1 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.11.297 | 1.11.297 | 1.11.297 | 1.11.297 | | Python | 2.7, 3.4 | 2.7, 3.4 | 2.7, 3.4 | 2.7, 3.4 | | Scala | 2.11.8 | 2.11.8 | 2.11.8 | 2.11.8 | | AmazonCloudWatchAgent | - | - | - | - | | Delta | - | - | - | - | | Flink | 1.4.2 | 1.4.2 | 1.4.2 | 1.4.0 | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 1.4.2 | 1.4.2 | 1.4.2 | 1.4.2 | | HCatalog | 2.3.2-amzn-2 | 2.3.2-amzn-2 | 2.3.2-amzn-2 | 2.3.2-amzn-2 | | Hadoop | 2.8.3-amzn-0 | 2.8.3-amzn-0 | 2.8.3-amzn-0 | 2.8.3-amzn-0 | | Hive | 2.3.2-amzn-2 | 2.3.2-amzn-2 | 2.3.2-amzn-2 | 2.3.2-amzn-2 | | Hudi | - | - | - | - | | Hue | 4.1.0 | 4.1.0 | 4.1.0 | 4.1.0 | | Iceberg | - | - | - | - | | JupyterEnterpriseGateway | - | - | - | - | | JupyterHub | 0.8.1 | 0.8.1 | 0.8.1 | - | | Livy | 0.4.0-incubating | 0.4.0-incubating | 0.4.0-incubating | 0.4.0-incubating | | MXNet | 1.1.0 | 1.1.0 | 1.1.0 | 1.0.0 | | Mahout | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | | Oozie | 4.3.0 | 4.3.0 | 4.3.0 | 4.3.0 | | Phoenix | 4.13.0-HBase-1.4 | 4.13.0-HBase-1.4 | 4.13.0-HBase-1.4 | 4.13.0-HBase-1.4 | | Pig | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | | Presto | 0.194 | 0.194 | 0.194 | 0.194 | | Spark | 2.3.0 | 2.3.0 | 2.3.0 | 2.3.0 | | Sqoop | 1.4.7 | 1.4.7 | 1.4.7 | 1.4.6 | | TensorFlow | - | - | - | - | | Tez | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | | Trino (PrestoSQL) | - | - | - | - | | Zeppelin | 0.7.3 | 0.7.3 | 0.7.3 | 0.7.3 | | ZooKeeper | 3.4.10 | 3.4.10 | 3.4.10 | 3.4.10 | ## 5.14.2 release notes This is a patch release to add AWS Signature Version 4 authentication for requests to Amazon S3. All applications and components are the same as the previous Amazon EMR release. **Important** In this release version, Amazon EMR uses AWS Signature Version 4 exclusively to authenticate requests to Amazon S3. For more information, see [Whats New](https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-whatsnew.html). ## 5.14.2 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | aws-sagemaker-spark-sdk | 1.0.1 | Amazon SageMaker Spark SDK | | emr-ddb | 4.5.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 2.4.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.4.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-s3-dist-cp | 2.10.0 | Distributed copy application optimized for Amazon S3. | | emrfs | 2.23.0 | Amazon S3 connector for Hadoop ecosystem applications. | | flink-client | 1.4.2 | Apache Flink command line client scripts and applications. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 2.8.3-amzn-1 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 2.8.3-amzn-1 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 2.8.3-amzn-1 | HDFS command-line client and library | | hadoop-hdfs-namenode | 2.8.3-amzn-1 | HDFS service for tracking file names and block locations. | | hadoop-httpfs-server | 2.8.3-amzn-1 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 2.8.3-amzn-1 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 2.8.3-amzn-1 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 2.8.3-amzn-1 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 2.8.3-amzn-1 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 2.8.3-amzn-1 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 1.4.2 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 1.4.2 | Service for serving one or more HBase regions. | | hbase-client | 1.4.2 | HBase command-line client. | | hbase-rest-server | 1.4.2 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 1.4.2 | Service providing a Thrift endpoint to HBase. | | hcatalog-client | 2.3.2-amzn-2 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 2.3.2-amzn-2 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 2.3.2-amzn-2 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 2.3.2-amzn-2 | Hive command line client. | | hive-hbase | 2.3.2-amzn-2 | Hive-hbase client. | | hive-metastore-server | 2.3.2-amzn-2 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server2 | 2.3.2-amzn-2 | Service for accepting Hive queries as web requests. | | hue-server | 4.1.0 | Web application for analyzing data using Hadoop ecosystem applications | | jupyterhub | 0.8.1 | Multi-user server for Jupyter notebooks | | livy-server | 0.4.0-incubating | REST interface for interacting with Apache Spark | | mahout-client | 0.13.0 | Library for machine learning. | | mxnet | 1.1.0 | A flexible, scalable, and efficient library for deep learning. | | mysql-server | 5.5.54\$1 | MySQL database server. | | nvidia-cuda | 9.1.85 | Nvidia drivers and Cuda toolkit | | oozie-client | 4.3.0 | Oozie command-line client. | | oozie-server | 4.3.0 | Service for accepting Oozie workflow requests. | | opencv | 3.4.0 | Open Source Computer Vision Library. | | phoenix-library | 4.13.0-HBase-1.4 | The phoenix libraries for server and client | | phoenix-query-server | 4.13.0-HBase-1.4 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.194 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.194 | Service for executing pieces of a query. | | pig-client | 0.17.0 | Pig command-line client. | | r | 3.4.1 | The R Project for Statistical Computing | | spark-client | 2.3.0 | Spark command-line clients. | | spark-history-server | 2.3.0 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 2.3.0 | In-memory execution engine for YARN. | | spark-yarn-slave | 2.3.0 | Apache Spark libraries needed by YARN slaves. | | sqoop-client | 1.4.7 | Apache Sqoop command-line client. | | tez-on-yarn | 0.8.4 | The tez YARN application and libraries. | | webserver | 2.4.25\$1 | Apache HTTP server. | | zeppelin-server | 0.7.3 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.4.10 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.4.10 | ZooKeeper command line client. | ## 5.14.2 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). **emr-5.14.2 classifications** | Classifications | Description | | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | | container-log4j | Change values in Hadoop YARN's container-log4j.properties file. | | core-site | Change values in Hadoop's core-site.xml file. | | emrfs-site | Change EMRFS settings. | | flink-conf | Change flink-conf.yaml settings. | | flink-log4j | Change Flink log4j.properties settings. | | flink-log4j-yarn-session | Change Flink log4j-yarn-session.properties settings. | | flink-log4j-cli | Change Flink log4j-cli.properties settings. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | | hadoop-ssl-server | Change hadoop ssl server configuration | | hadoop-ssl-client | Change hadoop ssl client configuration | | hbase | Amazon EMR-curated settings for Apache HBase. | | hbase-env | Change values in HBase's environment. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbase.properties file. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | | hbase-site | Change values in HBase's hbase-site.xml file. | | hdfs-encryption-zones | Configure HDFS encryption zones. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | | hcatalog-env | Change values in HCatalog's environment. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | | hcatalog-webhcat-log4j2 | Change values in HCatalog WebHCat's log4j2.properties. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | | hive-beeline-log4j2 | Change values in Hive's beeline-log4j2.properties file. | | hive-parquet-logging | Change values in Hive's parquet-logging.properties file. | | hive-env | Change values in the Hive environment. | | hive-exec-log4j2 | Change values in Hive's hive-exec-log4j2.properties file. | | hive-llap-daemon-log4j2 | Change values in Hive's llap-daemon-log4j2.properties file. | | hive-log4j2 | Change values in Hive's hive-log4j2.properties file. | | hive-site | Change values in Hive's hive-site.xml file | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | | hue-ini | Change values in Hue's ini file | | httpfs-env | Change values in the HTTPFS environment. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | | jupyter-notebook-conf | Change values in Jupyter Notebook's jupyter\$1notebook\$1config.py file. | | jupyter-hub-conf | Change values in JupyterHubs's jupyterhub\$1config.py file. | | jupyter-sparkmagic-conf | Change values in Sparkmagic's config.json file. | | livy-conf | Change values in Livy's livy.conf file. | | livy-env | Change values in the Livy environment. | | livy-log4j | Change Livy log4j.properties settings. | | mapred-env | Change values in the MapReduce application's environment. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | | oozie-env | Change values in Oozie's environment. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | | oozie-site | Change values in Oozie's oozie-site.xml file. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | | pig-env | Change values in the Pig environment. | | pig-properties | Change values in Pig's pig.properties file. | | pig-log4j | Change values in Pig's log4j.properties file. | | presto-log | Change values in Presto's log.properties file. | | presto-config | Change values in Presto's config.properties file. | | presto-env | Change values in Presto's presto-env.sh file. | | presto-node | Change values in Presto's node.properties file. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | | presto-connector-hive | Change values in Presto's hive.properties file. | | presto-connector-jmx | Change values in Presto's jmx.properties file. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | | presto-connector-localfile | Change values in Presto's localfile.properties file. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | | presto-connector-redis | Change values in Presto's redis.properties file. | | presto-connector-redshift | Change values in Presto's redshift.properties file. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | | spark | Amazon EMR-curated settings for Apache Spark. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | | spark-env | Change values in the Spark environment. | | spark-hive-site | Change values in Spark's hive-site.xml file | | spark-log4j | Change values in Spark's log4j.properties file. | | spark-metrics | Change values in Spark's metrics.properties file. | | sqoop-env | Change values in Sqoop's environment. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | | tez-site | Change values in Tez's tez-site.xml file. | | yarn-env | Change values in the YARN environment. | | yarn-site | Change values in YARN's yarn-site.xml file. | | zeppelin-env | Change values in the Zeppelin environment. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | # Amazon EMR release 5.14.1 ## 5.14.1 application versions This release includes the following applications: [https://flink.apache.org/](https://flink.apache.org/), [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [http://gethue.com/](http://gethue.com/), [https://jupyterhub.readthedocs.io/en/latest/#](https://jupyterhub.readthedocs.io/en/latest/#), [https://livy.incubator.apache.org/](https://livy.incubator.apache.org/), [https://mxnet.incubator.apache.org/](https://mxnet.incubator.apache.org/), [http://mahout.apache.org/](http://mahout.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-5.14.1 | emr-5.14.0 | emr-5.13.1 | emr-5.13.0 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.11.297 | 1.11.297 | 1.11.297 | 1.11.297 | | Python | 2.7, 3.4 | 2.7, 3.4 | 2.7, 3.4 | 2.7, 3.4 | | Scala | 2.11.8 | 2.11.8 | 2.11.8 | 2.11.8 | | AmazonCloudWatchAgent | - | - | - | - | | Delta | - | - | - | - | | Flink | 1.4.2 | 1.4.2 | 1.4.0 | 1.4.0 | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 1.4.2 | 1.4.2 | 1.4.2 | 1.4.2 | | HCatalog | 2.3.2-amzn-2 | 2.3.2-amzn-2 | 2.3.2-amzn-2 | 2.3.2-amzn-2 | | Hadoop | 2.8.3-amzn-0 | 2.8.3-amzn-0 | 2.8.3-amzn-0 | 2.8.3-amzn-0 | | Hive | 2.3.2-amzn-2 | 2.3.2-amzn-2 | 2.3.2-amzn-2 | 2.3.2-amzn-2 | | Hudi | - | - | - | - | | Hue | 4.1.0 | 4.1.0 | 4.1.0 | 4.1.0 | | Iceberg | - | - | - | - | | JupyterEnterpriseGateway | - | - | - | - | | JupyterHub | 0.8.1 | 0.8.1 | - | - | | Livy | 0.4.0-incubating | 0.4.0-incubating | 0.4.0-incubating | 0.4.0-incubating | | MXNet | 1.1.0 | 1.1.0 | 1.0.0 | 1.0.0 | | Mahout | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | | Oozie | 4.3.0 | 4.3.0 | 4.3.0 | 4.3.0 | | Phoenix | 4.13.0-HBase-1.4 | 4.13.0-HBase-1.4 | 4.13.0-HBase-1.4 | 4.13.0-HBase-1.4 | | Pig | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | | Presto | 0.194 | 0.194 | 0.194 | 0.194 | | Spark | 2.3.0 | 2.3.0 | 2.3.0 | 2.3.0 | | Sqoop | 1.4.7 | 1.4.7 | 1.4.6 | 1.4.6 | | TensorFlow | - | - | - | - | | Tez | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | | Trino (PrestoSQL) | - | - | - | - | | Zeppelin | 0.7.3 | 0.7.3 | 0.7.3 | 0.7.3 | | ZooKeeper | 3.4.10 | 3.4.10 | 3.4.10 | 3.4.10 | ## 5.14.1 release notes The following release notes include information for Amazon EMR release 5.14.1. Changes are relative to 5.14.0. Initial release date: October 17, 2018 Updated the default AMI for Amazon EMR to address potential security vulnerabilities. ## 5.14.1 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | aws-sagemaker-spark-sdk | 1.0.1 | Amazon SageMaker Spark SDK | | emr-ddb | 4.5.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 2.4.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.4.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-s3-dist-cp | 2.10.0 | Distributed copy application optimized for Amazon S3. | | emrfs | 2.23.0 | Amazon S3 connector for Hadoop ecosystem applications. | | flink-client | 1.4.2 | Apache Flink command line client scripts and applications. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 2.8.3-amzn-1 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 2.8.3-amzn-1 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 2.8.3-amzn-1 | HDFS command-line client and library | | hadoop-hdfs-namenode | 2.8.3-amzn-1 | HDFS service for tracking file names and block locations. | | hadoop-httpfs-server | 2.8.3-amzn-1 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 2.8.3-amzn-1 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 2.8.3-amzn-1 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 2.8.3-amzn-1 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 2.8.3-amzn-1 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 2.8.3-amzn-1 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 1.4.2 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 1.4.2 | Service for serving one or more HBase regions. | | hbase-client | 1.4.2 | HBase command-line client. | | hbase-rest-server | 1.4.2 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 1.4.2 | Service providing a Thrift endpoint to HBase. | | hcatalog-client | 2.3.2-amzn-2 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 2.3.2-amzn-2 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 2.3.2-amzn-2 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 2.3.2-amzn-2 | Hive command line client. | | hive-hbase | 2.3.2-amzn-2 | Hive-hbase client. | | hive-metastore-server | 2.3.2-amzn-2 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server2 | 2.3.2-amzn-2 | Service for accepting Hive queries as web requests. | | hue-server | 4.1.0 | Web application for analyzing data using Hadoop ecosystem applications | | jupyterhub | 0.8.1 | Multi-user server for Jupyter notebooks | | livy-server | 0.4.0-incubating | REST interface for interacting with Apache Spark | | mahout-client | 0.13.0 | Library for machine learning. | | mxnet | 1.1.0 | A flexible, scalable, and efficient library for deep learning. | | mysql-server | 5.5.54\$1 | MySQL database server. | | nvidia-cuda | 9.1.85 | Nvidia drivers and Cuda toolkit | | oozie-client | 4.3.0 | Oozie command-line client. | | oozie-server | 4.3.0 | Service for accepting Oozie workflow requests. | | opencv | 3.4.0 | Open Source Computer Vision Library. | | phoenix-library | 4.13.0-HBase-1.4 | The phoenix libraries for server and client | | phoenix-query-server | 4.13.0-HBase-1.4 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.194 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.194 | Service for executing pieces of a query. | | pig-client | 0.17.0 | Pig command-line client. | | r | 3.4.1 | The R Project for Statistical Computing | | spark-client | 2.3.0 | Spark command-line clients. | | spark-history-server | 2.3.0 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 2.3.0 | In-memory execution engine for YARN. | | spark-yarn-slave | 2.3.0 | Apache Spark libraries needed by YARN slaves. | | sqoop-client | 1.4.7 | Apache Sqoop command-line client. | | tez-on-yarn | 0.8.4 | The tez YARN application and libraries. | | webserver | 2.4.25\$1 | Apache HTTP server. | | zeppelin-server | 0.7.3 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.4.10 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.4.10 | ZooKeeper command line client. | ## 5.14.1 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). **emr-5.14.1 classifications** | Classifications | Description | | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | | container-log4j | Change values in Hadoop YARN's container-log4j.properties file. | | core-site | Change values in Hadoop's core-site.xml file. | | emrfs-site | Change EMRFS settings. | | flink-conf | Change flink-conf.yaml settings. | | flink-log4j | Change Flink log4j.properties settings. | | flink-log4j-yarn-session | Change Flink log4j-yarn-session.properties settings. | | flink-log4j-cli | Change Flink log4j-cli.properties settings. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | | hadoop-ssl-server | Change hadoop ssl server configuration | | hadoop-ssl-client | Change hadoop ssl client configuration | | hbase | Amazon EMR-curated settings for Apache HBase. | | hbase-env | Change values in HBase's environment. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbase.properties file. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | | hbase-site | Change values in HBase's hbase-site.xml file. | | hdfs-encryption-zones | Configure HDFS encryption zones. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | | hcatalog-env | Change values in HCatalog's environment. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | | hcatalog-webhcat-log4j2 | Change values in HCatalog WebHCat's log4j2.properties. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | | hive-beeline-log4j2 | Change values in Hive's beeline-log4j2.properties file. | | hive-parquet-logging | Change values in Hive's parquet-logging.properties file. | | hive-env | Change values in the Hive environment. | | hive-exec-log4j2 | Change values in Hive's hive-exec-log4j2.properties file. | | hive-llap-daemon-log4j2 | Change values in Hive's llap-daemon-log4j2.properties file. | | hive-log4j2 | Change values in Hive's hive-log4j2.properties file. | | hive-site | Change values in Hive's hive-site.xml file | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | | hue-ini | Change values in Hue's ini file | | httpfs-env | Change values in the HTTPFS environment. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | | jupyter-notebook-conf | Change values in Jupyter Notebook's jupyter\$1notebook\$1config.py file. | | jupyter-hub-conf | Change values in JupyterHubs's jupyterhub\$1config.py file. | | jupyter-sparkmagic-conf | Change values in Sparkmagic's config.json file. | | livy-conf | Change values in Livy's livy.conf file. | | livy-env | Change values in the Livy environment. | | livy-log4j | Change Livy log4j.properties settings. | | mapred-env | Change values in the MapReduce application's environment. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | | oozie-env | Change values in Oozie's environment. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | | oozie-site | Change values in Oozie's oozie-site.xml file. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | | pig-env | Change values in the Pig environment. | | pig-properties | Change values in Pig's pig.properties file. | | pig-log4j | Change values in Pig's log4j.properties file. | | presto-log | Change values in Presto's log.properties file. | | presto-config | Change values in Presto's config.properties file. | | presto-env | Change values in Presto's presto-env.sh file. | | presto-node | Change values in Presto's node.properties file. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | | presto-connector-hive | Change values in Presto's hive.properties file. | | presto-connector-jmx | Change values in Presto's jmx.properties file. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | | presto-connector-localfile | Change values in Presto's localfile.properties file. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | | presto-connector-redis | Change values in Presto's redis.properties file. | | presto-connector-redshift | Change values in Presto's redshift.properties file. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | | spark | Amazon EMR-curated settings for Apache Spark. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | | spark-env | Change values in the Spark environment. | | spark-hive-site | Change values in Spark's hive-site.xml file | | spark-log4j | Change values in Spark's log4j.properties file. | | spark-metrics | Change values in Spark's metrics.properties file. | | sqoop-env | Change values in Sqoop's environment. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | | tez-site | Change values in Tez's tez-site.xml file. | | yarn-env | Change values in the YARN environment. | | yarn-site | Change values in YARN's yarn-site.xml file. | | zeppelin-env | Change values in the Zeppelin environment. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | # Amazon EMR release 5.14.0 ## 5.14.0 application versions This release includes the following applications: [https://flink.apache.org/](https://flink.apache.org/), [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [http://gethue.com/](http://gethue.com/), [https://jupyterhub.readthedocs.io/en/latest/#](https://jupyterhub.readthedocs.io/en/latest/#), [https://livy.incubator.apache.org/](https://livy.incubator.apache.org/), [https://mxnet.incubator.apache.org/](https://mxnet.incubator.apache.org/), [http://mahout.apache.org/](http://mahout.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-5.14.0 | emr-5.13.1 | emr-5.13.0 | emr-5.12.3 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.11.297 | 1.11.297 | 1.11.297 | 1.11.267 | | Python | 2.7, 3.4 | 2.7, 3.4 | 2.7, 3.4 | 2.7, 3.4 | | Scala | 2.11.8 | 2.11.8 | 2.11.8 | 2.11.8 | | AmazonCloudWatchAgent | - | - | - | - | | Delta | - | - | - | - | | Flink | 1.4.2 | 1.4.0 | 1.4.0 | 1.4.0 | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 1.4.2 | 1.4.2 | 1.4.2 | 1.4.0 | | HCatalog | 2.3.2-amzn-2 | 2.3.2-amzn-2 | 2.3.2-amzn-2 | 2.3.2-amzn-1 | | Hadoop | 2.8.3-amzn-0 | 2.8.3-amzn-0 | 2.8.3-amzn-0 | 2.8.3-amzn-0 | | Hive | 2.3.2-amzn-2 | 2.3.2-amzn-2 | 2.3.2-amzn-2 | 2.3.2-amzn-1 | | Hudi | - | - | - | - | | Hue | 4.1.0 | 4.1.0 | 4.1.0 | 4.1.0 | | Iceberg | - | - | - | - | | JupyterEnterpriseGateway | - | - | - | - | | JupyterHub | 0.8.1 | - | - | - | | Livy | 0.4.0-incubating | 0.4.0-incubating | 0.4.0-incubating | 0.4.0-incubating | | MXNet | 1.1.0 | 1.0.0 | 1.0.0 | 1.0.0 | | Mahout | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | | Oozie | 4.3.0 | 4.3.0 | 4.3.0 | 4.3.0 | | Phoenix | 4.13.0-HBase-1.4 | 4.13.0-HBase-1.4 | 4.13.0-HBase-1.4 | 4.13.0-HBase-1.4 | | Pig | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | | Presto | 0.194 | 0.194 | 0.194 | 0.188 | | Spark | 2.3.0 | 2.3.0 | 2.3.0 | 2.2.1 | | Sqoop | 1.4.7 | 1.4.6 | 1.4.6 | 1.4.6 | | TensorFlow | - | - | - | - | | Tez | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | | Trino (PrestoSQL) | - | - | - | - | | Zeppelin | 0.7.3 | 0.7.3 | 0.7.3 | 0.7.3 | | ZooKeeper | 3.4.10 | 3.4.10 | 3.4.10 | 3.4.10 | ## 5.14.0 release notes The following release notes include information for Amazon EMR release 5.14.0. Changes are relative to 5.13.0. Initial release date: June 4, 2018 **Upgrades** + Upgraded Apache Flink to 1.4.2 + Upgraded Apache MXnet to 1.1.0 + Upgraded Apache Sqoop to 1.4.7 **New features** + Added JupyterHub support. For more information, see [JupyterHub](emr-jupyterhub.md). **Changes, enhancements, and resolved issues** + EMRFS + The userAgent string in requests to Amazon S3 has been updated to contain the user and group information of the invoking principal. This can be used with AWS CloudTrail logs for more comprehensive request tracking. + HBase + Included [HBASE-20447](https://issues.apache.org/jira/browse/HBASE-20447), which addresses an issue that could cause cache issues, especially with split Regions. + MXnet + Added OpenCV libraries. + Spark + When Spark writes Parquet files to an Amazon S3 location using EMRFS, the FileOutputCommitter algorithm has been updated to use version 2 instead of version 1. This reduces the number of renames, which improves application performance. This change does not affect: + Applications other than Spark. + Applications that write to other file systems, such as HDFS (which still use version 1 of FileOutputCommitter). + Applications that use other output formats, such as text or csv, that already use EMRFS direct write. **Known issues** + JupyterHub + Using configuration classifications to set up JupyterHub and individual Jupyter notebooks when you create a cluster is not supported. Edit the jupyterhub\$1config.py file and jupyter\$1notebook\$1config.py files for each user manually. For more information, see [Configuring JupyterHub](emr-jupyterhub-configure.md). + JupyterHub fails to start on clusters within a private subnet, failing with the message `Error: ENOENT: no such file or directory, open '/etc/jupyter/conf/server.crt' `. This is caused by an error in the script that generates self-signed certificates. Use the following workaround to generate self-signed certificates. All commands are executed while connected to the primary node. 1. Copy the certificate generation script from the container to the primary node: ``` sudo docker cp jupyterhub:/tmp/gen_self_signed_cert.sh ./ ``` 1. Use a text editor to change line 23 to change public hostname to local hostname as shown below: ``` local hostname=$(curl -s $EC2_METADATA_SERVICE_URI/local-hostname) ``` 1. Run the script to generate self-signed certificates: ``` sudo bash ./gen_self_signed_cert.sh ``` 1. Move the certificate files that the script generates to the `/etc/jupyter/conf/` directory: ``` sudo mv /tmp/server.crt /tmp/server.key /etc/jupyter/conf/ ``` You can `tail` the `jupyter.log` file to verify that JupyterHub restarted and is returning a 200 response code. For example: ``` tail -f /var/log/jupyter/jupyter.log ``` This should return a response similar to the following: ``` # [I 2018-06-14 18:56:51.356 JupyterHub app:1581] JupyterHub is now running at https://:9443/ # 19:01:51.359 - info: [ConfigProxy] 200 GET /api/routes ``` + After the primary node reboots or the instance controller restarts, the CloudWatch metrics will not be collected and the automatic scaling feature will not be available in Amazon EMR version 5.14.0, 5.15.0, or 5.16.0. This issue is fixed in Amazon EMR 5.17.0. ## 5.14.0 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | aws-sagemaker-spark-sdk | 1.0.1 | Amazon SageMaker Spark SDK | | emr-ddb | 4.5.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 2.4.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.4.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-s3-dist-cp | 2.10.0 | Distributed copy application optimized for Amazon S3. | | emrfs | 2.23.0 | Amazon S3 connector for Hadoop ecosystem applications. | | flink-client | 1.4.2 | Apache Flink command line client scripts and applications. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 2.8.3-amzn-1 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 2.8.3-amzn-1 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 2.8.3-amzn-1 | HDFS command-line client and library | | hadoop-hdfs-namenode | 2.8.3-amzn-1 | HDFS service for tracking file names and block locations. | | hadoop-httpfs-server | 2.8.3-amzn-1 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 2.8.3-amzn-1 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 2.8.3-amzn-1 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 2.8.3-amzn-1 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 2.8.3-amzn-1 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 2.8.3-amzn-1 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 1.4.2 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 1.4.2 | Service for serving one or more HBase regions. | | hbase-client | 1.4.2 | HBase command-line client. | | hbase-rest-server | 1.4.2 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 1.4.2 | Service providing a Thrift endpoint to HBase. | | hcatalog-client | 2.3.2-amzn-2 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 2.3.2-amzn-2 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 2.3.2-amzn-2 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 2.3.2-amzn-2 | Hive command line client. | | hive-hbase | 2.3.2-amzn-2 | Hive-hbase client. | | hive-metastore-server | 2.3.2-amzn-2 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server2 | 2.3.2-amzn-2 | Service for accepting Hive queries as web requests. | | hue-server | 4.1.0 | Web application for analyzing data using Hadoop ecosystem applications | | jupyterhub | 0.8.1 | Multi-user server for Jupyter notebooks | | livy-server | 0.4.0-incubating | REST interface for interacting with Apache Spark | | mahout-client | 0.13.0 | Library for machine learning. | | mxnet | 1.1.0 | A flexible, scalable, and efficient library for deep learning. | | mysql-server | 5.5.54\$1 | MySQL database server. | | nvidia-cuda | 9.1.85 | Nvidia drivers and Cuda toolkit | | oozie-client | 4.3.0 | Oozie command-line client. | | oozie-server | 4.3.0 | Service for accepting Oozie workflow requests. | | opencv | 3.4.0 | Open Source Computer Vision Library. | | phoenix-library | 4.13.0-HBase-1.4 | The phoenix libraries for server and client | | phoenix-query-server | 4.13.0-HBase-1.4 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.194 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.194 | Service for executing pieces of a query. | | pig-client | 0.17.0 | Pig command-line client. | | r | 3.4.1 | The R Project for Statistical Computing | | spark-client | 2.3.0 | Spark command-line clients. | | spark-history-server | 2.3.0 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 2.3.0 | In-memory execution engine for YARN. | | spark-yarn-slave | 2.3.0 | Apache Spark libraries needed by YARN slaves. | | sqoop-client | 1.4.7 | Apache Sqoop command-line client. | | tez-on-yarn | 0.8.4 | The tez YARN application and libraries. | | webserver | 2.4.25\$1 | Apache HTTP server. | | zeppelin-server | 0.7.3 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.4.10 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.4.10 | ZooKeeper command line client. | ## 5.14.0 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). **emr-5.14.0 classifications** | Classifications | Description | | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | | container-log4j | Change values in Hadoop YARN's container-log4j.properties file. | | core-site | Change values in Hadoop's core-site.xml file. | | emrfs-site | Change EMRFS settings. | | flink-conf | Change flink-conf.yaml settings. | | flink-log4j | Change Flink log4j.properties settings. | | flink-log4j-yarn-session | Change Flink log4j-yarn-session.properties settings. | | flink-log4j-cli | Change Flink log4j-cli.properties settings. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | | hadoop-ssl-server | Change hadoop ssl server configuration | | hadoop-ssl-client | Change hadoop ssl client configuration | | hbase | Amazon EMR-curated settings for Apache HBase. | | hbase-env | Change values in HBase's environment. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbase.properties file. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | | hbase-site | Change values in HBase's hbase-site.xml file. | | hdfs-encryption-zones | Configure HDFS encryption zones. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | | hcatalog-env | Change values in HCatalog's environment. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | | hcatalog-webhcat-log4j2 | Change values in HCatalog WebHCat's log4j2.properties. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | | hive-beeline-log4j2 | Change values in Hive's beeline-log4j2.properties file. | | hive-parquet-logging | Change values in Hive's parquet-logging.properties file. | | hive-env | Change values in the Hive environment. | | hive-exec-log4j2 | Change values in Hive's hive-exec-log4j2.properties file. | | hive-llap-daemon-log4j2 | Change values in Hive's llap-daemon-log4j2.properties file. | | hive-log4j2 | Change values in Hive's hive-log4j2.properties file. | | hive-site | Change values in Hive's hive-site.xml file | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | | hue-ini | Change values in Hue's ini file | | httpfs-env | Change values in the HTTPFS environment. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | | jupyter-notebook-conf | Change values in Jupyter Notebook's jupyter\$1notebook\$1config.py file. | | jupyter-hub-conf | Change values in JupyterHubs's jupyterhub\$1config.py file. | | jupyter-sparkmagic-conf | Change values in Sparkmagic's config.json file. | | livy-conf | Change values in Livy's livy.conf file. | | livy-env | Change values in the Livy environment. | | livy-log4j | Change Livy log4j.properties settings. | | mapred-env | Change values in the MapReduce application's environment. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | | oozie-env | Change values in Oozie's environment. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | | oozie-site | Change values in Oozie's oozie-site.xml file. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | | pig-env | Change values in the Pig environment. | | pig-properties | Change values in Pig's pig.properties file. | | pig-log4j | Change values in Pig's log4j.properties file. | | presto-log | Change values in Presto's log.properties file. | | presto-config | Change values in Presto's config.properties file. | | presto-env | Change values in Presto's presto-env.sh file. | | presto-node | Change values in Presto's node.properties file. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | | presto-connector-hive | Change values in Presto's hive.properties file. | | presto-connector-jmx | Change values in Presto's jmx.properties file. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | | presto-connector-localfile | Change values in Presto's localfile.properties file. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | | presto-connector-redis | Change values in Presto's redis.properties file. | | presto-connector-redshift | Change values in Presto's redshift.properties file. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | | spark | Amazon EMR-curated settings for Apache Spark. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | | spark-env | Change values in the Spark environment. | | spark-hive-site | Change values in Spark's hive-site.xml file | | spark-log4j | Change values in Spark's log4j.properties file. | | spark-metrics | Change values in Spark's metrics.properties file. | | sqoop-env | Change values in Sqoop's environment. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | | tez-site | Change values in Tez's tez-site.xml file. | | yarn-env | Change values in the YARN environment. | | yarn-site | Change values in YARN's yarn-site.xml file. | | zeppelin-env | Change values in the Zeppelin environment. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | # Amazon EMR release 5.13.1 ## 5.13.1 application versions This release includes the following applications: [https://flink.apache.org/](https://flink.apache.org/), [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [http://gethue.com/](http://gethue.com/), [https://livy.incubator.apache.org/](https://livy.incubator.apache.org/), [https://mxnet.incubator.apache.org/](https://mxnet.incubator.apache.org/), [http://mahout.apache.org/](http://mahout.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-5.13.1 | emr-5.13.0 | emr-5.12.3 | emr-5.12.2 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.11.297 | 1.11.297 | 1.11.267 | 1.11.267 | | Python | 2.7, 3.4 | 2.7, 3.4 | 2.7, 3.4 | 2.7, 3.4 | | Scala | 2.11.8 | 2.11.8 | 2.11.8 | 2.11.8 | | AmazonCloudWatchAgent | - | - | - | - | | Delta | - | - | - | - | | Flink | 1.4.0 | 1.4.0 | 1.4.0 | 1.4.0 | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 1.4.2 | 1.4.2 | 1.4.0 | 1.4.0 | | HCatalog | 2.3.2-amzn-2 | 2.3.2-amzn-2 | 2.3.2-amzn-1 | 2.3.2-amzn-1 | | Hadoop | 2.8.3-amzn-0 | 2.8.3-amzn-0 | 2.8.3-amzn-0 | 2.8.3-amzn-0 | | Hive | 2.3.2-amzn-2 | 2.3.2-amzn-2 | 2.3.2-amzn-1 | 2.3.2-amzn-1 | | Hudi | - | - | - | - | | Hue | 4.1.0 | 4.1.0 | 4.1.0 | 4.1.0 | | Iceberg | - | - | - | - | | JupyterEnterpriseGateway | - | - | - | - | | JupyterHub | - | - | - | - | | Livy | 0.4.0-incubating | 0.4.0-incubating | 0.4.0-incubating | 0.4.0-incubating | | MXNet | 1.0.0 | 1.0.0 | 1.0.0 | 1.0.0 | | Mahout | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | | Oozie | 4.3.0 | 4.3.0 | 4.3.0 | 4.3.0 | | Phoenix | 4.13.0-HBase-1.4 | 4.13.0-HBase-1.4 | 4.13.0-HBase-1.4 | 4.13.0-HBase-1.4 | | Pig | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | | Presto | 0.194 | 0.194 | 0.188 | 0.188 | | Spark | 2.3.0 | 2.3.0 | 2.2.1 | 2.2.1 | | Sqoop | 1.4.6 | 1.4.6 | 1.4.6 | 1.4.6 | | TensorFlow | - | - | - | - | | Tez | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | | Trino (PrestoSQL) | - | - | - | - | | Zeppelin | 0.7.3 | 0.7.3 | 0.7.3 | 0.7.3 | | ZooKeeper | 3.4.10 | 3.4.10 | 3.4.10 | 3.4.10 | ## 5.13.1 release notes This is a patch release to add AWS Signature Version 4 authentication for requests to Amazon S3. All applications and components are the same as the previous Amazon EMR release. **Important** In this release version, Amazon EMR uses AWS Signature Version 4 exclusively to authenticate requests to Amazon S3. For more information, see [Whats New](https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-whatsnew.html). ## 5.13.1 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | aws-sagemaker-spark-sdk | 1.0.1 | Amazon SageMaker Spark SDK | | emr-ddb | 4.5.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 2.4.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.4.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-s3-dist-cp | 2.10.0 | Distributed copy application optimized for Amazon S3. | | emrfs | 2.22.0 | Amazon S3 connector for Hadoop ecosystem applications. | | flink-client | 1.4.0 | Apache Flink command line client scripts and applications. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 2.8.3-amzn-0 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 2.8.3-amzn-0 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 2.8.3-amzn-0 | HDFS command-line client and library | | hadoop-hdfs-namenode | 2.8.3-amzn-0 | HDFS service for tracking file names and block locations. | | hadoop-httpfs-server | 2.8.3-amzn-0 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 2.8.3-amzn-0 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 2.8.3-amzn-0 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 2.8.3-amzn-0 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 2.8.3-amzn-0 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 2.8.3-amzn-0 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 1.4.2 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 1.4.2 | Service for serving one or more HBase regions. | | hbase-client | 1.4.2 | HBase command-line client. | | hbase-rest-server | 1.4.2 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 1.4.2 | Service providing a Thrift endpoint to HBase. | | hcatalog-client | 2.3.2-amzn-2 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 2.3.2-amzn-2 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 2.3.2-amzn-2 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 2.3.2-amzn-2 | Hive command line client. | | hive-hbase | 2.3.2-amzn-2 | Hive-hbase client. | | hive-metastore-server | 2.3.2-amzn-2 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server2 | 2.3.2-amzn-2 | Service for accepting Hive queries as web requests. | | hue-server | 4.1.0 | Web application for analyzing data using Hadoop ecosystem applications | | livy-server | 0.4.0-incubating | REST interface for interacting with Apache Spark | | mahout-client | 0.13.0 | Library for machine learning. | | mxnet | 1.0.0 | A flexible, scalable, and efficient library for deep learning. | | mysql-server | 5.5.54\$1 | MySQL database server. | | nvidia-cuda | 9.1.85 | Nvidia drivers and Cuda toolkit | | oozie-client | 4.3.0 | Oozie command-line client. | | oozie-server | 4.3.0 | Service for accepting Oozie workflow requests. | | phoenix-library | 4.13.0-HBase-1.4 | The phoenix libraries for server and client | | phoenix-query-server | 4.13.0-HBase-1.4 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.194 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.194 | Service for executing pieces of a query. | | pig-client | 0.17.0 | Pig command-line client. | | r | 3.4.1 | The R Project for Statistical Computing | | spark-client | 2.3.0 | Spark command-line clients. | | spark-history-server | 2.3.0 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 2.3.0 | In-memory execution engine for YARN. | | spark-yarn-slave | 2.3.0 | Apache Spark libraries needed by YARN slaves. | | sqoop-client | 1.4.6 | Apache Sqoop command-line client. | | tez-on-yarn | 0.8.4 | The tez YARN application and libraries. | | webserver | 2.4.25\$1 | Apache HTTP server. | | zeppelin-server | 0.7.3 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.4.10 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.4.10 | ZooKeeper command line client. | ## 5.13.1 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). **emr-5.13.1 classifications** | Classifications | Description | | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | | core-site | Change values in Hadoop's core-site.xml file. | | emrfs-site | Change EMRFS settings. | | flink-conf | Change flink-conf.yaml settings. | | flink-log4j | Change Flink log4j.properties settings. | | flink-log4j-yarn-session | Change Flink log4j-yarn-session.properties settings. | | flink-log4j-cli | Change Flink log4j-cli.properties settings. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | | hadoop-ssl-server | Change hadoop ssl server configuration | | hadoop-ssl-client | Change hadoop ssl client configuration | | hbase | Amazon EMR-curated settings for Apache HBase. | | hbase-env | Change values in HBase's environment. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbase.properties file. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | | hbase-site | Change values in HBase's hbase-site.xml file. | | hdfs-encryption-zones | Configure HDFS encryption zones. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | | hcatalog-env | Change values in HCatalog's environment. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | | hcatalog-webhcat-log4j2 | Change values in HCatalog WebHCat's log4j2.properties. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | | hive-beeline-log4j2 | Change values in Hive's beeline-log4j2.properties file. | | hive-parquet-logging | Change values in Hive's parquet-logging.properties file. | | hive-env | Change values in the Hive environment. | | hive-exec-log4j2 | Change values in Hive's hive-exec-log4j2.properties file. | | hive-llap-daemon-log4j2 | Change values in Hive's llap-daemon-log4j2.properties file. | | hive-log4j2 | Change values in Hive's hive-log4j2.properties file. | | hive-site | Change values in Hive's hive-site.xml file | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | | hue-ini | Change values in Hue's ini file | | httpfs-env | Change values in the HTTPFS environment. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | | livy-conf | Change values in Livy's livy.conf file. | | livy-env | Change values in the Livy environment. | | livy-log4j | Change Livy log4j.properties settings. | | mapred-env | Change values in the MapReduce application's environment. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | | oozie-env | Change values in Oozie's environment. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | | oozie-site | Change values in Oozie's oozie-site.xml file. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | | pig-env | Change values in the Pig environment. | | pig-properties | Change values in Pig's pig.properties file. | | pig-log4j | Change values in Pig's log4j.properties file. | | presto-log | Change values in Presto's log.properties file. | | presto-config | Change values in Presto's config.properties file. | | presto-env | Change values in Presto's presto-env.sh file. | | presto-node | Change values in Presto's node.properties file. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | | presto-connector-hive | Change values in Presto's hive.properties file. | | presto-connector-jmx | Change values in Presto's jmx.properties file. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | | presto-connector-localfile | Change values in Presto's localfile.properties file. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | | presto-connector-redis | Change values in Presto's redis.properties file. | | presto-connector-redshift | Change values in Presto's redshift.properties file. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | | spark | Amazon EMR-curated settings for Apache Spark. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | | spark-env | Change values in the Spark environment. | | spark-hive-site | Change values in Spark's hive-site.xml file | | spark-log4j | Change values in Spark's log4j.properties file. | | spark-metrics | Change values in Spark's metrics.properties file. | | sqoop-env | Change values in Sqoop's environment. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | | tez-site | Change values in Tez's tez-site.xml file. | | yarn-env | Change values in the YARN environment. | | yarn-site | Change values in YARN's yarn-site.xml file. | | zeppelin-env | Change values in the Zeppelin environment. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | # Amazon EMR release 5.13.0 ## 5.13.0 application versions This release includes the following applications: [https://flink.apache.org/](https://flink.apache.org/), [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [http://gethue.com/](http://gethue.com/), [https://livy.incubator.apache.org/](https://livy.incubator.apache.org/), [https://mxnet.incubator.apache.org/](https://mxnet.incubator.apache.org/), [http://mahout.apache.org/](http://mahout.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-5.13.0 | emr-5.12.3 | emr-5.12.2 | emr-5.12.1 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.11.297 | 1.11.267 | 1.11.267 | 1.11.267 | | Python | 2.7, 3.4 | 2.7, 3.4 | 2.7, 3.4 | 2.7, 3.4 | | Scala | 2.11.8 | 2.11.8 | 2.11.8 | 2.11.8 | | AmazonCloudWatchAgent | - | - | - | - | | Delta | - | - | - | - | | Flink | 1.4.0 | 1.4.0 | 1.4.0 | 1.4.0 | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 1.4.2 | 1.4.0 | 1.4.0 | 1.4.0 | | HCatalog | 2.3.2-amzn-2 | 2.3.2-amzn-1 | 2.3.2-amzn-1 | 2.3.2-amzn-1 | | Hadoop | 2.8.3-amzn-0 | 2.8.3-amzn-0 | 2.8.3-amzn-0 | 2.8.3-amzn-0 | | Hive | 2.3.2-amzn-2 | 2.3.2-amzn-1 | 2.3.2-amzn-1 | 2.3.2-amzn-1 | | Hudi | - | - | - | - | | Hue | 4.1.0 | 4.1.0 | 4.1.0 | 4.1.0 | | Iceberg | - | - | - | - | | JupyterEnterpriseGateway | - | - | - | - | | JupyterHub | - | - | - | - | | Livy | 0.4.0-incubating | 0.4.0-incubating | 0.4.0-incubating | 0.4.0-incubating | | MXNet | 1.0.0 | 1.0.0 | 1.0.0 | 1.0.0 | | Mahout | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | | Oozie | 4.3.0 | 4.3.0 | 4.3.0 | 4.3.0 | | Phoenix | 4.13.0-HBase-1.4 | 4.13.0-HBase-1.4 | 4.13.0-HBase-1.4 | 4.13.0-HBase-1.4 | | Pig | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | | Presto | 0.194 | 0.188 | 0.188 | 0.188 | | Spark | 2.3.0 | 2.2.1 | 2.2.1 | 2.2.1 | | Sqoop | 1.4.6 | 1.4.6 | 1.4.6 | 1.4.6 | | TensorFlow | - | - | - | - | | Tez | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | | Trino (PrestoSQL) | - | - | - | - | | Zeppelin | 0.7.3 | 0.7.3 | 0.7.3 | 0.7.3 | | ZooKeeper | 3.4.10 | 3.4.10 | 3.4.10 | 3.4.10 | ## 5.13.0 release notes The following release notes include information for the Amazon EMR release 5.13.0. Changes are relative to 5.12.0. **Upgrades** + Upgraded Spark to 2.3.0 + Upgraded HBase to 1.4.2 + Upgraded Presto to 0.194 + Upgraded AWS SDK for Java to 1.11.297 **Changes, enhancements, and resolved issues** + Hive + Backported [HIVE-15436](https://issues.apache.org/jira/browse/HIVE-15436). Enhanced Hive APIs to return only views. **Known issues** + MXNet does not currently have OpenCV libraries. ## 5.13.0 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | aws-sagemaker-spark-sdk | 1.0.1 | Amazon SageMaker Spark SDK | | emr-ddb | 4.5.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 2.4.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.4.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-s3-dist-cp | 2.10.0 | Distributed copy application optimized for Amazon S3. | | emrfs | 2.22.0 | Amazon S3 connector for Hadoop ecosystem applications. | | flink-client | 1.4.0 | Apache Flink command line client scripts and applications. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 2.8.3-amzn-0 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 2.8.3-amzn-0 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 2.8.3-amzn-0 | HDFS command-line client and library | | hadoop-hdfs-namenode | 2.8.3-amzn-0 | HDFS service for tracking file names and block locations. | | hadoop-httpfs-server | 2.8.3-amzn-0 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 2.8.3-amzn-0 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 2.8.3-amzn-0 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 2.8.3-amzn-0 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 2.8.3-amzn-0 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 2.8.3-amzn-0 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 1.4.2 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 1.4.2 | Service for serving one or more HBase regions. | | hbase-client | 1.4.2 | HBase command-line client. | | hbase-rest-server | 1.4.2 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 1.4.2 | Service providing a Thrift endpoint to HBase. | | hcatalog-client | 2.3.2-amzn-2 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 2.3.2-amzn-2 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 2.3.2-amzn-2 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 2.3.2-amzn-2 | Hive command line client. | | hive-hbase | 2.3.2-amzn-2 | Hive-hbase client. | | hive-metastore-server | 2.3.2-amzn-2 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server2 | 2.3.2-amzn-2 | Service for accepting Hive queries as web requests. | | hue-server | 4.1.0 | Web application for analyzing data using Hadoop ecosystem applications | | livy-server | 0.4.0-incubating | REST interface for interacting with Apache Spark | | mahout-client | 0.13.0 | Library for machine learning. | | mxnet | 1.0.0 | A flexible, scalable, and efficient library for deep learning. | | mysql-server | 5.5.54\$1 | MySQL database server. | | nvidia-cuda | 9.1.85 | Nvidia drivers and Cuda toolkit | | oozie-client | 4.3.0 | Oozie command-line client. | | oozie-server | 4.3.0 | Service for accepting Oozie workflow requests. | | phoenix-library | 4.13.0-HBase-1.4 | The phoenix libraries for server and client | | phoenix-query-server | 4.13.0-HBase-1.4 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.194 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.194 | Service for executing pieces of a query. | | pig-client | 0.17.0 | Pig command-line client. | | r | 3.4.1 | The R Project for Statistical Computing | | spark-client | 2.3.0 | Spark command-line clients. | | spark-history-server | 2.3.0 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 2.3.0 | In-memory execution engine for YARN. | | spark-yarn-slave | 2.3.0 | Apache Spark libraries needed by YARN slaves. | | sqoop-client | 1.4.6 | Apache Sqoop command-line client. | | tez-on-yarn | 0.8.4 | The tez YARN application and libraries. | | webserver | 2.4.25\$1 | Apache HTTP server. | | zeppelin-server | 0.7.3 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.4.10 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.4.10 | ZooKeeper command line client. | ## 5.13.0 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). **emr-5.13.0 classifications** | Classifications | Description | | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | | core-site | Change values in Hadoop's core-site.xml file. | | emrfs-site | Change EMRFS settings. | | flink-conf | Change flink-conf.yaml settings. | | flink-log4j | Change Flink log4j.properties settings. | | flink-log4j-yarn-session | Change Flink log4j-yarn-session.properties settings. | | flink-log4j-cli | Change Flink log4j-cli.properties settings. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | | hadoop-ssl-server | Change hadoop ssl server configuration | | hadoop-ssl-client | Change hadoop ssl client configuration | | hbase | Amazon EMR-curated settings for Apache HBase. | | hbase-env | Change values in HBase's environment. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbase.properties file. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | | hbase-site | Change values in HBase's hbase-site.xml file. | | hdfs-encryption-zones | Configure HDFS encryption zones. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | | hcatalog-env | Change values in HCatalog's environment. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | | hcatalog-webhcat-log4j2 | Change values in HCatalog WebHCat's log4j2.properties. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | | hive-beeline-log4j2 | Change values in Hive's beeline-log4j2.properties file. | | hive-parquet-logging | Change values in Hive's parquet-logging.properties file. | | hive-env | Change values in the Hive environment. | | hive-exec-log4j2 | Change values in Hive's hive-exec-log4j2.properties file. | | hive-llap-daemon-log4j2 | Change values in Hive's llap-daemon-log4j2.properties file. | | hive-log4j2 | Change values in Hive's hive-log4j2.properties file. | | hive-site | Change values in Hive's hive-site.xml file | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | | hue-ini | Change values in Hue's ini file | | httpfs-env | Change values in the HTTPFS environment. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | | livy-conf | Change values in Livy's livy.conf file. | | livy-env | Change values in the Livy environment. | | livy-log4j | Change Livy log4j.properties settings. | | mapred-env | Change values in the MapReduce application's environment. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | | oozie-env | Change values in Oozie's environment. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | | oozie-site | Change values in Oozie's oozie-site.xml file. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | | pig-env | Change values in the Pig environment. | | pig-properties | Change values in Pig's pig.properties file. | | pig-log4j | Change values in Pig's log4j.properties file. | | presto-log | Change values in Presto's log.properties file. | | presto-config | Change values in Presto's config.properties file. | | presto-env | Change values in Presto's presto-env.sh file. | | presto-node | Change values in Presto's node.properties file. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | | presto-connector-hive | Change values in Presto's hive.properties file. | | presto-connector-jmx | Change values in Presto's jmx.properties file. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | | presto-connector-localfile | Change values in Presto's localfile.properties file. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | | presto-connector-redis | Change values in Presto's redis.properties file. | | presto-connector-redshift | Change values in Presto's redshift.properties file. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | | spark | Amazon EMR-curated settings for Apache Spark. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | | spark-env | Change values in the Spark environment. | | spark-hive-site | Change values in Spark's hive-site.xml file | | spark-log4j | Change values in Spark's log4j.properties file. | | spark-metrics | Change values in Spark's metrics.properties file. | | sqoop-env | Change values in Sqoop's environment. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | | tez-site | Change values in Tez's tez-site.xml file. | | yarn-env | Change values in the YARN environment. | | yarn-site | Change values in YARN's yarn-site.xml file. | | zeppelin-env | Change values in the Zeppelin environment. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | # Amazon EMR release 5.12.3 ## 5.12.3 application versions This release includes the following applications: [https://flink.apache.org/](https://flink.apache.org/), [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [http://gethue.com/](http://gethue.com/), [https://livy.incubator.apache.org/](https://livy.incubator.apache.org/), [https://mxnet.incubator.apache.org/](https://mxnet.incubator.apache.org/), [http://mahout.apache.org/](http://mahout.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-5.12.3 | emr-5.12.2 | emr-5.12.1 | emr-5.12.0 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.11.267 | 1.11.267 | 1.11.267 | 1.11.267 | | Python | 2.7, 3.4 | 2.7, 3.4 | 2.7, 3.4 | 2.7, 3.4 | | Scala | 2.11.8 | 2.11.8 | 2.11.8 | 2.11.8 | | AmazonCloudWatchAgent | - | - | - | - | | Delta | - | - | - | - | | Flink | 1.4.0 | 1.4.0 | 1.4.0 | 1.4.0 | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 1.4.0 | 1.4.0 | 1.4.0 | 1.4.0 | | HCatalog | 2.3.2-amzn-1 | 2.3.2-amzn-1 | 2.3.2-amzn-1 | 2.3.2-amzn-1 | | Hadoop | 2.8.3-amzn-0 | 2.8.3-amzn-0 | 2.8.3-amzn-0 | 2.8.3-amzn-0 | | Hive | 2.3.2-amzn-1 | 2.3.2-amzn-1 | 2.3.2-amzn-1 | 2.3.2-amzn-1 | | Hudi | - | - | - | - | | Hue | 4.1.0 | 4.1.0 | 4.1.0 | 4.1.0 | | Iceberg | - | - | - | - | | JupyterEnterpriseGateway | - | - | - | - | | JupyterHub | - | - | - | - | | Livy | 0.4.0-incubating | 0.4.0-incubating | 0.4.0-incubating | 0.4.0-incubating | | MXNet | 1.0.0 | 1.0.0 | 1.0.0 | 1.0.0 | | Mahout | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | | Oozie | 4.3.0 | 4.3.0 | 4.3.0 | 4.3.0 | | Phoenix | 4.13.0-HBase-1.4 | 4.13.0-HBase-1.4 | 4.13.0-HBase-1.4 | 4.13.0-HBase-1.4 | | Pig | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | | Presto | 0.188 | 0.188 | 0.188 | 0.188 | | Spark | 2.2.1 | 2.2.1 | 2.2.1 | 2.2.1 | | Sqoop | 1.4.6 | 1.4.6 | 1.4.6 | 1.4.6 | | TensorFlow | - | - | - | - | | Tez | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | | Trino (PrestoSQL) | - | - | - | - | | Zeppelin | 0.7.3 | 0.7.3 | 0.7.3 | 0.7.3 | | ZooKeeper | 3.4.10 | 3.4.10 | 3.4.10 | 3.4.10 | ## 5.12.3 release notes This is a patch release to add AWS Signature Version 4 authentication for requests to Amazon S3. All applications and components are the same as the previous Amazon EMR release. **Important** In this release version, Amazon EMR uses AWS Signature Version 4 exclusively to authenticate requests to Amazon S3. For more information, see [Whats New](https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-whatsnew.html). ## 5.12.3 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | aws-sagemaker-spark-sdk | 1.0.1 | Amazon SageMaker Spark SDK | | emr-ddb | 4.5.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 2.4.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.4.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-s3-dist-cp | 2.9.0 | Distributed copy application optimized for Amazon S3. | | emrfs | 2.21.0 | Amazon S3 connector for Hadoop ecosystem applications. | | flink-client | 1.4.0 | Apache Flink command line client scripts and applications. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 2.8.3-amzn-0 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 2.8.3-amzn-0 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 2.8.3-amzn-0 | HDFS command-line client and library | | hadoop-hdfs-namenode | 2.8.3-amzn-0 | HDFS service for tracking file names and block locations. | | hadoop-httpfs-server | 2.8.3-amzn-0 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 2.8.3-amzn-0 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 2.8.3-amzn-0 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 2.8.3-amzn-0 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 2.8.3-amzn-0 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 2.8.3-amzn-0 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 1.4.0 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 1.4.0 | Service for serving one or more HBase regions. | | hbase-client | 1.4.0 | HBase command-line client. | | hbase-rest-server | 1.4.0 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 1.4.0 | Service providing a Thrift endpoint to HBase. | | hcatalog-client | 2.3.2-amzn-1 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 2.3.2-amzn-1 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 2.3.2-amzn-1 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 2.3.2-amzn-1 | Hive command line client. | | hive-hbase | 2.3.2-amzn-1 | Hive-hbase client. | | hive-metastore-server | 2.3.2-amzn-1 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server2 | 2.3.2-amzn-1 | Service for accepting Hive queries as web requests. | | hue-server | 4.1.0 | Web application for analyzing data using Hadoop ecosystem applications | | livy-server | 0.4.0-incubating | REST interface for interacting with Apache Spark | | mahout-client | 0.13.0 | Library for machine learning. | | mxnet | 1.0.0 | A flexible, scalable, and efficient library for deep learning. | | mysql-server | 5.5.54\$1 | MySQL database server. | | nvidia-cuda | 9.1.85 | Nvidia drivers and Cuda toolkit | | oozie-client | 4.3.0 | Oozie command-line client. | | oozie-server | 4.3.0 | Service for accepting Oozie workflow requests. | | phoenix-library | 4.13.0-HBase-1.4 | The phoenix libraries for server and client | | phoenix-query-server | 4.13.0-HBase-1.4 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.188 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.188 | Service for executing pieces of a query. | | pig-client | 0.17.0 | Pig command-line client. | | spark-client | 2.2.1 | Spark command-line clients. | | spark-history-server | 2.2.1 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 2.2.1 | In-memory execution engine for YARN. | | spark-yarn-slave | 2.2.1 | Apache Spark libraries needed by YARN slaves. | | sqoop-client | 1.4.6 | Apache Sqoop command-line client. | | tez-on-yarn | 0.8.4 | The tez YARN application and libraries. | | webserver | 2.4.25\$1 | Apache HTTP server. | | zeppelin-server | 0.7.3 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.4.10 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.4.10 | ZooKeeper command line client. | ## 5.12.3 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). **emr-5.12.3 classifications** | Classifications | Description | | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | | core-site | Change values in Hadoop's core-site.xml file. | | emrfs-site | Change EMRFS settings. | | flink-conf | Change flink-conf.yaml settings. | | flink-log4j | Change Flink log4j.properties settings. | | flink-log4j-yarn-session | Change Flink log4j-yarn-session.properties settings. | | flink-log4j-cli | Change Flink log4j-cli.properties settings. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | | hadoop-ssl-server | Change hadoop ssl server configuration | | hadoop-ssl-client | Change hadoop ssl client configuration | | hbase | Amazon EMR-curated settings for Apache HBase. | | hbase-env | Change values in HBase's environment. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbase.properties file. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | | hbase-site | Change values in HBase's hbase-site.xml file. | | hdfs-encryption-zones | Configure HDFS encryption zones. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | | hcatalog-env | Change values in HCatalog's environment. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | | hcatalog-webhcat-log4j2 | Change values in HCatalog WebHCat's log4j2.properties. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | | hive-beeline-log4j2 | Change values in Hive's beeline-log4j2.properties file. | | hive-parquet-logging | Change values in Hive's parquet-logging.properties file. | | hive-env | Change values in the Hive environment. | | hive-exec-log4j2 | Change values in Hive's hive-exec-log4j2.properties file. | | hive-llap-daemon-log4j2 | Change values in Hive's llap-daemon-log4j2.properties file. | | hive-log4j2 | Change values in Hive's hive-log4j2.properties file. | | hive-site | Change values in Hive's hive-site.xml file | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | | hue-ini | Change values in Hue's ini file | | httpfs-env | Change values in the HTTPFS environment. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | | livy-conf | Change values in Livy's livy.conf file. | | livy-env | Change values in the Livy environment. | | livy-log4j | Change Livy log4j.properties settings. | | mapred-env | Change values in the MapReduce application's environment. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | | oozie-env | Change values in Oozie's environment. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | | oozie-site | Change values in Oozie's oozie-site.xml file. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | | pig-env | Change values in the Pig environment. | | pig-properties | Change values in Pig's pig.properties file. | | pig-log4j | Change values in Pig's log4j.properties file. | | presto-log | Change values in Presto's log.properties file. | | presto-config | Change values in Presto's config.properties file. | | presto-env | Change values in Presto's presto-env.sh file. | | presto-node | Change values in Presto's node.properties file. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | | presto-connector-hive | Change values in Presto's hive.properties file. | | presto-connector-jmx | Change values in Presto's jmx.properties file. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | | presto-connector-localfile | Change values in Presto's localfile.properties file. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | | presto-connector-redis | Change values in Presto's redis.properties file. | | presto-connector-redshift | Change values in Presto's redshift.properties file. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | | spark | Amazon EMR-curated settings for Apache Spark. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | | spark-env | Change values in the Spark environment. | | spark-hive-site | Change values in Spark's hive-site.xml file | | spark-log4j | Change values in Spark's log4j.properties file. | | spark-metrics | Change values in Spark's metrics.properties file. | | sqoop-env | Change values in Sqoop's environment. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | | tez-site | Change values in Tez's tez-site.xml file. | | yarn-env | Change values in the YARN environment. | | yarn-site | Change values in YARN's yarn-site.xml file. | | zeppelin-env | Change values in the Zeppelin environment. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | # Amazon EMR release 5.12.2 ## 5.12.2 application versions This release includes the following applications: [https://flink.apache.org/](https://flink.apache.org/), [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [http://gethue.com/](http://gethue.com/), [https://livy.incubator.apache.org/](https://livy.incubator.apache.org/), [https://mxnet.incubator.apache.org/](https://mxnet.incubator.apache.org/), [http://mahout.apache.org/](http://mahout.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-5.12.2 | emr-5.12.1 | emr-5.12.0 | emr-5.11.4 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.11.267 | 1.11.267 | 1.11.267 | 1.11.238 | | Python | 2.7, 3.4 | 2.7, 3.4 | 2.7, 3.4 | 2.7, 3.4 | | Scala | 2.11.8 | 2.11.8 | 2.11.8 | 2.11.8 | | AmazonCloudWatchAgent | - | - | - | - | | Delta | - | - | - | - | | Flink | 1.4.0 | 1.4.0 | 1.4.0 | 1.3.2 | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 1.4.0 | 1.4.0 | 1.4.0 | 1.3.1 | | HCatalog | 2.3.2-amzn-1 | 2.3.2-amzn-1 | 2.3.2-amzn-1 | 2.3.2-amzn-0 | | Hadoop | 2.8.3-amzn-0 | 2.8.3-amzn-0 | 2.8.3-amzn-0 | 2.7.3-amzn-6 | | Hive | 2.3.2-amzn-1 | 2.3.2-amzn-1 | 2.3.2-amzn-1 | 2.3.2-amzn-0 | | Hudi | - | - | - | - | | Hue | 4.1.0 | 4.1.0 | 4.1.0 | 4.0.1 | | Iceberg | - | - | - | - | | JupyterEnterpriseGateway | - | - | - | - | | JupyterHub | - | - | - | - | | Livy | 0.4.0-incubating | 0.4.0-incubating | 0.4.0-incubating | 0.4.0-incubating | | MXNet | 1.0.0 | 1.0.0 | 1.0.0 | 0.12.0 | | Mahout | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | | Oozie | 4.3.0 | 4.3.0 | 4.3.0 | 4.3.0 | | Phoenix | 4.13.0-HBase-1.4 | 4.13.0-HBase-1.4 | 4.13.0-HBase-1.4 | 4.11.0-HBase-1.3 | | Pig | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | | Presto | 0.188 | 0.188 | 0.188 | 0.187 | | Spark | 2.2.1 | 2.2.1 | 2.2.1 | 2.2.1 | | Sqoop | 1.4.6 | 1.4.6 | 1.4.6 | 1.4.6 | | TensorFlow | - | - | - | - | | Tez | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | | Trino (PrestoSQL) | - | - | - | - | | Zeppelin | 0.7.3 | 0.7.3 | 0.7.3 | 0.7.3 | | ZooKeeper | 3.4.10 | 3.4.10 | 3.4.10 | 3.4.10 | ## 5.12.2 release notes The following release notes include information for Amazon EMR release 5.12.2. Changes are relative to 5.12.1. Initial release date: August 29, 2018 **Changes, enhancements, and resolved issues** + This release addresses a potential security vulnerability. ## 5.12.2 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | aws-sagemaker-spark-sdk | 1.0.1 | Amazon SageMaker Spark SDK | | emr-ddb | 4.5.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 2.4.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.4.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-s3-dist-cp | 2.9.0 | Distributed copy application optimized for Amazon S3. | | emrfs | 2.21.0 | Amazon S3 connector for Hadoop ecosystem applications. | | flink-client | 1.4.0 | Apache Flink command line client scripts and applications. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 2.8.3-amzn-0 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 2.8.3-amzn-0 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 2.8.3-amzn-0 | HDFS command-line client and library | | hadoop-hdfs-namenode | 2.8.3-amzn-0 | HDFS service for tracking file names and block locations. | | hadoop-httpfs-server | 2.8.3-amzn-0 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 2.8.3-amzn-0 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 2.8.3-amzn-0 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 2.8.3-amzn-0 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 2.8.3-amzn-0 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 2.8.3-amzn-0 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 1.4.0 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 1.4.0 | Service for serving one or more HBase regions. | | hbase-client | 1.4.0 | HBase command-line client. | | hbase-rest-server | 1.4.0 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 1.4.0 | Service providing a Thrift endpoint to HBase. | | hcatalog-client | 2.3.2-amzn-1 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 2.3.2-amzn-1 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 2.3.2-amzn-1 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 2.3.2-amzn-1 | Hive command line client. | | hive-hbase | 2.3.2-amzn-1 | Hive-hbase client. | | hive-metastore-server | 2.3.2-amzn-1 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server2 | 2.3.2-amzn-1 | Service for accepting Hive queries as web requests. | | hue-server | 4.1.0 | Web application for analyzing data using Hadoop ecosystem applications | | livy-server | 0.4.0-incubating | REST interface for interacting with Apache Spark | | mahout-client | 0.13.0 | Library for machine learning. | | mxnet | 1.0.0 | A flexible, scalable, and efficient library for deep learning. | | mysql-server | 5.5.54\$1 | MySQL database server. | | nvidia-cuda | 9.1.85 | Nvidia drivers and Cuda toolkit | | oozie-client | 4.3.0 | Oozie command-line client. | | oozie-server | 4.3.0 | Service for accepting Oozie workflow requests. | | phoenix-library | 4.13.0-HBase-1.4 | The phoenix libraries for server and client | | phoenix-query-server | 4.13.0-HBase-1.4 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.188 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.188 | Service for executing pieces of a query. | | pig-client | 0.17.0 | Pig command-line client. | | spark-client | 2.2.1 | Spark command-line clients. | | spark-history-server | 2.2.1 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 2.2.1 | In-memory execution engine for YARN. | | spark-yarn-slave | 2.2.1 | Apache Spark libraries needed by YARN slaves. | | sqoop-client | 1.4.6 | Apache Sqoop command-line client. | | tez-on-yarn | 0.8.4 | The tez YARN application and libraries. | | webserver | 2.4.25\$1 | Apache HTTP server. | | zeppelin-server | 0.7.3 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.4.10 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.4.10 | ZooKeeper command line client. | ## 5.12.2 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). **emr-5.12.2 classifications** | Classifications | Description | | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | | core-site | Change values in Hadoop's core-site.xml file. | | emrfs-site | Change EMRFS settings. | | flink-conf | Change flink-conf.yaml settings. | | flink-log4j | Change Flink log4j.properties settings. | | flink-log4j-yarn-session | Change Flink log4j-yarn-session.properties settings. | | flink-log4j-cli | Change Flink log4j-cli.properties settings. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | | hadoop-ssl-server | Change hadoop ssl server configuration | | hadoop-ssl-client | Change hadoop ssl client configuration | | hbase | Amazon EMR-curated settings for Apache HBase. | | hbase-env | Change values in HBase's environment. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbase.properties file. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | | hbase-site | Change values in HBase's hbase-site.xml file. | | hdfs-encryption-zones | Configure HDFS encryption zones. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | | hcatalog-env | Change values in HCatalog's environment. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | | hcatalog-webhcat-log4j2 | Change values in HCatalog WebHCat's log4j2.properties. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | | hive-beeline-log4j2 | Change values in Hive's beeline-log4j2.properties file. | | hive-parquet-logging | Change values in Hive's parquet-logging.properties file. | | hive-env | Change values in the Hive environment. | | hive-exec-log4j2 | Change values in Hive's hive-exec-log4j2.properties file. | | hive-llap-daemon-log4j2 | Change values in Hive's llap-daemon-log4j2.properties file. | | hive-log4j2 | Change values in Hive's hive-log4j2.properties file. | | hive-site | Change values in Hive's hive-site.xml file | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | | hue-ini | Change values in Hue's ini file | | httpfs-env | Change values in the HTTPFS environment. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | | livy-conf | Change values in Livy's livy.conf file. | | livy-env | Change values in the Livy environment. | | livy-log4j | Change Livy log4j.properties settings. | | mapred-env | Change values in the MapReduce application's environment. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | | oozie-env | Change values in Oozie's environment. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | | oozie-site | Change values in Oozie's oozie-site.xml file. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | | pig-env | Change values in the Pig environment. | | pig-properties | Change values in Pig's pig.properties file. | | pig-log4j | Change values in Pig's log4j.properties file. | | presto-log | Change values in Presto's log.properties file. | | presto-config | Change values in Presto's config.properties file. | | presto-env | Change values in Presto's presto-env.sh file. | | presto-node | Change values in Presto's node.properties file. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | | presto-connector-hive | Change values in Presto's hive.properties file. | | presto-connector-jmx | Change values in Presto's jmx.properties file. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | | presto-connector-localfile | Change values in Presto's localfile.properties file. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | | presto-connector-redis | Change values in Presto's redis.properties file. | | presto-connector-redshift | Change values in Presto's redshift.properties file. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | | spark | Amazon EMR-curated settings for Apache Spark. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | | spark-env | Change values in the Spark environment. | | spark-hive-site | Change values in Spark's hive-site.xml file | | spark-log4j | Change values in Spark's log4j.properties file. | | spark-metrics | Change values in Spark's metrics.properties file. | | sqoop-env | Change values in Sqoop's environment. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | | tez-site | Change values in Tez's tez-site.xml file. | | yarn-env | Change values in the YARN environment. | | yarn-site | Change values in YARN's yarn-site.xml file. | | zeppelin-env | Change values in the Zeppelin environment. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | # Amazon EMR release 5.12.1 ## 5.12.1 application versions This release includes the following applications: [https://flink.apache.org/](https://flink.apache.org/), [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [http://gethue.com/](http://gethue.com/), [https://livy.incubator.apache.org/](https://livy.incubator.apache.org/), [https://mxnet.incubator.apache.org/](https://mxnet.incubator.apache.org/), [http://mahout.apache.org/](http://mahout.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-5.12.1 | emr-5.12.0 | emr-5.11.4 | emr-5.11.3 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.11.267 | 1.11.267 | 1.11.238 | 1.11.238 | | Python | 2.7, 3.4 | 2.7, 3.4 | 2.7, 3.4 | 2.7, 3.4 | | Scala | 2.11.8 | 2.11.8 | 2.11.8 | 2.11.8 | | AmazonCloudWatchAgent | - | - | - | - | | Delta | - | - | - | - | | Flink | 1.4.0 | 1.4.0 | 1.3.2 | 1.3.2 | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 1.4.0 | 1.4.0 | 1.3.1 | 1.3.1 | | HCatalog | 2.3.2-amzn-1 | 2.3.2-amzn-1 | 2.3.2-amzn-0 | 2.3.2-amzn-0 | | Hadoop | 2.8.3-amzn-0 | 2.8.3-amzn-0 | 2.7.3-amzn-6 | 2.7.3-amzn-6 | | Hive | 2.3.2-amzn-1 | 2.3.2-amzn-1 | 2.3.2-amzn-0 | 2.3.2-amzn-0 | | Hudi | - | - | - | - | | Hue | 4.1.0 | 4.1.0 | 4.0.1 | 4.0.1 | | Iceberg | - | - | - | - | | JupyterEnterpriseGateway | - | - | - | - | | JupyterHub | - | - | - | - | | Livy | 0.4.0-incubating | 0.4.0-incubating | 0.4.0-incubating | 0.4.0-incubating | | MXNet | 1.0.0 | 1.0.0 | 0.12.0 | 0.12.0 | | Mahout | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | | Oozie | 4.3.0 | 4.3.0 | 4.3.0 | 4.3.0 | | Phoenix | 4.13.0-HBase-1.4 | 4.13.0-HBase-1.4 | 4.11.0-HBase-1.3 | 4.11.0-HBase-1.3 | | Pig | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | | Presto | 0.188 | 0.188 | 0.187 | 0.187 | | Spark | 2.2.1 | 2.2.1 | 2.2.1 | 2.2.1 | | Sqoop | 1.4.6 | 1.4.6 | 1.4.6 | 1.4.6 | | TensorFlow | - | - | - | - | | Tez | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | | Trino (PrestoSQL) | - | - | - | - | | Zeppelin | 0.7.3 | 0.7.3 | 0.7.3 | 0.7.3 | | ZooKeeper | 3.4.10 | 3.4.10 | 3.4.10 | 3.4.10 | ## 5.12.1 release notes The following release notes include information for Amazon EMR release 5.12.1. Changes are relative to 5.12.0. Initial release date: March 29, 2018 **Changes, enhancements, and resolved issues** + Updated the Amazon Linux kernel of the defaultAmazon Linux AMI for Amazon EMR to address potential vulnerabilities. ## 5.12.1 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | aws-sagemaker-spark-sdk | 1.0.1 | Amazon SageMaker Spark SDK | | emr-ddb | 4.5.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 2.4.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.4.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-s3-dist-cp | 2.9.0 | Distributed copy application optimized for Amazon S3. | | emrfs | 2.21.0 | Amazon S3 connector for Hadoop ecosystem applications. | | flink-client | 1.4.0 | Apache Flink command line client scripts and applications. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 2.8.3-amzn-0 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 2.8.3-amzn-0 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 2.8.3-amzn-0 | HDFS command-line client and library | | hadoop-hdfs-namenode | 2.8.3-amzn-0 | HDFS service for tracking file names and block locations. | | hadoop-httpfs-server | 2.8.3-amzn-0 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 2.8.3-amzn-0 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 2.8.3-amzn-0 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 2.8.3-amzn-0 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 2.8.3-amzn-0 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 2.8.3-amzn-0 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 1.4.0 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 1.4.0 | Service for serving one or more HBase regions. | | hbase-client | 1.4.0 | HBase command-line client. | | hbase-rest-server | 1.4.0 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 1.4.0 | Service providing a Thrift endpoint to HBase. | | hcatalog-client | 2.3.2-amzn-1 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 2.3.2-amzn-1 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 2.3.2-amzn-1 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 2.3.2-amzn-1 | Hive command line client. | | hive-hbase | 2.3.2-amzn-1 | Hive-hbase client. | | hive-metastore-server | 2.3.2-amzn-1 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server2 | 2.3.2-amzn-1 | Service for accepting Hive queries as web requests. | | hue-server | 4.1.0 | Web application for analyzing data using Hadoop ecosystem applications | | livy-server | 0.4.0-incubating | REST interface for interacting with Apache Spark | | mahout-client | 0.13.0 | Library for machine learning. | | mxnet | 1.0.0 | A flexible, scalable, and efficient library for deep learning. | | mysql-server | 5.5.54\$1 | MySQL database server. | | nvidia-cuda | 9.1.85 | Nvidia drivers and Cuda toolkit | | oozie-client | 4.3.0 | Oozie command-line client. | | oozie-server | 4.3.0 | Service for accepting Oozie workflow requests. | | phoenix-library | 4.13.0-HBase-1.4 | The phoenix libraries for server and client | | phoenix-query-server | 4.13.0-HBase-1.4 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.188 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.188 | Service for executing pieces of a query. | | pig-client | 0.17.0 | Pig command-line client. | | spark-client | 2.2.1 | Spark command-line clients. | | spark-history-server | 2.2.1 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 2.2.1 | In-memory execution engine for YARN. | | spark-yarn-slave | 2.2.1 | Apache Spark libraries needed by YARN slaves. | | sqoop-client | 1.4.6 | Apache Sqoop command-line client. | | tez-on-yarn | 0.8.4 | The tez YARN application and libraries. | | webserver | 2.4.25\$1 | Apache HTTP server. | | zeppelin-server | 0.7.3 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.4.10 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.4.10 | ZooKeeper command line client. | ## 5.12.1 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). **emr-5.12.1 classifications** | Classifications | Description | | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | | core-site | Change values in Hadoop's core-site.xml file. | | emrfs-site | Change EMRFS settings. | | flink-conf | Change flink-conf.yaml settings. | | flink-log4j | Change Flink log4j.properties settings. | | flink-log4j-yarn-session | Change Flink log4j-yarn-session.properties settings. | | flink-log4j-cli | Change Flink log4j-cli.properties settings. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | | hadoop-ssl-server | Change hadoop ssl server configuration | | hadoop-ssl-client | Change hadoop ssl client configuration | | hbase | Amazon EMR-curated settings for Apache HBase. | | hbase-env | Change values in HBase's environment. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbase.properties file. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | | hbase-site | Change values in HBase's hbase-site.xml file. | | hdfs-encryption-zones | Configure HDFS encryption zones. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | | hcatalog-env | Change values in HCatalog's environment. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | | hcatalog-webhcat-log4j2 | Change values in HCatalog WebHCat's log4j2.properties. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | | hive-beeline-log4j2 | Change values in Hive's beeline-log4j2.properties file. | | hive-parquet-logging | Change values in Hive's parquet-logging.properties file. | | hive-env | Change values in the Hive environment. | | hive-exec-log4j2 | Change values in Hive's hive-exec-log4j2.properties file. | | hive-llap-daemon-log4j2 | Change values in Hive's llap-daemon-log4j2.properties file. | | hive-log4j2 | Change values in Hive's hive-log4j2.properties file. | | hive-site | Change values in Hive's hive-site.xml file | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | | hue-ini | Change values in Hue's ini file | | httpfs-env | Change values in the HTTPFS environment. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | | livy-conf | Change values in Livy's livy.conf file. | | livy-env | Change values in the Livy environment. | | livy-log4j | Change Livy log4j.properties settings. | | mapred-env | Change values in the MapReduce application's environment. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | | oozie-env | Change values in Oozie's environment. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | | oozie-site | Change values in Oozie's oozie-site.xml file. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | | pig-env | Change values in the Pig environment. | | pig-properties | Change values in Pig's pig.properties file. | | pig-log4j | Change values in Pig's log4j.properties file. | | presto-log | Change values in Presto's log.properties file. | | presto-config | Change values in Presto's config.properties file. | | presto-env | Change values in Presto's presto-env.sh file. | | presto-node | Change values in Presto's node.properties file. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | | presto-connector-hive | Change values in Presto's hive.properties file. | | presto-connector-jmx | Change values in Presto's jmx.properties file. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | | presto-connector-localfile | Change values in Presto's localfile.properties file. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | | presto-connector-redis | Change values in Presto's redis.properties file. | | presto-connector-redshift | Change values in Presto's redshift.properties file. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | | spark | Amazon EMR-curated settings for Apache Spark. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | | spark-env | Change values in the Spark environment. | | spark-hive-site | Change values in Spark's hive-site.xml file | | spark-log4j | Change values in Spark's log4j.properties file. | | spark-metrics | Change values in Spark's metrics.properties file. | | sqoop-env | Change values in Sqoop's environment. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | | tez-site | Change values in Tez's tez-site.xml file. | | yarn-env | Change values in the YARN environment. | | yarn-site | Change values in YARN's yarn-site.xml file. | | zeppelin-env | Change values in the Zeppelin environment. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | # Amazon EMR release 5.12.0 ## 5.12.0 application versions This release includes the following applications: [https://flink.apache.org/](https://flink.apache.org/), [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [http://gethue.com/](http://gethue.com/), [https://livy.incubator.apache.org/](https://livy.incubator.apache.org/), [https://mxnet.incubator.apache.org/](https://mxnet.incubator.apache.org/), [http://mahout.apache.org/](http://mahout.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-5.12.0 | emr-5.11.4 | emr-5.11.3 | emr-5.11.2 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.11.267 | 1.11.238 | 1.11.238 | 1.11.238 | | Python | 2.7, 3.4 | 2.7, 3.4 | 2.7, 3.4 | 2.7, 3.4 | | Scala | 2.11.8 | 2.11.8 | 2.11.8 | 2.11.8 | | AmazonCloudWatchAgent | - | - | - | - | | Delta | - | - | - | - | | Flink | 1.4.0 | 1.3.2 | 1.3.2 | 1.3.2 | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 1.4.0 | 1.3.1 | 1.3.1 | 1.3.1 | | HCatalog | 2.3.2-amzn-1 | 2.3.2-amzn-0 | 2.3.2-amzn-0 | 2.3.2-amzn-0 | | Hadoop | 2.8.3-amzn-0 | 2.7.3-amzn-6 | 2.7.3-amzn-6 | 2.7.3-amzn-6 | | Hive | 2.3.2-amzn-1 | 2.3.2-amzn-0 | 2.3.2-amzn-0 | 2.3.2-amzn-0 | | Hudi | - | - | - | - | | Hue | 4.1.0 | 4.0.1 | 4.0.1 | 4.0.1 | | Iceberg | - | - | - | - | | JupyterEnterpriseGateway | - | - | - | - | | JupyterHub | - | - | - | - | | Livy | 0.4.0-incubating | 0.4.0-incubating | 0.4.0-incubating | 0.4.0-incubating | | MXNet | 1.0.0 | 0.12.0 | 0.12.0 | 0.12.0 | | Mahout | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | | Oozie | 4.3.0 | 4.3.0 | 4.3.0 | 4.3.0 | | Phoenix | 4.13.0-HBase-1.4 | 4.11.0-HBase-1.3 | 4.11.0-HBase-1.3 | 4.11.0-HBase-1.3 | | Pig | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | | Presto | 0.188 | 0.187 | 0.187 | 0.187 | | Spark | 2.2.1 | 2.2.1 | 2.2.1 | 2.2.1 | | Sqoop | 1.4.6 | 1.4.6 | 1.4.6 | 1.4.6 | | TensorFlow | - | - | - | - | | Tez | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | | Trino (PrestoSQL) | - | - | - | - | | Zeppelin | 0.7.3 | 0.7.3 | 0.7.3 | 0.7.3 | | ZooKeeper | 3.4.10 | 3.4.10 | 3.4.10 | 3.4.10 | ## 5.12.0 release notes The following release notes include information for the Amazon EMR release 5.12.0. Changes are relative to 5.11.1. **Upgrades** + AWS SDK for Java 1.11.238 ⇒ 1.11.267. For more information, see the [AWS SDK for Java Change Log](https://github.com/aws/aws-sdk-java/blob/master/CHANGELOG.md) on GitHub. + Hadoop 2.7.3 ⇒ 2.8.3. For more information, see [Apache Hadoop Releases](http://hadoop.apache.org/releases.html). + Flink 1.3.2 ⇒ 1.4.0. For more information, see the [Apache Flink 1.4.0 Release Announcement](https://flink.apache.org/news/2017/12/12/release-1.4.0.html). + HBase 1.3.1 ⇒ 1.4.0. For more information, see the [HBase Release Announcement](http://mail-archives.apache.org/mod_mbox/www-announce/201712.mbox/%3CCA+RK=_AU+tB=7SU1HRbeKVEd-sKA5WcJo3oa43vQ6PMB3L9pgQ@mail.gmail.com%3E). + Hue 4.0.1 ⇒ 4.1.0. For more information, see the [Release Notes](https://docs.gethue.com/releases/release-notes-4.10.0/). + MxNet 0.12.0 ⇒ 1.0.0. For more information, see the [MXNet Change Log](https://github.com/apache/incubator-mxnet/releases/tag/1.0.0) on GitHub. + Presto 0.187 ⇒ 0.188. For more information, see the [Release Notes](https://prestodb.io/docs/current/release/release-0.188.html). **Changes, enhancements, and resolved issues** + **Hadoop** + The `yarn.resourcemanager.decommissioning.timeout` property has changed to `yarn.resourcemanager.nodemanager-graceful-decommission-timeout-secs`. You can use this property to customize cluster scale-down. For more information, see [Cluster Scale-Down](https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-scaledown-behavior.html) in the *Amazon EMR Management Guide*. + The Hadoop CLI added the `-d` option to the `cp` (copy) command, which specifies direct copy. You can use this to avoid creating an intermediary `.COPYING` file, which makes copying data between Amazon S3 faster. For more information, see [HADOOP-12384](https://issues.apache.org/jira/browse/HADOOP-12384). + **Pig** + Added the `pig-env` configuration classification, which simplifies the configuration of Pig environment properties. For more information, see [Configure applications](emr-configure-apps.md). + **Presto** + Added the `presto-connector-redshift` configuration classification, which you can use to configure values in the Presto `redshift.properties` configuration file. For more information, see [Redshift Connector](https://prestodb.io/docs/current/connector/redshift.html) in Presto documentation, and [Configure applications](emr-configure-apps.md). + Presto support for EMRFS has been added and is the default configuration. Earlier Amazon EMR releases used PrestoS3FileSystem, which was the only option. For more information, see [EMRFS and PrestoS3FileSystem configuration](emr-presto-considerations.md#emr-presto-prestos3). **Note** If you query underlying data in Amazon S3 with Amazon EMR version 5.12.0, Presto errors can occur. This is because Presto fails to pick up configuration classification values from `emrfs-site.xml`. As a workaround, create an `emrfs` subdirectory under `usr/lib/presto/plugin/hive-hadoop2/` and create a symlink in `usr/lib/presto/plugin/hive-hadoop2/emrfs` to the existing `/usr/share/aws/emr/emrfs/conf/emrfs-site.xml` file. Then restart the presto-server process (`sudo presto-server stop` followed by `sudo presto-server start`). + **Spark** + Backported [SPARK-22036: BigDecimal multiplication sometimes returns null](https://issues.apache.org/jira/browse/SPARK-22036). **Known issues** + MXNet does not include OpenCV libraries. + SparkR is not available for clusters created using a custom AMI because R is not installed by default on cluster nodes. ## 5.12.0 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | aws-sagemaker-spark-sdk | 1.0.1 | Amazon SageMaker Spark SDK | | emr-ddb | 4.5.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 2.4.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.4.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-s3-dist-cp | 2.9.0 | Distributed copy application optimized for Amazon S3. | | emrfs | 2.21.0 | Amazon S3 connector for Hadoop ecosystem applications. | | flink-client | 1.4.0 | Apache Flink command line client scripts and applications. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 2.8.3-amzn-0 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 2.8.3-amzn-0 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 2.8.3-amzn-0 | HDFS command-line client and library | | hadoop-hdfs-namenode | 2.8.3-amzn-0 | HDFS service for tracking file names and block locations. | | hadoop-httpfs-server | 2.8.3-amzn-0 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 2.8.3-amzn-0 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 2.8.3-amzn-0 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 2.8.3-amzn-0 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 2.8.3-amzn-0 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 2.8.3-amzn-0 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 1.4.0 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 1.4.0 | Service for serving one or more HBase regions. | | hbase-client | 1.4.0 | HBase command-line client. | | hbase-rest-server | 1.4.0 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 1.4.0 | Service providing a Thrift endpoint to HBase. | | hcatalog-client | 2.3.2-amzn-1 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 2.3.2-amzn-1 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 2.3.2-amzn-1 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 2.3.2-amzn-1 | Hive command line client. | | hive-hbase | 2.3.2-amzn-1 | Hive-hbase client. | | hive-metastore-server | 2.3.2-amzn-1 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server2 | 2.3.2-amzn-1 | Service for accepting Hive queries as web requests. | | hue-server | 4.1.0 | Web application for analyzing data using Hadoop ecosystem applications | | livy-server | 0.4.0-incubating | REST interface for interacting with Apache Spark | | mahout-client | 0.13.0 | Library for machine learning. | | mxnet | 1.0.0 | A flexible, scalable, and efficient library for deep learning. | | mysql-server | 5.5.54\$1 | MySQL database server. | | nvidia-cuda | 9.1.85 | Nvidia drivers and Cuda toolkit | | oozie-client | 4.3.0 | Oozie command-line client. | | oozie-server | 4.3.0 | Service for accepting Oozie workflow requests. | | phoenix-library | 4.13.0-HBase-1.4 | The phoenix libraries for server and client | | phoenix-query-server | 4.13.0-HBase-1.4 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.188 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.188 | Service for executing pieces of a query. | | pig-client | 0.17.0 | Pig command-line client. | | spark-client | 2.2.1 | Spark command-line clients. | | spark-history-server | 2.2.1 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 2.2.1 | In-memory execution engine for YARN. | | spark-yarn-slave | 2.2.1 | Apache Spark libraries needed by YARN slaves. | | sqoop-client | 1.4.6 | Apache Sqoop command-line client. | | tez-on-yarn | 0.8.4 | The tez YARN application and libraries. | | webserver | 2.4.25\$1 | Apache HTTP server. | | zeppelin-server | 0.7.3 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.4.10 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.4.10 | ZooKeeper command line client. | ## 5.12.0 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). **emr-5.12.0 classifications** | Classifications | Description | | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | | core-site | Change values in Hadoop's core-site.xml file. | | emrfs-site | Change EMRFS settings. | | flink-conf | Change flink-conf.yaml settings. | | flink-log4j | Change Flink log4j.properties settings. | | flink-log4j-yarn-session | Change Flink log4j-yarn-session.properties settings. | | flink-log4j-cli | Change Flink log4j-cli.properties settings. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | | hadoop-ssl-server | Change hadoop ssl server configuration | | hadoop-ssl-client | Change hadoop ssl client configuration | | hbase | Amazon EMR-curated settings for Apache HBase. | | hbase-env | Change values in HBase's environment. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbase.properties file. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | | hbase-site | Change values in HBase's hbase-site.xml file. | | hdfs-encryption-zones | Configure HDFS encryption zones. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | | hcatalog-env | Change values in HCatalog's environment. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | | hcatalog-webhcat-log4j2 | Change values in HCatalog WebHCat's log4j2.properties. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | | hive-beeline-log4j2 | Change values in Hive's beeline-log4j2.properties file. | | hive-parquet-logging | Change values in Hive's parquet-logging.properties file. | | hive-env | Change values in the Hive environment. | | hive-exec-log4j2 | Change values in Hive's hive-exec-log4j2.properties file. | | hive-llap-daemon-log4j2 | Change values in Hive's llap-daemon-log4j2.properties file. | | hive-log4j2 | Change values in Hive's hive-log4j2.properties file. | | hive-site | Change values in Hive's hive-site.xml file | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | | hue-ini | Change values in Hue's ini file | | httpfs-env | Change values in the HTTPFS environment. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | | livy-conf | Change values in Livy's livy.conf file. | | livy-env | Change values in the Livy environment. | | livy-log4j | Change Livy log4j.properties settings. | | mapred-env | Change values in the MapReduce application's environment. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | | oozie-env | Change values in Oozie's environment. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | | oozie-site | Change values in Oozie's oozie-site.xml file. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | | pig-env | Change values in the Pig environment. | | pig-properties | Change values in Pig's pig.properties file. | | pig-log4j | Change values in Pig's log4j.properties file. | | presto-log | Change values in Presto's log.properties file. | | presto-config | Change values in Presto's config.properties file. | | presto-env | Change values in Presto's presto-env.sh file. | | presto-node | Change values in Presto's node.properties file. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | | presto-connector-hive | Change values in Presto's hive.properties file. | | presto-connector-jmx | Change values in Presto's jmx.properties file. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | | presto-connector-localfile | Change values in Presto's localfile.properties file. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | | presto-connector-redis | Change values in Presto's redis.properties file. | | presto-connector-redshift | Change values in Presto's redshift.properties file. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | | spark | Amazon EMR-curated settings for Apache Spark. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | | spark-env | Change values in the Spark environment. | | spark-hive-site | Change values in Spark's hive-site.xml file | | spark-log4j | Change values in Spark's log4j.properties file. | | spark-metrics | Change values in Spark's metrics.properties file. | | sqoop-env | Change values in Sqoop's environment. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | | tez-site | Change values in Tez's tez-site.xml file. | | yarn-env | Change values in the YARN environment. | | yarn-site | Change values in YARN's yarn-site.xml file. | | zeppelin-env | Change values in the Zeppelin environment. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | # Amazon EMR release 5.11.4 ## 5.11.4 application versions This release includes the following applications: [https://flink.apache.org/](https://flink.apache.org/), [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [http://gethue.com/](http://gethue.com/), [https://livy.incubator.apache.org/](https://livy.incubator.apache.org/), [https://mxnet.incubator.apache.org/](https://mxnet.incubator.apache.org/), [http://mahout.apache.org/](http://mahout.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-5.11.4 | emr-5.11.3 | emr-5.11.2 | emr-5.11.1 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.11.238 | 1.11.238 | 1.11.238 | 1.11.238 | | Python | 2.7, 3.4 | 2.7, 3.4 | 2.7, 3.4 | 2.7, 3.4 | | Scala | 2.11.8 | 2.11.8 | 2.11.8 | 2.11.8 | | AmazonCloudWatchAgent | - | - | - | - | | Delta | - | - | - | - | | Flink | 1.3.2 | 1.3.2 | 1.3.2 | 1.3.2 | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 1.3.1 | 1.3.1 | 1.3.1 | 1.3.1 | | HCatalog | 2.3.2-amzn-0 | 2.3.2-amzn-0 | 2.3.2-amzn-0 | 2.3.2-amzn-0 | | Hadoop | 2.7.3-amzn-6 | 2.7.3-amzn-6 | 2.7.3-amzn-6 | 2.7.3-amzn-6 | | Hive | 2.3.2-amzn-0 | 2.3.2-amzn-0 | 2.3.2-amzn-0 | 2.3.2-amzn-0 | | Hudi | - | - | - | - | | Hue | 4.0.1 | 4.0.1 | 4.0.1 | 4.0.1 | | Iceberg | - | - | - | - | | JupyterEnterpriseGateway | - | - | - | - | | JupyterHub | - | - | - | - | | Livy | 0.4.0-incubating | 0.4.0-incubating | 0.4.0-incubating | 0.4.0-incubating | | MXNet | 0.12.0 | 0.12.0 | 0.12.0 | 0.12.0 | | Mahout | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | | Oozie | 4.3.0 | 4.3.0 | 4.3.0 | 4.3.0 | | Phoenix | 4.11.0-HBase-1.3 | 4.11.0-HBase-1.3 | 4.11.0-HBase-1.3 | 4.11.0-HBase-1.3 | | Pig | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | | Presto | 0.187 | 0.187 | 0.187 | 0.187 | | Spark | 2.2.1 | 2.2.1 | 2.2.1 | 2.2.1 | | Sqoop | 1.4.6 | 1.4.6 | 1.4.6 | 1.4.6 | | TensorFlow | - | - | - | - | | Tez | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | | Trino (PrestoSQL) | - | - | - | - | | Zeppelin | 0.7.3 | 0.7.3 | 0.7.3 | 0.7.3 | | ZooKeeper | 3.4.10 | 3.4.10 | 3.4.10 | 3.4.10 | ## 5.11.4 release notes This is a patch release to add AWS Signature Version 4 authentication for requests to Amazon S3. All applications and components are the same as the previous Amazon EMR release. **Important** In this release version, Amazon EMR uses AWS Signature Version 4 exclusively to authenticate requests to Amazon S3. For more information, see [Whats New](https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-whatsnew.html). ## 5.11.4 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | aws-sagemaker-spark-sdk | 1.0 | Amazon SageMaker Spark SDK | | emr-ddb | 4.5.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 2.4.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.4.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-s3-dist-cp | 2.8.0 | Distributed copy application optimized for Amazon S3. | | emrfs | 2.20.0 | Amazon S3 connector for Hadoop ecosystem applications. | | flink-client | 1.3.2 | Apache Flink command line client scripts and applications. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 2.7.3-amzn-6 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 2.7.3-amzn-6 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 2.7.3-amzn-6 | HDFS command-line client and library | | hadoop-hdfs-namenode | 2.7.3-amzn-6 | HDFS service for tracking file names and block locations. | | hadoop-httpfs-server | 2.7.3-amzn-6 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 2.7.3-amzn-6 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 2.7.3-amzn-6 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 2.7.3-amzn-6 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 2.7.3-amzn-6 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 2.7.3-amzn-6 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 1.3.1 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 1.3.1 | Service for serving one or more HBase regions. | | hbase-client | 1.3.1 | HBase command-line client. | | hbase-rest-server | 1.3.1 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 1.3.1 | Service providing a Thrift endpoint to HBase. | | hcatalog-client | 2.3.2-amzn-0 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 2.3.2-amzn-0 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 2.3.2-amzn-0 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 2.3.2-amzn-0 | Hive command line client. | | hive-hbase | 2.3.2-amzn-0 | Hive-hbase client. | | hive-metastore-server | 2.3.2-amzn-0 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server2 | 2.3.2-amzn-0 | Service for accepting Hive queries as web requests. | | hue-server | 4.0.1 | Web application for analyzing data using Hadoop ecosystem applications | | livy-server | 0.4.0-incubating | REST interface for interacting with Apache Spark | | mahout-client | 0.13.0 | Library for machine learning. | | mxnet | 0.12.0 | A flexible, scalable, and efficient library for deep learning. | | mysql-server | 5.5.54\$1 | MySQL database server. | | nvidia-cuda | 9.0.176 | Nvidia drivers and Cuda toolkit | | oozie-client | 4.3.0 | Oozie command-line client. | | oozie-server | 4.3.0 | Service for accepting Oozie workflow requests. | | phoenix-library | 4.11.0-HBase-1.3 | The phoenix libraries for server and client | | phoenix-query-server | 4.11.0-HBase-1.3 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.187 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.187 | Service for executing pieces of a query. | | pig-client | 0.17.0 | Pig command-line client. | | spark-client | 2.2.1 | Spark command-line clients. | | spark-history-server | 2.2.1 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 2.2.1 | In-memory execution engine for YARN. | | spark-yarn-slave | 2.2.1 | Apache Spark libraries needed by YARN slaves. | | sqoop-client | 1.4.6 | Apache Sqoop command-line client. | | tez-on-yarn | 0.8.4 | The tez YARN application and libraries. | | webserver | 2.4.25\$1 | Apache HTTP server. | | zeppelin-server | 0.7.3 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.4.10 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.4.10 | ZooKeeper command line client. | ## 5.11.4 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). **emr-5.11.4 classifications** | Classifications | Description | | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | | core-site | Change values in Hadoop's core-site.xml file. | | emrfs-site | Change EMRFS settings. | | flink-conf | Change flink-conf.yaml settings. | | flink-log4j | Change Flink log4j.properties settings. | | flink-log4j-yarn-session | Change Flink log4j-yarn-session.properties settings. | | flink-log4j-cli | Change Flink log4j-cli.properties settings. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | | hadoop-ssl-server | Change hadoop ssl server configuration | | hadoop-ssl-client | Change hadoop ssl client configuration | | hbase | Amazon EMR-curated settings for Apache HBase. | | hbase-env | Change values in HBase's environment. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbase.properties file. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | | hbase-site | Change values in HBase's hbase-site.xml file. | | hdfs-encryption-zones | Configure HDFS encryption zones. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | | hcatalog-env | Change values in HCatalog's environment. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | | hcatalog-webhcat-log4j2 | Change values in HCatalog WebHCat's log4j2.properties. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | | hive-beeline-log4j2 | Change values in Hive's beeline-log4j2.properties file. | | hive-parquet-logging | Change values in Hive's parquet-logging.properties file. | | hive-env | Change values in the Hive environment. | | hive-exec-log4j2 | Change values in Hive's hive-exec-log4j2.properties file. | | hive-llap-daemon-log4j2 | Change values in Hive's llap-daemon-log4j2.properties file. | | hive-log4j2 | Change values in Hive's hive-log4j2.properties file. | | hive-site | Change values in Hive's hive-site.xml file | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | | hue-ini | Change values in Hue's ini file | | httpfs-env | Change values in the HTTPFS environment. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | | livy-conf | Change values in Livy's livy.conf file. | | livy-env | Change values in the Livy environment. | | livy-log4j | Change Livy log4j.properties settings. | | mapred-env | Change values in the MapReduce application's environment. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | | oozie-env | Change values in Oozie's environment. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | | oozie-site | Change values in Oozie's oozie-site.xml file. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | | pig-properties | Change values in Pig's pig.properties file. | | pig-log4j | Change values in Pig's log4j.properties file. | | presto-log | Change values in Presto's log.properties file. | | presto-config | Change values in Presto's config.properties file. | | presto-env | Change values in Presto's presto-env.sh file. | | presto-node | Change values in Presto's node.properties file. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | | presto-connector-hive | Change values in Presto's hive.properties file. | | presto-connector-jmx | Change values in Presto's jmx.properties file. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | | presto-connector-localfile | Change values in Presto's localfile.properties file. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | | presto-connector-redis | Change values in Presto's redis.properties file. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | | spark | Amazon EMR-curated settings for Apache Spark. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | | spark-env | Change values in the Spark environment. | | spark-hive-site | Change values in Spark's hive-site.xml file | | spark-log4j | Change values in Spark's log4j.properties file. | | spark-metrics | Change values in Spark's metrics.properties file. | | sqoop-env | Change values in Sqoop's environment. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | | tez-site | Change values in Tez's tez-site.xml file. | | yarn-env | Change values in the YARN environment. | | yarn-site | Change values in YARN's yarn-site.xml file. | | zeppelin-env | Change values in the Zeppelin environment. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | # Amazon EMR release 5.11.3 ## 5.11.3 application versions This release includes the following applications: [https://flink.apache.org/](https://flink.apache.org/), [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [http://gethue.com/](http://gethue.com/), [https://livy.incubator.apache.org/](https://livy.incubator.apache.org/), [https://mxnet.incubator.apache.org/](https://mxnet.incubator.apache.org/), [http://mahout.apache.org/](http://mahout.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-5.11.3 | emr-5.11.2 | emr-5.11.1 | emr-5.11.0 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.11.238 | 1.11.238 | 1.11.238 | 1.11.238 | | Python | 2.7, 3.4 | 2.7, 3.4 | 2.7, 3.4 | 2.7, 3.4 | | Scala | 2.11.8 | 2.11.8 | 2.11.8 | 2.11.8 | | AmazonCloudWatchAgent | - | - | - | - | | Delta | - | - | - | - | | Flink | 1.3.2 | 1.3.2 | 1.3.2 | 1.3.2 | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 1.3.1 | 1.3.1 | 1.3.1 | 1.3.1 | | HCatalog | 2.3.2-amzn-0 | 2.3.2-amzn-0 | 2.3.2-amzn-0 | 2.3.2-amzn-0 | | Hadoop | 2.7.3-amzn-6 | 2.7.3-amzn-6 | 2.7.3-amzn-6 | 2.7.3-amzn-6 | | Hive | 2.3.2-amzn-0 | 2.3.2-amzn-0 | 2.3.2-amzn-0 | 2.3.2-amzn-0 | | Hudi | - | - | - | - | | Hue | 4.0.1 | 4.0.1 | 4.0.1 | 4.0.1 | | Iceberg | - | - | - | - | | JupyterEnterpriseGateway | - | - | - | - | | JupyterHub | - | - | - | - | | Livy | 0.4.0-incubating | 0.4.0-incubating | 0.4.0-incubating | 0.4.0-incubating | | MXNet | 0.12.0 | 0.12.0 | 0.12.0 | 0.12.0 | | Mahout | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | | Oozie | 4.3.0 | 4.3.0 | 4.3.0 | 4.3.0 | | Phoenix | 4.11.0-HBase-1.3 | 4.11.0-HBase-1.3 | 4.11.0-HBase-1.3 | 4.11.0-HBase-1.3 | | Pig | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | | Presto | 0.187 | 0.187 | 0.187 | 0.187 | | Spark | 2.2.1 | 2.2.1 | 2.2.1 | 2.2.1 | | Sqoop | 1.4.6 | 1.4.6 | 1.4.6 | 1.4.6 | | TensorFlow | - | - | - | - | | Tez | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | | Trino (PrestoSQL) | - | - | - | - | | Zeppelin | 0.7.3 | 0.7.3 | 0.7.3 | 0.7.3 | | ZooKeeper | 3.4.10 | 3.4.10 | 3.4.10 | 3.4.10 | ## 5.11.3 release notes The following release notes include information for Amazon EMR release 5.11.3. Changes are relative to 5.11.2. Initial release date: July 18, 2019 **Changes, enhancements, and resolved issues** + Updated the default Amazon Linux AMI for Amazon EMR to include important Linux kernel security updates, including the TCP SACK Denial of Service Issue ([AWS-2019-005](https://aws.amazon.com/security/security-bulletins/AWS-2019-005/)). ## 5.11.3 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | aws-sagemaker-spark-sdk | 1.0 | Amazon SageMaker Spark SDK | | emr-ddb | 4.5.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 2.4.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.4.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-s3-dist-cp | 2.8.0 | Distributed copy application optimized for Amazon S3. | | emrfs | 2.20.0 | Amazon S3 connector for Hadoop ecosystem applications. | | flink-client | 1.3.2 | Apache Flink command line client scripts and applications. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 2.7.3-amzn-6 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 2.7.3-amzn-6 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 2.7.3-amzn-6 | HDFS command-line client and library | | hadoop-hdfs-namenode | 2.7.3-amzn-6 | HDFS service for tracking file names and block locations. | | hadoop-httpfs-server | 2.7.3-amzn-6 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 2.7.3-amzn-6 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 2.7.3-amzn-6 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 2.7.3-amzn-6 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 2.7.3-amzn-6 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 2.7.3-amzn-6 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 1.3.1 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 1.3.1 | Service for serving one or more HBase regions. | | hbase-client | 1.3.1 | HBase command-line client. | | hbase-rest-server | 1.3.1 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 1.3.1 | Service providing a Thrift endpoint to HBase. | | hcatalog-client | 2.3.2-amzn-0 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 2.3.2-amzn-0 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 2.3.2-amzn-0 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 2.3.2-amzn-0 | Hive command line client. | | hive-hbase | 2.3.2-amzn-0 | Hive-hbase client. | | hive-metastore-server | 2.3.2-amzn-0 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server2 | 2.3.2-amzn-0 | Service for accepting Hive queries as web requests. | | hue-server | 4.0.1 | Web application for analyzing data using Hadoop ecosystem applications | | livy-server | 0.4.0-incubating | REST interface for interacting with Apache Spark | | mahout-client | 0.13.0 | Library for machine learning. | | mxnet | 0.12.0 | A flexible, scalable, and efficient library for deep learning. | | mysql-server | 5.5.54\$1 | MySQL database server. | | nvidia-cuda | 9.0.176 | Nvidia drivers and Cuda toolkit | | oozie-client | 4.3.0 | Oozie command-line client. | | oozie-server | 4.3.0 | Service for accepting Oozie workflow requests. | | phoenix-library | 4.11.0-HBase-1.3 | The phoenix libraries for server and client | | phoenix-query-server | 4.11.0-HBase-1.3 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.187 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.187 | Service for executing pieces of a query. | | pig-client | 0.17.0 | Pig command-line client. | | spark-client | 2.2.1 | Spark command-line clients. | | spark-history-server | 2.2.1 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 2.2.1 | In-memory execution engine for YARN. | | spark-yarn-slave | 2.2.1 | Apache Spark libraries needed by YARN slaves. | | sqoop-client | 1.4.6 | Apache Sqoop command-line client. | | tez-on-yarn | 0.8.4 | The tez YARN application and libraries. | | webserver | 2.4.25\$1 | Apache HTTP server. | | zeppelin-server | 0.7.3 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.4.10 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.4.10 | ZooKeeper command line client. | ## 5.11.3 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). **emr-5.11.3 classifications** | Classifications | Description | | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | | core-site | Change values in Hadoop's core-site.xml file. | | emrfs-site | Change EMRFS settings. | | flink-conf | Change flink-conf.yaml settings. | | flink-log4j | Change Flink log4j.properties settings. | | flink-log4j-yarn-session | Change Flink log4j-yarn-session.properties settings. | | flink-log4j-cli | Change Flink log4j-cli.properties settings. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | | hadoop-ssl-server | Change hadoop ssl server configuration | | hadoop-ssl-client | Change hadoop ssl client configuration | | hbase | Amazon EMR-curated settings for Apache HBase. | | hbase-env | Change values in HBase's environment. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbase.properties file. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | | hbase-site | Change values in HBase's hbase-site.xml file. | | hdfs-encryption-zones | Configure HDFS encryption zones. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | | hcatalog-env | Change values in HCatalog's environment. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | | hcatalog-webhcat-log4j2 | Change values in HCatalog WebHCat's log4j2.properties. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | | hive-beeline-log4j2 | Change values in Hive's beeline-log4j2.properties file. | | hive-parquet-logging | Change values in Hive's parquet-logging.properties file. | | hive-env | Change values in the Hive environment. | | hive-exec-log4j2 | Change values in Hive's hive-exec-log4j2.properties file. | | hive-llap-daemon-log4j2 | Change values in Hive's llap-daemon-log4j2.properties file. | | hive-log4j2 | Change values in Hive's hive-log4j2.properties file. | | hive-site | Change values in Hive's hive-site.xml file | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | | hue-ini | Change values in Hue's ini file | | httpfs-env | Change values in the HTTPFS environment. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | | livy-conf | Change values in Livy's livy.conf file. | | livy-env | Change values in the Livy environment. | | livy-log4j | Change Livy log4j.properties settings. | | mapred-env | Change values in the MapReduce application's environment. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | | oozie-env | Change values in Oozie's environment. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | | oozie-site | Change values in Oozie's oozie-site.xml file. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | | pig-properties | Change values in Pig's pig.properties file. | | pig-log4j | Change values in Pig's log4j.properties file. | | presto-log | Change values in Presto's log.properties file. | | presto-config | Change values in Presto's config.properties file. | | presto-env | Change values in Presto's presto-env.sh file. | | presto-node | Change values in Presto's node.properties file. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | | presto-connector-hive | Change values in Presto's hive.properties file. | | presto-connector-jmx | Change values in Presto's jmx.properties file. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | | presto-connector-localfile | Change values in Presto's localfile.properties file. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | | presto-connector-redis | Change values in Presto's redis.properties file. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | | spark | Amazon EMR-curated settings for Apache Spark. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | | spark-env | Change values in the Spark environment. | | spark-hive-site | Change values in Spark's hive-site.xml file | | spark-log4j | Change values in Spark's log4j.properties file. | | spark-metrics | Change values in Spark's metrics.properties file. | | sqoop-env | Change values in Sqoop's environment. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | | tez-site | Change values in Tez's tez-site.xml file. | | yarn-env | Change values in the YARN environment. | | yarn-site | Change values in YARN's yarn-site.xml file. | | zeppelin-env | Change values in the Zeppelin environment. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | # Amazon EMR release 5.11.2 ## 5.11.2 application versions This release includes the following applications: [https://flink.apache.org/](https://flink.apache.org/), [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [http://gethue.com/](http://gethue.com/), [https://livy.incubator.apache.org/](https://livy.incubator.apache.org/), [https://mxnet.incubator.apache.org/](https://mxnet.incubator.apache.org/), [http://mahout.apache.org/](http://mahout.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-5.11.2 | emr-5.11.1 | emr-5.11.0 | emr-5.10.1 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.11.238 | 1.11.238 | 1.11.238 | 1.11.221 | | Python | 2.7, 3.4 | 2.7, 3.4 | 2.7, 3.4 | 2.7, 3.4 | | Scala | 2.11.8 | 2.11.8 | 2.11.8 | 2.11.8 | | AmazonCloudWatchAgent | - | - | - | - | | Delta | - | - | - | - | | Flink | 1.3.2 | 1.3.2 | 1.3.2 | 1.3.2 | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 1.3.1 | 1.3.1 | 1.3.1 | 1.3.1 | | HCatalog | 2.3.2-amzn-0 | 2.3.2-amzn-0 | 2.3.2-amzn-0 | 2.3.1-amzn-0 | | Hadoop | 2.7.3-amzn-6 | 2.7.3-amzn-6 | 2.7.3-amzn-6 | 2.7.3-amzn-5 | | Hive | 2.3.2-amzn-0 | 2.3.2-amzn-0 | 2.3.2-amzn-0 | 2.3.1-amzn-0 | | Hudi | - | - | - | - | | Hue | 4.0.1 | 4.0.1 | 4.0.1 | 4.0.1 | | Iceberg | - | - | - | - | | JupyterEnterpriseGateway | - | - | - | - | | JupyterHub | - | - | - | - | | Livy | 0.4.0-incubating | 0.4.0-incubating | 0.4.0-incubating | 0.4.0-incubating | | MXNet | 0.12.0 | 0.12.0 | 0.12.0 | 0.12.0 | | Mahout | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | | Oozie | 4.3.0 | 4.3.0 | 4.3.0 | 4.3.0 | | Phoenix | 4.11.0-HBase-1.3 | 4.11.0-HBase-1.3 | 4.11.0-HBase-1.3 | 4.11.0-HBase-1.3 | | Pig | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | | Presto | 0.187 | 0.187 | 0.187 | 0.187 | | Spark | 2.2.1 | 2.2.1 | 2.2.1 | 2.2.0 | | Sqoop | 1.4.6 | 1.4.6 | 1.4.6 | 1.4.6 | | TensorFlow | - | - | - | - | | Tez | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | | Trino (PrestoSQL) | - | - | - | - | | Zeppelin | 0.7.3 | 0.7.3 | 0.7.3 | 0.7.3 | | ZooKeeper | 3.4.10 | 3.4.10 | 3.4.10 | 3.4.10 | ## 5.11.2 release notes The following release notes include information for Amazon EMR release 5.11.2. Changes are relative to 5.11.1. Initial release date: August 29, 2018 **Changes, enhancements, and resolved issues** + This release addresses a potential security vulnerability. ## 5.11.2 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | aws-sagemaker-spark-sdk | 1.0 | Amazon SageMaker Spark SDK | | emr-ddb | 4.5.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 2.4.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.4.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-s3-dist-cp | 2.8.0 | Distributed copy application optimized for Amazon S3. | | emrfs | 2.20.0 | Amazon S3 connector for Hadoop ecosystem applications. | | flink-client | 1.3.2 | Apache Flink command line client scripts and applications. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 2.7.3-amzn-6 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 2.7.3-amzn-6 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 2.7.3-amzn-6 | HDFS command-line client and library | | hadoop-hdfs-namenode | 2.7.3-amzn-6 | HDFS service for tracking file names and block locations. | | hadoop-httpfs-server | 2.7.3-amzn-6 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 2.7.3-amzn-6 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 2.7.3-amzn-6 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 2.7.3-amzn-6 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 2.7.3-amzn-6 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 2.7.3-amzn-6 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 1.3.1 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 1.3.1 | Service for serving one or more HBase regions. | | hbase-client | 1.3.1 | HBase command-line client. | | hbase-rest-server | 1.3.1 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 1.3.1 | Service providing a Thrift endpoint to HBase. | | hcatalog-client | 2.3.2-amzn-0 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 2.3.2-amzn-0 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 2.3.2-amzn-0 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 2.3.2-amzn-0 | Hive command line client. | | hive-hbase | 2.3.2-amzn-0 | Hive-hbase client. | | hive-metastore-server | 2.3.2-amzn-0 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server2 | 2.3.2-amzn-0 | Service for accepting Hive queries as web requests. | | hue-server | 4.0.1 | Web application for analyzing data using Hadoop ecosystem applications | | livy-server | 0.4.0-incubating | REST interface for interacting with Apache Spark | | mahout-client | 0.13.0 | Library for machine learning. | | mxnet | 0.12.0 | A flexible, scalable, and efficient library for deep learning. | | mysql-server | 5.5.54\$1 | MySQL database server. | | nvidia-cuda | 9.0.176 | Nvidia drivers and Cuda toolkit | | oozie-client | 4.3.0 | Oozie command-line client. | | oozie-server | 4.3.0 | Service for accepting Oozie workflow requests. | | phoenix-library | 4.11.0-HBase-1.3 | The phoenix libraries for server and client | | phoenix-query-server | 4.11.0-HBase-1.3 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.187 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.187 | Service for executing pieces of a query. | | pig-client | 0.17.0 | Pig command-line client. | | spark-client | 2.2.1 | Spark command-line clients. | | spark-history-server | 2.2.1 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 2.2.1 | In-memory execution engine for YARN. | | spark-yarn-slave | 2.2.1 | Apache Spark libraries needed by YARN slaves. | | sqoop-client | 1.4.6 | Apache Sqoop command-line client. | | tez-on-yarn | 0.8.4 | The tez YARN application and libraries. | | webserver | 2.4.25\$1 | Apache HTTP server. | | zeppelin-server | 0.7.3 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.4.10 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.4.10 | ZooKeeper command line client. | ## 5.11.2 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). **emr-5.11.2 classifications** | Classifications | Description | | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | | core-site | Change values in Hadoop's core-site.xml file. | | emrfs-site | Change EMRFS settings. | | flink-conf | Change flink-conf.yaml settings. | | flink-log4j | Change Flink log4j.properties settings. | | flink-log4j-yarn-session | Change Flink log4j-yarn-session.properties settings. | | flink-log4j-cli | Change Flink log4j-cli.properties settings. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | | hadoop-ssl-server | Change hadoop ssl server configuration | | hadoop-ssl-client | Change hadoop ssl client configuration | | hbase | Amazon EMR-curated settings for Apache HBase. | | hbase-env | Change values in HBase's environment. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbase.properties file. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | | hbase-site | Change values in HBase's hbase-site.xml file. | | hdfs-encryption-zones | Configure HDFS encryption zones. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | | hcatalog-env | Change values in HCatalog's environment. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | | hcatalog-webhcat-log4j2 | Change values in HCatalog WebHCat's log4j2.properties. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | | hive-beeline-log4j2 | Change values in Hive's beeline-log4j2.properties file. | | hive-parquet-logging | Change values in Hive's parquet-logging.properties file. | | hive-env | Change values in the Hive environment. | | hive-exec-log4j2 | Change values in Hive's hive-exec-log4j2.properties file. | | hive-llap-daemon-log4j2 | Change values in Hive's llap-daemon-log4j2.properties file. | | hive-log4j2 | Change values in Hive's hive-log4j2.properties file. | | hive-site | Change values in Hive's hive-site.xml file | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | | hue-ini | Change values in Hue's ini file | | httpfs-env | Change values in the HTTPFS environment. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | | livy-conf | Change values in Livy's livy.conf file. | | livy-env | Change values in the Livy environment. | | livy-log4j | Change Livy log4j.properties settings. | | mapred-env | Change values in the MapReduce application's environment. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | | oozie-env | Change values in Oozie's environment. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | | oozie-site | Change values in Oozie's oozie-site.xml file. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | | pig-properties | Change values in Pig's pig.properties file. | | pig-log4j | Change values in Pig's log4j.properties file. | | presto-log | Change values in Presto's log.properties file. | | presto-config | Change values in Presto's config.properties file. | | presto-env | Change values in Presto's presto-env.sh file. | | presto-node | Change values in Presto's node.properties file. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | | presto-connector-hive | Change values in Presto's hive.properties file. | | presto-connector-jmx | Change values in Presto's jmx.properties file. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | | presto-connector-localfile | Change values in Presto's localfile.properties file. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | | presto-connector-redis | Change values in Presto's redis.properties file. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | | spark | Amazon EMR-curated settings for Apache Spark. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | | spark-env | Change values in the Spark environment. | | spark-hive-site | Change values in Spark's hive-site.xml file | | spark-log4j | Change values in Spark's log4j.properties file. | | spark-metrics | Change values in Spark's metrics.properties file. | | sqoop-env | Change values in Sqoop's environment. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | | tez-site | Change values in Tez's tez-site.xml file. | | yarn-env | Change values in the YARN environment. | | yarn-site | Change values in YARN's yarn-site.xml file. | | zeppelin-env | Change values in the Zeppelin environment. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | # Amazon EMR release 5.11.1 ## 5.11.1 application versions This release includes the following applications: [https://flink.apache.org/](https://flink.apache.org/), [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [http://gethue.com/](http://gethue.com/), [https://livy.incubator.apache.org/](https://livy.incubator.apache.org/), [https://mxnet.incubator.apache.org/](https://mxnet.incubator.apache.org/), [http://mahout.apache.org/](http://mahout.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-5.11.1 | emr-5.11.0 | emr-5.10.1 | emr-5.10.0 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.11.238 | 1.11.238 | 1.11.221 | 1.11.221 | | Python | 2.7, 3.4 | 2.7, 3.4 | 2.7, 3.4 | 2.7, 3.4 | | Scala | 2.11.8 | 2.11.8 | 2.11.8 | 2.11.8 | | AmazonCloudWatchAgent | - | - | - | - | | Delta | - | - | - | - | | Flink | 1.3.2 | 1.3.2 | 1.3.2 | 1.3.2 | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 1.3.1 | 1.3.1 | 1.3.1 | 1.3.1 | | HCatalog | 2.3.2-amzn-0 | 2.3.2-amzn-0 | 2.3.1-amzn-0 | 2.3.1-amzn-0 | | Hadoop | 2.7.3-amzn-6 | 2.7.3-amzn-6 | 2.7.3-amzn-5 | 2.7.3-amzn-5 | | Hive | 2.3.2-amzn-0 | 2.3.2-amzn-0 | 2.3.1-amzn-0 | 2.3.1-amzn-0 | | Hudi | - | - | - | - | | Hue | 4.0.1 | 4.0.1 | 4.0.1 | 4.0.1 | | Iceberg | - | - | - | - | | JupyterEnterpriseGateway | - | - | - | - | | JupyterHub | - | - | - | - | | Livy | 0.4.0-incubating | 0.4.0-incubating | 0.4.0-incubating | 0.4.0-incubating | | MXNet | 0.12.0 | 0.12.0 | 0.12.0 | 0.12.0 | | Mahout | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | | Oozie | 4.3.0 | 4.3.0 | 4.3.0 | 4.3.0 | | Phoenix | 4.11.0-HBase-1.3 | 4.11.0-HBase-1.3 | 4.11.0-HBase-1.3 | 4.11.0-HBase-1.3 | | Pig | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | | Presto | 0.187 | 0.187 | 0.187 | 0.187 | | Spark | 2.2.1 | 2.2.1 | 2.2.0 | 2.2.0 | | Sqoop | 1.4.6 | 1.4.6 | 1.4.6 | 1.4.6 | | TensorFlow | - | - | - | - | | Tez | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | | Trino (PrestoSQL) | - | - | - | - | | Zeppelin | 0.7.3 | 0.7.3 | 0.7.3 | 0.7.3 | | ZooKeeper | 3.4.10 | 3.4.10 | 3.4.10 | 3.4.10 | ## 5.11.1 release notes The following release notes include information for the Amazon EMR 5.11.1 release. Changes are relative to the Amazon EMR 5.8.0 release. Initial release date: January 22, 2018 **Changes, enhancements, and resolved issues** + Updated the Amazon Linux kernel of the defaultAmazon Linux AMI for Amazon EMR to address vulnerabilities associated with speculative execution (CVE-2017-5715, CVE-2017-5753, and CVE-2017-5754). For more information, see [https://aws.amazon.com/security/security-bulletins/AWS-2018-013/](https://aws.amazon.com/security/security-bulletins/AWS-2018-013/). ## 5.11.1 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | aws-sagemaker-spark-sdk | 1.0 | Amazon SageMaker Spark SDK | | emr-ddb | 4.5.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 2.4.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.4.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-s3-dist-cp | 2.8.0 | Distributed copy application optimized for Amazon S3. | | emrfs | 2.20.0 | Amazon S3 connector for Hadoop ecosystem applications. | | flink-client | 1.3.2 | Apache Flink command line client scripts and applications. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 2.7.3-amzn-6 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 2.7.3-amzn-6 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 2.7.3-amzn-6 | HDFS command-line client and library | | hadoop-hdfs-namenode | 2.7.3-amzn-6 | HDFS service for tracking file names and block locations. | | hadoop-httpfs-server | 2.7.3-amzn-6 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 2.7.3-amzn-6 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 2.7.3-amzn-6 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 2.7.3-amzn-6 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 2.7.3-amzn-6 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 2.7.3-amzn-6 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 1.3.1 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 1.3.1 | Service for serving one or more HBase regions. | | hbase-client | 1.3.1 | HBase command-line client. | | hbase-rest-server | 1.3.1 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 1.3.1 | Service providing a Thrift endpoint to HBase. | | hcatalog-client | 2.3.2-amzn-0 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 2.3.2-amzn-0 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 2.3.2-amzn-0 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 2.3.2-amzn-0 | Hive command line client. | | hive-hbase | 2.3.2-amzn-0 | Hive-hbase client. | | hive-metastore-server | 2.3.2-amzn-0 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server2 | 2.3.2-amzn-0 | Service for accepting Hive queries as web requests. | | hue-server | 4.0.1 | Web application for analyzing data using Hadoop ecosystem applications | | livy-server | 0.4.0-incubating | REST interface for interacting with Apache Spark | | mahout-client | 0.13.0 | Library for machine learning. | | mxnet | 0.12.0 | A flexible, scalable, and efficient library for deep learning. | | mysql-server | 5.5.54\$1 | MySQL database server. | | nvidia-cuda | 9.0.176 | Nvidia drivers and Cuda toolkit | | oozie-client | 4.3.0 | Oozie command-line client. | | oozie-server | 4.3.0 | Service for accepting Oozie workflow requests. | | phoenix-library | 4.11.0-HBase-1.3 | The phoenix libraries for server and client | | phoenix-query-server | 4.11.0-HBase-1.3 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.187 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.187 | Service for executing pieces of a query. | | pig-client | 0.17.0 | Pig command-line client. | | spark-client | 2.2.1 | Spark command-line clients. | | spark-history-server | 2.2.1 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 2.2.1 | In-memory execution engine for YARN. | | spark-yarn-slave | 2.2.1 | Apache Spark libraries needed by YARN slaves. | | sqoop-client | 1.4.6 | Apache Sqoop command-line client. | | tez-on-yarn | 0.8.4 | The tez YARN application and libraries. | | webserver | 2.4.25\$1 | Apache HTTP server. | | zeppelin-server | 0.7.3 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.4.10 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.4.10 | ZooKeeper command line client. | ## 5.11.1 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). **emr-5.11.1 classifications** | Classifications | Description | | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | | core-site | Change values in Hadoop's core-site.xml file. | | emrfs-site | Change EMRFS settings. | | flink-conf | Change flink-conf.yaml settings. | | flink-log4j | Change Flink log4j.properties settings. | | flink-log4j-yarn-session | Change Flink log4j-yarn-session.properties settings. | | flink-log4j-cli | Change Flink log4j-cli.properties settings. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | | hadoop-ssl-server | Change hadoop ssl server configuration | | hadoop-ssl-client | Change hadoop ssl client configuration | | hbase | Amazon EMR-curated settings for Apache HBase. | | hbase-env | Change values in HBase's environment. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbase.properties file. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | | hbase-site | Change values in HBase's hbase-site.xml file. | | hdfs-encryption-zones | Configure HDFS encryption zones. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | | hcatalog-env | Change values in HCatalog's environment. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | | hcatalog-webhcat-log4j2 | Change values in HCatalog WebHCat's log4j2.properties. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | | hive-beeline-log4j2 | Change values in Hive's beeline-log4j2.properties file. | | hive-parquet-logging | Change values in Hive's parquet-logging.properties file. | | hive-env | Change values in the Hive environment. | | hive-exec-log4j2 | Change values in Hive's hive-exec-log4j2.properties file. | | hive-llap-daemon-log4j2 | Change values in Hive's llap-daemon-log4j2.properties file. | | hive-log4j2 | Change values in Hive's hive-log4j2.properties file. | | hive-site | Change values in Hive's hive-site.xml file | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | | hue-ini | Change values in Hue's ini file | | httpfs-env | Change values in the HTTPFS environment. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | | livy-conf | Change values in Livy's livy.conf file. | | livy-env | Change values in the Livy environment. | | livy-log4j | Change Livy log4j.properties settings. | | mapred-env | Change values in the MapReduce application's environment. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | | oozie-env | Change values in Oozie's environment. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | | oozie-site | Change values in Oozie's oozie-site.xml file. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | | pig-properties | Change values in Pig's pig.properties file. | | pig-log4j | Change values in Pig's log4j.properties file. | | presto-log | Change values in Presto's log.properties file. | | presto-config | Change values in Presto's config.properties file. | | presto-env | Change values in Presto's presto-env.sh file. | | presto-node | Change values in Presto's node.properties file. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | | presto-connector-hive | Change values in Presto's hive.properties file. | | presto-connector-jmx | Change values in Presto's jmx.properties file. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | | presto-connector-localfile | Change values in Presto's localfile.properties file. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | | presto-connector-redis | Change values in Presto's redis.properties file. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | | spark | Amazon EMR-curated settings for Apache Spark. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | | spark-env | Change values in the Spark environment. | | spark-hive-site | Change values in Spark's hive-site.xml file | | spark-log4j | Change values in Spark's log4j.properties file. | | spark-metrics | Change values in Spark's metrics.properties file. | | sqoop-env | Change values in Sqoop's environment. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | | tez-site | Change values in Tez's tez-site.xml file. | | yarn-env | Change values in the YARN environment. | | yarn-site | Change values in YARN's yarn-site.xml file. | | zeppelin-env | Change values in the Zeppelin environment. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | # Amazon EMR release 5.11.0 ## 5.11.0 application versions This release includes the following applications: [https://flink.apache.org/](https://flink.apache.org/), [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [http://gethue.com/](http://gethue.com/), [https://livy.incubator.apache.org/](https://livy.incubator.apache.org/), [https://mxnet.incubator.apache.org/](https://mxnet.incubator.apache.org/), [http://mahout.apache.org/](http://mahout.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-5.11.0 | emr-5.10.1 | emr-5.10.0 | emr-5.9.1 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.11.238 | 1.11.221 | 1.11.221 | 1.11.183 | | Python | 2.7, 3.4 | 2.7, 3.4 | 2.7, 3.4 | Not tracked | | Scala | 2.11.8 | 2.11.8 | 2.11.8 | 2.11.8 | | AmazonCloudWatchAgent | - | - | - | - | | Delta | - | - | - | - | | Flink | 1.3.2 | 1.3.2 | 1.3.2 | 1.3.2 | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 1.3.1 | 1.3.1 | 1.3.1 | 1.3.1 | | HCatalog | 2.3.2-amzn-0 | 2.3.1-amzn-0 | 2.3.1-amzn-0 | 2.3.0-amzn-0 | | Hadoop | 2.7.3-amzn-6 | 2.7.3-amzn-5 | 2.7.3-amzn-5 | 2.7.3-amzn-4 | | Hive | 2.3.2-amzn-0 | 2.3.1-amzn-0 | 2.3.1-amzn-0 | 2.3.0-amzn-0 | | Hudi | - | - | - | - | | Hue | 4.0.1 | 4.0.1 | 4.0.1 | 4.0.1 | | Iceberg | - | - | - | - | | JupyterEnterpriseGateway | - | - | - | - | | JupyterHub | - | - | - | - | | Livy | 0.4.0-incubating | 0.4.0-incubating | 0.4.0-incubating | 0.4.0-incubating | | MXNet | 0.12.0 | 0.12.0 | 0.12.0 | - | | Mahout | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | | Oozie | 4.3.0 | 4.3.0 | 4.3.0 | 4.3.0 | | Phoenix | 4.11.0-HBase-1.3 | 4.11.0-HBase-1.3 | 4.11.0-HBase-1.3 | 4.11.0-HBase-1.3 | | Pig | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | | Presto | 0.187 | 0.187 | 0.187 | 0.184 | | Spark | 2.2.1 | 2.2.0 | 2.2.0 | 2.2.0 | | Sqoop | 1.4.6 | 1.4.6 | 1.4.6 | 1.4.6 | | TensorFlow | - | - | - | - | | Tez | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | | Trino (PrestoSQL) | - | - | - | - | | Zeppelin | 0.7.3 | 0.7.3 | 0.7.3 | 0.7.2 | | ZooKeeper | 3.4.10 | 3.4.10 | 3.4.10 | 3.4.10 | ## 5.11.0 release notes The following release notes include information for the Amazon EMR release 5.11.0. Changes are relative to 5.10.0. **Upgrades** + Hive 2.3.2 + Spark 2.2.1 + SDK for Java 1.11.238 **New features** + Spark + Added `spark.decommissioning.timeout.threshold` setting, which improves Spark decommissioning behavior when using Spot Instances. For more information, see [Configuring node decommissioning behavior](emr-spark-configure.md#spark-decommissioning). + Added the `aws-sagemaker-spark-sdk` component to Spark, which installs Amazon SageMaker Spark and associated dependencies for Spark integration with [Amazon SageMaker](https://aws.amazon.com/sagemaker/). You can use Amazon SageMaker Spark to construct Spark machine learning (ML) pipelines using Amazon SageMaker stages. For more information, see the [SageMaker Spark Readme](https://github.com/aws/sagemaker-spark/blob/master/README.md) on GitHub and [Using Apache Spark with Amazon SageMaker](https://docs.aws.amazon.com/sagemaker/latest/dg/apache-spark.html) in the *Amazon SageMaker Developer Guide*. **Known issues** + MXNet does not include OpenCV libraries. + Hive 2.3.2 sets `hive.compute.query.using.stats=true` by default. This causes queries to get data from existing statistics rather than directly from data, which could be confusing. For example, if you have a table with `hive.compute.query.using.stats=true` and upload new files to the table `LOCATION`, running a `SELECT COUNT(*)` query on the table returns the count from the statistics, rather than picking up the added rows. As a workaround, use the `ANALYZE TABLE` command to gather new statistics, or set `hive.compute.query.using.stats=false`. For more information, see [Statistics in Hive](https://cwiki.apache.org/confluence/display/Hive/StatsDev#StatsDev-StatisticsinHive) in the Apache Hive documentation. ## 5.11.0 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | aws-sagemaker-spark-sdk | 1.0 | Amazon SageMaker Spark SDK | | emr-ddb | 4.5.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 2.4.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.4.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-s3-dist-cp | 2.8.0 | Distributed copy application optimized for Amazon S3. | | emrfs | 2.20.0 | Amazon S3 connector for Hadoop ecosystem applications. | | flink-client | 1.3.2 | Apache Flink command line client scripts and applications. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 2.7.3-amzn-6 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 2.7.3-amzn-6 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 2.7.3-amzn-6 | HDFS command-line client and library | | hadoop-hdfs-namenode | 2.7.3-amzn-6 | HDFS service for tracking file names and block locations. | | hadoop-httpfs-server | 2.7.3-amzn-6 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 2.7.3-amzn-6 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 2.7.3-amzn-6 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 2.7.3-amzn-6 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 2.7.3-amzn-6 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 2.7.3-amzn-6 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 1.3.1 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 1.3.1 | Service for serving one or more HBase regions. | | hbase-client | 1.3.1 | HBase command-line client. | | hbase-rest-server | 1.3.1 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 1.3.1 | Service providing a Thrift endpoint to HBase. | | hcatalog-client | 2.3.2-amzn-0 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 2.3.2-amzn-0 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 2.3.2-amzn-0 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 2.3.2-amzn-0 | Hive command line client. | | hive-hbase | 2.3.2-amzn-0 | Hive-hbase client. | | hive-metastore-server | 2.3.2-amzn-0 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server2 | 2.3.2-amzn-0 | Service for accepting Hive queries as web requests. | | hue-server | 4.0.1 | Web application for analyzing data using Hadoop ecosystem applications | | livy-server | 0.4.0-incubating | REST interface for interacting with Apache Spark | | mahout-client | 0.13.0 | Library for machine learning. | | mxnet | 0.12.0 | A flexible, scalable, and efficient library for deep learning. | | mysql-server | 5.5.54\$1 | MySQL database server. | | nvidia-cuda | 9.0.176 | Nvidia drivers and Cuda toolkit | | oozie-client | 4.3.0 | Oozie command-line client. | | oozie-server | 4.3.0 | Service for accepting Oozie workflow requests. | | phoenix-library | 4.11.0-HBase-1.3 | The phoenix libraries for server and client | | phoenix-query-server | 4.11.0-HBase-1.3 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.187 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.187 | Service for executing pieces of a query. | | pig-client | 0.17.0 | Pig command-line client. | | spark-client | 2.2.1 | Spark command-line clients. | | spark-history-server | 2.2.1 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 2.2.1 | In-memory execution engine for YARN. | | spark-yarn-slave | 2.2.1 | Apache Spark libraries needed by YARN slaves. | | sqoop-client | 1.4.6 | Apache Sqoop command-line client. | | tez-on-yarn | 0.8.4 | The tez YARN application and libraries. | | webserver | 2.4.25\$1 | Apache HTTP server. | | zeppelin-server | 0.7.3 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.4.10 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.4.10 | ZooKeeper command line client. | ## 5.11.0 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). **emr-5.11.0 classifications** | Classifications | Description | | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | | core-site | Change values in Hadoop's core-site.xml file. | | emrfs-site | Change EMRFS settings. | | flink-conf | Change flink-conf.yaml settings. | | flink-log4j | Change Flink log4j.properties settings. | | flink-log4j-yarn-session | Change Flink log4j-yarn-session.properties settings. | | flink-log4j-cli | Change Flink log4j-cli.properties settings. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | | hadoop-ssl-server | Change hadoop ssl server configuration | | hadoop-ssl-client | Change hadoop ssl client configuration | | hbase | Amazon EMR-curated settings for Apache HBase. | | hbase-env | Change values in HBase's environment. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbase.properties file. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | | hbase-site | Change values in HBase's hbase-site.xml file. | | hdfs-encryption-zones | Configure HDFS encryption zones. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | | hcatalog-env | Change values in HCatalog's environment. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | | hcatalog-webhcat-log4j2 | Change values in HCatalog WebHCat's log4j2.properties. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | | hive-beeline-log4j2 | Change values in Hive's beeline-log4j2.properties file. | | hive-parquet-logging | Change values in Hive's parquet-logging.properties file. | | hive-env | Change values in the Hive environment. | | hive-exec-log4j2 | Change values in Hive's hive-exec-log4j2.properties file. | | hive-llap-daemon-log4j2 | Change values in Hive's llap-daemon-log4j2.properties file. | | hive-log4j2 | Change values in Hive's hive-log4j2.properties file. | | hive-site | Change values in Hive's hive-site.xml file | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | | hue-ini | Change values in Hue's ini file | | httpfs-env | Change values in the HTTPFS environment. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | | livy-conf | Change values in Livy's livy.conf file. | | livy-env | Change values in the Livy environment. | | livy-log4j | Change Livy log4j.properties settings. | | mapred-env | Change values in the MapReduce application's environment. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | | oozie-env | Change values in Oozie's environment. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | | oozie-site | Change values in Oozie's oozie-site.xml file. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | | pig-properties | Change values in Pig's pig.properties file. | | pig-log4j | Change values in Pig's log4j.properties file. | | presto-log | Change values in Presto's log.properties file. | | presto-config | Change values in Presto's config.properties file. | | presto-env | Change values in Presto's presto-env.sh file. | | presto-node | Change values in Presto's node.properties file. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | | presto-connector-hive | Change values in Presto's hive.properties file. | | presto-connector-jmx | Change values in Presto's jmx.properties file. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | | presto-connector-localfile | Change values in Presto's localfile.properties file. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | | presto-connector-redis | Change values in Presto's redis.properties file. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | | spark | Amazon EMR-curated settings for Apache Spark. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | | spark-env | Change values in the Spark environment. | | spark-hive-site | Change values in Spark's hive-site.xml file | | spark-log4j | Change values in Spark's log4j.properties file. | | spark-metrics | Change values in Spark's metrics.properties file. | | sqoop-env | Change values in Sqoop's environment. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | | tez-site | Change values in Tez's tez-site.xml file. | | yarn-env | Change values in the YARN environment. | | yarn-site | Change values in YARN's yarn-site.xml file. | | zeppelin-env | Change values in the Zeppelin environment. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | # Amazon EMR release 5.10.1 ## 5.10.1 application versions This release includes the following applications: [https://flink.apache.org/](https://flink.apache.org/), [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [http://gethue.com/](http://gethue.com/), [https://livy.incubator.apache.org/](https://livy.incubator.apache.org/), [https://mxnet.incubator.apache.org/](https://mxnet.incubator.apache.org/), [http://mahout.apache.org/](http://mahout.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-5.10.1 | emr-5.10.0 | emr-5.9.1 | emr-5.9.0 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.11.221 | 1.11.221 | 1.11.183 | 1.11.183 | | Python | 2.7, 3.4 | 2.7, 3.4 | Not tracked | Not tracked | | Scala | 2.11.8 | 2.11.8 | 2.11.8 | 2.11.8 | | AmazonCloudWatchAgent | - | - | - | - | | Delta | - | - | - | - | | Flink | 1.3.2 | 1.3.2 | 1.3.2 | 1.3.2 | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 1.3.1 | 1.3.1 | 1.3.1 | 1.3.1 | | HCatalog | 2.3.1-amzn-0 | 2.3.1-amzn-0 | 2.3.0-amzn-0 | 2.3.0-amzn-0 | | Hadoop | 2.7.3-amzn-5 | 2.7.3-amzn-5 | 2.7.3-amzn-4 | 2.7.3-amzn-4 | | Hive | 2.3.1-amzn-0 | 2.3.1-amzn-0 | 2.3.0-amzn-0 | 2.3.0-amzn-0 | | Hudi | - | - | - | - | | Hue | 4.0.1 | 4.0.1 | 4.0.1 | 4.0.1 | | Iceberg | - | - | - | - | | JupyterEnterpriseGateway | - | - | - | - | | JupyterHub | - | - | - | - | | Livy | 0.4.0-incubating | 0.4.0-incubating | 0.4.0-incubating | 0.4.0-incubating | | MXNet | 0.12.0 | 0.12.0 | - | - | | Mahout | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | | Oozie | 4.3.0 | 4.3.0 | 4.3.0 | 4.3.0 | | Phoenix | 4.11.0-HBase-1.3 | 4.11.0-HBase-1.3 | 4.11.0-HBase-1.3 | 4.11.0-HBase-1.3 | | Pig | 0.17.0 | 0.17.0 | 0.17.0 | 0.17.0 | | Presto | 0.187 | 0.187 | 0.184 | 0.184 | | Spark | 2.2.0 | 2.2.0 | 2.2.0 | 2.2.0 | | Sqoop | 1.4.6 | 1.4.6 | 1.4.6 | 1.4.6 | | TensorFlow | - | - | - | - | | Tez | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | | Trino (PrestoSQL) | - | - | - | - | | Zeppelin | 0.7.3 | 0.7.3 | 0.7.2 | 0.7.2 | | ZooKeeper | 3.4.10 | 3.4.10 | 3.4.10 | 3.4.10 | ## 5.10.1 release notes This is a patch release to add AWS Signature Version 4 authentication for requests to Amazon S3. All applications and components are the same as the previous Amazon EMR release. **Important** In this release version, Amazon EMR uses AWS Signature Version 4 exclusively to authenticate requests to Amazon S3. For more information, see [Whats New](https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-whatsnew.html). ## 5.10.1 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | emr-ddb | 4.5.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 2.4.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.4.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-s3-dist-cp | 2.7.0 | Distributed copy application optimized for Amazon S3. | | emrfs | 2.20.0 | Amazon S3 connector for Hadoop ecosystem applications. | | flink-client | 1.3.2 | Apache Flink command line client scripts and applications. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 2.7.3-amzn-5 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 2.7.3-amzn-5 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 2.7.3-amzn-5 | HDFS command-line client and library | | hadoop-hdfs-namenode | 2.7.3-amzn-5 | HDFS service for tracking file names and block locations. | | hadoop-httpfs-server | 2.7.3-amzn-5 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 2.7.3-amzn-5 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 2.7.3-amzn-5 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 2.7.3-amzn-5 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 2.7.3-amzn-5 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 2.7.3-amzn-5 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 1.3.1 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 1.3.1 | Service for serving one or more HBase regions. | | hbase-client | 1.3.1 | HBase command-line client. | | hbase-rest-server | 1.3.1 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 1.3.1 | Service providing a Thrift endpoint to HBase. | | hcatalog-client | 2.3.1-amzn-0 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 2.3.1-amzn-0 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 2.3.1-amzn-0 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 2.3.1-amzn-0 | Hive command line client. | | hive-hbase | 2.3.1-amzn-0 | Hive-hbase client. | | hive-metastore-server | 2.3.1-amzn-0 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server2 | 2.3.1-amzn-0 | Service for accepting Hive queries as web requests. | | hue-server | 4.0.1 | Web application for analyzing data using Hadoop ecosystem applications | | livy-server | 0.4.0-incubating | REST interface for interacting with Apache Spark | | mahout-client | 0.13.0 | Library for machine learning. | | mxnet | 0.12.0 | A flexible, scalable, and efficient library for deep learning. | | mysql-server | 5.5.54\$1 | MySQL database server. | | nvidia-cuda | 9.0.176 | Nvidia drivers and Cuda toolkit | | oozie-client | 4.3.0 | Oozie command-line client. | | oozie-server | 4.3.0 | Service for accepting Oozie workflow requests. | | phoenix-library | 4.11.0-HBase-1.3 | The phoenix libraries for server and client | | phoenix-query-server | 4.11.0-HBase-1.3 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.187 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.187 | Service for executing pieces of a query. | | pig-client | 0.17.0 | Pig command-line client. | | spark-client | 2.2.0 | Spark command-line clients. | | spark-history-server | 2.2.0 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 2.2.0 | In-memory execution engine for YARN. | | spark-yarn-slave | 2.2.0 | Apache Spark libraries needed by YARN slaves. | | sqoop-client | 1.4.6 | Apache Sqoop command-line client. | | tez-on-yarn | 0.8.4 | The tez YARN application and libraries. | | webserver | 2.4.25\$1 | Apache HTTP server. | | zeppelin-server | 0.7.3 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.4.10 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.4.10 | ZooKeeper command line client. | ## 5.10.1 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). **emr-5.10.1 classifications** | Classifications | Description | | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | | core-site | Change values in Hadoop's core-site.xml file. | | emrfs-site | Change EMRFS settings. | | flink-conf | Change flink-conf.yaml settings. | | flink-log4j | Change Flink log4j.properties settings. | | flink-log4j-yarn-session | Change Flink log4j-yarn-session.properties settings. | | flink-log4j-cli | Change Flink log4j-cli.properties settings. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | | hadoop-ssl-server | Change hadoop ssl server configuration | | hadoop-ssl-client | Change hadoop ssl client configuration | | hbase | Amazon EMR-curated settings for Apache HBase. | | hbase-env | Change values in HBase's environment. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbase.properties file. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | | hbase-site | Change values in HBase's hbase-site.xml file. | | hdfs-encryption-zones | Configure HDFS encryption zones. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | | hcatalog-env | Change values in HCatalog's environment. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | | hcatalog-webhcat-log4j2 | Change values in HCatalog WebHCat's log4j2.properties. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | | hive-beeline-log4j2 | Change values in Hive's beeline-log4j2.properties file. | | hive-parquet-logging | Change values in Hive's parquet-logging.properties file. | | hive-env | Change values in the Hive environment. | | hive-exec-log4j2 | Change values in Hive's hive-exec-log4j2.properties file. | | hive-llap-daemon-log4j2 | Change values in Hive's llap-daemon-log4j2.properties file. | | hive-log4j2 | Change values in Hive's hive-log4j2.properties file. | | hive-site | Change values in Hive's hive-site.xml file | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | | hue-ini | Change values in Hue's ini file | | httpfs-env | Change values in the HTTPFS environment. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | | livy-conf | Change values in Livy's livy.conf file. | | livy-env | Change values in the Livy environment. | | livy-log4j | Change Livy log4j.properties settings. | | mapred-env | Change values in the MapReduce application's environment. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | | oozie-env | Change values in Oozie's environment. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | | oozie-site | Change values in Oozie's oozie-site.xml file. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | | pig-properties | Change values in Pig's pig.properties file. | | pig-log4j | Change values in Pig's log4j.properties file. | | presto-log | Change values in Presto's log.properties file. | | presto-config | Change values in Presto's config.properties file. | | presto-env | Change values in Presto's presto-env.sh file. | | presto-node | Change values in Presto's node.properties file. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | | presto-connector-hive | Change values in Presto's hive.properties file. | | presto-connector-jmx | Change values in Presto's jmx.properties file. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | | presto-connector-localfile | Change values in Presto's localfile.properties file. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | | presto-connector-redis | Change values in Presto's redis.properties file. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | | spark | Amazon EMR-curated settings for Apache Spark. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | | spark-env | Change values in the Spark environment. | | spark-hive-site | Change values in Spark's hive-site.xml file | | spark-log4j | Change values in Spark's log4j.properties file. | | spark-metrics | Change values in Spark's metrics.properties file. | | sqoop-env | Change values in Sqoop's environment. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | | tez-site | Change values in Tez's tez-site.xml file. | | yarn-env | Change values in the YARN environment. | | yarn-site | Change values in YARN's yarn-site.xml file. | | zeppelin-env | Change values in the Zeppelin environment. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | # Amazon EMR release 5.10.0 ## 5.10.0 application versions This release includes the following applications: [https://flink.apache.org/](https://flink.apache.org/), [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [http://gethue.com/](http://gethue.com/), [https://livy.incubator.apache.org/](https://livy.incubator.apache.org/), [https://mxnet.incubator.apache.org/](https://mxnet.incubator.apache.org/), [http://mahout.apache.org/](http://mahout.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-5.10.0 | emr-5.9.1 | emr-5.9.0 | emr-5.8.3 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.11.221 | 1.11.183 | 1.11.183 | 1.11.160 | | Python | 2.7, 3.4 | Not tracked | Not tracked | Not tracked | | Scala | 2.11.8 | 2.11.8 | 2.11.8 | 2.11.8 | | AmazonCloudWatchAgent | - | - | - | - | | Delta | - | - | - | - | | Flink | 1.3.2 | 1.3.2 | 1.3.2 | 1.3.1 | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 1.3.1 | 1.3.1 | 1.3.1 | 1.3.1 | | HCatalog | 2.3.1-amzn-0 | 2.3.0-amzn-0 | 2.3.0-amzn-0 | 2.3.0-amzn-0 | | Hadoop | 2.7.3-amzn-5 | 2.7.3-amzn-4 | 2.7.3-amzn-4 | 2.7.3-amzn-3 | | Hive | 2.3.1-amzn-0 | 2.3.0-amzn-0 | 2.3.0-amzn-0 | 2.3.0-amzn-0 | | Hudi | - | - | - | - | | Hue | 4.0.1 | 4.0.1 | 4.0.1 | 3.12.0 | | Iceberg | - | - | - | - | | JupyterEnterpriseGateway | - | - | - | - | | JupyterHub | - | - | - | - | | Livy | 0.4.0-incubating | 0.4.0-incubating | 0.4.0-incubating | - | | MXNet | 0.12.0 | - | - | - | | Mahout | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | | Oozie | 4.3.0 | 4.3.0 | 4.3.0 | 4.3.0 | | Phoenix | 4.11.0-HBase-1.3 | 4.11.0-HBase-1.3 | 4.11.0-HBase-1.3 | 4.11.0-HBase-1.3 | | Pig | 0.17.0 | 0.17.0 | 0.17.0 | 0.16.0-amzn-1 | | Presto | 0.187 | 0.184 | 0.184 | 0.170 | | Spark | 2.2.0 | 2.2.0 | 2.2.0 | 2.2.0 | | Sqoop | 1.4.6 | 1.4.6 | 1.4.6 | 1.4.6 | | TensorFlow | - | - | - | - | | Tez | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | | Trino (PrestoSQL) | - | - | - | - | | Zeppelin | 0.7.3 | 0.7.2 | 0.7.2 | 0.7.2 | | ZooKeeper | 3.4.10 | 3.4.10 | 3.4.10 | 3.4.10 | ## 5.10.0 release notes The following release notes include information for the Amazon EMR version 5.10.0 release. Changes are relative to the Amazon EMR 5.9.0 release. **Upgrades** + AWS SDK for Java 1.11.221 + Hive 2.3.1 + Presto 0.187 **New features** + Added support for Kerberos authentication. For more information, see [Use Kerberos Authentication](https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-kerberos.html) in the *Amazon EMR Management Guide* + Added support for IAM roles for EMRFS. For more information, see [Configure IAM Roles for EMRFS Requests to Amazon S3](https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-emrfs-iam-role.html) in the *Amazon EMR Management Guide* + Added support for GPU-based P2 and P3 instance types. For more information, see [Amazon EC2 P2 Instances](https://aws.amazon.com/ec2/instance-types/p2/) and [Amazon EC2 P3 Instances](https://aws.amazon.com/ec2/instance-types/p3/). NVIDIA driver 384.81 and CUDA driver 9.0.176 are installed on these instance types by default. + Added support for [Apache MXNet](emr-mxnet.md). **Changes, enhancements, and resolved issues** + Presto + Added support for using the AWS Glue Data Catalog as the default Hive metastore. For more information, see [Using Presto with the AWS Glue Data Catalog](https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-presto.html#emr-presto-glue). + Added support for [geospatial functions](https://prestodb.io/docs/current/functions/geospatial.html). + Added [spill to disk](https://prestodb.io/docs/current/admin/spill.html) support for joins. + Added support for the [Redshift connector](https://prestodb.io/docs/current/connector/redshift.html). + Spark + Backported [SPARK-20640](https://issues.apache.org/jira/browse/SPARK-20640), which makes the rpc timeout and the retries for shuffle registration values configurable using `spark.shuffle.registration.timeout` and `spark.shuffle.registration.maxAttempts` properties. + Backported [SPARK-21549](https://issues.apache.org/jira/browse/SPARK-21549), which corrects an error that occurs when writing custom OutputFormat to non-HDFS locations. + Backported [Hadoop-13270](https://issues.apache.org/jira/browse/HADOOP-13270) + The Numpy, Scipy, and Matplotlib libraries have been removed from the base Amazon EMR AMI. If these libraries are required for your application, they are available in the application repository, so you can use a bootstrap action to install them on all nodes using `yum install`. + The Amazon EMR base AMI no longer has application RPM packages included, so the RPM packages are no longer present on cluster nodes. Custom AMIs and the Amazon EMR base AMI now reference the RPM package repository in Amazon S3. + Because of the introduction of per-second billing in Amazon EC2, the default **Scale down behavior** is now **Terminate at task completion** rather than **Terminate at instance hour**. For more information, see [Configure Cluster Scale-Down](https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-scaledown-behavior.html). **Known issues** + MXNet does not include OpenCV libraries. + Hive 2.3.1 sets `hive.compute.query.using.stats=true` by default. This causes queries to get data from existing statistics rather than directly from data, which could be confusing. For example, if you have a table with `hive.compute.query.using.stats=true` and upload new files to the table `LOCATION`, running a `SELECT COUNT(*)` query on the table returns the count from the statistics, rather than picking up the added rows. As a workaround, use the `ANALYZE TABLE` command to gather new statistics, or set `hive.compute.query.using.stats=false`. For more information, see [Statistics in Hive](https://cwiki.apache.org/confluence/display/Hive/StatsDev#StatsDev-StatisticsinHive) in the Apache Hive documentation. ## 5.10.0 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | emr-ddb | 4.5.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 2.4.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.4.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-s3-dist-cp | 2.7.0 | Distributed copy application optimized for Amazon S3. | | emrfs | 2.20.0 | Amazon S3 connector for Hadoop ecosystem applications. | | flink-client | 1.3.2 | Apache Flink command line client scripts and applications. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 2.7.3-amzn-5 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 2.7.3-amzn-5 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 2.7.3-amzn-5 | HDFS command-line client and library | | hadoop-hdfs-namenode | 2.7.3-amzn-5 | HDFS service for tracking file names and block locations. | | hadoop-httpfs-server | 2.7.3-amzn-5 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 2.7.3-amzn-5 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 2.7.3-amzn-5 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 2.7.3-amzn-5 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 2.7.3-amzn-5 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 2.7.3-amzn-5 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 1.3.1 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 1.3.1 | Service for serving one or more HBase regions. | | hbase-client | 1.3.1 | HBase command-line client. | | hbase-rest-server | 1.3.1 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 1.3.1 | Service providing a Thrift endpoint to HBase. | | hcatalog-client | 2.3.1-amzn-0 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 2.3.1-amzn-0 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 2.3.1-amzn-0 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 2.3.1-amzn-0 | Hive command line client. | | hive-hbase | 2.3.1-amzn-0 | Hive-hbase client. | | hive-metastore-server | 2.3.1-amzn-0 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server2 | 2.3.1-amzn-0 | Service for accepting Hive queries as web requests. | | hue-server | 4.0.1 | Web application for analyzing data using Hadoop ecosystem applications | | livy-server | 0.4.0-incubating | REST interface for interacting with Apache Spark | | mahout-client | 0.13.0 | Library for machine learning. | | mxnet | 0.12.0 | A flexible, scalable, and efficient library for deep learning. | | mysql-server | 5.5.54\$1 | MySQL database server. | | nvidia-cuda | 9.0.176 | Nvidia drivers and Cuda toolkit | | oozie-client | 4.3.0 | Oozie command-line client. | | oozie-server | 4.3.0 | Service for accepting Oozie workflow requests. | | phoenix-library | 4.11.0-HBase-1.3 | The phoenix libraries for server and client | | phoenix-query-server | 4.11.0-HBase-1.3 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.187 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.187 | Service for executing pieces of a query. | | pig-client | 0.17.0 | Pig command-line client. | | spark-client | 2.2.0 | Spark command-line clients. | | spark-history-server | 2.2.0 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 2.2.0 | In-memory execution engine for YARN. | | spark-yarn-slave | 2.2.0 | Apache Spark libraries needed by YARN slaves. | | sqoop-client | 1.4.6 | Apache Sqoop command-line client. | | tez-on-yarn | 0.8.4 | The tez YARN application and libraries. | | webserver | 2.4.25\$1 | Apache HTTP server. | | zeppelin-server | 0.7.3 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.4.10 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.4.10 | ZooKeeper command line client. | ## 5.10.0 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). **emr-5.10.0 classifications** | Classifications | Description | | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | | core-site | Change values in Hadoop's core-site.xml file. | | emrfs-site | Change EMRFS settings. | | flink-conf | Change flink-conf.yaml settings. | | flink-log4j | Change Flink log4j.properties settings. | | flink-log4j-yarn-session | Change Flink log4j-yarn-session.properties settings. | | flink-log4j-cli | Change Flink log4j-cli.properties settings. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | | hadoop-ssl-server | Change hadoop ssl server configuration | | hadoop-ssl-client | Change hadoop ssl client configuration | | hbase | Amazon EMR-curated settings for Apache HBase. | | hbase-env | Change values in HBase's environment. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbase.properties file. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | | hbase-site | Change values in HBase's hbase-site.xml file. | | hdfs-encryption-zones | Configure HDFS encryption zones. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | | hcatalog-env | Change values in HCatalog's environment. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | | hcatalog-webhcat-log4j2 | Change values in HCatalog WebHCat's log4j2.properties. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | | hive-beeline-log4j2 | Change values in Hive's beeline-log4j2.properties file. | | hive-parquet-logging | Change values in Hive's parquet-logging.properties file. | | hive-env | Change values in the Hive environment. | | hive-exec-log4j2 | Change values in Hive's hive-exec-log4j2.properties file. | | hive-llap-daemon-log4j2 | Change values in Hive's llap-daemon-log4j2.properties file. | | hive-log4j2 | Change values in Hive's hive-log4j2.properties file. | | hive-site | Change values in Hive's hive-site.xml file | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | | hue-ini | Change values in Hue's ini file | | httpfs-env | Change values in the HTTPFS environment. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | | livy-conf | Change values in Livy's livy.conf file. | | livy-env | Change values in the Livy environment. | | livy-log4j | Change Livy log4j.properties settings. | | mapred-env | Change values in the MapReduce application's environment. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | | oozie-env | Change values in Oozie's environment. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | | oozie-site | Change values in Oozie's oozie-site.xml file. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | | pig-properties | Change values in Pig's pig.properties file. | | pig-log4j | Change values in Pig's log4j.properties file. | | presto-log | Change values in Presto's log.properties file. | | presto-config | Change values in Presto's config.properties file. | | presto-env | Change values in Presto's presto-env.sh file. | | presto-node | Change values in Presto's node.properties file. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | | presto-connector-hive | Change values in Presto's hive.properties file. | | presto-connector-jmx | Change values in Presto's jmx.properties file. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | | presto-connector-localfile | Change values in Presto's localfile.properties file. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | | presto-connector-redis | Change values in Presto's redis.properties file. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | | spark | Amazon EMR-curated settings for Apache Spark. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | | spark-env | Change values in the Spark environment. | | spark-hive-site | Change values in Spark's hive-site.xml file | | spark-log4j | Change values in Spark's log4j.properties file. | | spark-metrics | Change values in Spark's metrics.properties file. | | sqoop-env | Change values in Sqoop's environment. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | | tez-site | Change values in Tez's tez-site.xml file. | | yarn-env | Change values in the YARN environment. | | yarn-site | Change values in YARN's yarn-site.xml file. | | zeppelin-env | Change values in the Zeppelin environment. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | # Amazon EMR release 5.9.1 ## 5.9.1 application versions This release includes the following applications: [https://flink.apache.org/](https://flink.apache.org/), [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [http://gethue.com/](http://gethue.com/), [https://livy.incubator.apache.org/](https://livy.incubator.apache.org/), [http://mahout.apache.org/](http://mahout.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-5.9.1 | emr-5.9.0 | emr-5.8.3 | emr-5.8.2 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.11.183 | 1.11.183 | 1.11.160 | 1.11.160 | | Python | Not tracked | Not tracked | Not tracked | Not tracked | | Scala | 2.11.8 | 2.11.8 | 2.11.8 | 2.11.8 | | AmazonCloudWatchAgent | - | - | - | - | | Delta | - | - | - | - | | Flink | 1.3.2 | 1.3.2 | 1.3.1 | 1.3.1 | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 1.3.1 | 1.3.1 | 1.3.1 | 1.3.1 | | HCatalog | 2.3.0-amzn-0 | 2.3.0-amzn-0 | 2.3.0-amzn-0 | 2.3.0-amzn-0 | | Hadoop | 2.7.3-amzn-4 | 2.7.3-amzn-4 | 2.7.3-amzn-3 | 2.7.3-amzn-3 | | Hive | 2.3.0-amzn-0 | 2.3.0-amzn-0 | 2.3.0-amzn-0 | 2.3.0-amzn-0 | | Hudi | - | - | - | - | | Hue | 4.0.1 | 4.0.1 | 3.12.0 | 3.12.0 | | Iceberg | - | - | - | - | | JupyterEnterpriseGateway | - | - | - | - | | JupyterHub | - | - | - | - | | Livy | 0.4.0-incubating | 0.4.0-incubating | - | - | | MXNet | - | - | - | - | | Mahout | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | | Oozie | 4.3.0 | 4.3.0 | 4.3.0 | 4.3.0 | | Phoenix | 4.11.0-HBase-1.3 | 4.11.0-HBase-1.3 | 4.11.0-HBase-1.3 | 4.11.0-HBase-1.3 | | Pig | 0.17.0 | 0.17.0 | 0.16.0-amzn-1 | 0.16.0-amzn-1 | | Presto | 0.184 | 0.184 | 0.170 | 0.170 | | Spark | 2.2.0 | 2.2.0 | 2.2.0 | 2.2.0 | | Sqoop | 1.4.6 | 1.4.6 | 1.4.6 | 1.4.6 | | TensorFlow | - | - | - | - | | Tez | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | | Trino (PrestoSQL) | - | - | - | - | | Zeppelin | 0.7.2 | 0.7.2 | 0.7.2 | 0.7.2 | | ZooKeeper | 3.4.10 | 3.4.10 | 3.4.10 | 3.4.10 | ## 5.9.1 release notes This is a patch release to add AWS Signature Version 4 authentication for requests to Amazon S3. All applications and components are the same as the previous Amazon EMR release. **Important** In this release version, Amazon EMR uses AWS Signature Version 4 exclusively to authenticate requests to Amazon S3. For more information, see [Whats New](https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-whatsnew.html). ## 5.9.1 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | emr-ddb | 4.4.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 2.4.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.4.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-s3-dist-cp | 2.7.0 | Distributed copy application optimized for Amazon S3. | | emrfs | 2.19.0 | Amazon S3 connector for Hadoop ecosystem applications. | | flink-client | 1.3.2 | Apache Flink command line client scripts and applications. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 2.7.3-amzn-4 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 2.7.3-amzn-4 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 2.7.3-amzn-4 | HDFS command-line client and library | | hadoop-hdfs-namenode | 2.7.3-amzn-4 | HDFS service for tracking file names and block locations. | | hadoop-httpfs-server | 2.7.3-amzn-4 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 2.7.3-amzn-4 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 2.7.3-amzn-4 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 2.7.3-amzn-4 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 2.7.3-amzn-4 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 2.7.3-amzn-4 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 1.3.1 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 1.3.1 | Service for serving one or more HBase regions. | | hbase-client | 1.3.1 | HBase command-line client. | | hbase-rest-server | 1.3.1 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 1.3.1 | Service providing a Thrift endpoint to HBase. | | hcatalog-client | 2.3.0-amzn-0 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 2.3.0-amzn-0 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 2.3.0-amzn-0 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 2.3.0-amzn-0 | Hive command line client. | | hive-hbase | 2.3.0-amzn-0 | Hive-hbase client. | | hive-metastore-server | 2.3.0-amzn-0 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server2 | 2.3.0-amzn-0 | Service for accepting Hive queries as web requests. | | hue-server | 4.0.1 | Web application for analyzing data using Hadoop ecosystem applications | | livy-server | 0.4.0-incubating | REST interface for interacting with Apache Spark | | mahout-client | 0.13.0 | Library for machine learning. | | mysql-server | 5.5.54\$1 | MySQL database server. | | oozie-client | 4.3.0 | Oozie command-line client. | | oozie-server | 4.3.0 | Service for accepting Oozie workflow requests. | | phoenix-library | 4.11.0-HBase-1.3 | The phoenix libraries for server and client | | phoenix-query-server | 4.11.0-HBase-1.3 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.184 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.184 | Service for executing pieces of a query. | | pig-client | 0.17.0 | Pig command-line client. | | spark-client | 2.2.0 | Spark command-line clients. | | spark-history-server | 2.2.0 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 2.2.0 | In-memory execution engine for YARN. | | spark-yarn-slave | 2.2.0 | Apache Spark libraries needed by YARN slaves. | | sqoop-client | 1.4.6 | Apache Sqoop command-line client. | | tez-on-yarn | 0.8.4 | The tez YARN application and libraries. | | webserver | 2.4.25\$1 | Apache HTTP server. | | zeppelin-server | 0.7.2 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.4.10 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.4.10 | ZooKeeper command line client. | ## 5.9.1 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). **emr-5.9.1 classifications** | Classifications | Description | | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | | core-site | Change values in Hadoop's core-site.xml file. | | emrfs-site | Change EMRFS settings. | | flink-conf | Change flink-conf.yaml settings. | | flink-log4j | Change Flink log4j.properties settings. | | flink-log4j-yarn-session | Change Flink log4j-yarn-session.properties settings. | | flink-log4j-cli | Change Flink log4j-cli.properties settings. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | | hadoop-ssl-server | Change hadoop ssl server configuration | | hadoop-ssl-client | Change hadoop ssl client configuration | | hbase | Amazon EMR-curated settings for Apache HBase. | | hbase-env | Change values in HBase's environment. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbase.properties file. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | | hbase-site | Change values in HBase's hbase-site.xml file. | | hdfs-encryption-zones | Configure HDFS encryption zones. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | | hcatalog-env | Change values in HCatalog's environment. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | | hcatalog-webhcat-log4j2 | Change values in HCatalog WebHCat's log4j2.properties. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | | hive-beeline-log4j2 | Change values in Hive's beeline-log4j2.properties file. | | hive-parquet-logging | Change values in Hive's parquet-logging.properties file. | | hive-env | Change values in the Hive environment. | | hive-exec-log4j2 | Change values in Hive's hive-exec-log4j2.properties file. | | hive-llap-daemon-log4j2 | Change values in Hive's llap-daemon-log4j2.properties file. | | hive-log4j2 | Change values in Hive's hive-log4j2.properties file. | | hive-site | Change values in Hive's hive-site.xml file | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | | hue-ini | Change values in Hue's ini file | | httpfs-env | Change values in the HTTPFS environment. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | | livy-conf | Change values in Livy's livy.conf file. | | livy-env | Change values in the Livy environment. | | livy-log4j | Change Livy log4j.properties settings. | | mapred-env | Change values in the MapReduce application's environment. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | | oozie-env | Change values in Oozie's environment. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | | oozie-site | Change values in Oozie's oozie-site.xml file. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | | pig-properties | Change values in Pig's pig.properties file. | | pig-log4j | Change values in Pig's log4j.properties file. | | presto-log | Change values in Presto's log.properties file. | | presto-config | Change values in Presto's config.properties file. | | presto-env | Change values in Presto's presto-env.sh file. | | presto-node | Change values in Presto's node.properties file. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | | presto-connector-hive | Change values in Presto's hive.properties file. | | presto-connector-jmx | Change values in Presto's jmx.properties file. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | | presto-connector-localfile | Change values in Presto's localfile.properties file. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | | presto-connector-redis | Change values in Presto's redis.properties file. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | | spark | Amazon EMR-curated settings for Apache Spark. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | | spark-env | Change values in the Spark environment. | | spark-hive-site | Change values in Spark's hive-site.xml file | | spark-log4j | Change values in Spark's log4j.properties file. | | spark-metrics | Change values in Spark's metrics.properties file. | | sqoop-env | Change values in Sqoop's environment. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | | tez-site | Change values in Tez's tez-site.xml file. | | yarn-env | Change values in the YARN environment. | | yarn-site | Change values in YARN's yarn-site.xml file. | | zeppelin-env | Change values in the Zeppelin environment. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | # Amazon EMR release 5.9.0 ## 5.9.0 application versions This release includes the following applications: [https://flink.apache.org/](https://flink.apache.org/), [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [http://gethue.com/](http://gethue.com/), [https://livy.incubator.apache.org/](https://livy.incubator.apache.org/), [http://mahout.apache.org/](http://mahout.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-5.9.0 | emr-5.8.3 | emr-5.8.2 | emr-5.8.1 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.11.183 | 1.11.160 | 1.11.160 | 1.11.160 | | Python | Not tracked | Not tracked | Not tracked | Not tracked | | Scala | 2.11.8 | 2.11.8 | 2.11.8 | 2.11.8 | | AmazonCloudWatchAgent | - | - | - | - | | Delta | - | - | - | - | | Flink | 1.3.2 | 1.3.1 | 1.3.1 | 1.3.1 | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 1.3.1 | 1.3.1 | 1.3.1 | 1.3.1 | | HCatalog | 2.3.0-amzn-0 | 2.3.0-amzn-0 | 2.3.0-amzn-0 | 2.3.0-amzn-0 | | Hadoop | 2.7.3-amzn-4 | 2.7.3-amzn-3 | 2.7.3-amzn-3 | 2.7.3-amzn-3 | | Hive | 2.3.0-amzn-0 | 2.3.0-amzn-0 | 2.3.0-amzn-0 | 2.3.0-amzn-0 | | Hudi | - | - | - | - | | Hue | 4.0.1 | 3.12.0 | 3.12.0 | 3.12.0 | | Iceberg | - | - | - | - | | JupyterEnterpriseGateway | - | - | - | - | | JupyterHub | - | - | - | - | | Livy | 0.4.0-incubating | - | - | - | | MXNet | - | - | - | - | | Mahout | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | | Oozie | 4.3.0 | 4.3.0 | 4.3.0 | 4.3.0 | | Phoenix | 4.11.0-HBase-1.3 | 4.11.0-HBase-1.3 | 4.11.0-HBase-1.3 | 4.11.0-HBase-1.3 | | Pig | 0.17.0 | 0.16.0-amzn-1 | 0.16.0-amzn-1 | 0.16.0-amzn-1 | | Presto | 0.184 | 0.170 | 0.170 | 0.170 | | Spark | 2.2.0 | 2.2.0 | 2.2.0 | 2.2.0 | | Sqoop | 1.4.6 | 1.4.6 | 1.4.6 | 1.4.6 | | TensorFlow | - | - | - | - | | Tez | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | | Trino (PrestoSQL) | - | - | - | - | | Zeppelin | 0.7.2 | 0.7.2 | 0.7.2 | 0.7.2 | | ZooKeeper | 3.4.10 | 3.4.10 | 3.4.10 | 3.4.10 | ## 5.9.0 release notes The following release notes include information for the Amazon EMR version 5.9.0 release. Changes are relative to the Amazon EMR 5.8.0 release. Release date: October 5, 2017 Latest feature update: October 12, 2017 **Upgrades** + AWS SDK for Java version 1.11.183 + Flink 1.3.2 + Hue 4.0.1 + Pig 0.17.0 + Presto 0.184 **New features** + Added Livy support (version 0.4.0-incubating). For more information, see [Apache Livy](emr-livy.md). + Added support for Hue Notebook for Spark. + Added support for i3-series Amazon EC2 instances (October 12, 2017). **Changes, enhancements, and resolved issues** + Spark + Added a new set of features that help ensure Spark handles node termination because of a manual resize or an automatic scaling policy request more gracefully. For more information, see [Configuring node decommissioning behavior](emr-spark-configure.md#spark-decommissioning). + SSL is used instead of 3DES for in-transit encryption for the block transfer service, which enhances performance when using Amazon EC2 instance types with AES-NI. + Backported [SPARK-21494](https://issues.apache.org/jira/browse/SPARK-21494). + Zeppelin + Backported [ZEPPELIN-2377](https://issues.apache.org/jira/browse/ZEPPELIN-2377). + HBase + Added patch [HBASE-18533](https://issues.apache.org/jira/browse/HBASE-18533), which allows additional values for HBase BucketCache configuration using the `hbase-site` configuration classification. + Hue + Added AWS Glue Data Catalog support for the Hive query editor in Hue. + By default, superusers in Hue can access all files that Amazon EMR IAM roles are allowed to access. Newly created users do not automatically have permissions to access the Amazon S3 filebrowser and must have the `filebrowser.s3_access` permissions enabled for their group. + Resolved an issue that caused underlying JSON data created using AWS Glue Data Catalog to be inaccessible. **Known issues** + Cluster launch fails when all applications are installed and the default Amazon EBS root volume size is not changed. As a workaround, use the `aws emr create-cluster` command from the AWS CLI and specify a larger `--ebs-root-volume-size` parameter. + Hive 2.3.0 sets `hive.compute.query.using.stats=true` by default. This causes queries to get data from existing statistics rather than directly from data, which could be confusing. For example, if you have a table with `hive.compute.query.using.stats=true` and upload new files to the table `LOCATION`, running a `SELECT COUNT(*)` query on the table returns the count from the statistics, rather than picking up the added rows. As a workaround, use the `ANALYZE TABLE` command to gather new statistics, or set `hive.compute.query.using.stats=false`. For more information, see [Statistics in Hive](https://cwiki.apache.org/confluence/display/Hive/StatsDev#StatsDev-StatisticsinHive) in the Apache Hive documentation. ## 5.9.0 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | emr-ddb | 4.4.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 2.4.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.4.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-s3-dist-cp | 2.7.0 | Distributed copy application optimized for Amazon S3. | | emrfs | 2.19.0 | Amazon S3 connector for Hadoop ecosystem applications. | | flink-client | 1.3.2 | Apache Flink command line client scripts and applications. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 2.7.3-amzn-4 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 2.7.3-amzn-4 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 2.7.3-amzn-4 | HDFS command-line client and library | | hadoop-hdfs-namenode | 2.7.3-amzn-4 | HDFS service for tracking file names and block locations. | | hadoop-httpfs-server | 2.7.3-amzn-4 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 2.7.3-amzn-4 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 2.7.3-amzn-4 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 2.7.3-amzn-4 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 2.7.3-amzn-4 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 2.7.3-amzn-4 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 1.3.1 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 1.3.1 | Service for serving one or more HBase regions. | | hbase-client | 1.3.1 | HBase command-line client. | | hbase-rest-server | 1.3.1 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 1.3.1 | Service providing a Thrift endpoint to HBase. | | hcatalog-client | 2.3.0-amzn-0 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 2.3.0-amzn-0 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 2.3.0-amzn-0 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 2.3.0-amzn-0 | Hive command line client. | | hive-hbase | 2.3.0-amzn-0 | Hive-hbase client. | | hive-metastore-server | 2.3.0-amzn-0 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server2 | 2.3.0-amzn-0 | Service for accepting Hive queries as web requests. | | hue-server | 4.0.1 | Web application for analyzing data using Hadoop ecosystem applications | | livy-server | 0.4.0-incubating | REST interface for interacting with Apache Spark | | mahout-client | 0.13.0 | Library for machine learning. | | mysql-server | 5.5.54\$1 | MySQL database server. | | oozie-client | 4.3.0 | Oozie command-line client. | | oozie-server | 4.3.0 | Service for accepting Oozie workflow requests. | | phoenix-library | 4.11.0-HBase-1.3 | The phoenix libraries for server and client | | phoenix-query-server | 4.11.0-HBase-1.3 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.184 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.184 | Service for executing pieces of a query. | | pig-client | 0.17.0 | Pig command-line client. | | spark-client | 2.2.0 | Spark command-line clients. | | spark-history-server | 2.2.0 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 2.2.0 | In-memory execution engine for YARN. | | spark-yarn-slave | 2.2.0 | Apache Spark libraries needed by YARN slaves. | | sqoop-client | 1.4.6 | Apache Sqoop command-line client. | | tez-on-yarn | 0.8.4 | The tez YARN application and libraries. | | webserver | 2.4.25\$1 | Apache HTTP server. | | zeppelin-server | 0.7.2 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.4.10 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.4.10 | ZooKeeper command line client. | ## 5.9.0 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). **emr-5.9.0 classifications** | Classifications | Description | | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | | core-site | Change values in Hadoop's core-site.xml file. | | emrfs-site | Change EMRFS settings. | | flink-conf | Change flink-conf.yaml settings. | | flink-log4j | Change Flink log4j.properties settings. | | flink-log4j-yarn-session | Change Flink log4j-yarn-session.properties settings. | | flink-log4j-cli | Change Flink log4j-cli.properties settings. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | | hadoop-ssl-server | Change hadoop ssl server configuration | | hadoop-ssl-client | Change hadoop ssl client configuration | | hbase | Amazon EMR-curated settings for Apache HBase. | | hbase-env | Change values in HBase's environment. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbase.properties file. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | | hbase-site | Change values in HBase's hbase-site.xml file. | | hdfs-encryption-zones | Configure HDFS encryption zones. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | | hcatalog-env | Change values in HCatalog's environment. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | | hcatalog-webhcat-log4j2 | Change values in HCatalog WebHCat's log4j2.properties. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | | hive-beeline-log4j2 | Change values in Hive's beeline-log4j2.properties file. | | hive-parquet-logging | Change values in Hive's parquet-logging.properties file. | | hive-env | Change values in the Hive environment. | | hive-exec-log4j2 | Change values in Hive's hive-exec-log4j2.properties file. | | hive-llap-daemon-log4j2 | Change values in Hive's llap-daemon-log4j2.properties file. | | hive-log4j2 | Change values in Hive's hive-log4j2.properties file. | | hive-site | Change values in Hive's hive-site.xml file | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | | hue-ini | Change values in Hue's ini file | | httpfs-env | Change values in the HTTPFS environment. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | | livy-conf | Change values in Livy's livy.conf file. | | livy-env | Change values in the Livy environment. | | livy-log4j | Change Livy log4j.properties settings. | | mapred-env | Change values in the MapReduce application's environment. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | | oozie-env | Change values in Oozie's environment. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | | oozie-site | Change values in Oozie's oozie-site.xml file. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | | pig-properties | Change values in Pig's pig.properties file. | | pig-log4j | Change values in Pig's log4j.properties file. | | presto-log | Change values in Presto's log.properties file. | | presto-config | Change values in Presto's config.properties file. | | presto-env | Change values in Presto's presto-env.sh file. | | presto-node | Change values in Presto's node.properties file. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | | presto-connector-hive | Change values in Presto's hive.properties file. | | presto-connector-jmx | Change values in Presto's jmx.properties file. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | | presto-connector-localfile | Change values in Presto's localfile.properties file. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | | presto-connector-redis | Change values in Presto's redis.properties file. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | | spark | Amazon EMR-curated settings for Apache Spark. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | | spark-env | Change values in the Spark environment. | | spark-hive-site | Change values in Spark's hive-site.xml file | | spark-log4j | Change values in Spark's log4j.properties file. | | spark-metrics | Change values in Spark's metrics.properties file. | | sqoop-env | Change values in Sqoop's environment. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | | tez-site | Change values in Tez's tez-site.xml file. | | yarn-env | Change values in the YARN environment. | | yarn-site | Change values in YARN's yarn-site.xml file. | | zeppelin-env | Change values in the Zeppelin environment. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | # Amazon EMR release 5.8.3 ## 5.8.3 application versions This release includes the following applications: [https://flink.apache.org/](https://flink.apache.org/), [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [http://gethue.com/](http://gethue.com/), [http://mahout.apache.org/](http://mahout.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-5.8.3 | emr-5.8.2 | emr-5.8.1 | emr-5.8.0 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.11.160 | 1.11.160 | 1.11.160 | 1.11.160 | | Python | Not tracked | Not tracked | Not tracked | Not tracked | | Scala | 2.11.8 | 2.11.8 | 2.11.8 | 2.11.8 | | AmazonCloudWatchAgent | - | - | - | - | | Delta | - | - | - | - | | Flink | 1.3.1 | 1.3.1 | 1.3.1 | 1.3.1 | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 1.3.1 | 1.3.1 | 1.3.1 | 1.3.1 | | HCatalog | 2.3.0-amzn-0 | 2.3.0-amzn-0 | 2.3.0-amzn-0 | 2.3.0-amzn-0 | | Hadoop | 2.7.3-amzn-3 | 2.7.3-amzn-3 | 2.7.3-amzn-3 | 2.7.3-amzn-3 | | Hive | 2.3.0-amzn-0 | 2.3.0-amzn-0 | 2.3.0-amzn-0 | 2.3.0-amzn-0 | | Hudi | - | - | - | - | | Hue | 3.12.0 | 3.12.0 | 3.12.0 | 3.12.0 | | Iceberg | - | - | - | - | | JupyterEnterpriseGateway | - | - | - | - | | JupyterHub | - | - | - | - | | Livy | - | - | - | - | | MXNet | - | - | - | - | | Mahout | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | | Oozie | 4.3.0 | 4.3.0 | 4.3.0 | 4.3.0 | | Phoenix | 4.11.0-HBase-1.3 | 4.11.0-HBase-1.3 | 4.11.0-HBase-1.3 | 4.11.0-HBase-1.3 | | Pig | 0.16.0-amzn-1 | 0.16.0-amzn-1 | 0.16.0-amzn-1 | 0.16.0-amzn-1 | | Presto | 0.170 | 0.170 | 0.170 | 0.170 | | Spark | 2.2.0 | 2.2.0 | 2.2.0 | 2.2.0 | | Sqoop | 1.4.6 | 1.4.6 | 1.4.6 | 1.4.6 | | TensorFlow | - | - | - | - | | Tez | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | | Trino (PrestoSQL) | - | - | - | - | | Zeppelin | 0.7.2 | 0.7.2 | 0.7.2 | 0.7.2 | | ZooKeeper | 3.4.10 | 3.4.10 | 3.4.10 | 3.4.10 | ## 5.8.3 release notes This is a patch release to add AWS Signature Version 4 authentication for requests to Amazon S3. All applications and components are the same as the previous Amazon EMR release. **Important** In this release version, Amazon EMR uses AWS Signature Version 4 exclusively to authenticate requests to Amazon S3. For more information, see [Whats New](https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-whatsnew.html). ## 5.8.3 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | emr-ddb | 4.4.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 2.4.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.4.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-s3-dist-cp | 2.6.0 | Distributed copy application optimized for Amazon S3. | | emrfs | 2.18.0 | Amazon S3 connector for Hadoop ecosystem applications. | | flink-client | 1.3.1 | Apache Flink command line client scripts and applications. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 2.7.3-amzn-3 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 2.7.3-amzn-3 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 2.7.3-amzn-3 | HDFS command-line client and library | | hadoop-hdfs-namenode | 2.7.3-amzn-3 | HDFS service for tracking file names and block locations. | | hadoop-httpfs-server | 2.7.3-amzn-3 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 2.7.3-amzn-3 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 2.7.3-amzn-3 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 2.7.3-amzn-3 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 2.7.3-amzn-3 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 2.7.3-amzn-3 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 1.3.1 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 1.3.1 | Service for serving one or more HBase regions. | | hbase-client | 1.3.1 | HBase command-line client. | | hbase-rest-server | 1.3.1 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 1.3.1 | Service providing a Thrift endpoint to HBase. | | hcatalog-client | 2.3.0-amzn-0 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 2.3.0-amzn-0 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 2.3.0-amzn-0 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 2.3.0-amzn-0 | Hive command line client. | | hive-hbase | 2.3.0-amzn-0 | Hive-hbase client. | | hive-metastore-server | 2.3.0-amzn-0 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server2 | 2.3.0-amzn-0 | Service for accepting Hive queries as web requests. | | hue-server | 3.12.0 | Web application for analyzing data using Hadoop ecosystem applications | | mahout-client | 0.13.0 | Library for machine learning. | | mysql-server | 5.5.54\$1 | MySQL database server. | | oozie-client | 4.3.0 | Oozie command-line client. | | oozie-server | 4.3.0 | Service for accepting Oozie workflow requests. | | phoenix-library | 4.11.0-HBase-1.3 | The phoenix libraries for server and client | | phoenix-query-server | 4.11.0-HBase-1.3 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.170 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.170 | Service for executing pieces of a query. | | pig-client | 0.16.0-amzn-1 | Pig command-line client. | | spark-client | 2.2.0 | Spark command-line clients. | | spark-history-server | 2.2.0 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 2.2.0 | In-memory execution engine for YARN. | | spark-yarn-slave | 2.2.0 | Apache Spark libraries needed by YARN slaves. | | sqoop-client | 1.4.6 | Apache Sqoop command-line client. | | tez-on-yarn | 0.8.4 | The tez YARN application and libraries. | | webserver | 2.4.25\$1 | Apache HTTP server. | | zeppelin-server | 0.7.2 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.4.10 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.4.10 | ZooKeeper command line client. | ## 5.8.3 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). **emr-5.8.3 classifications** | Classifications | Description | | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | | core-site | Change values in Hadoop's core-site.xml file. | | emrfs-site | Change EMRFS settings. | | flink-conf | Change flink-conf.yaml settings. | | flink-log4j | Change Flink log4j.properties settings. | | flink-log4j-yarn-session | Change Flink log4j-yarn-session.properties settings. | | flink-log4j-cli | Change Flink log4j-cli.properties settings. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | | hadoop-ssl-server | Change hadoop ssl server configuration | | hadoop-ssl-client | Change hadoop ssl client configuration | | hbase | Amazon EMR-curated settings for Apache HBase. | | hbase-env | Change values in HBase's environment. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbase.properties file. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | | hbase-site | Change values in HBase's hbase-site.xml file. | | hdfs-encryption-zones | Configure HDFS encryption zones. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | | hcatalog-env | Change values in HCatalog's environment. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | | hcatalog-webhcat-log4j2 | Change values in HCatalog WebHCat's log4j2.properties. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | | hive-beeline-log4j2 | Change values in Hive's beeline-log4j2.properties file. | | hive-parquet-logging | Change values in Hive's parquet-logging.properties file. | | hive-env | Change values in the Hive environment. | | hive-exec-log4j2 | Change values in Hive's hive-exec-log4j2.properties file. | | hive-llap-daemon-log4j2 | Change values in Hive's llap-daemon-log4j2.properties file. | | hive-log4j2 | Change values in Hive's hive-log4j2.properties file. | | hive-site | Change values in Hive's hive-site.xml file | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | | hue-ini | Change values in Hue's ini file | | httpfs-env | Change values in the HTTPFS environment. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | | mapred-env | Change values in the MapReduce application's environment. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | | oozie-env | Change values in Oozie's environment. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | | oozie-site | Change values in Oozie's oozie-site.xml file. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | | pig-properties | Change values in Pig's pig.properties file. | | pig-log4j | Change values in Pig's log4j.properties file. | | presto-log | Change values in Presto's log.properties file. | | presto-config | Change values in Presto's config.properties file. | | presto-env | Change values in Presto's presto-env.sh file. | | presto-node | Change values in Presto's node.properties file. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | | presto-connector-hive | Change values in Presto's hive.properties file. | | presto-connector-jmx | Change values in Presto's jmx.properties file. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | | presto-connector-localfile | Change values in Presto's localfile.properties file. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | | presto-connector-redis | Change values in Presto's redis.properties file. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | | spark | Amazon EMR-curated settings for Apache Spark. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | | spark-env | Change values in the Spark environment. | | spark-hive-site | Change values in Spark's hive-site.xml file | | spark-log4j | Change values in Spark's log4j.properties file. | | spark-metrics | Change values in Spark's metrics.properties file. | | sqoop-env | Change values in Sqoop's environment. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | | tez-site | Change values in Tez's tez-site.xml file. | | yarn-env | Change values in the YARN environment. | | yarn-site | Change values in YARN's yarn-site.xml file. | | zeppelin-env | Change values in the Zeppelin environment. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | # Amazon EMR release 5.8.2 ## 5.8.2 application versions This release includes the following applications: [https://flink.apache.org/](https://flink.apache.org/), [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [http://gethue.com/](http://gethue.com/), [http://mahout.apache.org/](http://mahout.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-5.8.2 | emr-5.8.1 | emr-5.8.0 | emr-5.7.1 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.11.160 | 1.11.160 | 1.11.160 | 1.10.75 | | Python | Not tracked | Not tracked | Not tracked | Not tracked | | Scala | 2.11.8 | 2.11.8 | 2.11.8 | 2.11.8 | | AmazonCloudWatchAgent | - | - | - | - | | Delta | - | - | - | - | | Flink | 1.3.1 | 1.3.1 | 1.3.1 | 1.3.0 | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 1.3.1 | 1.3.1 | 1.3.1 | 1.3.1 | | HCatalog | 2.3.0-amzn-0 | 2.3.0-amzn-0 | 2.3.0-amzn-0 | 2.1.1-amzn-0 | | Hadoop | 2.7.3-amzn-3 | 2.7.3-amzn-3 | 2.7.3-amzn-3 | 2.7.3-amzn-2 | | Hive | 2.3.0-amzn-0 | 2.3.0-amzn-0 | 2.3.0-amzn-0 | 2.1.1-amzn-0 | | Hudi | - | - | - | - | | Hue | 3.12.0 | 3.12.0 | 3.12.0 | 3.12.0 | | Iceberg | - | - | - | - | | JupyterEnterpriseGateway | - | - | - | - | | JupyterHub | - | - | - | - | | Livy | - | - | - | - | | MXNet | - | - | - | - | | Mahout | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | | Oozie | 4.3.0 | 4.3.0 | 4.3.0 | 4.3.0 | | Phoenix | 4.11.0-HBase-1.3 | 4.11.0-HBase-1.3 | 4.11.0-HBase-1.3 | 4.11.0-HBase-1.3 | | Pig | 0.16.0-amzn-1 | 0.16.0-amzn-1 | 0.16.0-amzn-1 | 0.16.0-amzn-0 | | Presto | 0.170 | 0.170 | 0.170 | 0.170 | | Spark | 2.2.0 | 2.2.0 | 2.2.0 | 2.1.1 | | Sqoop | 1.4.6 | 1.4.6 | 1.4.6 | 1.4.6 | | TensorFlow | - | - | - | - | | Tez | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | | Trino (PrestoSQL) | - | - | - | - | | Zeppelin | 0.7.2 | 0.7.2 | 0.7.2 | 0.7.2 | | ZooKeeper | 3.4.10 | 3.4.10 | 3.4.10 | 3.4.10 | ## 5.8.2 release notes The following release notes include information for Amazon EMR release 5.8.2. Changes are relative to 5.8.1. Initial release date: March 29, 2018 **Changes, enhancements, and resolved issues** + Updated the Amazon Linux kernel of the defaultAmazon Linux AMI for Amazon EMR to address potential vulnerabilities. ## 5.8.2 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | emr-ddb | 4.4.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 2.4.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.4.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-s3-dist-cp | 2.6.0 | Distributed copy application optimized for Amazon S3. | | emrfs | 2.18.0 | Amazon S3 connector for Hadoop ecosystem applications. | | flink-client | 1.3.1 | Apache Flink command line client scripts and applications. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 2.7.3-amzn-3 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 2.7.3-amzn-3 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 2.7.3-amzn-3 | HDFS command-line client and library | | hadoop-hdfs-namenode | 2.7.3-amzn-3 | HDFS service for tracking file names and block locations. | | hadoop-httpfs-server | 2.7.3-amzn-3 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 2.7.3-amzn-3 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 2.7.3-amzn-3 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 2.7.3-amzn-3 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 2.7.3-amzn-3 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 2.7.3-amzn-3 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 1.3.1 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 1.3.1 | Service for serving one or more HBase regions. | | hbase-client | 1.3.1 | HBase command-line client. | | hbase-rest-server | 1.3.1 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 1.3.1 | Service providing a Thrift endpoint to HBase. | | hcatalog-client | 2.3.0-amzn-0 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 2.3.0-amzn-0 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 2.3.0-amzn-0 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 2.3.0-amzn-0 | Hive command line client. | | hive-hbase | 2.3.0-amzn-0 | Hive-hbase client. | | hive-metastore-server | 2.3.0-amzn-0 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server2 | 2.3.0-amzn-0 | Service for accepting Hive queries as web requests. | | hue-server | 3.12.0 | Web application for analyzing data using Hadoop ecosystem applications | | mahout-client | 0.13.0 | Library for machine learning. | | mysql-server | 5.5.54\$1 | MySQL database server. | | oozie-client | 4.3.0 | Oozie command-line client. | | oozie-server | 4.3.0 | Service for accepting Oozie workflow requests. | | phoenix-library | 4.11.0-HBase-1.3 | The phoenix libraries for server and client | | phoenix-query-server | 4.11.0-HBase-1.3 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.170 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.170 | Service for executing pieces of a query. | | pig-client | 0.16.0-amzn-1 | Pig command-line client. | | spark-client | 2.2.0 | Spark command-line clients. | | spark-history-server | 2.2.0 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 2.2.0 | In-memory execution engine for YARN. | | spark-yarn-slave | 2.2.0 | Apache Spark libraries needed by YARN slaves. | | sqoop-client | 1.4.6 | Apache Sqoop command-line client. | | tez-on-yarn | 0.8.4 | The tez YARN application and libraries. | | webserver | 2.4.25\$1 | Apache HTTP server. | | zeppelin-server | 0.7.2 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.4.10 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.4.10 | ZooKeeper command line client. | ## 5.8.2 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). **emr-5.8.2 classifications** | Classifications | Description | | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | | core-site | Change values in Hadoop's core-site.xml file. | | emrfs-site | Change EMRFS settings. | | flink-conf | Change flink-conf.yaml settings. | | flink-log4j | Change Flink log4j.properties settings. | | flink-log4j-yarn-session | Change Flink log4j-yarn-session.properties settings. | | flink-log4j-cli | Change Flink log4j-cli.properties settings. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | | hadoop-ssl-server | Change hadoop ssl server configuration | | hadoop-ssl-client | Change hadoop ssl client configuration | | hbase | Amazon EMR-curated settings for Apache HBase. | | hbase-env | Change values in HBase's environment. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbase.properties file. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | | hbase-site | Change values in HBase's hbase-site.xml file. | | hdfs-encryption-zones | Configure HDFS encryption zones. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | | hcatalog-env | Change values in HCatalog's environment. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | | hcatalog-webhcat-log4j2 | Change values in HCatalog WebHCat's log4j2.properties. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | | hive-beeline-log4j2 | Change values in Hive's beeline-log4j2.properties file. | | hive-parquet-logging | Change values in Hive's parquet-logging.properties file. | | hive-env | Change values in the Hive environment. | | hive-exec-log4j2 | Change values in Hive's hive-exec-log4j2.properties file. | | hive-llap-daemon-log4j2 | Change values in Hive's llap-daemon-log4j2.properties file. | | hive-log4j2 | Change values in Hive's hive-log4j2.properties file. | | hive-site | Change values in Hive's hive-site.xml file | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | | hue-ini | Change values in Hue's ini file | | httpfs-env | Change values in the HTTPFS environment. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | | mapred-env | Change values in the MapReduce application's environment. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | | oozie-env | Change values in Oozie's environment. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | | oozie-site | Change values in Oozie's oozie-site.xml file. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | | pig-properties | Change values in Pig's pig.properties file. | | pig-log4j | Change values in Pig's log4j.properties file. | | presto-log | Change values in Presto's log.properties file. | | presto-config | Change values in Presto's config.properties file. | | presto-env | Change values in Presto's presto-env.sh file. | | presto-node | Change values in Presto's node.properties file. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | | presto-connector-hive | Change values in Presto's hive.properties file. | | presto-connector-jmx | Change values in Presto's jmx.properties file. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | | presto-connector-localfile | Change values in Presto's localfile.properties file. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | | presto-connector-redis | Change values in Presto's redis.properties file. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | | spark | Amazon EMR-curated settings for Apache Spark. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | | spark-env | Change values in the Spark environment. | | spark-hive-site | Change values in Spark's hive-site.xml file | | spark-log4j | Change values in Spark's log4j.properties file. | | spark-metrics | Change values in Spark's metrics.properties file. | | sqoop-env | Change values in Sqoop's environment. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | | tez-site | Change values in Tez's tez-site.xml file. | | yarn-env | Change values in the YARN environment. | | yarn-site | Change values in YARN's yarn-site.xml file. | | zeppelin-env | Change values in the Zeppelin environment. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | # Amazon EMR release 5.8.1 ## 5.8.1 application versions This release includes the following applications: [https://flink.apache.org/](https://flink.apache.org/), [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [http://gethue.com/](http://gethue.com/), [http://mahout.apache.org/](http://mahout.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-5.8.1 | emr-5.8.0 | emr-5.7.1 | emr-5.7.0 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.11.160 | 1.11.160 | 1.10.75 | 1.10.75 | | Python | Not tracked | Not tracked | Not tracked | Not tracked | | Scala | 2.11.8 | 2.11.8 | 2.11.8 | 2.11.8 | | AmazonCloudWatchAgent | - | - | - | - | | Delta | - | - | - | - | | Flink | 1.3.1 | 1.3.1 | 1.3.0 | 1.3.0 | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 1.3.1 | 1.3.1 | 1.3.1 | 1.3.1 | | HCatalog | 2.3.0-amzn-0 | 2.3.0-amzn-0 | 2.1.1-amzn-0 | 2.1.1-amzn-0 | | Hadoop | 2.7.3-amzn-3 | 2.7.3-amzn-3 | 2.7.3-amzn-2 | 2.7.3-amzn-2 | | Hive | 2.3.0-amzn-0 | 2.3.0-amzn-0 | 2.1.1-amzn-0 | 2.1.1-amzn-0 | | Hudi | - | - | - | - | | Hue | 3.12.0 | 3.12.0 | 3.12.0 | 3.12.0 | | Iceberg | - | - | - | - | | JupyterEnterpriseGateway | - | - | - | - | | JupyterHub | - | - | - | - | | Livy | - | - | - | - | | MXNet | - | - | - | - | | Mahout | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | | Oozie | 4.3.0 | 4.3.0 | 4.3.0 | 4.3.0 | | Phoenix | 4.11.0-HBase-1.3 | 4.11.0-HBase-1.3 | 4.11.0-HBase-1.3 | 4.11.0-HBase-1.3 | | Pig | 0.16.0-amzn-1 | 0.16.0-amzn-1 | 0.16.0-amzn-0 | 0.16.0-amzn-0 | | Presto | 0.170 | 0.170 | 0.170 | 0.170 | | Spark | 2.2.0 | 2.2.0 | 2.1.1 | 2.1.1 | | Sqoop | 1.4.6 | 1.4.6 | 1.4.6 | 1.4.6 | | TensorFlow | - | - | - | - | | Tez | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | | Trino (PrestoSQL) | - | - | - | - | | Zeppelin | 0.7.2 | 0.7.2 | 0.7.2 | 0.7.2 | | ZooKeeper | 3.4.10 | 3.4.10 | 3.4.10 | 3.4.10 | ## 5.8.1 release notes The following release notes include information for the Amazon EMR 5.8.1 release. Changes are relative to the Amazon EMR 5.8.0 release. Initial release date: January 22, 2018 **Changes, enhancements, and resolved issues** + Updated the Amazon Linux kernel of the defaultAmazon Linux AMI for Amazon EMR to address vulnerabilities associated with speculative execution (CVE-2017-5715, CVE-2017-5753, and CVE-2017-5754). For more information, see [https://aws.amazon.com/security/security-bulletins/AWS-2018-013/](https://aws.amazon.com/security/security-bulletins/AWS-2018-013/). ## 5.8.1 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | emr-ddb | 4.4.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 2.4.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.4.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-s3-dist-cp | 2.6.0 | Distributed copy application optimized for Amazon S3. | | emrfs | 2.18.0 | Amazon S3 connector for Hadoop ecosystem applications. | | flink-client | 1.3.1 | Apache Flink command line client scripts and applications. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 2.7.3-amzn-3 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 2.7.3-amzn-3 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 2.7.3-amzn-3 | HDFS command-line client and library | | hadoop-hdfs-namenode | 2.7.3-amzn-3 | HDFS service for tracking file names and block locations. | | hadoop-httpfs-server | 2.7.3-amzn-3 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 2.7.3-amzn-3 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 2.7.3-amzn-3 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 2.7.3-amzn-3 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 2.7.3-amzn-3 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 2.7.3-amzn-3 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 1.3.1 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 1.3.1 | Service for serving one or more HBase regions. | | hbase-client | 1.3.1 | HBase command-line client. | | hbase-rest-server | 1.3.1 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 1.3.1 | Service providing a Thrift endpoint to HBase. | | hcatalog-client | 2.3.0-amzn-0 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 2.3.0-amzn-0 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 2.3.0-amzn-0 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 2.3.0-amzn-0 | Hive command line client. | | hive-hbase | 2.3.0-amzn-0 | Hive-hbase client. | | hive-metastore-server | 2.3.0-amzn-0 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server2 | 2.3.0-amzn-0 | Service for accepting Hive queries as web requests. | | hue-server | 3.12.0 | Web application for analyzing data using Hadoop ecosystem applications | | mahout-client | 0.13.0 | Library for machine learning. | | mysql-server | 5.5.54\$1 | MySQL database server. | | oozie-client | 4.3.0 | Oozie command-line client. | | oozie-server | 4.3.0 | Service for accepting Oozie workflow requests. | | phoenix-library | 4.11.0-HBase-1.3 | The phoenix libraries for server and client | | phoenix-query-server | 4.11.0-HBase-1.3 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.170 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.170 | Service for executing pieces of a query. | | pig-client | 0.16.0-amzn-1 | Pig command-line client. | | spark-client | 2.2.0 | Spark command-line clients. | | spark-history-server | 2.2.0 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 2.2.0 | In-memory execution engine for YARN. | | spark-yarn-slave | 2.2.0 | Apache Spark libraries needed by YARN slaves. | | sqoop-client | 1.4.6 | Apache Sqoop command-line client. | | tez-on-yarn | 0.8.4 | The tez YARN application and libraries. | | webserver | 2.4.25\$1 | Apache HTTP server. | | zeppelin-server | 0.7.2 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.4.10 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.4.10 | ZooKeeper command line client. | ## 5.8.1 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). **emr-5.8.1 classifications** | Classifications | Description | | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | | core-site | Change values in Hadoop's core-site.xml file. | | emrfs-site | Change EMRFS settings. | | flink-conf | Change flink-conf.yaml settings. | | flink-log4j | Change Flink log4j.properties settings. | | flink-log4j-yarn-session | Change Flink log4j-yarn-session.properties settings. | | flink-log4j-cli | Change Flink log4j-cli.properties settings. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | | hadoop-ssl-server | Change hadoop ssl server configuration | | hadoop-ssl-client | Change hadoop ssl client configuration | | hbase | Amazon EMR-curated settings for Apache HBase. | | hbase-env | Change values in HBase's environment. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbase.properties file. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | | hbase-site | Change values in HBase's hbase-site.xml file. | | hdfs-encryption-zones | Configure HDFS encryption zones. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | | hcatalog-env | Change values in HCatalog's environment. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | | hcatalog-webhcat-log4j2 | Change values in HCatalog WebHCat's log4j2.properties. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | | hive-beeline-log4j2 | Change values in Hive's beeline-log4j2.properties file. | | hive-parquet-logging | Change values in Hive's parquet-logging.properties file. | | hive-env | Change values in the Hive environment. | | hive-exec-log4j2 | Change values in Hive's hive-exec-log4j2.properties file. | | hive-llap-daemon-log4j2 | Change values in Hive's llap-daemon-log4j2.properties file. | | hive-log4j2 | Change values in Hive's hive-log4j2.properties file. | | hive-site | Change values in Hive's hive-site.xml file | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | | hue-ini | Change values in Hue's ini file | | httpfs-env | Change values in the HTTPFS environment. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | | mapred-env | Change values in the MapReduce application's environment. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | | oozie-env | Change values in Oozie's environment. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | | oozie-site | Change values in Oozie's oozie-site.xml file. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | | pig-properties | Change values in Pig's pig.properties file. | | pig-log4j | Change values in Pig's log4j.properties file. | | presto-log | Change values in Presto's log.properties file. | | presto-config | Change values in Presto's config.properties file. | | presto-env | Change values in Presto's presto-env.sh file. | | presto-node | Change values in Presto's node.properties file. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | | presto-connector-hive | Change values in Presto's hive.properties file. | | presto-connector-jmx | Change values in Presto's jmx.properties file. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | | presto-connector-localfile | Change values in Presto's localfile.properties file. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | | presto-connector-redis | Change values in Presto's redis.properties file. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | | spark | Amazon EMR-curated settings for Apache Spark. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | | spark-env | Change values in the Spark environment. | | spark-hive-site | Change values in Spark's hive-site.xml file | | spark-log4j | Change values in Spark's log4j.properties file. | | spark-metrics | Change values in Spark's metrics.properties file. | | sqoop-env | Change values in Sqoop's environment. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | | tez-site | Change values in Tez's tez-site.xml file. | | yarn-env | Change values in the YARN environment. | | yarn-site | Change values in YARN's yarn-site.xml file. | | zeppelin-env | Change values in the Zeppelin environment. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | # Amazon EMR release 5.8.0 ## 5.8.0 application versions This release includes the following applications: [https://flink.apache.org/](https://flink.apache.org/), [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [http://gethue.com/](http://gethue.com/), [http://mahout.apache.org/](http://mahout.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-5.8.0 | emr-5.7.1 | emr-5.7.0 | emr-5.6.1 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.11.160 | 1.10.75 | 1.10.75 | 1.10.75 | | Python | Not tracked | Not tracked | Not tracked | Not tracked | | Scala | 2.11.8 | 2.11.8 | 2.11.8 | 2.11.8 | | AmazonCloudWatchAgent | - | - | - | - | | Delta | - | - | - | - | | Flink | 1.3.1 | 1.3.0 | 1.3.0 | 1.2.1 | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 1.3.1 | 1.3.1 | 1.3.1 | 1.3.0 | | HCatalog | 2.3.0-amzn-0 | 2.1.1-amzn-0 | 2.1.1-amzn-0 | 2.1.1-amzn-0 | | Hadoop | 2.7.3-amzn-3 | 2.7.3-amzn-2 | 2.7.3-amzn-2 | 2.7.3-amzn-2 | | Hive | 2.3.0-amzn-0 | 2.1.1-amzn-0 | 2.1.1-amzn-0 | 2.1.1-amzn-0 | | Hudi | - | - | - | - | | Hue | 3.12.0 | 3.12.0 | 3.12.0 | 3.12.0 | | Iceberg | - | - | - | - | | JupyterEnterpriseGateway | - | - | - | - | | JupyterHub | - | - | - | - | | Livy | - | - | - | - | | MXNet | - | - | - | - | | Mahout | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | | Oozie | 4.3.0 | 4.3.0 | 4.3.0 | 4.3.0 | | Phoenix | 4.11.0-HBase-1.3 | 4.11.0-HBase-1.3 | 4.11.0-HBase-1.3 | 4.9.0-HBase-1.2 | | Pig | 0.16.0-amzn-1 | 0.16.0-amzn-0 | 0.16.0-amzn-0 | 0.16.0-amzn-0 | | Presto | 0.170 | 0.170 | 0.170 | 0.170 | | Spark | 2.2.0 | 2.1.1 | 2.1.1 | 2.1.1 | | Sqoop | 1.4.6 | 1.4.6 | 1.4.6 | 1.4.6 | | TensorFlow | - | - | - | - | | Tez | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | | Trino (PrestoSQL) | - | - | - | - | | Zeppelin | 0.7.2 | 0.7.2 | 0.7.2 | 0.7.1 | | ZooKeeper | 3.4.10 | 3.4.10 | 3.4.10 | 3.4.10 | ## 5.8.0 release notes The following release notes include information for the Amazon EMR version 5.8.0 release. Changes are relative to the Amazon EMR 5.7.0 release. Initial release date: August 10, 2017 Latest feature update: September 25, 2017 **Upgrades** + AWS SDK 1.11.160 + Flink 1.3.1 + Hive 2.3.0. For more information, see [Release Notes](https://issues.apache.org/jira/secure/ConfigureReleaseNote.jspa?projectId=12310843&version=12340269) on the Apache Hive site. + Spark 2.2.0. For more information, see [Release Notes](https://spark.apache.org/releases/spark-release-2-2-0.html) on the Apache Spark site. **New features** + Added support for viewing application history (September 25, 2017). For more information, see [Viewing Application History](https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-cluster-application-history.html) in the *Amazon EMR Management Guide*. **Changes, enhancements, and resolved issues** + **Integration with AWS Glue Data Catalog** + Added ability for Hive and Spark SQL to use AWS Glue Data Catalog as the Hive metadata store. For more information, see [Using the AWS Glue Data Catalog as the metastore for Hive](emr-hive-metastore-glue.md) and [Use AWS Glue Data Catalog with Spark on Amazon EMR](emr-spark-glue.md). + Added **Application history** to cluster details, which allows you to view historical data for YARN applications and additional details for Spark applications. For more information, see [View Application History](https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-cluster-application-history.html) in the *Amazon EMR Management Guide*. + **Oozie** + Backported [OOZIE-2748](https://issues.apache.org/jira/browse/OOZIE-2748). + **Hue** + Backported [HUE-5859](https://issues.cloudera.org/browse/HUE-5859) + **HBase** + Added patch to expose the HBase master server start time through Java Management Extensions (JMX) using `getMasterInitializedTime`. + Added patch that improves cluster start time. **Known issues** + Cluster launch fails when all applications are installed and the default Amazon EBS root volume size is not changed. As a workaround, use the `aws emr create-cluster` command from the AWS CLI and specify a larger `--ebs-root-volume-size` parameter. + Hive 2.3.0 sets `hive.compute.query.using.stats=true` by default. This causes queries to get data from existing statistics rather than directly from data, which could be confusing. For example, if you have a table with `hive.compute.query.using.stats=true` and upload new files to the table `LOCATION`, running a `SELECT COUNT(*)` query on the table returns the count from the statistics, rather than picking up the added rows. As a workaround, use the `ANALYZE TABLE` command to gather new statistics, or set `hive.compute.query.using.stats=false`. For more information, see [Statistics in Hive](https://cwiki.apache.org/confluence/display/Hive/StatsDev#StatsDev-StatisticsinHive) in the Apache Hive documentation. + **Spark**—When using Spark, there is a file handler leak issue with the apppusher daemon, which can appear for a long-running Spark job after several hours or days. To fix the issue, connect to the primary node and type `sudo /etc/init.d/apppusher stop`. This stops that apppusher daemon, which Amazon EMR will restart automatically. + **Application history** + Historical data for dead Spark executors is not available. + Application history is not available for clusters that use a security configuration to enable in-flight encryption. ## 5.8.0 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | emr-ddb | 4.4.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 2.4.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.4.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-s3-dist-cp | 2.6.0 | Distributed copy application optimized for Amazon S3. | | emrfs | 2.18.0 | Amazon S3 connector for Hadoop ecosystem applications. | | flink-client | 1.3.1 | Apache Flink command line client scripts and applications. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 2.7.3-amzn-3 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 2.7.3-amzn-3 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 2.7.3-amzn-3 | HDFS command-line client and library | | hadoop-hdfs-namenode | 2.7.3-amzn-3 | HDFS service for tracking file names and block locations. | | hadoop-httpfs-server | 2.7.3-amzn-3 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 2.7.3-amzn-3 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 2.7.3-amzn-3 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 2.7.3-amzn-3 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 2.7.3-amzn-3 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 2.7.3-amzn-3 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 1.3.1 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 1.3.1 | Service for serving one or more HBase regions. | | hbase-client | 1.3.1 | HBase command-line client. | | hbase-rest-server | 1.3.1 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 1.3.1 | Service providing a Thrift endpoint to HBase. | | hcatalog-client | 2.3.0-amzn-0 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 2.3.0-amzn-0 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 2.3.0-amzn-0 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 2.3.0-amzn-0 | Hive command line client. | | hive-hbase | 2.3.0-amzn-0 | Hive-hbase client. | | hive-metastore-server | 2.3.0-amzn-0 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server2 | 2.3.0-amzn-0 | Service for accepting Hive queries as web requests. | | hue-server | 3.12.0 | Web application for analyzing data using Hadoop ecosystem applications | | mahout-client | 0.13.0 | Library for machine learning. | | mysql-server | 5.5.54\$1 | MySQL database server. | | oozie-client | 4.3.0 | Oozie command-line client. | | oozie-server | 4.3.0 | Service for accepting Oozie workflow requests. | | phoenix-library | 4.11.0-HBase-1.3 | The phoenix libraries for server and client | | phoenix-query-server | 4.11.0-HBase-1.3 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.170 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.170 | Service for executing pieces of a query. | | pig-client | 0.16.0-amzn-1 | Pig command-line client. | | spark-client | 2.2.0 | Spark command-line clients. | | spark-history-server | 2.2.0 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 2.2.0 | In-memory execution engine for YARN. | | spark-yarn-slave | 2.2.0 | Apache Spark libraries needed by YARN slaves. | | sqoop-client | 1.4.6 | Apache Sqoop command-line client. | | tez-on-yarn | 0.8.4 | The tez YARN application and libraries. | | webserver | 2.4.25\$1 | Apache HTTP server. | | zeppelin-server | 0.7.2 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.4.10 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.4.10 | ZooKeeper command line client. | ## 5.8.0 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). **emr-5.8.0 classifications** | Classifications | Description | | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | | core-site | Change values in Hadoop's core-site.xml file. | | emrfs-site | Change EMRFS settings. | | flink-conf | Change flink-conf.yaml settings. | | flink-log4j | Change Flink log4j.properties settings. | | flink-log4j-yarn-session | Change Flink log4j-yarn-session.properties settings. | | flink-log4j-cli | Change Flink log4j-cli.properties settings. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | | hadoop-ssl-server | Change hadoop ssl server configuration | | hadoop-ssl-client | Change hadoop ssl client configuration | | hbase | Amazon EMR-curated settings for Apache HBase. | | hbase-env | Change values in HBase's environment. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbase.properties file. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | | hbase-site | Change values in HBase's hbase-site.xml file. | | hdfs-encryption-zones | Configure HDFS encryption zones. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | | hcatalog-env | Change values in HCatalog's environment. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | | hcatalog-webhcat-log4j2 | Change values in HCatalog WebHCat's log4j2.properties. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | | hive-beeline-log4j2 | Change values in Hive's beeline-log4j2.properties file. | | hive-parquet-logging | Change values in Hive's parquet-logging.properties file. | | hive-env | Change values in the Hive environment. | | hive-exec-log4j2 | Change values in Hive's hive-exec-log4j2.properties file. | | hive-llap-daemon-log4j2 | Change values in Hive's llap-daemon-log4j2.properties file. | | hive-log4j2 | Change values in Hive's hive-log4j2.properties file. | | hive-site | Change values in Hive's hive-site.xml file | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | | hue-ini | Change values in Hue's ini file | | httpfs-env | Change values in the HTTPFS environment. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | | mapred-env | Change values in the MapReduce application's environment. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | | oozie-env | Change values in Oozie's environment. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | | oozie-site | Change values in Oozie's oozie-site.xml file. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | | pig-properties | Change values in Pig's pig.properties file. | | pig-log4j | Change values in Pig's log4j.properties file. | | presto-log | Change values in Presto's log.properties file. | | presto-config | Change values in Presto's config.properties file. | | presto-env | Change values in Presto's presto-env.sh file. | | presto-node | Change values in Presto's node.properties file. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | | presto-connector-hive | Change values in Presto's hive.properties file. | | presto-connector-jmx | Change values in Presto's jmx.properties file. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | | presto-connector-localfile | Change values in Presto's localfile.properties file. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | | presto-connector-redis | Change values in Presto's redis.properties file. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | | spark | Amazon EMR-curated settings for Apache Spark. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | | spark-env | Change values in the Spark environment. | | spark-hive-site | Change values in Spark's hive-site.xml file | | spark-log4j | Change values in Spark's log4j.properties file. | | spark-metrics | Change values in Spark's metrics.properties file. | | sqoop-env | Change values in Sqoop's environment. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | | tez-site | Change values in Tez's tez-site.xml file. | | yarn-env | Change values in the YARN environment. | | yarn-site | Change values in YARN's yarn-site.xml file. | | zeppelin-env | Change values in the Zeppelin environment. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | # Amazon EMR release 5.7.1 ## 5.7.1 application versions This release includes the following applications: [https://flink.apache.org/](https://flink.apache.org/), [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [http://gethue.com/](http://gethue.com/), [http://mahout.apache.org/](http://mahout.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-5.7.1 | emr-5.7.0 | emr-5.6.1 | emr-5.6.0 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.10.75 | 1.10.75 | 1.10.75 | 1.10.75 | | Python | Not tracked | Not tracked | Not tracked | Not tracked | | Scala | 2.11.8 | 2.11.8 | 2.11.8 | 2.11.8 | | AmazonCloudWatchAgent | - | - | - | - | | Delta | - | - | - | - | | Flink | 1.3.0 | 1.3.0 | 1.2.1 | 1.2.1 | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 1.3.1 | 1.3.1 | 1.3.0 | 1.3.0 | | HCatalog | 2.1.1-amzn-0 | 2.1.1-amzn-0 | 2.1.1-amzn-0 | 2.1.1-amzn-0 | | Hadoop | 2.7.3-amzn-2 | 2.7.3-amzn-2 | 2.7.3-amzn-2 | 2.7.3-amzn-2 | | Hive | 2.1.1-amzn-0 | 2.1.1-amzn-0 | 2.1.1-amzn-0 | 2.1.1-amzn-0 | | Hudi | - | - | - | - | | Hue | 3.12.0 | 3.12.0 | 3.12.0 | 3.12.0 | | Iceberg | - | - | - | - | | JupyterEnterpriseGateway | - | - | - | - | | JupyterHub | - | - | - | - | | Livy | - | - | - | - | | MXNet | - | - | - | - | | Mahout | 0.13.0 | 0.13.0 | 0.13.0 | 0.13.0 | | Oozie | 4.3.0 | 4.3.0 | 4.3.0 | 4.3.0 | | Phoenix | 4.11.0-HBase-1.3 | 4.11.0-HBase-1.3 | 4.9.0-HBase-1.2 | 4.9.0-HBase-1.2 | | Pig | 0.16.0-amzn-0 | 0.16.0-amzn-0 | 0.16.0-amzn-0 | 0.16.0-amzn-0 | | Presto | 0.170 | 0.170 | 0.170 | 0.170 | | Spark | 2.1.1 | 2.1.1 | 2.1.1 | 2.1.1 | | Sqoop | 1.4.6 | 1.4.6 | 1.4.6 | 1.4.6 | | TensorFlow | - | - | - | - | | Tez | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | | Trino (PrestoSQL) | - | - | - | - | | Zeppelin | 0.7.2 | 0.7.2 | 0.7.1 | 0.7.1 | | ZooKeeper | 3.4.10 | 3.4.10 | 3.4.10 | 3.4.10 | ## 5.7.1 release notes This is a patch release to add AWS Signature Version 4 authentication for requests to Amazon S3. All applications and components are the same as the previous Amazon EMR release. **Important** In this release version, Amazon EMR uses AWS Signature Version 4 exclusively to authenticate requests to Amazon S3. For more information, see [Whats New](https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-whatsnew.html). ## 5.7.1 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | emr-ddb | 4.3.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 2.3.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.3.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-s3-dist-cp | 2.5.0 | Distributed copy application optimized for Amazon S3. | | emrfs | 2.18.0 | Amazon S3 connector for Hadoop ecosystem applications. | | flink-client | 1.3.0 | Apache Flink command line client scripts and applications. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 2.7.3-amzn-2 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 2.7.3-amzn-2 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 2.7.3-amzn-2 | HDFS command-line client and library | | hadoop-hdfs-namenode | 2.7.3-amzn-2 | HDFS service for tracking file names and block locations. | | hadoop-httpfs-server | 2.7.3-amzn-2 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 2.7.3-amzn-2 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 2.7.3-amzn-2 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 2.7.3-amzn-2 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 2.7.3-amzn-2 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 2.7.3-amzn-2 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 1.3.1 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 1.3.1 | Service for serving one or more HBase regions. | | hbase-client | 1.3.1 | HBase command-line client. | | hbase-rest-server | 1.3.1 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 1.3.1 | Service providing a Thrift endpoint to HBase. | | hcatalog-client | 2.1.1-amzn-0 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 2.1.1-amzn-0 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 2.1.1-amzn-0 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 2.1.1-amzn-0 | Hive command line client. | | hive-hbase | 2.1.1-amzn-0 | Hive-hbase client. | | hive-metastore-server | 2.1.1-amzn-0 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server2 | 2.1.1-amzn-0 | Service for accepting Hive queries as web requests. | | hue-server | 3.12.0 | Web application for analyzing data using Hadoop ecosystem applications | | mahout-client | 0.13.0 | Library for machine learning. | | mysql-server | 5.5.54\$1 | MySQL database server. | | oozie-client | 4.3.0 | Oozie command-line client. | | oozie-server | 4.3.0 | Service for accepting Oozie workflow requests. | | phoenix-library | 4.11.0-HBase-1.3 | The phoenix libraries for server and client | | phoenix-query-server | 4.11.0-HBase-1.3 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.170 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.170 | Service for executing pieces of a query. | | pig-client | 0.16.0-amzn-0 | Pig command-line client. | | spark-client | 2.1.1 | Spark command-line clients. | | spark-history-server | 2.1.1 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 2.1.1 | In-memory execution engine for YARN. | | spark-yarn-slave | 2.1.1 | Apache Spark libraries needed by YARN slaves. | | sqoop-client | 1.4.6 | Apache Sqoop command-line client. | | tez-on-yarn | 0.8.4 | The tez YARN application and libraries. | | webserver | 2.4.25\$1 | Apache HTTP server. | | zeppelin-server | 0.7.2 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.4.10 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.4.10 | ZooKeeper command line client. | ## 5.7.1 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). **emr-5.7.1 classifications** | Classifications | Description | | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | | core-site | Change values in Hadoop's core-site.xml file. | | emrfs-site | Change EMRFS settings. | | flink-conf | Change flink-conf.yaml settings. | | flink-log4j | Change Flink log4j.properties settings. | | flink-log4j-yarn-session | Change Flink log4j-yarn-session.properties settings. | | flink-log4j-cli | Change Flink log4j-cli.properties settings. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | | hadoop-ssl-server | Change hadoop ssl server configuration | | hadoop-ssl-client | Change hadoop ssl client configuration | | hbase | Amazon EMR-curated settings for Apache HBase. | | hbase-env | Change values in HBase's environment. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbase.properties file. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | | hbase-site | Change values in HBase's hbase-site.xml file. | | hdfs-encryption-zones | Configure HDFS encryption zones. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | | hcatalog-env | Change values in HCatalog's environment. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | | hcatalog-webhcat-log4j2 | Change values in HCatalog WebHCat's log4j2.properties. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | | hive-beeline-log4j2 | Change values in Hive's beeline-log4j2.properties file. | | hive-parquet-logging | Change values in Hive's parquet-logging.properties file. | | hive-env | Change values in the Hive environment. | | hive-exec-log4j2 | Change values in Hive's hive-exec-log4j2.properties file. | | hive-llap-daemon-log4j2 | Change values in Hive's llap-daemon-log4j2.properties file. | | hive-log4j2 | Change values in Hive's hive-log4j2.properties file. | | hive-site | Change values in Hive's hive-site.xml file | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | | hue-ini | Change values in Hue's ini file | | httpfs-env | Change values in the HTTPFS environment. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | | mapred-env | Change values in the MapReduce application's environment. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | | oozie-env | Change values in Oozie's environment. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | | oozie-site | Change values in Oozie's oozie-site.xml file. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | | pig-properties | Change values in Pig's pig.properties file. | | pig-log4j | Change values in Pig's log4j.properties file. | | presto-log | Change values in Presto's log.properties file. | | presto-config | Change values in Presto's config.properties file. | | presto-env | Change values in Presto's presto-env.sh file. | | presto-node | Change values in Presto's node.properties file. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | | presto-connector-hive | Change values in Presto's hive.properties file. | | presto-connector-jmx | Change values in Presto's jmx.properties file. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | | presto-connector-localfile | Change values in Presto's localfile.properties file. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | | presto-connector-redis | Change values in Presto's redis.properties file. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | | spark | Amazon EMR-curated settings for Apache Spark. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | | spark-env | Change values in the Spark environment. | | spark-hive-site | Change values in Spark's hive-site.xml file | | spark-log4j | Change values in Spark's log4j.properties file. | | spark-metrics | Change values in Spark's metrics.properties file. | | sqoop-env | Change values in Sqoop's environment. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | | tez-site | Change values in Tez's tez-site.xml file. | | yarn-env | Change values in the YARN environment. | | yarn-site | Change values in YARN's yarn-site.xml file. | | zeppelin-env | Change values in the Zeppelin environment. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | # Amazon EMR release 5.7.0 ## 5.7.0 application versions This release includes the following applications: [https://flink.apache.org/](https://flink.apache.org/), [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [http://gethue.com/](http://gethue.com/), [http://mahout.apache.org/](http://mahout.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-5.7.0 | emr-5.6.1 | emr-5.6.0 | emr-5.5.4 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.10.75 | 1.10.75 | 1.10.75 | 1.10.75 | | Python | Not tracked | Not tracked | Not tracked | Not tracked | | Scala | 2.11.8 | 2.11.8 | 2.11.8 | 2.11.8 | | AmazonCloudWatchAgent | - | - | - | - | | Delta | - | - | - | - | | Flink | 1.3.0 | 1.2.1 | 1.2.1 | 1.2.0 | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 1.3.1 | 1.3.0 | 1.3.0 | 1.3.0 | | HCatalog | 2.1.1-amzn-0 | 2.1.1-amzn-0 | 2.1.1-amzn-0 | 2.1.1-amzn-0 | | Hadoop | 2.7.3-amzn-2 | 2.7.3-amzn-2 | 2.7.3-amzn-2 | 2.7.3-amzn-2 | | Hive | 2.1.1-amzn-0 | 2.1.1-amzn-0 | 2.1.1-amzn-0 | 2.1.1-amzn-0 | | Hudi | - | - | - | - | | Hue | 3.12.0 | 3.12.0 | 3.12.0 | 3.12.0 | | Iceberg | - | - | - | - | | JupyterEnterpriseGateway | - | - | - | - | | JupyterHub | - | - | - | - | | Livy | - | - | - | - | | MXNet | - | - | - | - | | Mahout | 0.13.0 | 0.13.0 | 0.13.0 | 0.12.2 | | Oozie | 4.3.0 | 4.3.0 | 4.3.0 | 4.3.0 | | Phoenix | 4.11.0-HBase-1.3 | 4.9.0-HBase-1.2 | 4.9.0-HBase-1.2 | 4.9.0-HBase-1.2 | | Pig | 0.16.0-amzn-0 | 0.16.0-amzn-0 | 0.16.0-amzn-0 | 0.16.0-amzn-0 | | Presto | 0.170 | 0.170 | 0.170 | 0.170 | | Spark | 2.1.1 | 2.1.1 | 2.1.1 | 2.1.0 | | Sqoop | 1.4.6 | 1.4.6 | 1.4.6 | 1.4.6 | | TensorFlow | - | - | - | - | | Tez | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | | Trino (PrestoSQL) | - | - | - | - | | Zeppelin | 0.7.2 | 0.7.1 | 0.7.1 | 0.7.1 | | ZooKeeper | 3.4.10 | 3.4.10 | 3.4.10 | 3.4.10 | ## 5.7.0 release notes The following release notes include information for the Amazon EMR 5.7.0 release. Changes are relative to the Amazon EMR 5.6.0 release. Release date: July 13, 2017 **Upgrades** + Flink 1.3.0 + Phoenix 4.11.0 + Zeppelin 0.7.2 **New features** + Added the ability to specify a customAmazon Linux AMI when you create a cluster. For more information, see [Using a Custom AMI](https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-custom-ami.html). **Changes, enhancements, and resolved issues** + **HBase** + Added capability to configure HBase read-replica clusters. See [Using a Read-Replica Cluster.](https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-hbase-s3.html#emr-hbase-s3-read-replica) + Multiple bug fixes and enhancements + **Presto** - added ability to configure `node.properties`. + **YARN** - added ability to configure `container-log4j.properties` + **Sqoop** - backported [SQOOP-2880](https://issues.apache.org/jira/browse/SQOOP-2880), which introduces an argument that allows you to set the Sqoop temporary directory. ## 5.7.0 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | emr-ddb | 4.3.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 2.3.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.3.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-s3-dist-cp | 2.5.0 | Distributed copy application optimized for Amazon S3. | | emrfs | 2.18.0 | Amazon S3 connector for Hadoop ecosystem applications. | | flink-client | 1.3.0 | Apache Flink command line client scripts and applications. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 2.7.3-amzn-2 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 2.7.3-amzn-2 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 2.7.3-amzn-2 | HDFS command-line client and library | | hadoop-hdfs-namenode | 2.7.3-amzn-2 | HDFS service for tracking file names and block locations. | | hadoop-httpfs-server | 2.7.3-amzn-2 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 2.7.3-amzn-2 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 2.7.3-amzn-2 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 2.7.3-amzn-2 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 2.7.3-amzn-2 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 2.7.3-amzn-2 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 1.3.1 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 1.3.1 | Service for serving one or more HBase regions. | | hbase-client | 1.3.1 | HBase command-line client. | | hbase-rest-server | 1.3.1 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 1.3.1 | Service providing a Thrift endpoint to HBase. | | hcatalog-client | 2.1.1-amzn-0 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 2.1.1-amzn-0 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 2.1.1-amzn-0 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 2.1.1-amzn-0 | Hive command line client. | | hive-hbase | 2.1.1-amzn-0 | Hive-hbase client. | | hive-metastore-server | 2.1.1-amzn-0 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server2 | 2.1.1-amzn-0 | Service for accepting Hive queries as web requests. | | hue-server | 3.12.0 | Web application for analyzing data using Hadoop ecosystem applications | | mahout-client | 0.13.0 | Library for machine learning. | | mysql-server | 5.5.54\$1 | MySQL database server. | | oozie-client | 4.3.0 | Oozie command-line client. | | oozie-server | 4.3.0 | Service for accepting Oozie workflow requests. | | phoenix-library | 4.11.0-HBase-1.3 | The phoenix libraries for server and client | | phoenix-query-server | 4.11.0-HBase-1.3 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.170 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.170 | Service for executing pieces of a query. | | pig-client | 0.16.0-amzn-0 | Pig command-line client. | | spark-client | 2.1.1 | Spark command-line clients. | | spark-history-server | 2.1.1 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 2.1.1 | In-memory execution engine for YARN. | | spark-yarn-slave | 2.1.1 | Apache Spark libraries needed by YARN slaves. | | sqoop-client | 1.4.6 | Apache Sqoop command-line client. | | tez-on-yarn | 0.8.4 | The tez YARN application and libraries. | | webserver | 2.4.25\$1 | Apache HTTP server. | | zeppelin-server | 0.7.2 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.4.10 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.4.10 | ZooKeeper command line client. | ## 5.7.0 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). **emr-5.7.0 classifications** | Classifications | Description | | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | | core-site | Change values in Hadoop's core-site.xml file. | | emrfs-site | Change EMRFS settings. | | flink-conf | Change flink-conf.yaml settings. | | flink-log4j | Change Flink log4j.properties settings. | | flink-log4j-yarn-session | Change Flink log4j-yarn-session.properties settings. | | flink-log4j-cli | Change Flink log4j-cli.properties settings. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | | hadoop-ssl-server | Change hadoop ssl server configuration | | hadoop-ssl-client | Change hadoop ssl client configuration | | hbase | Amazon EMR-curated settings for Apache HBase. | | hbase-env | Change values in HBase's environment. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbase.properties file. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | | hbase-site | Change values in HBase's hbase-site.xml file. | | hdfs-encryption-zones | Configure HDFS encryption zones. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | | hcatalog-env | Change values in HCatalog's environment. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | | hcatalog-webhcat-log4j2 | Change values in HCatalog WebHCat's log4j2.properties. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | | hive-beeline-log4j2 | Change values in Hive's beeline-log4j2.properties file. | | hive-parquet-logging | Change values in Hive's parquet-logging.properties file. | | hive-env | Change values in the Hive environment. | | hive-exec-log4j2 | Change values in Hive's hive-exec-log4j2.properties file. | | hive-llap-daemon-log4j2 | Change values in Hive's llap-daemon-log4j2.properties file. | | hive-log4j2 | Change values in Hive's hive-log4j2.properties file. | | hive-site | Change values in Hive's hive-site.xml file | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | | hue-ini | Change values in Hue's ini file | | httpfs-env | Change values in the HTTPFS environment. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | | mapred-env | Change values in the MapReduce application's environment. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | | oozie-env | Change values in Oozie's environment. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | | oozie-site | Change values in Oozie's oozie-site.xml file. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | | pig-properties | Change values in Pig's pig.properties file. | | pig-log4j | Change values in Pig's log4j.properties file. | | presto-log | Change values in Presto's log.properties file. | | presto-config | Change values in Presto's config.properties file. | | presto-env | Change values in Presto's presto-env.sh file. | | presto-node | Change values in Presto's node.properties file. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | | presto-connector-hive | Change values in Presto's hive.properties file. | | presto-connector-jmx | Change values in Presto's jmx.properties file. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | | presto-connector-localfile | Change values in Presto's localfile.properties file. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | | presto-connector-redis | Change values in Presto's redis.properties file. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | | spark | Amazon EMR-curated settings for Apache Spark. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | | spark-env | Change values in the Spark environment. | | spark-hive-site | Change values in Spark's hive-site.xml file | | spark-log4j | Change values in Spark's log4j.properties file. | | spark-metrics | Change values in Spark's metrics.properties file. | | sqoop-env | Change values in Sqoop's environment. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | | tez-site | Change values in Tez's tez-site.xml file. | | yarn-env | Change values in the YARN environment. | | yarn-site | Change values in YARN's yarn-site.xml file. | | zeppelin-env | Change values in the Zeppelin environment. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | # Amazon EMR release 5.6.1 ## 5.6.1 application versions This release includes the following applications: [https://flink.apache.org/](https://flink.apache.org/), [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [http://gethue.com/](http://gethue.com/), [http://mahout.apache.org/](http://mahout.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-5.6.1 | emr-5.6.0 | emr-5.5.4 | emr-5.5.3 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.10.75 | 1.10.75 | 1.10.75 | 1.10.75 | | Python | Not tracked | Not tracked | Not tracked | Not tracked | | Scala | 2.11.8 | 2.11.8 | 2.11.8 | 2.11.8 | | AmazonCloudWatchAgent | - | - | - | - | | Delta | - | - | - | - | | Flink | 1.2.1 | 1.2.1 | 1.2.0 | 1.2.0 | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 1.3.0 | 1.3.0 | 1.3.0 | 1.3.0 | | HCatalog | 2.1.1-amzn-0 | 2.1.1-amzn-0 | 2.1.1-amzn-0 | 2.1.1-amzn-0 | | Hadoop | 2.7.3-amzn-2 | 2.7.3-amzn-2 | 2.7.3-amzn-2 | 2.7.3-amzn-2 | | Hive | 2.1.1-amzn-0 | 2.1.1-amzn-0 | 2.1.1-amzn-0 | 2.1.1-amzn-0 | | Hudi | - | - | - | - | | Hue | 3.12.0 | 3.12.0 | 3.12.0 | 3.12.0 | | Iceberg | - | - | - | - | | JupyterEnterpriseGateway | - | - | - | - | | JupyterHub | - | - | - | - | | Livy | - | - | - | - | | MXNet | - | - | - | - | | Mahout | 0.13.0 | 0.13.0 | 0.12.2 | 0.12.2 | | Oozie | 4.3.0 | 4.3.0 | 4.3.0 | 4.3.0 | | Phoenix | 4.9.0-HBase-1.2 | 4.9.0-HBase-1.2 | 4.9.0-HBase-1.2 | 4.9.0-HBase-1.2 | | Pig | 0.16.0-amzn-0 | 0.16.0-amzn-0 | 0.16.0-amzn-0 | 0.16.0-amzn-0 | | Presto | 0.170 | 0.170 | 0.170 | 0.170 | | Spark | 2.1.1 | 2.1.1 | 2.1.0 | 2.1.0 | | Sqoop | 1.4.6 | 1.4.6 | 1.4.6 | 1.4.6 | | TensorFlow | - | - | - | - | | Tez | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | | Trino (PrestoSQL) | - | - | - | - | | Zeppelin | 0.7.1 | 0.7.1 | 0.7.1 | 0.7.1 | | ZooKeeper | 3.4.10 | 3.4.10 | 3.4.10 | 3.4.10 | ## 5.6.1 release notes This is a patch release to add AWS Signature Version 4 authentication for requests to Amazon S3. All applications and components are the same as the previous Amazon EMR release. **Important** In this release version, Amazon EMR uses AWS Signature Version 4 exclusively to authenticate requests to Amazon S3. For more information, see [Whats New](https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-whatsnew.html). ## 5.6.1 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | emr-ddb | 4.3.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 2.3.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.3.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-s3-dist-cp | 2.5.0 | Distributed copy application optimized for Amazon S3. | | emrfs | 2.17.0 | Amazon S3 connector for Hadoop ecosystem applications. | | flink-client | 1.2.1 | Apache Flink command line client scripts and applications. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 2.7.3-amzn-2 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 2.7.3-amzn-2 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 2.7.3-amzn-2 | HDFS command-line client and library | | hadoop-hdfs-namenode | 2.7.3-amzn-2 | HDFS service for tracking file names and block locations. | | hadoop-httpfs-server | 2.7.3-amzn-2 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 2.7.3-amzn-2 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 2.7.3-amzn-2 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 2.7.3-amzn-2 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 2.7.3-amzn-2 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 2.7.3-amzn-2 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 1.3.1 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 1.3.1 | Service for serving one or more HBase regions. | | hbase-client | 1.3.1 | HBase command-line client. | | hbase-rest-server | 1.3.1 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 1.3.1 | Service providing a Thrift endpoint to HBase. | | hcatalog-client | 2.1.1-amzn-0 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 2.1.1-amzn-0 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 2.1.1-amzn-0 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 2.1.1-amzn-0 | Hive command line client. | | hive-hbase | 2.1.1-amzn-0 | Hive-hbase client. | | hive-metastore-server | 2.1.1-amzn-0 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server2 | 2.1.1-amzn-0 | Service for accepting Hive queries as web requests. | | hue-server | 3.12.0 | Web application for analyzing data using Hadoop ecosystem applications | | mahout-client | 0.13.0 | Library for machine learning. | | mysql-server | 5.5.54\$1 | MySQL database server. | | oozie-client | 4.3.0 | Oozie command-line client. | | oozie-server | 4.3.0 | Service for accepting Oozie workflow requests. | | phoenix-library | 4.9.0-HBase-1.2 | The phoenix libraries for server and client | | phoenix-query-server | 4.9.0-HBase-1.2 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.170 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.170 | Service for executing pieces of a query. | | pig-client | 0.16.0-amzn-0 | Pig command-line client. | | spark-client | 2.1.1 | Spark command-line clients. | | spark-history-server | 2.1.1 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 2.1.1 | In-memory execution engine for YARN. | | spark-yarn-slave | 2.1.1 | Apache Spark libraries needed by YARN slaves. | | sqoop-client | 1.4.6 | Apache Sqoop command-line client. | | tez-on-yarn | 0.8.4 | The tez YARN application and libraries. | | webserver | 2.4.25\$1 | Apache HTTP server. | | zeppelin-server | 0.7.1 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.4.10 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.4.10 | ZooKeeper command line client. | ## 5.6.1 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). **emr-5.6.1 classifications** | Classifications | Description | | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | | core-site | Change values in Hadoop's core-site.xml file. | | emrfs-site | Change EMRFS settings. | | flink-conf | Change flink-conf.yaml settings. | | flink-log4j | Change Flink log4j.properties settings. | | flink-log4j-yarn-session | Change Flink log4j-yarn-session.properties settings. | | flink-log4j-cli | Change Flink log4j-cli.properties settings. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | | hadoop-ssl-server | Change hadoop ssl server configuration | | hadoop-ssl-client | Change hadoop ssl client configuration | | hbase | Amazon EMR-curated settings for Apache HBase. | | hbase-env | Change values in HBase's environment. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbase.properties file. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | | hbase-site | Change values in HBase's hbase-site.xml file. | | hdfs-encryption-zones | Configure HDFS encryption zones. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | | hcatalog-env | Change values in HCatalog's environment. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | | hcatalog-webhcat-log4j2 | Change values in HCatalog WebHCat's log4j2.properties. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | | hive-beeline-log4j2 | Change values in Hive's beeline-log4j2.properties file. | | hive-parquet-logging | Change values in Hive's parquet-logging.properties file. | | hive-env | Change values in the Hive environment. | | hive-exec-log4j2 | Change values in Hive's hive-exec-log4j2.properties file. | | hive-llap-daemon-log4j2 | Change values in Hive's llap-daemon-log4j2.properties file. | | hive-log4j2 | Change values in Hive's hive-log4j2.properties file. | | hive-site | Change values in Hive's hive-site.xml file | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | | hue-ini | Change values in Hue's ini file | | httpfs-env | Change values in the HTTPFS environment. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | | mapred-env | Change values in the MapReduce application's environment. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | | oozie-env | Change values in Oozie's environment. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | | oozie-site | Change values in Oozie's oozie-site.xml file. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | | pig-properties | Change values in Pig's pig.properties file. | | pig-log4j | Change values in Pig's log4j.properties file. | | presto-log | Change values in Presto's log.properties file. | | presto-config | Change values in Presto's config.properties file. | | presto-env | Change values in Presto's presto-env.sh file. | | presto-node | Change values in Presto's node.properties file. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | | presto-connector-hive | Change values in Presto's hive.properties file. | | presto-connector-jmx | Change values in Presto's jmx.properties file. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | | presto-connector-localfile | Change values in Presto's localfile.properties file. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | | presto-connector-redis | Change values in Presto's redis.properties file. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | | spark | Amazon EMR-curated settings for Apache Spark. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | | spark-env | Change values in the Spark environment. | | spark-hive-site | Change values in Spark's hive-site.xml file | | spark-log4j | Change values in Spark's log4j.properties file. | | spark-metrics | Change values in Spark's metrics.properties file. | | sqoop-env | Change values in Sqoop's environment. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | | tez-site | Change values in Tez's tez-site.xml file. | | yarn-env | Change values in the YARN environment. | | yarn-site | Change values in YARN's yarn-site.xml file. | | zeppelin-env | Change values in the Zeppelin environment. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | # Amazon EMR release 5.6.0 ## 5.6.0 application versions This release includes the following applications: [https://flink.apache.org/](https://flink.apache.org/), [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [http://gethue.com/](http://gethue.com/), [http://mahout.apache.org/](http://mahout.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-5.6.0 | emr-5.5.4 | emr-5.5.3 | emr-5.5.2 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.10.75 | 1.10.75 | 1.10.75 | 1.10.75 | | Python | Not tracked | Not tracked | Not tracked | Not tracked | | Scala | 2.11.8 | 2.11.8 | 2.11.8 | 2.11.8 | | AmazonCloudWatchAgent | - | - | - | - | | Delta | - | - | - | - | | Flink | 1.2.1 | 1.2.0 | 1.2.0 | 1.2.0 | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 1.3.0 | 1.3.0 | 1.3.0 | 1.3.0 | | HCatalog | 2.1.1-amzn-0 | 2.1.1-amzn-0 | 2.1.1-amzn-0 | 2.1.1-amzn-0 | | Hadoop | 2.7.3-amzn-2 | 2.7.3-amzn-2 | 2.7.3-amzn-2 | 2.7.3-amzn-2 | | Hive | 2.1.1-amzn-0 | 2.1.1-amzn-0 | 2.1.1-amzn-0 | 2.1.1-amzn-0 | | Hudi | - | - | - | - | | Hue | 3.12.0 | 3.12.0 | 3.12.0 | 3.12.0 | | Iceberg | - | - | - | - | | JupyterEnterpriseGateway | - | - | - | - | | JupyterHub | - | - | - | - | | Livy | - | - | - | - | | MXNet | - | - | - | - | | Mahout | 0.13.0 | 0.12.2 | 0.12.2 | 0.12.2 | | Oozie | 4.3.0 | 4.3.0 | 4.3.0 | 4.3.0 | | Phoenix | 4.9.0-HBase-1.2 | 4.9.0-HBase-1.2 | 4.9.0-HBase-1.2 | 4.9.0-HBase-1.2 | | Pig | 0.16.0-amzn-0 | 0.16.0-amzn-0 | 0.16.0-amzn-0 | 0.16.0-amzn-0 | | Presto | 0.170 | 0.170 | 0.170 | 0.170 | | Spark | 2.1.1 | 2.1.0 | 2.1.0 | 2.1.0 | | Sqoop | 1.4.6 | 1.4.6 | 1.4.6 | 1.4.6 | | TensorFlow | - | - | - | - | | Tez | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | | Trino (PrestoSQL) | - | - | - | - | | Zeppelin | 0.7.1 | 0.7.1 | 0.7.1 | 0.7.1 | | ZooKeeper | 3.4.10 | 3.4.10 | 3.4.10 | 3.4.10 | ## 5.6.0 release notes The following release notes include information for the Amazon EMR 5.6.0 release. Changes are relative to the Amazon EMR 5.5.0 release. Release date: June 5, 2017 **Upgrades** + Flink 1.2.1 + HBase 1.3.1 + Mahout 0.13.0. This is the first version of Mahout to support Spark 2.x in Amazon EMR version 5.0 and later. + Spark 2.1.1 **Changes, enhancements, and resolved issues** + **Presto** + Added the ability to enable SSL/TLS secured communication between Presto nodes by enabling in-transit encryption using a security configuration. For more information, see [In-transit Data Encryption](https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-data-encryption-options.html#emr-encryption-intransit). + Backported [Presto 7661](https://github.com/prestodb/presto/pull/7661/commits), which adds the `VERBOSE` option to the `EXPLAIN ANALYZE` statement to report more detailed, low-level statistics about a query plan. ## 5.6.0 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | emr-ddb | 4.3.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 2.3.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.3.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-s3-dist-cp | 2.5.0 | Distributed copy application optimized for Amazon S3. | | emrfs | 2.17.0 | Amazon S3 connector for Hadoop ecosystem applications. | | flink-client | 1.2.1 | Apache Flink command line client scripts and applications. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 2.7.3-amzn-2 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 2.7.3-amzn-2 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 2.7.3-amzn-2 | HDFS command-line client and library | | hadoop-hdfs-namenode | 2.7.3-amzn-2 | HDFS service for tracking file names and block locations. | | hadoop-httpfs-server | 2.7.3-amzn-2 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 2.7.3-amzn-2 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 2.7.3-amzn-2 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 2.7.3-amzn-2 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 2.7.3-amzn-2 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 2.7.3-amzn-2 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 1.3.1 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 1.3.1 | Service for serving one or more HBase regions. | | hbase-client | 1.3.1 | HBase command-line client. | | hbase-rest-server | 1.3.1 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 1.3.1 | Service providing a Thrift endpoint to HBase. | | hcatalog-client | 2.1.1-amzn-0 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 2.1.1-amzn-0 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 2.1.1-amzn-0 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 2.1.1-amzn-0 | Hive command line client. | | hive-hbase | 2.1.1-amzn-0 | Hive-hbase client. | | hive-metastore-server | 2.1.1-amzn-0 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server2 | 2.1.1-amzn-0 | Service for accepting Hive queries as web requests. | | hue-server | 3.12.0 | Web application for analyzing data using Hadoop ecosystem applications | | mahout-client | 0.13.0 | Library for machine learning. | | mysql-server | 5.5.54\$1 | MySQL database server. | | oozie-client | 4.3.0 | Oozie command-line client. | | oozie-server | 4.3.0 | Service for accepting Oozie workflow requests. | | phoenix-library | 4.9.0-HBase-1.2 | The phoenix libraries for server and client | | phoenix-query-server | 4.9.0-HBase-1.2 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.170 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.170 | Service for executing pieces of a query. | | pig-client | 0.16.0-amzn-0 | Pig command-line client. | | spark-client | 2.1.1 | Spark command-line clients. | | spark-history-server | 2.1.1 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 2.1.1 | In-memory execution engine for YARN. | | spark-yarn-slave | 2.1.1 | Apache Spark libraries needed by YARN slaves. | | sqoop-client | 1.4.6 | Apache Sqoop command-line client. | | tez-on-yarn | 0.8.4 | The tez YARN application and libraries. | | webserver | 2.4.25\$1 | Apache HTTP server. | | zeppelin-server | 0.7.1 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.4.10 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.4.10 | ZooKeeper command line client. | ## 5.6.0 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). **emr-5.6.0 classifications** | Classifications | Description | | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | | core-site | Change values in Hadoop's core-site.xml file. | | emrfs-site | Change EMRFS settings. | | flink-conf | Change flink-conf.yaml settings. | | flink-log4j | Change Flink log4j.properties settings. | | flink-log4j-yarn-session | Change Flink log4j-yarn-session.properties settings. | | flink-log4j-cli | Change Flink log4j-cli.properties settings. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | | hadoop-ssl-server | Change hadoop ssl server configuration | | hadoop-ssl-client | Change hadoop ssl client configuration | | hbase | Amazon EMR-curated settings for Apache HBase. | | hbase-env | Change values in HBase's environment. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbase.properties file. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | | hbase-site | Change values in HBase's hbase-site.xml file. | | hdfs-encryption-zones | Configure HDFS encryption zones. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | | hcatalog-env | Change values in HCatalog's environment. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | | hcatalog-webhcat-log4j2 | Change values in HCatalog WebHCat's log4j2.properties. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | | hive-beeline-log4j2 | Change values in Hive's beeline-log4j2.properties file. | | hive-parquet-logging | Change values in Hive's parquet-logging.properties file. | | hive-env | Change values in the Hive environment. | | hive-exec-log4j2 | Change values in Hive's hive-exec-log4j2.properties file. | | hive-llap-daemon-log4j2 | Change values in Hive's llap-daemon-log4j2.properties file. | | hive-log4j2 | Change values in Hive's hive-log4j2.properties file. | | hive-site | Change values in Hive's hive-site.xml file | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | | hue-ini | Change values in Hue's ini file | | httpfs-env | Change values in the HTTPFS environment. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | | mapred-env | Change values in the MapReduce application's environment. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | | oozie-env | Change values in Oozie's environment. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | | oozie-site | Change values in Oozie's oozie-site.xml file. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | | pig-properties | Change values in Pig's pig.properties file. | | pig-log4j | Change values in Pig's log4j.properties file. | | presto-log | Change values in Presto's log.properties file. | | presto-config | Change values in Presto's config.properties file. | | presto-env | Change values in Presto's presto-env.sh file. | | presto-node | Change values in Presto's node.properties file. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | | presto-connector-hive | Change values in Presto's hive.properties file. | | presto-connector-jmx | Change values in Presto's jmx.properties file. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | | presto-connector-localfile | Change values in Presto's localfile.properties file. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | | presto-connector-redis | Change values in Presto's redis.properties file. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | | spark | Amazon EMR-curated settings for Apache Spark. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | | spark-env | Change values in the Spark environment. | | spark-hive-site | Change values in Spark's hive-site.xml file | | spark-log4j | Change values in Spark's log4j.properties file. | | spark-metrics | Change values in Spark's metrics.properties file. | | sqoop-env | Change values in Sqoop's environment. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | | tez-site | Change values in Tez's tez-site.xml file. | | yarn-env | Change values in the YARN environment. | | yarn-site | Change values in YARN's yarn-site.xml file. | | zeppelin-env | Change values in the Zeppelin environment. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | # Amazon EMR release 5.5.4 ## 5.5.4 application versions This release includes the following applications: [https://flink.apache.org/](https://flink.apache.org/), [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [http://gethue.com/](http://gethue.com/), [http://mahout.apache.org/](http://mahout.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-5.5.4 | emr-5.5.3 | emr-5.5.2 | emr-5.5.1 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.10.75 | 1.10.75 | 1.10.75 | 1.10.75 | | Python | Not tracked | Not tracked | Not tracked | Not tracked | | Scala | 2.11.8 | 2.11.8 | 2.11.8 | 2.11.8 | | AmazonCloudWatchAgent | - | - | - | - | | Delta | - | - | - | - | | Flink | 1.2.0 | 1.2.0 | 1.2.0 | 1.2.0 | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 1.3.0 | 1.3.0 | 1.3.0 | 1.3.0 | | HCatalog | 2.1.1-amzn-0 | 2.1.1-amzn-0 | 2.1.1-amzn-0 | 2.1.1-amzn-0 | | Hadoop | 2.7.3-amzn-2 | 2.7.3-amzn-2 | 2.7.3-amzn-2 | 2.7.3-amzn-2 | | Hive | 2.1.1-amzn-0 | 2.1.1-amzn-0 | 2.1.1-amzn-0 | 2.1.1-amzn-0 | | Hudi | - | - | - | - | | Hue | 3.12.0 | 3.12.0 | 3.12.0 | 3.12.0 | | Iceberg | - | - | - | - | | JupyterEnterpriseGateway | - | - | - | - | | JupyterHub | - | - | - | - | | Livy | - | - | - | - | | MXNet | - | - | - | - | | Mahout | 0.12.2 | 0.12.2 | 0.12.2 | 0.12.2 | | Oozie | 4.3.0 | 4.3.0 | 4.3.0 | 4.3.0 | | Phoenix | 4.9.0-HBase-1.2 | 4.9.0-HBase-1.2 | 4.9.0-HBase-1.2 | 4.9.0-HBase-1.2 | | Pig | 0.16.0-amzn-0 | 0.16.0-amzn-0 | 0.16.0-amzn-0 | 0.16.0-amzn-0 | | Presto | 0.170 | 0.170 | 0.170 | 0.170 | | Spark | 2.1.0 | 2.1.0 | 2.1.0 | 2.1.0 | | Sqoop | 1.4.6 | 1.4.6 | 1.4.6 | 1.4.6 | | TensorFlow | - | - | - | - | | Tez | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | | Trino (PrestoSQL) | - | - | - | - | | Zeppelin | 0.7.1 | 0.7.1 | 0.7.1 | 0.7.1 | | ZooKeeper | 3.4.10 | 3.4.10 | 3.4.10 | 3.4.10 | ## 5.5.4 release notes This is a patch release to add AWS Signature Version 4 authentication for requests to Amazon S3. All applications and components are the same as the previous Amazon EMR release. **Important** In this release version, Amazon EMR uses AWS Signature Version 4 exclusively to authenticate requests to Amazon S3. For more information, see [Whats New](https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-whatsnew.html). ## 5.5.4 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | emr-ddb | 4.3.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 2.3.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.3.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-s3-dist-cp | 2.5.0 | Distributed copy application optimized for Amazon S3. | | emrfs | 2.16.0 | Amazon S3 connector for Hadoop ecosystem applications. | | flink-client | 1.2.0 | Apache Flink command line client scripts and applications. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 2.7.3-amzn-2 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 2.7.3-amzn-2 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 2.7.3-amzn-2 | HDFS command-line client and library | | hadoop-hdfs-namenode | 2.7.3-amzn-2 | HDFS service for tracking file names and block locations. | | hadoop-httpfs-server | 2.7.3-amzn-2 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 2.7.3-amzn-2 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 2.7.3-amzn-2 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 2.7.3-amzn-2 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 2.7.3-amzn-2 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 2.7.3-amzn-2 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 1.3.0 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 1.3.0 | Service for serving one or more HBase regions. | | hbase-client | 1.3.0 | HBase command-line client. | | hbase-rest-server | 1.3.0 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 1.3.0 | Service providing a Thrift endpoint to HBase. | | hcatalog-client | 2.1.1-amzn-0 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 2.1.1-amzn-0 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 2.1.1-amzn-0 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 2.1.1-amzn-0 | Hive command line client. | | hive-hbase | 2.1.1-amzn-0 | Hive-hbase client. | | hive-metastore-server | 2.1.1-amzn-0 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server2 | 2.1.1-amzn-0 | Service for accepting Hive queries as web requests. | | hue-server | 3.12.0 | Web application for analyzing data using Hadoop ecosystem applications | | mahout-client | 0.12.2 | Library for machine learning. | | mysql-server | 5.5.54\$1 | MySQL database server. | | oozie-client | 4.3.0 | Oozie command-line client. | | oozie-server | 4.3.0 | Service for accepting Oozie workflow requests. | | phoenix-library | 4.9.0-HBase-1.2 | The phoenix libraries for server and client | | phoenix-query-server | 4.9.0-HBase-1.2 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.170 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.170 | Service for executing pieces of a query. | | pig-client | 0.16.0-amzn-0 | Pig command-line client. | | spark-client | 2.1.0 | Spark command-line clients. | | spark-history-server | 2.1.0 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 2.1.0 | In-memory execution engine for YARN. | | spark-yarn-slave | 2.1.0 | Apache Spark libraries needed by YARN slaves. | | sqoop-client | 1.4.6 | Apache Sqoop command-line client. | | tez-on-yarn | 0.8.4 | The tez YARN application and libraries. | | webserver | 2.4.25\$1 | Apache HTTP server. | | zeppelin-server | 0.7.1 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.4.10 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.4.10 | ZooKeeper command line client. | ## 5.5.4 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). **emr-5.5.4 classifications** | Classifications | Description | | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | | core-site | Change values in Hadoop's core-site.xml file. | | emrfs-site | Change EMRFS settings. | | flink-conf | Change flink-conf.yaml settings. | | flink-log4j | Change Flink log4j.properties settings. | | flink-log4j-yarn-session | Change Flink log4j-yarn-session.properties settings. | | flink-log4j-cli | Change Flink log4j-cli.properties settings. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | | hadoop-ssl-server | Change hadoop ssl server configuration | | hadoop-ssl-client | Change hadoop ssl client configuration | | hbase | Amazon EMR-curated settings for Apache HBase. | | hbase-env | Change values in HBase's environment. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbaase.properties file. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | | hbase-site | Change values in HBase's hbase-site.xml file. | | hdfs-encryption-zones | Configure HDFS encryption zones. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | | hcatalog-env | Change values in HCatalog's environment. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | | hcatalog-webhcat-log4j2 | Change values in HCatalog WebHCat's log4j2.properties. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | | hive-beeline-log4j2 | Change values in Hive's beeline-log4j2.properties file. | | hive-parquet-logging | Change values in Hive's parquet-logging.properties file. | | hive-env | Change values in the Hive environment. | | hive-exec-log4j2 | Change values in Hive's hive-exec-log4j2.properties file. | | hive-llap-daemon-log4j2 | Change values in Hive's llap-daemon-log4j2.properties file. | | hive-log4j2 | Change values in Hive's hive-log4j2.properties file. | | hive-site | Change values in Hive's hive-site.xml file | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | | hue-ini | Change values in Hue's ini file | | httpfs-env | Change values in the HTTPFS environment. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | | mapred-env | Change values in the MapReduce application's environment. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | | oozie-env | Change values in Oozie's environment. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | | oozie-site | Change values in Oozie's oozie-site.xml file. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | | pig-properties | Change values in Pig's pig.properties file. | | pig-log4j | Change values in Pig's log4j.properties file. | | presto-log | Change values in Presto's log.properties file. | | presto-config | Change values in Presto's config.properties file. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | | presto-connector-hive | Change values in Presto's hive.properties file. | | presto-connector-jmx | Change values in Presto's jmx.properties file. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | | presto-connector-localfile | Change values in Presto's localfile.properties file. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | | presto-connector-redis | Change values in Presto's redis.properties file. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | | spark | Amazon EMR-curated settings for Apache Spark. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | | spark-env | Change values in the Spark environment. | | spark-hive-site | Change values in Spark's hive-site.xml file | | spark-log4j | Change values in Spark's log4j.properties file. | | spark-metrics | Change values in Spark's metrics.properties file. | | sqoop-env | Change values in Sqoop's environment. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | | tez-site | Change values in Tez's tez-site.xml file. | | yarn-env | Change values in the YARN environment. | | yarn-site | Change values in YARN's yarn-site.xml file. | | zeppelin-env | Change values in the Zeppelin environment. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | # Amazon EMR release 5.5.3 ## 5.5.3 application versions This release includes the following applications: [https://flink.apache.org/](https://flink.apache.org/), [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [http://gethue.com/](http://gethue.com/), [http://mahout.apache.org/](http://mahout.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-5.5.3 | emr-5.5.2 | emr-5.5.1 | emr-5.5.0 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.10.75 | 1.10.75 | 1.10.75 | 1.10.75 | | Python | Not tracked | Not tracked | Not tracked | Not tracked | | Scala | 2.11.8 | 2.11.8 | 2.11.8 | 2.11.8 | | AmazonCloudWatchAgent | - | - | - | - | | Delta | - | - | - | - | | Flink | 1.2.0 | 1.2.0 | 1.2.0 | 1.2.0 | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 1.3.0 | 1.3.0 | 1.3.0 | 1.3.0 | | HCatalog | 2.1.1-amzn-0 | 2.1.1-amzn-0 | 2.1.1-amzn-0 | 2.1.1-amzn-0 | | Hadoop | 2.7.3-amzn-2 | 2.7.3-amzn-2 | 2.7.3-amzn-2 | 2.7.3-amzn-2 | | Hive | 2.1.1-amzn-0 | 2.1.1-amzn-0 | 2.1.1-amzn-0 | 2.1.1-amzn-0 | | Hudi | - | - | - | - | | Hue | 3.12.0 | 3.12.0 | 3.12.0 | 3.12.0 | | Iceberg | - | - | - | - | | JupyterEnterpriseGateway | - | - | - | - | | JupyterHub | - | - | - | - | | Livy | - | - | - | - | | MXNet | - | - | - | - | | Mahout | 0.12.2 | 0.12.2 | 0.12.2 | 0.12.2 | | Oozie | 4.3.0 | 4.3.0 | 4.3.0 | 4.3.0 | | Phoenix | 4.9.0-HBase-1.2 | 4.9.0-HBase-1.2 | 4.9.0-HBase-1.2 | 4.9.0-HBase-1.2 | | Pig | 0.16.0-amzn-0 | 0.16.0-amzn-0 | 0.16.0-amzn-0 | 0.16.0-amzn-0 | | Presto | 0.170 | 0.170 | 0.170 | 0.170 | | Spark | 2.1.0 | 2.1.0 | 2.1.0 | 2.1.0 | | Sqoop | 1.4.6 | 1.4.6 | 1.4.6 | 1.4.6 | | TensorFlow | - | - | - | - | | Tez | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | | Trino (PrestoSQL) | - | - | - | - | | Zeppelin | 0.7.1 | 0.7.1 | 0.7.1 | 0.7.1 | | ZooKeeper | 3.4.10 | 3.4.10 | 3.4.10 | 3.4.10 | ## 5.5.3 release notes The following release notes include information for Amazon EMR release 5.5.3. Changes are relative to 5.5.2. Initial release date: August 29, 2018 **Changes, enhancements, and resolved issues** + This release addresses a potential security vulnerability. ## 5.5.3 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | emr-ddb | 4.3.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 2.3.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.3.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-s3-dist-cp | 2.5.0 | Distributed copy application optimized for Amazon S3. | | emrfs | 2.16.0 | Amazon S3 connector for Hadoop ecosystem applications. | | flink-client | 1.2.0 | Apache Flink command line client scripts and applications. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 2.7.3-amzn-2 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 2.7.3-amzn-2 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 2.7.3-amzn-2 | HDFS command-line client and library | | hadoop-hdfs-namenode | 2.7.3-amzn-2 | HDFS service for tracking file names and block locations. | | hadoop-httpfs-server | 2.7.3-amzn-2 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 2.7.3-amzn-2 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 2.7.3-amzn-2 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 2.7.3-amzn-2 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 2.7.3-amzn-2 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 2.7.3-amzn-2 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 1.3.0 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 1.3.0 | Service for serving one or more HBase regions. | | hbase-client | 1.3.0 | HBase command-line client. | | hbase-rest-server | 1.3.0 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 1.3.0 | Service providing a Thrift endpoint to HBase. | | hcatalog-client | 2.1.1-amzn-0 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 2.1.1-amzn-0 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 2.1.1-amzn-0 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 2.1.1-amzn-0 | Hive command line client. | | hive-hbase | 2.1.1-amzn-0 | Hive-hbase client. | | hive-metastore-server | 2.1.1-amzn-0 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server2 | 2.1.1-amzn-0 | Service for accepting Hive queries as web requests. | | hue-server | 3.12.0 | Web application for analyzing data using Hadoop ecosystem applications | | mahout-client | 0.12.2 | Library for machine learning. | | mysql-server | 5.5.54\$1 | MySQL database server. | | oozie-client | 4.3.0 | Oozie command-line client. | | oozie-server | 4.3.0 | Service for accepting Oozie workflow requests. | | phoenix-library | 4.9.0-HBase-1.2 | The phoenix libraries for server and client | | phoenix-query-server | 4.9.0-HBase-1.2 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.170 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.170 | Service for executing pieces of a query. | | pig-client | 0.16.0-amzn-0 | Pig command-line client. | | spark-client | 2.1.0 | Spark command-line clients. | | spark-history-server | 2.1.0 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 2.1.0 | In-memory execution engine for YARN. | | spark-yarn-slave | 2.1.0 | Apache Spark libraries needed by YARN slaves. | | sqoop-client | 1.4.6 | Apache Sqoop command-line client. | | tez-on-yarn | 0.8.4 | The tez YARN application and libraries. | | webserver | 2.4.25\$1 | Apache HTTP server. | | zeppelin-server | 0.7.1 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.4.10 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.4.10 | ZooKeeper command line client. | ## 5.5.3 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). **emr-5.5.3 classifications** | Classifications | Description | | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | | core-site | Change values in Hadoop's core-site.xml file. | | emrfs-site | Change EMRFS settings. | | flink-conf | Change flink-conf.yaml settings. | | flink-log4j | Change Flink log4j.properties settings. | | flink-log4j-yarn-session | Change Flink log4j-yarn-session.properties settings. | | flink-log4j-cli | Change Flink log4j-cli.properties settings. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | | hadoop-ssl-server | Change hadoop ssl server configuration | | hadoop-ssl-client | Change hadoop ssl client configuration | | hbase | Amazon EMR-curated settings for Apache HBase. | | hbase-env | Change values in HBase's environment. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbaase.properties file. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | | hbase-site | Change values in HBase's hbase-site.xml file. | | hdfs-encryption-zones | Configure HDFS encryption zones. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | | hcatalog-env | Change values in HCatalog's environment. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | | hcatalog-webhcat-log4j2 | Change values in HCatalog WebHCat's log4j2.properties. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | | hive-beeline-log4j2 | Change values in Hive's beeline-log4j2.properties file. | | hive-parquet-logging | Change values in Hive's parquet-logging.properties file. | | hive-env | Change values in the Hive environment. | | hive-exec-log4j2 | Change values in Hive's hive-exec-log4j2.properties file. | | hive-llap-daemon-log4j2 | Change values in Hive's llap-daemon-log4j2.properties file. | | hive-log4j2 | Change values in Hive's hive-log4j2.properties file. | | hive-site | Change values in Hive's hive-site.xml file | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | | hue-ini | Change values in Hue's ini file | | httpfs-env | Change values in the HTTPFS environment. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | | mapred-env | Change values in the MapReduce application's environment. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | | oozie-env | Change values in Oozie's environment. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | | oozie-site | Change values in Oozie's oozie-site.xml file. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | | pig-properties | Change values in Pig's pig.properties file. | | pig-log4j | Change values in Pig's log4j.properties file. | | presto-log | Change values in Presto's log.properties file. | | presto-config | Change values in Presto's config.properties file. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | | presto-connector-hive | Change values in Presto's hive.properties file. | | presto-connector-jmx | Change values in Presto's jmx.properties file. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | | presto-connector-localfile | Change values in Presto's localfile.properties file. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | | presto-connector-redis | Change values in Presto's redis.properties file. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | | spark | Amazon EMR-curated settings for Apache Spark. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | | spark-env | Change values in the Spark environment. | | spark-hive-site | Change values in Spark's hive-site.xml file | | spark-log4j | Change values in Spark's log4j.properties file. | | spark-metrics | Change values in Spark's metrics.properties file. | | sqoop-env | Change values in Sqoop's environment. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | | tez-site | Change values in Tez's tez-site.xml file. | | yarn-env | Change values in the YARN environment. | | yarn-site | Change values in YARN's yarn-site.xml file. | | zeppelin-env | Change values in the Zeppelin environment. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | # Amazon EMR release 5.5.2 ## 5.5.2 application versions This release includes the following applications: [https://flink.apache.org/](https://flink.apache.org/), [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [http://gethue.com/](http://gethue.com/), [http://mahout.apache.org/](http://mahout.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-5.5.2 | emr-5.5.1 | emr-5.5.0 | emr-5.4.1 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.10.75 | 1.10.75 | 1.10.75 | 1.10.75 | | Python | Not tracked | Not tracked | Not tracked | Not tracked | | Scala | 2.11.8 | 2.11.8 | 2.11.8 | 2.11.8 | | AmazonCloudWatchAgent | - | - | - | - | | Delta | - | - | - | - | | Flink | 1.2.0 | 1.2.0 | 1.2.0 | 1.2.0 | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 1.3.0 | 1.3.0 | 1.3.0 | 1.3.0 | | HCatalog | 2.1.1-amzn-0 | 2.1.1-amzn-0 | 2.1.1-amzn-0 | 2.1.1-amzn-0 | | Hadoop | 2.7.3-amzn-2 | 2.7.3-amzn-2 | 2.7.3-amzn-2 | 2.7.3-amzn-1 | | Hive | 2.1.1-amzn-0 | 2.1.1-amzn-0 | 2.1.1-amzn-0 | 2.1.1-amzn-0 | | Hudi | - | - | - | - | | Hue | 3.12.0 | 3.12.0 | 3.12.0 | 3.11.0 | | Iceberg | - | - | - | - | | JupyterEnterpriseGateway | - | - | - | - | | JupyterHub | - | - | - | - | | Livy | - | - | - | - | | MXNet | - | - | - | - | | Mahout | 0.12.2 | 0.12.2 | 0.12.2 | 0.12.2 | | Oozie | 4.3.0 | 4.3.0 | 4.3.0 | 4.3.0 | | Phoenix | 4.9.0-HBase-1.2 | 4.9.0-HBase-1.2 | 4.9.0-HBase-1.2 | 4.9.0-HBase-1.2 | | Pig | 0.16.0-amzn-0 | 0.16.0-amzn-0 | 0.16.0-amzn-0 | 0.16.0-amzn-0 | | Presto | 0.170 | 0.170 | 0.170 | 0.166 | | Spark | 2.1.0 | 2.1.0 | 2.1.0 | 2.1.0 | | Sqoop | 1.4.6 | 1.4.6 | 1.4.6 | 1.4.6 | | TensorFlow | - | - | - | - | | Tez | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | | Trino (PrestoSQL) | - | - | - | - | | Zeppelin | 0.7.1 | 0.7.1 | 0.7.1 | 0.7.0 | | ZooKeeper | 3.4.10 | 3.4.10 | 3.4.10 | 3.4.9 | ## 5.5.2 release notes The following release notes include information for Amazon EMR release 5.5.2. Changes are relative to 5.5.1. Initial release date: March 29, 2018 **Changes, enhancements, and resolved issues** + Updated the Amazon Linux kernel of the defaultAmazon Linux AMI for Amazon EMR to address potential vulnerabilities. ## 5.5.2 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | emr-ddb | 4.3.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 2.3.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.3.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-s3-dist-cp | 2.5.0 | Distributed copy application optimized for Amazon S3. | | emrfs | 2.16.0 | Amazon S3 connector for Hadoop ecosystem applications. | | flink-client | 1.2.0 | Apache Flink command line client scripts and applications. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 2.7.3-amzn-2 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 2.7.3-amzn-2 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 2.7.3-amzn-2 | HDFS command-line client and library | | hadoop-hdfs-namenode | 2.7.3-amzn-2 | HDFS service for tracking file names and block locations. | | hadoop-httpfs-server | 2.7.3-amzn-2 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 2.7.3-amzn-2 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 2.7.3-amzn-2 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 2.7.3-amzn-2 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 2.7.3-amzn-2 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 2.7.3-amzn-2 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 1.3.0 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 1.3.0 | Service for serving one or more HBase regions. | | hbase-client | 1.3.0 | HBase command-line client. | | hbase-rest-server | 1.3.0 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 1.3.0 | Service providing a Thrift endpoint to HBase. | | hcatalog-client | 2.1.1-amzn-0 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 2.1.1-amzn-0 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 2.1.1-amzn-0 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 2.1.1-amzn-0 | Hive command line client. | | hive-hbase | 2.1.1-amzn-0 | Hive-hbase client. | | hive-metastore-server | 2.1.1-amzn-0 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server2 | 2.1.1-amzn-0 | Service for accepting Hive queries as web requests. | | hue-server | 3.12.0 | Web application for analyzing data using Hadoop ecosystem applications | | mahout-client | 0.12.2 | Library for machine learning. | | mysql-server | 5.5.54\$1 | MySQL database server. | | oozie-client | 4.3.0 | Oozie command-line client. | | oozie-server | 4.3.0 | Service for accepting Oozie workflow requests. | | phoenix-library | 4.9.0-HBase-1.2 | The phoenix libraries for server and client | | phoenix-query-server | 4.9.0-HBase-1.2 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.170 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.170 | Service for executing pieces of a query. | | pig-client | 0.16.0-amzn-0 | Pig command-line client. | | spark-client | 2.1.0 | Spark command-line clients. | | spark-history-server | 2.1.0 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 2.1.0 | In-memory execution engine for YARN. | | spark-yarn-slave | 2.1.0 | Apache Spark libraries needed by YARN slaves. | | sqoop-client | 1.4.6 | Apache Sqoop command-line client. | | tez-on-yarn | 0.8.4 | The tez YARN application and libraries. | | webserver | 2.4.25\$1 | Apache HTTP server. | | zeppelin-server | 0.7.1 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.4.10 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.4.10 | ZooKeeper command line client. | ## 5.5.2 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). **emr-5.5.2 classifications** | Classifications | Description | | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | | core-site | Change values in Hadoop's core-site.xml file. | | emrfs-site | Change EMRFS settings. | | flink-conf | Change flink-conf.yaml settings. | | flink-log4j | Change Flink log4j.properties settings. | | flink-log4j-yarn-session | Change Flink log4j-yarn-session.properties settings. | | flink-log4j-cli | Change Flink log4j-cli.properties settings. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | | hadoop-ssl-server | Change hadoop ssl server configuration | | hadoop-ssl-client | Change hadoop ssl client configuration | | hbase | Amazon EMR-curated settings for Apache HBase. | | hbase-env | Change values in HBase's environment. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbaase.properties file. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | | hbase-site | Change values in HBase's hbase-site.xml file. | | hdfs-encryption-zones | Configure HDFS encryption zones. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | | hcatalog-env | Change values in HCatalog's environment. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | | hcatalog-webhcat-log4j2 | Change values in HCatalog WebHCat's log4j2.properties. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | | hive-beeline-log4j2 | Change values in Hive's beeline-log4j2.properties file. | | hive-parquet-logging | Change values in Hive's parquet-logging.properties file. | | hive-env | Change values in the Hive environment. | | hive-exec-log4j2 | Change values in Hive's hive-exec-log4j2.properties file. | | hive-llap-daemon-log4j2 | Change values in Hive's llap-daemon-log4j2.properties file. | | hive-log4j2 | Change values in Hive's hive-log4j2.properties file. | | hive-site | Change values in Hive's hive-site.xml file | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | | hue-ini | Change values in Hue's ini file | | httpfs-env | Change values in the HTTPFS environment. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | | mapred-env | Change values in the MapReduce application's environment. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | | oozie-env | Change values in Oozie's environment. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | | oozie-site | Change values in Oozie's oozie-site.xml file. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | | pig-properties | Change values in Pig's pig.properties file. | | pig-log4j | Change values in Pig's log4j.properties file. | | presto-log | Change values in Presto's log.properties file. | | presto-config | Change values in Presto's config.properties file. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | | presto-connector-hive | Change values in Presto's hive.properties file. | | presto-connector-jmx | Change values in Presto's jmx.properties file. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | | presto-connector-localfile | Change values in Presto's localfile.properties file. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | | presto-connector-redis | Change values in Presto's redis.properties file. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | | spark | Amazon EMR-curated settings for Apache Spark. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | | spark-env | Change values in the Spark environment. | | spark-hive-site | Change values in Spark's hive-site.xml file | | spark-log4j | Change values in Spark's log4j.properties file. | | spark-metrics | Change values in Spark's metrics.properties file. | | sqoop-env | Change values in Sqoop's environment. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | | tez-site | Change values in Tez's tez-site.xml file. | | yarn-env | Change values in the YARN environment. | | yarn-site | Change values in YARN's yarn-site.xml file. | | zeppelin-env | Change values in the Zeppelin environment. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | # Amazon EMR release 5.5.1 ## 5.5.1 application versions This release includes the following applications: [https://flink.apache.org/](https://flink.apache.org/), [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [http://gethue.com/](http://gethue.com/), [http://mahout.apache.org/](http://mahout.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-5.5.1 | emr-5.5.0 | emr-5.4.1 | emr-5.4.0 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.10.75 | 1.10.75 | 1.10.75 | 1.10.75 | | Python | Not tracked | Not tracked | Not tracked | Not tracked | | Scala | 2.11.8 | 2.11.8 | 2.11.8 | 2.11.8 | | AmazonCloudWatchAgent | - | - | - | - | | Delta | - | - | - | - | | Flink | 1.2.0 | 1.2.0 | 1.2.0 | 1.2.0 | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 1.3.0 | 1.3.0 | 1.3.0 | 1.3.0 | | HCatalog | 2.1.1-amzn-0 | 2.1.1-amzn-0 | 2.1.1-amzn-0 | 2.1.1-amzn-0 | | Hadoop | 2.7.3-amzn-2 | 2.7.3-amzn-2 | 2.7.3-amzn-1 | 2.7.3-amzn-1 | | Hive | 2.1.1-amzn-0 | 2.1.1-amzn-0 | 2.1.1-amzn-0 | 2.1.1-amzn-0 | | Hudi | - | - | - | - | | Hue | 3.12.0 | 3.12.0 | 3.11.0 | 3.11.0 | | Iceberg | - | - | - | - | | JupyterEnterpriseGateway | - | - | - | - | | JupyterHub | - | - | - | - | | Livy | - | - | - | - | | MXNet | - | - | - | - | | Mahout | 0.12.2 | 0.12.2 | 0.12.2 | 0.12.2 | | Oozie | 4.3.0 | 4.3.0 | 4.3.0 | 4.3.0 | | Phoenix | 4.9.0-HBase-1.2 | 4.9.0-HBase-1.2 | 4.9.0-HBase-1.2 | 4.9.0-HBase-1.2 | | Pig | 0.16.0-amzn-0 | 0.16.0-amzn-0 | 0.16.0-amzn-0 | 0.16.0-amzn-0 | | Presto | 0.170 | 0.170 | 0.166 | 0.166 | | Spark | 2.1.0 | 2.1.0 | 2.1.0 | 2.1.0 | | Sqoop | 1.4.6 | 1.4.6 | 1.4.6 | 1.4.6 | | TensorFlow | - | - | - | - | | Tez | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | | Trino (PrestoSQL) | - | - | - | - | | Zeppelin | 0.7.1 | 0.7.1 | 0.7.0 | 0.7.0 | | ZooKeeper | 3.4.10 | 3.4.10 | 3.4.9 | 3.4.9 | ## 5.5.1 release notes The following release notes include information for the Amazon EMR 5.5.1 release. Changes are relative to the Amazon EMR 5.5.0 release. Initial release date: January 22, 2018 **Changes, enhancements, and resolved issues** + Updated the Amazon Linux kernel of the defaultAmazon Linux AMI for Amazon EMR to address vulnerabilities associated with speculative execution (CVE-2017-5715, CVE-2017-5753, and CVE-2017-5754). For more information, see [https://aws.amazon.com/security/security-bulletins/AWS-2018-013/](https://aws.amazon.com/security/security-bulletins/AWS-2018-013/). ## 5.5.1 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | emr-ddb | 4.3.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 2.3.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.3.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-s3-dist-cp | 2.5.0 | Distributed copy application optimized for Amazon S3. | | emrfs | 2.16.0 | Amazon S3 connector for Hadoop ecosystem applications. | | flink-client | 1.2.0 | Apache Flink command line client scripts and applications. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 2.7.3-amzn-2 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 2.7.3-amzn-2 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 2.7.3-amzn-2 | HDFS command-line client and library | | hadoop-hdfs-namenode | 2.7.3-amzn-2 | HDFS service for tracking file names and block locations. | | hadoop-httpfs-server | 2.7.3-amzn-2 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 2.7.3-amzn-2 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 2.7.3-amzn-2 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 2.7.3-amzn-2 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 2.7.3-amzn-2 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 2.7.3-amzn-2 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 1.3.0 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 1.3.0 | Service for serving one or more HBase regions. | | hbase-client | 1.3.0 | HBase command-line client. | | hbase-rest-server | 1.3.0 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 1.3.0 | Service providing a Thrift endpoint to HBase. | | hcatalog-client | 2.1.1-amzn-0 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 2.1.1-amzn-0 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 2.1.1-amzn-0 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 2.1.1-amzn-0 | Hive command line client. | | hive-hbase | 2.1.1-amzn-0 | Hive-hbase client. | | hive-metastore-server | 2.1.1-amzn-0 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server2 | 2.1.1-amzn-0 | Service for accepting Hive queries as web requests. | | hue-server | 3.12.0 | Web application for analyzing data using Hadoop ecosystem applications | | mahout-client | 0.12.2 | Library for machine learning. | | mysql-server | 5.5.54\$1 | MySQL database server. | | oozie-client | 4.3.0 | Oozie command-line client. | | oozie-server | 4.3.0 | Service for accepting Oozie workflow requests. | | phoenix-library | 4.9.0-HBase-1.2 | The phoenix libraries for server and client | | phoenix-query-server | 4.9.0-HBase-1.2 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.170 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.170 | Service for executing pieces of a query. | | pig-client | 0.16.0-amzn-0 | Pig command-line client. | | spark-client | 2.1.0 | Spark command-line clients. | | spark-history-server | 2.1.0 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 2.1.0 | In-memory execution engine for YARN. | | spark-yarn-slave | 2.1.0 | Apache Spark libraries needed by YARN slaves. | | sqoop-client | 1.4.6 | Apache Sqoop command-line client. | | tez-on-yarn | 0.8.4 | The tez YARN application and libraries. | | webserver | 2.4.25\$1 | Apache HTTP server. | | zeppelin-server | 0.7.1 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.4.10 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.4.10 | ZooKeeper command line client. | ## 5.5.1 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). **emr-5.5.1 classifications** | Classifications | Description | | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | | core-site | Change values in Hadoop's core-site.xml file. | | emrfs-site | Change EMRFS settings. | | flink-conf | Change flink-conf.yaml settings. | | flink-log4j | Change Flink log4j.properties settings. | | flink-log4j-yarn-session | Change Flink log4j-yarn-session.properties settings. | | flink-log4j-cli | Change Flink log4j-cli.properties settings. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | | hadoop-ssl-server | Change hadoop ssl server configuration | | hadoop-ssl-client | Change hadoop ssl client configuration | | hbase | Amazon EMR-curated settings for Apache HBase. | | hbase-env | Change values in HBase's environment. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbaase.properties file. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | | hbase-site | Change values in HBase's hbase-site.xml file. | | hdfs-encryption-zones | Configure HDFS encryption zones. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | | hcatalog-env | Change values in HCatalog's environment. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | | hcatalog-webhcat-log4j2 | Change values in HCatalog WebHCat's log4j2.properties. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | | hive-beeline-log4j2 | Change values in Hive's beeline-log4j2.properties file. | | hive-parquet-logging | Change values in Hive's parquet-logging.properties file. | | hive-env | Change values in the Hive environment. | | hive-exec-log4j2 | Change values in Hive's hive-exec-log4j2.properties file. | | hive-llap-daemon-log4j2 | Change values in Hive's llap-daemon-log4j2.properties file. | | hive-log4j2 | Change values in Hive's hive-log4j2.properties file. | | hive-site | Change values in Hive's hive-site.xml file | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | | hue-ini | Change values in Hue's ini file | | httpfs-env | Change values in the HTTPFS environment. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | | mapred-env | Change values in the MapReduce application's environment. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | | oozie-env | Change values in Oozie's environment. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | | oozie-site | Change values in Oozie's oozie-site.xml file. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | | pig-properties | Change values in Pig's pig.properties file. | | pig-log4j | Change values in Pig's log4j.properties file. | | presto-log | Change values in Presto's log.properties file. | | presto-config | Change values in Presto's config.properties file. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | | presto-connector-hive | Change values in Presto's hive.properties file. | | presto-connector-jmx | Change values in Presto's jmx.properties file. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | | presto-connector-localfile | Change values in Presto's localfile.properties file. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | | presto-connector-redis | Change values in Presto's redis.properties file. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | | spark | Amazon EMR-curated settings for Apache Spark. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | | spark-env | Change values in the Spark environment. | | spark-hive-site | Change values in Spark's hive-site.xml file | | spark-log4j | Change values in Spark's log4j.properties file. | | spark-metrics | Change values in Spark's metrics.properties file. | | sqoop-env | Change values in Sqoop's environment. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | | tez-site | Change values in Tez's tez-site.xml file. | | yarn-env | Change values in the YARN environment. | | yarn-site | Change values in YARN's yarn-site.xml file. | | zeppelin-env | Change values in the Zeppelin environment. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | # Amazon EMR release 5.5.0 ## 5.5.0 application versions This release includes the following applications: [https://flink.apache.org/](https://flink.apache.org/), [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [http://gethue.com/](http://gethue.com/), [http://mahout.apache.org/](http://mahout.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-5.5.0 | emr-5.4.1 | emr-5.4.0 | emr-5.3.2 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.10.75 | 1.10.75 | 1.10.75 | 1.10.75 | | Python | Not tracked | Not tracked | Not tracked | Not tracked | | Scala | 2.11.8 | 2.11.8 | 2.11.8 | 2.11.8 | | AmazonCloudWatchAgent | - | - | - | - | | Delta | - | - | - | - | | Flink | 1.2.0 | 1.2.0 | 1.2.0 | 1.1.4 | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 1.3.0 | 1.3.0 | 1.3.0 | 1.2.3 | | HCatalog | 2.1.1-amzn-0 | 2.1.1-amzn-0 | 2.1.1-amzn-0 | 2.1.1-amzn-0 | | Hadoop | 2.7.3-amzn-2 | 2.7.3-amzn-1 | 2.7.3-amzn-1 | 2.7.3-amzn-1 | | Hive | 2.1.1-amzn-0 | 2.1.1-amzn-0 | 2.1.1-amzn-0 | 2.1.1-amzn-0 | | Hudi | - | - | - | - | | Hue | 3.12.0 | 3.11.0 | 3.11.0 | 3.11.0 | | Iceberg | - | - | - | - | | JupyterEnterpriseGateway | - | - | - | - | | JupyterHub | - | - | - | - | | Livy | - | - | - | - | | MXNet | - | - | - | - | | Mahout | 0.12.2 | 0.12.2 | 0.12.2 | 0.12.2 | | Oozie | 4.3.0 | 4.3.0 | 4.3.0 | 4.3.0 | | Phoenix | 4.9.0-HBase-1.2 | 4.9.0-HBase-1.2 | 4.9.0-HBase-1.2 | 4.7.0-HBase-1.2 | | Pig | 0.16.0-amzn-0 | 0.16.0-amzn-0 | 0.16.0-amzn-0 | 0.16.0-amzn-0 | | Presto | 0.170 | 0.166 | 0.166 | 0.157.1 | | Spark | 2.1.0 | 2.1.0 | 2.1.0 | 2.1.0 | | Sqoop | 1.4.6 | 1.4.6 | 1.4.6 | 1.4.6 | | TensorFlow | - | - | - | - | | Tez | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | | Trino (PrestoSQL) | - | - | - | - | | Zeppelin | 0.7.1 | 0.7.0 | 0.7.0 | 0.6.2 | | ZooKeeper | 3.4.10 | 3.4.9 | 3.4.9 | 3.4.9 | ## 5.5.0 release notes The following release notes include information for the Amazon EMR 5.5.0 release. Changes are relative to the Amazon EMR 5.4.0 release. Release date: April 26, 2017 **Upgrades** + Hue 3.12 + Presto 0.170 + Zeppelin 0.7.1 + ZooKeeper 3.4.10 **Changes, enhancements, and resolved issues** + **Spark** + Backported Spark Patch [(SPARK-20115) Fix DAGScheduler to recompute all the lost shuffle blocks when external shuffle service is unavailable](https://issues.apache.org/jira/browse/SPARK-20115) to version 2.1.0 of Spark, which is included in this release. + **Flink** + Flink is now built with Scala 2.11. If you use the Scala API and libraries, we recommend that you use Scala 2.11 in your projects. + Addressed an issue where `HADOOP_CONF_DIR` and `YARN_CONF_DIR` defaults were not properly set, so `start-scala-shell.sh` failed to work. Also added the ability to set these values using `env.hadoop.conf.dir` and `env.yarn.conf.dir` in `/etc/flink/conf/flink-conf.yaml` or the `flink-conf` configuration classification. + Introduced a new EMR-specific command, `flink-scala-shell` as a wrapper for `start-scala-shell.sh`. We recommend using this command instead of `start-scala-shell`. The new command simplifies execution. For example, `flink-scala-shell -n 2` starts a Flink Scala shell with a task parallelism of 2. + Introduced a new EMR-specific command, `flink-yarn-session` as a wrapper for `yarn-session.sh`. We recommend using this command instead of `yarn-session`. The new command simplifies execution. For example, `flink-yarn-session -d -n 2` starts a long-running Flink session in a detached state with two task managers. + Addressed [(FLINK-6125) Commons httpclient is not shaded anymore in Flink 1.2](https://issues.apache.org/jira/browse/FLINK-6125). + **Presto** + Added support for LDAP authentication. Using LDAP with Presto on Amazon EMR requires that you enable HTTPS access for the Presto coordinator (`http-server.https.enabled=true` in `config.properties`). For configuration details, see [LDAP Authentication](https://prestodb.io/docs/current/security/ldap.html) in Presto documentation. + Added support for `SHOW GRANTS`. + **Amazon EMR Base Linux AMI** + Amazon EMR releases are now based on Amazon Linux 2017.03. For more information, see [Amazon Linux AMI 2017.03 Release Notes](https://aws.amazon.com/amazon-linux-ami/2017.03-release-notes/). + Removed Python 2.6 from the Amazon EMR base Linux image. Python 2.7 and 3.4 are installed by default. You can install Python 2.6 manually if necessary. ## 5.5.0 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | emr-ddb | 4.3.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 2.3.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.3.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-s3-dist-cp | 2.5.0 | Distributed copy application optimized for Amazon S3. | | emrfs | 2.16.0 | Amazon S3 connector for Hadoop ecosystem applications. | | flink-client | 1.2.0 | Apache Flink command line client scripts and applications. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 2.7.3-amzn-2 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 2.7.3-amzn-2 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 2.7.3-amzn-2 | HDFS command-line client and library | | hadoop-hdfs-namenode | 2.7.3-amzn-2 | HDFS service for tracking file names and block locations. | | hadoop-httpfs-server | 2.7.3-amzn-2 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 2.7.3-amzn-2 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 2.7.3-amzn-2 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 2.7.3-amzn-2 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 2.7.3-amzn-2 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 2.7.3-amzn-2 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 1.3.0 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 1.3.0 | Service for serving one or more HBase regions. | | hbase-client | 1.3.0 | HBase command-line client. | | hbase-rest-server | 1.3.0 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 1.3.0 | Service providing a Thrift endpoint to HBase. | | hcatalog-client | 2.1.1-amzn-0 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 2.1.1-amzn-0 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 2.1.1-amzn-0 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 2.1.1-amzn-0 | Hive command line client. | | hive-hbase | 2.1.1-amzn-0 | Hive-hbase client. | | hive-metastore-server | 2.1.1-amzn-0 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server2 | 2.1.1-amzn-0 | Service for accepting Hive queries as web requests. | | hue-server | 3.12.0 | Web application for analyzing data using Hadoop ecosystem applications | | mahout-client | 0.12.2 | Library for machine learning. | | mysql-server | 5.5.54\$1 | MySQL database server. | | oozie-client | 4.3.0 | Oozie command-line client. | | oozie-server | 4.3.0 | Service for accepting Oozie workflow requests. | | phoenix-library | 4.9.0-HBase-1.2 | The phoenix libraries for server and client | | phoenix-query-server | 4.9.0-HBase-1.2 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.170 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.170 | Service for executing pieces of a query. | | pig-client | 0.16.0-amzn-0 | Pig command-line client. | | spark-client | 2.1.0 | Spark command-line clients. | | spark-history-server | 2.1.0 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 2.1.0 | In-memory execution engine for YARN. | | spark-yarn-slave | 2.1.0 | Apache Spark libraries needed by YARN slaves. | | sqoop-client | 1.4.6 | Apache Sqoop command-line client. | | tez-on-yarn | 0.8.4 | The tez YARN application and libraries. | | webserver | 2.4.25\$1 | Apache HTTP server. | | zeppelin-server | 0.7.1 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.4.10 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.4.10 | ZooKeeper command line client. | ## 5.5.0 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). **emr-5.5.0 classifications** | Classifications | Description | | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | | core-site | Change values in Hadoop's core-site.xml file. | | emrfs-site | Change EMRFS settings. | | flink-conf | Change flink-conf.yaml settings. | | flink-log4j | Change Flink log4j.properties settings. | | flink-log4j-yarn-session | Change Flink log4j-yarn-session.properties settings. | | flink-log4j-cli | Change Flink log4j-cli.properties settings. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | | hadoop-ssl-server | Change hadoop ssl server configuration | | hadoop-ssl-client | Change hadoop ssl client configuration | | hbase | Amazon EMR-curated settings for Apache HBase. | | hbase-env | Change values in HBase's environment. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbaase.properties file. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | | hbase-site | Change values in HBase's hbase-site.xml file. | | hdfs-encryption-zones | Configure HDFS encryption zones. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | | hcatalog-env | Change values in HCatalog's environment. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | | hcatalog-webhcat-log4j2 | Change values in HCatalog WebHCat's log4j2.properties. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | | hive-beeline-log4j2 | Change values in Hive's beeline-log4j2.properties file. | | hive-parquet-logging | Change values in Hive's parquet-logging.properties file. | | hive-env | Change values in the Hive environment. | | hive-exec-log4j2 | Change values in Hive's hive-exec-log4j2.properties file. | | hive-llap-daemon-log4j2 | Change values in Hive's llap-daemon-log4j2.properties file. | | hive-log4j2 | Change values in Hive's hive-log4j2.properties file. | | hive-site | Change values in Hive's hive-site.xml file | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | | hue-ini | Change values in Hue's ini file | | httpfs-env | Change values in the HTTPFS environment. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | | mapred-env | Change values in the MapReduce application's environment. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | | oozie-env | Change values in Oozie's environment. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | | oozie-site | Change values in Oozie's oozie-site.xml file. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | | pig-properties | Change values in Pig's pig.properties file. | | pig-log4j | Change values in Pig's log4j.properties file. | | presto-log | Change values in Presto's log.properties file. | | presto-config | Change values in Presto's config.properties file. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | | presto-connector-hive | Change values in Presto's hive.properties file. | | presto-connector-jmx | Change values in Presto's jmx.properties file. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | | presto-connector-localfile | Change values in Presto's localfile.properties file. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | | presto-connector-redis | Change values in Presto's redis.properties file. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | | spark | Amazon EMR-curated settings for Apache Spark. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | | spark-env | Change values in the Spark environment. | | spark-hive-site | Change values in Spark's hive-site.xml file | | spark-log4j | Change values in Spark's log4j.properties file. | | spark-metrics | Change values in Spark's metrics.properties file. | | sqoop-env | Change values in Sqoop's environment. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | | tez-site | Change values in Tez's tez-site.xml file. | | yarn-env | Change values in the YARN environment. | | yarn-site | Change values in YARN's yarn-site.xml file. | | zeppelin-env | Change values in the Zeppelin environment. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | # Amazon EMR release 5.4.1 ## 5.4.1 application versions This release includes the following applications: [https://flink.apache.org/](https://flink.apache.org/), [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [http://gethue.com/](http://gethue.com/), [http://mahout.apache.org/](http://mahout.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-5.4.1 | emr-5.4.0 | emr-5.3.2 | emr-5.3.1 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.10.75 | 1.10.75 | 1.10.75 | 1.10.75 | | Python | Not tracked | Not tracked | Not tracked | Not tracked | | Scala | 2.11.8 | 2.11.8 | 2.11.8 | 2.11.8 | | AmazonCloudWatchAgent | - | - | - | - | | Delta | - | - | - | - | | Flink | 1.2.0 | 1.2.0 | 1.1.4 | 1.1.4 | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 1.3.0 | 1.3.0 | 1.2.3 | 1.2.3 | | HCatalog | 2.1.1-amzn-0 | 2.1.1-amzn-0 | 2.1.1-amzn-0 | 2.1.1-amzn-0 | | Hadoop | 2.7.3-amzn-1 | 2.7.3-amzn-1 | 2.7.3-amzn-1 | 2.7.3-amzn-1 | | Hive | 2.1.1-amzn-0 | 2.1.1-amzn-0 | 2.1.1-amzn-0 | 2.1.1-amzn-0 | | Hudi | - | - | - | - | | Hue | 3.11.0 | 3.11.0 | 3.11.0 | 3.11.0 | | Iceberg | - | - | - | - | | JupyterEnterpriseGateway | - | - | - | - | | JupyterHub | - | - | - | - | | Livy | - | - | - | - | | MXNet | - | - | - | - | | Mahout | 0.12.2 | 0.12.2 | 0.12.2 | 0.12.2 | | Oozie | 4.3.0 | 4.3.0 | 4.3.0 | 4.3.0 | | Phoenix | 4.9.0-HBase-1.2 | 4.9.0-HBase-1.2 | 4.7.0-HBase-1.2 | 4.7.0-HBase-1.2 | | Pig | 0.16.0-amzn-0 | 0.16.0-amzn-0 | 0.16.0-amzn-0 | 0.16.0-amzn-0 | | Presto | 0.166 | 0.166 | 0.157.1 | 0.157.1 | | Spark | 2.1.0 | 2.1.0 | 2.1.0 | 2.1.0 | | Sqoop | 1.4.6 | 1.4.6 | 1.4.6 | 1.4.6 | | TensorFlow | - | - | - | - | | Tez | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | | Trino (PrestoSQL) | - | - | - | - | | Zeppelin | 0.7.0 | 0.7.0 | 0.6.2 | 0.6.2 | | ZooKeeper | 3.4.9 | 3.4.9 | 3.4.9 | 3.4.9 | ## 5.4.1 release notes This is a patch release to add AWS Signature Version 4 authentication for requests to Amazon S3. All applications and components are the same as the previous Amazon EMR release. **Important** In this release version, Amazon EMR uses AWS Signature Version 4 exclusively to authenticate requests to Amazon S3. For more information, see [Whats New](https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-whatsnew.html). ## 5.4.1 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | emr-ddb | 4.2.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 2.3.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.2.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-s3-dist-cp | 2.4.0 | Distributed copy application optimized for Amazon S3. | | emrfs | 2.15.0 | Amazon S3 connector for Hadoop ecosystem applications. | | flink-client | 1.2.0 | Apache Flink command line client scripts and applications. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 2.7.3-amzn-1 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 2.7.3-amzn-1 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 2.7.3-amzn-1 | HDFS command-line client and library | | hadoop-hdfs-namenode | 2.7.3-amzn-1 | HDFS service for tracking file names and block locations. | | hadoop-httpfs-server | 2.7.3-amzn-1 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 2.7.3-amzn-1 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 2.7.3-amzn-1 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 2.7.3-amzn-1 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 2.7.3-amzn-1 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 2.7.3-amzn-1 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 1.3.0 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 1.3.0 | Service for serving one or more HBase regions. | | hbase-client | 1.3.0 | HBase command-line client. | | hbase-rest-server | 1.3.0 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 1.3.0 | Service providing a Thrift endpoint to HBase. | | hcatalog-client | 2.1.1-amzn-0 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 2.1.1-amzn-0 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 2.1.1-amzn-0 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 2.1.1-amzn-0 | Hive command line client. | | hive-hbase | 2.1.1-amzn-0 | Hive-hbase client. | | hive-metastore-server | 2.1.1-amzn-0 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server2 | 2.1.1-amzn-0 | Service for accepting Hive queries as web requests. | | hue-server | 3.11.0 | Web application for analyzing data using Hadoop ecosystem applications | | mahout-client | 0.12.2 | Library for machine learning. | | mysql-server | 5.5.54\$1 | MySQL database server. | | oozie-client | 4.3.0 | Oozie command-line client. | | oozie-server | 4.3.0 | Service for accepting Oozie workflow requests. | | phoenix-library | 4.9.0-HBase-1.2 | The phoenix libraries for server and client | | phoenix-query-server | 4.9.0-HBase-1.2 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.166 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.166 | Service for executing pieces of a query. | | pig-client | 0.16.0-amzn-0 | Pig command-line client. | | spark-client | 2.1.0 | Spark command-line clients. | | spark-history-server | 2.1.0 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 2.1.0 | In-memory execution engine for YARN. | | spark-yarn-slave | 2.1.0 | Apache Spark libraries needed by YARN slaves. | | sqoop-client | 1.4.6 | Apache Sqoop command-line client. | | tez-on-yarn | 0.8.4 | The tez YARN application and libraries. | | webserver | 2.4.25\$1 | Apache HTTP server. | | zeppelin-server | 0.7.0 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.4.9 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.4.9 | ZooKeeper command line client. | ## 5.4.1 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). **emr-5.4.1 classifications** | Classifications | Description | | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | | core-site | Change values in Hadoop's core-site.xml file. | | emrfs-site | Change EMRFS settings. | | flink-conf | Change flink-conf.yaml settings. | | flink-log4j | Change Flink log4j.properties settings. | | flink-log4j-yarn-session | Change Flink log4j-yarn-session.properties settings. | | flink-log4j-cli | Change Flink log4j-cli.properties settings. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | | hadoop-ssl-server | Change hadoop ssl server configuration | | hadoop-ssl-client | Change hadoop ssl client configuration | | hbase | Amazon EMR-curated settings for Apache HBase. | | hbase-env | Change values in HBase's environment. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbaase.properties file. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | | hbase-site | Change values in HBase's hbase-site.xml file. | | hdfs-encryption-zones | Configure HDFS encryption zones. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | | hcatalog-env | Change values in HCatalog's environment. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | | hcatalog-webhcat-log4j2 | Change values in HCatalog WebHCat's log4j2.properties. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | | hive-beeline-log4j2 | Change values in Hive's beeline-log4j2.properties file. | | hive-parquet-logging | Change values in Hive's parquet-logging.properties file. | | hive-env | Change values in the Hive environment. | | hive-exec-log4j2 | Change values in Hive's hive-exec-log4j2.properties file. | | hive-llap-daemon-log4j2 | Change values in Hive's llap-daemon-log4j2.properties file. | | hive-log4j2 | Change values in Hive's hive-log4j2.properties file. | | hive-site | Change values in Hive's hive-site.xml file | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | | hue-ini | Change values in Hue's ini file | | httpfs-env | Change values in the HTTPFS environment. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | | mapred-env | Change values in the MapReduce application's environment. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | | oozie-env | Change values in Oozie's environment. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | | oozie-site | Change values in Oozie's oozie-site.xml file. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | | pig-properties | Change values in Pig's pig.properties file. | | pig-log4j | Change values in Pig's log4j.properties file. | | presto-log | Change values in Presto's log.properties file. | | presto-config | Change values in Presto's config.properties file. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | | presto-connector-hive | Change values in Presto's hive.properties file. | | presto-connector-jmx | Change values in Presto's jmx.properties file. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | | presto-connector-localfile | Change values in Presto's localfile.properties file. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | | presto-connector-redis | Change values in Presto's redis.properties file. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | | spark | Amazon EMR-curated settings for Apache Spark. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | | spark-env | Change values in the Spark environment. | | spark-hive-site | Change values in Spark's hive-site.xml file | | spark-log4j | Change values in Spark's log4j.properties file. | | spark-metrics | Change values in Spark's metrics.properties file. | | sqoop-env | Change values in Sqoop's environment. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | | tez-site | Change values in Tez's tez-site.xml file. | | yarn-env | Change values in the YARN environment. | | yarn-site | Change values in YARN's yarn-site.xml file. | | zeppelin-env | Change values in the Zeppelin environment. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | # Amazon EMR release 5.4.0 ## 5.4.0 application versions This release includes the following applications: [https://flink.apache.org/](https://flink.apache.org/), [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [http://gethue.com/](http://gethue.com/), [http://mahout.apache.org/](http://mahout.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-5.4.0 | emr-5.3.2 | emr-5.3.1 | emr-5.3.0 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.10.75 | 1.10.75 | 1.10.75 | 1.10.75 | | Python | Not tracked | Not tracked | Not tracked | Not tracked | | Scala | 2.11.8 | 2.11.8 | 2.11.8 | 2.11.8 | | AmazonCloudWatchAgent | - | - | - | - | | Delta | - | - | - | - | | Flink | 1.2.0 | 1.1.4 | 1.1.4 | 1.1.4 | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 1.3.0 | 1.2.3 | 1.2.3 | 1.2.3 | | HCatalog | 2.1.1-amzn-0 | 2.1.1-amzn-0 | 2.1.1-amzn-0 | 2.1.1-amzn-0 | | Hadoop | 2.7.3-amzn-1 | 2.7.3-amzn-1 | 2.7.3-amzn-1 | 2.7.3-amzn-1 | | Hive | 2.1.1-amzn-0 | 2.1.1-amzn-0 | 2.1.1-amzn-0 | 2.1.1-amzn-0 | | Hudi | - | - | - | - | | Hue | 3.11.0 | 3.11.0 | 3.11.0 | 3.11.0 | | Iceberg | - | - | - | - | | JupyterEnterpriseGateway | - | - | - | - | | JupyterHub | - | - | - | - | | Livy | - | - | - | - | | MXNet | - | - | - | - | | Mahout | 0.12.2 | 0.12.2 | 0.12.2 | 0.12.2 | | Oozie | 4.3.0 | 4.3.0 | 4.3.0 | 4.3.0 | | Phoenix | 4.9.0-HBase-1.2 | 4.7.0-HBase-1.2 | 4.7.0-HBase-1.2 | 4.7.0-HBase-1.2 | | Pig | 0.16.0-amzn-0 | 0.16.0-amzn-0 | 0.16.0-amzn-0 | 0.16.0-amzn-0 | | Presto | 0.166 | 0.157.1 | 0.157.1 | 0.157.1 | | Spark | 2.1.0 | 2.1.0 | 2.1.0 | 2.1.0 | | Sqoop | 1.4.6 | 1.4.6 | 1.4.6 | 1.4.6 | | TensorFlow | - | - | - | - | | Tez | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | | Trino (PrestoSQL) | - | - | - | - | | Zeppelin | 0.7.0 | 0.6.2 | 0.6.2 | 0.6.2 | | ZooKeeper | 3.4.9 | 3.4.9 | 3.4.9 | 3.4.9 | ## 5.4.0 release notes The following release notes include information for the Amazon EMR 5.4.0 release. Changes are relative to the Amazon EMR 5.3.0 release. Release date: March 08, 2017 **Upgrades** + Upgraded to Flink 1.2.0 + Upgraded to HBase 1.3.0 + Upgraded to Phoenix 4.9.0 **Note** If you upgrade from an earlier version of Amazon EMR to Amazon EMR version 5.4.0 or later and use secondary indexing, upgrade local indexes as described in the [Apache Phoenix documentation](https://phoenix.apache.org/secondary_indexing.html#Upgrading_Local_Indexes_created_before_4.8.0). Amazon EMR removes the required configurations from the `hbase-site` classification, but indexes need to be repopulated. Online and offline upgrade of indexes are supported. Online upgrades are the default, which means indexes are repopulated while initializing from Phoenix clients of version 4.8.0 or greater. To specify offline upgrades, set the `phoenix.client.localIndexUpgrade` configuration to false in the `phoenix-site` classification, and then SSH to the primary node to run `psql [zookeeper] -1`. + Upgraded to Presto 0.166 + Upgraded to Zeppelin 0.7.0 **Changes and enhancements** + Added support for r4 instances. See [Amazon EC2 Instance Types](https://aws.amazon.com/ec2/instance-types/). ## 5.4.0 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | emr-ddb | 4.2.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 2.3.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.2.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-s3-dist-cp | 2.4.0 | Distributed copy application optimized for Amazon S3. | | emrfs | 2.15.0 | Amazon S3 connector for Hadoop ecosystem applications. | | flink-client | 1.2.0 | Apache Flink command line client scripts and applications. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 2.7.3-amzn-1 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 2.7.3-amzn-1 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 2.7.3-amzn-1 | HDFS command-line client and library | | hadoop-hdfs-namenode | 2.7.3-amzn-1 | HDFS service for tracking file names and block locations. | | hadoop-httpfs-server | 2.7.3-amzn-1 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 2.7.3-amzn-1 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 2.7.3-amzn-1 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 2.7.3-amzn-1 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 2.7.3-amzn-1 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 2.7.3-amzn-1 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 1.3.0 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 1.3.0 | Service for serving one or more HBase regions. | | hbase-client | 1.3.0 | HBase command-line client. | | hbase-rest-server | 1.3.0 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 1.3.0 | Service providing a Thrift endpoint to HBase. | | hcatalog-client | 2.1.1-amzn-0 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 2.1.1-amzn-0 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 2.1.1-amzn-0 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 2.1.1-amzn-0 | Hive command line client. | | hive-hbase | 2.1.1-amzn-0 | Hive-hbase client. | | hive-metastore-server | 2.1.1-amzn-0 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server2 | 2.1.1-amzn-0 | Service for accepting Hive queries as web requests. | | hue-server | 3.11.0 | Web application for analyzing data using Hadoop ecosystem applications | | mahout-client | 0.12.2 | Library for machine learning. | | mysql-server | 5.5.54\$1 | MySQL database server. | | oozie-client | 4.3.0 | Oozie command-line client. | | oozie-server | 4.3.0 | Service for accepting Oozie workflow requests. | | phoenix-library | 4.9.0-HBase-1.2 | The phoenix libraries for server and client | | phoenix-query-server | 4.9.0-HBase-1.2 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.166 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.166 | Service for executing pieces of a query. | | pig-client | 0.16.0-amzn-0 | Pig command-line client. | | spark-client | 2.1.0 | Spark command-line clients. | | spark-history-server | 2.1.0 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 2.1.0 | In-memory execution engine for YARN. | | spark-yarn-slave | 2.1.0 | Apache Spark libraries needed by YARN slaves. | | sqoop-client | 1.4.6 | Apache Sqoop command-line client. | | tez-on-yarn | 0.8.4 | The tez YARN application and libraries. | | webserver | 2.4.25\$1 | Apache HTTP server. | | zeppelin-server | 0.7.0 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.4.9 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.4.9 | ZooKeeper command line client. | ## 5.4.0 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). **emr-5.4.0 classifications** | Classifications | Description | | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | | core-site | Change values in Hadoop's core-site.xml file. | | emrfs-site | Change EMRFS settings. | | flink-conf | Change flink-conf.yaml settings. | | flink-log4j | Change Flink log4j.properties settings. | | flink-log4j-yarn-session | Change Flink log4j-yarn-session.properties settings. | | flink-log4j-cli | Change Flink log4j-cli.properties settings. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | | hadoop-ssl-server | Change hadoop ssl server configuration | | hadoop-ssl-client | Change hadoop ssl client configuration | | hbase | Amazon EMR-curated settings for Apache HBase. | | hbase-env | Change values in HBase's environment. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbaase.properties file. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | | hbase-site | Change values in HBase's hbase-site.xml file. | | hdfs-encryption-zones | Configure HDFS encryption zones. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | | hcatalog-env | Change values in HCatalog's environment. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | | hcatalog-webhcat-log4j2 | Change values in HCatalog WebHCat's log4j2.properties. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | | hive-beeline-log4j2 | Change values in Hive's beeline-log4j2.properties file. | | hive-parquet-logging | Change values in Hive's parquet-logging.properties file. | | hive-env | Change values in the Hive environment. | | hive-exec-log4j2 | Change values in Hive's hive-exec-log4j2.properties file. | | hive-llap-daemon-log4j2 | Change values in Hive's llap-daemon-log4j2.properties file. | | hive-log4j2 | Change values in Hive's hive-log4j2.properties file. | | hive-site | Change values in Hive's hive-site.xml file | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | | hue-ini | Change values in Hue's ini file | | httpfs-env | Change values in the HTTPFS environment. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | | mapred-env | Change values in the MapReduce application's environment. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | | oozie-env | Change values in Oozie's environment. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | | oozie-site | Change values in Oozie's oozie-site.xml file. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | | pig-properties | Change values in Pig's pig.properties file. | | pig-log4j | Change values in Pig's log4j.properties file. | | presto-log | Change values in Presto's log.properties file. | | presto-config | Change values in Presto's config.properties file. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | | presto-connector-hive | Change values in Presto's hive.properties file. | | presto-connector-jmx | Change values in Presto's jmx.properties file. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | | presto-connector-localfile | Change values in Presto's localfile.properties file. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | | presto-connector-redis | Change values in Presto's redis.properties file. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | | spark | Amazon EMR-curated settings for Apache Spark. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | | spark-env | Change values in the Spark environment. | | spark-hive-site | Change values in Spark's hive-site.xml file | | spark-log4j | Change values in Spark's log4j.properties file. | | spark-metrics | Change values in Spark's metrics.properties file. | | sqoop-env | Change values in Sqoop's environment. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | | tez-site | Change values in Tez's tez-site.xml file. | | yarn-env | Change values in the YARN environment. | | yarn-site | Change values in YARN's yarn-site.xml file. | | zeppelin-env | Change values in the Zeppelin environment. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | # Amazon EMR release 5.3.2 ## 5.3.2 application versions This release includes the following applications: [https://flink.apache.org/](https://flink.apache.org/), [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [http://gethue.com/](http://gethue.com/), [http://mahout.apache.org/](http://mahout.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-5.3.2 | emr-5.3.1 | emr-5.3.0 | emr-5.2.3 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.10.75 | 1.10.75 | 1.10.75 | 1.10.75 | | Python | Not tracked | Not tracked | Not tracked | Not tracked | | Scala | 2.11.8 | 2.11.8 | 2.11.8 | 2.11.8 | | AmazonCloudWatchAgent | - | - | - | - | | Delta | - | - | - | - | | Flink | 1.1.4 | 1.1.4 | 1.1.4 | 1.1.3 | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 1.2.3 | 1.2.3 | 1.2.3 | 1.2.3 | | HCatalog | 2.1.1-amzn-0 | 2.1.1-amzn-0 | 2.1.1-amzn-0 | 2.1.0-amzn-0 | | Hadoop | 2.7.3-amzn-1 | 2.7.3-amzn-1 | 2.7.3-amzn-1 | 2.7.3-amzn-1 | | Hive | 2.1.1-amzn-0 | 2.1.1-amzn-0 | 2.1.1-amzn-0 | 2.1.0-amzn-0 | | Hudi | - | - | - | - | | Hue | 3.11.0 | 3.11.0 | 3.11.0 | 3.10.0-amzn-0 | | Iceberg | - | - | - | - | | JupyterEnterpriseGateway | - | - | - | - | | JupyterHub | - | - | - | - | | Livy | - | - | - | - | | MXNet | - | - | - | - | | Mahout | 0.12.2 | 0.12.2 | 0.12.2 | 0.12.2 | | Oozie | 4.3.0 | 4.3.0 | 4.3.0 | 4.2.0 | | Phoenix | 4.7.0-HBase-1.2 | 4.7.0-HBase-1.2 | 4.7.0-HBase-1.2 | 4.7.0-HBase-1.2 | | Pig | 0.16.0-amzn-0 | 0.16.0-amzn-0 | 0.16.0-amzn-0 | 0.16.0-amzn-0 | | Presto | 0.157.1 | 0.157.1 | 0.157.1 | 0.157.1 | | Spark | 2.1.0 | 2.1.0 | 2.1.0 | 2.0.2 | | Sqoop | 1.4.6 | 1.4.6 | 1.4.6 | 1.4.6 | | TensorFlow | - | - | - | - | | Tez | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | | Trino (PrestoSQL) | - | - | - | - | | Zeppelin | 0.6.2 | 0.6.2 | 0.6.2 | 0.6.2 | | ZooKeeper | 3.4.9 | 3.4.9 | 3.4.9 | 3.4.9 | ## 5.3.2 release notes This is a patch release to add AWS Signature Version 4 authentication for requests to Amazon S3. All applications and components are the same as the previous Amazon EMR release. **Important** In this release version, Amazon EMR uses AWS Signature Version 4 exclusively to authenticate requests to Amazon S3. For more information, see [Whats New](https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-whatsnew.html). ## 5.3.2 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | emr-ddb | 4.2.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 2.2.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.2.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-s3-dist-cp | 2.4.0 | Distributed copy application optimized for Amazon S3. | | emrfs | 2.14.0 | Amazon S3 connector for Hadoop ecosystem applications. | | flink-client | 1.1.4 | Apache Flink command line client scripts and applications. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 2.7.3-amzn-1 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 2.7.3-amzn-1 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 2.7.3-amzn-1 | HDFS command-line client and library | | hadoop-hdfs-namenode | 2.7.3-amzn-1 | HDFS service for tracking file names and block locations. | | hadoop-httpfs-server | 2.7.3-amzn-1 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 2.7.3-amzn-1 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 2.7.3-amzn-1 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 2.7.3-amzn-1 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 2.7.3-amzn-1 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 2.7.3-amzn-1 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 1.2.3 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 1.2.3 | Service for serving one or more HBase regions. | | hbase-client | 1.2.3 | HBase command-line client. | | hbase-rest-server | 1.2.3 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 1.2.3 | Service providing a Thrift endpoint to HBase. | | hcatalog-client | 2.1.1-amzn-0 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 2.1.1-amzn-0 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 2.1.1-amzn-0 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 2.1.1-amzn-0 | Hive command line client. | | hive-metastore-server | 2.1.1-amzn-0 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server | 2.1.1-amzn-0 | Service for accepting Hive queries as web requests. | | hue-server | 3.11.0 | Web application for analyzing data using Hadoop ecosystem applications | | mahout-client | 0.12.2 | Library for machine learning. | | mysql-server | 5.5.54\$1 | MySQL database server. | | oozie-client | 4.3.0 | Oozie command-line client. | | oozie-server | 4.3.0 | Service for accepting Oozie workflow requests. | | phoenix-library | 4.7.0-HBase-1.2 | The phoenix libraries for server and client | | phoenix-query-server | 4.7.0-HBase-1.2 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.157.1 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.157.1 | Service for executing pieces of a query. | | pig-client | 0.16.0-amzn-0 | Pig command-line client. | | spark-client | 2.1.0 | Spark command-line clients. | | spark-history-server | 2.1.0 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 2.1.0 | In-memory execution engine for YARN. | | spark-yarn-slave | 2.1.0 | Apache Spark libraries needed by YARN slaves. | | sqoop-client | 1.4.6 | Apache Sqoop command-line client. | | tez-on-yarn | 0.8.4 | The tez YARN application and libraries. | | webserver | 2.4.25\$1 | Apache HTTP server. | | zeppelin-server | 0.6.2 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.4.9 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.4.9 | ZooKeeper command line client. | ## 5.3.2 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). **emr-5.3.2 classifications** | Classifications | Description | | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | | core-site | Change values in Hadoop's core-site.xml file. | | emrfs-site | Change EMRFS settings. | | flink-conf | Change flink-conf.yaml settings. | | flink-log4j | Change Flink log4j.properties settings. | | flink-log4j-yarn-session | Change Flink log4j-yarn-session.properties settings. | | flink-log4j-cli | Change Flink log4j-cli.properties settings. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | | hadoop-ssl-server | Change hadoop ssl server configuration | | hadoop-ssl-client | Change hadoop ssl client configuration | | hbase | Amazon EMR-curated settings for Apache HBase. | | hbase-env | Change values in HBase's environment. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbaase.properties file. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | | hbase-site | Change values in HBase's hbase-site.xml file. | | hdfs-encryption-zones | Configure HDFS encryption zones. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | | hcatalog-env | Change values in HCatalog's environment. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | | hcatalog-webhcat-log4j2 | Change values in HCatalog WebHCat's log4j2.properties. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | | hive-beeline-log4j2 | Change values in Hive's beeline-log4j2.properties file. | | hive-parquet-logging | Change values in Hive's parquet-logging.properties file. | | hive-env | Change values in the Hive environment. | | hive-exec-log4j2 | Change values in Hive's hive-exec-log4j2.properties file. | | hive-llap-daemon-log4j2 | Change values in Hive's llap-daemon-log4j2.properties file. | | hive-log4j2 | Change values in Hive's hive-log4j2.properties file. | | hive-site | Change values in Hive's hive-site.xml file | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | | hue-ini | Change values in Hue's ini file | | httpfs-env | Change values in the HTTPFS environment. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | | mapred-env | Change values in the MapReduce application's environment. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | | oozie-env | Change values in Oozie's environment. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | | oozie-site | Change values in Oozie's oozie-site.xml file. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | | pig-properties | Change values in Pig's pig.properties file. | | pig-log4j | Change values in Pig's log4j.properties file. | | presto-log | Change values in Presto's log.properties file. | | presto-config | Change values in Presto's config.properties file. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | | presto-connector-hive | Change values in Presto's hive.properties file. | | presto-connector-jmx | Change values in Presto's jmx.properties file. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | | presto-connector-localfile | Change values in Presto's localfile.properties file. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | | presto-connector-redis | Change values in Presto's redis.properties file. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | | spark | Amazon EMR-curated settings for Apache Spark. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | | spark-env | Change values in the Spark environment. | | spark-hive-site | Change values in Spark's hive-site.xml file | | spark-log4j | Change values in Spark's log4j.properties file. | | spark-metrics | Change values in Spark's metrics.properties file. | | sqoop-env | Change values in Sqoop's environment. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | | tez-site | Change values in Tez's tez-site.xml file. | | yarn-env | Change values in the YARN environment. | | yarn-site | Change values in YARN's yarn-site.xml file. | | zeppelin-env | Change values in the Zeppelin environment. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | # Amazon EMR release 5.3.1 ## 5.3.1 application versions This release includes the following applications: [https://flink.apache.org/](https://flink.apache.org/), [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [http://gethue.com/](http://gethue.com/), [http://mahout.apache.org/](http://mahout.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-5.3.1 | emr-5.3.0 | emr-5.2.3 | emr-5.2.2 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.10.75 | 1.10.75 | 1.10.75 | 1.10.75 | | Python | Not tracked | Not tracked | Not tracked | Not tracked | | Scala | 2.11.8 | 2.11.8 | 2.11.8 | 2.11.8 | | AmazonCloudWatchAgent | - | - | - | - | | Delta | - | - | - | - | | Flink | 1.1.4 | 1.1.4 | 1.1.3 | 1.1.3 | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 1.2.3 | 1.2.3 | 1.2.3 | 1.2.3 | | HCatalog | 2.1.1-amzn-0 | 2.1.1-amzn-0 | 2.1.0-amzn-0 | 2.1.0-amzn-0 | | Hadoop | 2.7.3-amzn-1 | 2.7.3-amzn-1 | 2.7.3-amzn-1 | 2.7.3-amzn-1 | | Hive | 2.1.1-amzn-0 | 2.1.1-amzn-0 | 2.1.0-amzn-0 | 2.1.0-amzn-0 | | Hudi | - | - | - | - | | Hue | 3.11.0 | 3.11.0 | 3.10.0-amzn-0 | 3.10.0-amzn-0 | | Iceberg | - | - | - | - | | JupyterEnterpriseGateway | - | - | - | - | | JupyterHub | - | - | - | - | | Livy | - | - | - | - | | MXNet | - | - | - | - | | Mahout | 0.12.2 | 0.12.2 | 0.12.2 | 0.12.2 | | Oozie | 4.3.0 | 4.3.0 | 4.2.0 | 4.2.0 | | Phoenix | 4.7.0-HBase-1.2 | 4.7.0-HBase-1.2 | 4.7.0-HBase-1.2 | 4.7.0-HBase-1.2 | | Pig | 0.16.0-amzn-0 | 0.16.0-amzn-0 | 0.16.0-amzn-0 | 0.16.0-amzn-0 | | Presto | 0.157.1 | 0.157.1 | 0.157.1 | 0.157.1 | | Spark | 2.1.0 | 2.1.0 | 2.0.2 | 2.0.2 | | Sqoop | 1.4.6 | 1.4.6 | 1.4.6 | 1.4.6 | | TensorFlow | - | - | - | - | | Tez | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | | Trino (PrestoSQL) | - | - | - | - | | Zeppelin | 0.6.2 | 0.6.2 | 0.6.2 | 0.6.2 | | ZooKeeper | 3.4.9 | 3.4.9 | 3.4.9 | 3.4.9 | ## 5.3.1 release notes The following release notes include information for the Amazon EMR 5.3.1 release. Changes are relative to the Amazon EMR 5.3.0 release. Release date: February 7, 2017 Minor changes to backport Zeppelin patches and update the default AMI for Amazon EMR. ## 5.3.1 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | emr-ddb | 4.2.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 2.2.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.2.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-s3-dist-cp | 2.4.0 | Distributed copy application optimized for Amazon S3. | | emrfs | 2.14.0 | Amazon S3 connector for Hadoop ecosystem applications. | | flink-client | 1.1.4 | Apache Flink command line client scripts and applications. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 2.7.3-amzn-1 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 2.7.3-amzn-1 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 2.7.3-amzn-1 | HDFS command-line client and library | | hadoop-hdfs-namenode | 2.7.3-amzn-1 | HDFS service for tracking file names and block locations. | | hadoop-httpfs-server | 2.7.3-amzn-1 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 2.7.3-amzn-1 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 2.7.3-amzn-1 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 2.7.3-amzn-1 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 2.7.3-amzn-1 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 2.7.3-amzn-1 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 1.2.3 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 1.2.3 | Service for serving one or more HBase regions. | | hbase-client | 1.2.3 | HBase command-line client. | | hbase-rest-server | 1.2.3 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 1.2.3 | Service providing a Thrift endpoint to HBase. | | hcatalog-client | 2.1.1-amzn-0 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 2.1.1-amzn-0 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 2.1.1-amzn-0 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 2.1.1-amzn-0 | Hive command line client. | | hive-metastore-server | 2.1.1-amzn-0 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server | 2.1.1-amzn-0 | Service for accepting Hive queries as web requests. | | hue-server | 3.11.0 | Web application for analyzing data using Hadoop ecosystem applications | | mahout-client | 0.12.2 | Library for machine learning. | | mysql-server | 5.5.54\$1 | MySQL database server. | | oozie-client | 4.3.0 | Oozie command-line client. | | oozie-server | 4.3.0 | Service for accepting Oozie workflow requests. | | phoenix-library | 4.7.0-HBase-1.2 | The phoenix libraries for server and client | | phoenix-query-server | 4.7.0-HBase-1.2 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.157.1 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.157.1 | Service for executing pieces of a query. | | pig-client | 0.16.0-amzn-0 | Pig command-line client. | | spark-client | 2.1.0 | Spark command-line clients. | | spark-history-server | 2.1.0 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 2.1.0 | In-memory execution engine for YARN. | | spark-yarn-slave | 2.1.0 | Apache Spark libraries needed by YARN slaves. | | sqoop-client | 1.4.6 | Apache Sqoop command-line client. | | tez-on-yarn | 0.8.4 | The tez YARN application and libraries. | | webserver | 2.4.25\$1 | Apache HTTP server. | | zeppelin-server | 0.6.2 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.4.9 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.4.9 | ZooKeeper command line client. | ## 5.3.1 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). **emr-5.3.1 classifications** | Classifications | Description | | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | | core-site | Change values in Hadoop's core-site.xml file. | | emrfs-site | Change EMRFS settings. | | flink-conf | Change flink-conf.yaml settings. | | flink-log4j | Change Flink log4j.properties settings. | | flink-log4j-yarn-session | Change Flink log4j-yarn-session.properties settings. | | flink-log4j-cli | Change Flink log4j-cli.properties settings. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | | hadoop-ssl-server | Change hadoop ssl server configuration | | hadoop-ssl-client | Change hadoop ssl client configuration | | hbase | Amazon EMR-curated settings for Apache HBase. | | hbase-env | Change values in HBase's environment. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbaase.properties file. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | | hbase-site | Change values in HBase's hbase-site.xml file. | | hdfs-encryption-zones | Configure HDFS encryption zones. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | | hcatalog-env | Change values in HCatalog's environment. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | | hcatalog-webhcat-log4j2 | Change values in HCatalog WebHCat's log4j2.properties. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | | hive-beeline-log4j2 | Change values in Hive's beeline-log4j2.properties file. | | hive-parquet-logging | Change values in Hive's parquet-logging.properties file. | | hive-env | Change values in the Hive environment. | | hive-exec-log4j2 | Change values in Hive's hive-exec-log4j2.properties file. | | hive-llap-daemon-log4j2 | Change values in Hive's llap-daemon-log4j2.properties file. | | hive-log4j2 | Change values in Hive's hive-log4j2.properties file. | | hive-site | Change values in Hive's hive-site.xml file | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | | hue-ini | Change values in Hue's ini file | | httpfs-env | Change values in the HTTPFS environment. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | | mapred-env | Change values in the MapReduce application's environment. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | | oozie-env | Change values in Oozie's environment. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | | oozie-site | Change values in Oozie's oozie-site.xml file. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | | pig-properties | Change values in Pig's pig.properties file. | | pig-log4j | Change values in Pig's log4j.properties file. | | presto-log | Change values in Presto's log.properties file. | | presto-config | Change values in Presto's config.properties file. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | | presto-connector-hive | Change values in Presto's hive.properties file. | | presto-connector-jmx | Change values in Presto's jmx.properties file. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | | presto-connector-localfile | Change values in Presto's localfile.properties file. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | | presto-connector-redis | Change values in Presto's redis.properties file. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | | spark | Amazon EMR-curated settings for Apache Spark. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | | spark-env | Change values in the Spark environment. | | spark-hive-site | Change values in Spark's hive-site.xml file | | spark-log4j | Change values in Spark's log4j.properties file. | | spark-metrics | Change values in Spark's metrics.properties file. | | sqoop-env | Change values in Sqoop's environment. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | | tez-site | Change values in Tez's tez-site.xml file. | | yarn-env | Change values in the YARN environment. | | yarn-site | Change values in YARN's yarn-site.xml file. | | zeppelin-env | Change values in the Zeppelin environment. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | # Amazon EMR release 5.3.0 ## 5.3.0 application versions This release includes the following applications: [https://flink.apache.org/](https://flink.apache.org/), [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [http://gethue.com/](http://gethue.com/), [http://mahout.apache.org/](http://mahout.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-5.3.0 | emr-5.2.3 | emr-5.2.2 | emr-5.2.1 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.10.75 | 1.10.75 | 1.10.75 | 1.10.75 | | Python | Not tracked | Not tracked | Not tracked | Not tracked | | Scala | 2.11.8 | 2.11.8 | 2.11.8 | 2.11.8 | | AmazonCloudWatchAgent | - | - | - | - | | Delta | - | - | - | - | | Flink | 1.1.4 | 1.1.3 | 1.1.3 | 1.1.3 | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 1.2.3 | 1.2.3 | 1.2.3 | 1.2.3 | | HCatalog | 2.1.1-amzn-0 | 2.1.0-amzn-0 | 2.1.0-amzn-0 | 2.1.0-amzn-0 | | Hadoop | 2.7.3-amzn-1 | 2.7.3-amzn-1 | 2.7.3-amzn-1 | 2.7.3-amzn-1 | | Hive | 2.1.1-amzn-0 | 2.1.0-amzn-0 | 2.1.0-amzn-0 | 2.1.0-amzn-0 | | Hudi | - | - | - | - | | Hue | 3.11.0 | 3.10.0-amzn-0 | 3.10.0-amzn-0 | 3.10.0-amzn-0 | | Iceberg | - | - | - | - | | JupyterEnterpriseGateway | - | - | - | - | | JupyterHub | - | - | - | - | | Livy | - | - | - | - | | MXNet | - | - | - | - | | Mahout | 0.12.2 | 0.12.2 | 0.12.2 | 0.12.2 | | Oozie | 4.3.0 | 4.2.0 | 4.2.0 | 4.2.0 | | Phoenix | 4.7.0-HBase-1.2 | 4.7.0-HBase-1.2 | 4.7.0-HBase-1.2 | 4.7.0-HBase-1.2 | | Pig | 0.16.0-amzn-0 | 0.16.0-amzn-0 | 0.16.0-amzn-0 | 0.16.0-amzn-0 | | Presto | 0.157.1 | 0.157.1 | 0.157.1 | 0.157.1 | | Spark | 2.1.0 | 2.0.2 | 2.0.2 | 2.0.2 | | Sqoop | 1.4.6 | 1.4.6 | 1.4.6 | 1.4.6 | | TensorFlow | - | - | - | - | | Tez | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | | Trino (PrestoSQL) | - | - | - | - | | Zeppelin | 0.6.2 | 0.6.2 | 0.6.2 | 0.6.2 | | ZooKeeper | 3.4.9 | 3.4.9 | 3.4.9 | 3.4.9 | ## 5.3.0 release notes The following release notes include information for the Amazon EMR 5.3.0 release. Changes are relative to the Amazon EMR 5.2.1 release. Release date: January 26, 2017 **Upgrades** + Upgraded to Hive 2.1.1 + Upgraded to Hue 3.11.0 + Upgraded to Spark 2.1.0 + Upgraded to Oozie 4.3.0 + Upgraded to Flink 1.1.4 **Changes and enhancements** + Added a patch to Hue that allows you to use the `interpreters_shown_on_wheel` setting to configure what interpreters to show first on the Notebook selection wheel, regardless of their ordering in the `hue.ini` file. + Added the `hive-parquet-logging` configuration classification, which you can use to configure values in the Hive `parquet-logging.properties` file. ## 5.3.0 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | emr-ddb | 4.2.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 2.2.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.2.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-s3-dist-cp | 2.4.0 | Distributed copy application optimized for Amazon S3. | | emrfs | 2.14.0 | Amazon S3 connector for Hadoop ecosystem applications. | | flink-client | 1.1.4 | Apache Flink command line client scripts and applications. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 2.7.3-amzn-1 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 2.7.3-amzn-1 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 2.7.3-amzn-1 | HDFS command-line client and library | | hadoop-hdfs-namenode | 2.7.3-amzn-1 | HDFS service for tracking file names and block locations. | | hadoop-httpfs-server | 2.7.3-amzn-1 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 2.7.3-amzn-1 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 2.7.3-amzn-1 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 2.7.3-amzn-1 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 2.7.3-amzn-1 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 2.7.3-amzn-1 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 1.2.3 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 1.2.3 | Service for serving one or more HBase regions. | | hbase-client | 1.2.3 | HBase command-line client. | | hbase-rest-server | 1.2.3 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 1.2.3 | Service providing a Thrift endpoint to HBase. | | hcatalog-client | 2.1.1-amzn-0 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 2.1.1-amzn-0 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 2.1.1-amzn-0 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 2.1.1-amzn-0 | Hive command line client. | | hive-metastore-server | 2.1.1-amzn-0 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server | 2.1.1-amzn-0 | Service for accepting Hive queries as web requests. | | hue-server | 3.11.0 | Web application for analyzing data using Hadoop ecosystem applications | | mahout-client | 0.12.2 | Library for machine learning. | | mysql-server | 5.5.52 | MySQL database server. | | oozie-client | 4.3.0 | Oozie command-line client. | | oozie-server | 4.3.0 | Service for accepting Oozie workflow requests. | | phoenix-library | 4.7.0-HBase-1.2 | The phoenix libraries for server and client | | phoenix-query-server | 4.7.0-HBase-1.2 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.157.1 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.157.1 | Service for executing pieces of a query. | | pig-client | 0.16.0-amzn-0 | Pig command-line client. | | spark-client | 2.1.0 | Spark command-line clients. | | spark-history-server | 2.1.0 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 2.1.0 | In-memory execution engine for YARN. | | spark-yarn-slave | 2.1.0 | Apache Spark libraries needed by YARN slaves. | | sqoop-client | 1.4.6 | Apache Sqoop command-line client. | | tez-on-yarn | 0.8.4 | The tez YARN application and libraries. | | webserver | 2.4.25\$1 | Apache HTTP server. | | zeppelin-server | 0.6.2 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.4.9 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.4.9 | ZooKeeper command line client. | ## 5.3.0 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). **emr-5.3.0 classifications** | Classifications | Description | | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | | core-site | Change values in Hadoop's core-site.xml file. | | emrfs-site | Change EMRFS settings. | | flink-conf | Change flink-conf.yaml settings. | | flink-log4j | Change Flink log4j.properties settings. | | flink-log4j-yarn-session | Change Flink log4j-yarn-session.properties settings. | | flink-log4j-cli | Change Flink log4j-cli.properties settings. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | | hadoop-ssl-server | Change hadoop ssl server configuration | | hadoop-ssl-client | Change hadoop ssl client configuration | | hbase | Amazon EMR-curated settings for Apache HBase. | | hbase-env | Change values in HBase's environment. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbaase.properties file. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | | hbase-site | Change values in HBase's hbase-site.xml file. | | hdfs-encryption-zones | Configure HDFS encryption zones. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | | hcatalog-env | Change values in HCatalog's environment. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | | hcatalog-webhcat-log4j2 | Change values in HCatalog WebHCat's log4j2.properties. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | | hive-beeline-log4j2 | Change values in Hive's beeline-log4j2.properties file. | | hive-parquet-logging | Change values in Hive's parquet-logging.properties file. | | hive-env | Change values in the Hive environment. | | hive-exec-log4j2 | Change values in Hive's hive-exec-log4j2.properties file. | | hive-llap-daemon-log4j2 | Change values in Hive's llap-daemon-log4j2.properties file. | | hive-log4j2 | Change values in Hive's hive-log4j2.properties file. | | hive-site | Change values in Hive's hive-site.xml file | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | | hue-ini | Change values in Hue's ini file | | httpfs-env | Change values in the HTTPFS environment. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | | mapred-env | Change values in the MapReduce application's environment. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | | oozie-env | Change values in Oozie's environment. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | | oozie-site | Change values in Oozie's oozie-site.xml file. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | | pig-properties | Change values in Pig's pig.properties file. | | pig-log4j | Change values in Pig's log4j.properties file. | | presto-log | Change values in Presto's log.properties file. | | presto-config | Change values in Presto's config.properties file. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | | presto-connector-hive | Change values in Presto's hive.properties file. | | presto-connector-jmx | Change values in Presto's jmx.properties file. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | | presto-connector-localfile | Change values in Presto's localfile.properties file. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | | presto-connector-redis | Change values in Presto's redis.properties file. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | | spark | Amazon EMR-curated settings for Apache Spark. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | | spark-env | Change values in the Spark environment. | | spark-hive-site | Change values in Spark's hive-site.xml file | | spark-log4j | Change values in Spark's log4j.properties file. | | spark-metrics | Change values in Spark's metrics.properties file. | | sqoop-env | Change values in Sqoop's environment. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | | tez-site | Change values in Tez's tez-site.xml file. | | yarn-env | Change values in the YARN environment. | | yarn-site | Change values in YARN's yarn-site.xml file. | | zeppelin-env | Change values in the Zeppelin environment. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | # Amazon EMR release 5.2.3 ## 5.2.3 application versions This release includes the following applications: [https://flink.apache.org/](https://flink.apache.org/), [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [http://gethue.com/](http://gethue.com/), [http://mahout.apache.org/](http://mahout.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-5.2.3 | emr-5.2.2 | emr-5.2.1 | emr-5.2.0 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.10.75 | 1.10.75 | 1.10.75 | 1.10.75 | | Python | Not tracked | Not tracked | Not tracked | Not tracked | | Scala | 2.11.8 | 2.11.8 | 2.11.8 | 2.11.8 | | AmazonCloudWatchAgent | - | - | - | - | | Delta | - | - | - | - | | Flink | 1.1.3 | 1.1.3 | 1.1.3 | 1.1.3 | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 1.2.3 | 1.2.3 | 1.2.3 | 1.2.3 | | HCatalog | 2.1.0-amzn-0 | 2.1.0-amzn-0 | 2.1.0-amzn-0 | 2.1.0-amzn-0 | | Hadoop | 2.7.3-amzn-1 | 2.7.3-amzn-1 | 2.7.3-amzn-1 | 2.7.3-amzn-0 | | Hive | 2.1.0-amzn-0 | 2.1.0-amzn-0 | 2.1.0-amzn-0 | 2.1.0-amzn-0 | | Hudi | - | - | - | - | | Hue | 3.10.0-amzn-0 | 3.10.0-amzn-0 | 3.10.0-amzn-0 | 3.10.0-amzn-0 | | Iceberg | - | - | - | - | | JupyterEnterpriseGateway | - | - | - | - | | JupyterHub | - | - | - | - | | Livy | - | - | - | - | | MXNet | - | - | - | - | | Mahout | 0.12.2 | 0.12.2 | 0.12.2 | 0.12.2 | | Oozie | 4.2.0 | 4.2.0 | 4.2.0 | 4.2.0 | | Phoenix | 4.7.0-HBase-1.2 | 4.7.0-HBase-1.2 | 4.7.0-HBase-1.2 | 4.7.0-HBase-1.2 | | Pig | 0.16.0-amzn-0 | 0.16.0-amzn-0 | 0.16.0-amzn-0 | 0.16.0-amzn-0 | | Presto | 0.157.1 | 0.157.1 | 0.157.1 | 0.152.3 | | Spark | 2.0.2 | 2.0.2 | 2.0.2 | 2.0.2 | | Sqoop | 1.4.6 | 1.4.6 | 1.4.6 | 1.4.6 | | TensorFlow | - | - | - | - | | Tez | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | | Trino (PrestoSQL) | - | - | - | - | | Zeppelin | 0.6.2 | 0.6.2 | 0.6.2 | 0.6.2 | | ZooKeeper | 3.4.9 | 3.4.9 | 3.4.9 | 3.4.8 | ## 5.2.3 release notes This is a patch release to add AWS Signature Version 4 authentication for requests to Amazon S3. All applications and components are the same as the previous Amazon EMR release. **Important** In this release version, Amazon EMR uses AWS Signature Version 4 exclusively to authenticate requests to Amazon S3. For more information, see [Whats New](https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-whatsnew.html). ## 5.2.3 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | emr-ddb | 4.2.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 2.2.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.2.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-s3-dist-cp | 2.4.0 | Distributed copy application optimized for Amazon S3. | | emrfs | 2.13.0 | Amazon S3 connector for Hadoop ecosystem applications. | | flink-client | 1.1.3 | Apache Flink command line client scripts and applications. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 2.7.3-amzn-1 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 2.7.3-amzn-1 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 2.7.3-amzn-1 | HDFS command-line client and library | | hadoop-hdfs-namenode | 2.7.3-amzn-1 | HDFS service for tracking file names and block locations. | | hadoop-httpfs-server | 2.7.3-amzn-1 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 2.7.3-amzn-1 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 2.7.3-amzn-1 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 2.7.3-amzn-1 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 2.7.3-amzn-1 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 2.7.3-amzn-1 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 1.2.3 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 1.2.3 | Service for serving one or more HBase regions. | | hbase-client | 1.2.3 | HBase command-line client. | | hbase-rest-server | 1.2.3 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 1.2.3 | Service providing a Thrift endpoint to HBase. | | hcatalog-client | 2.1.0-amzn-0 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 2.1.0-amzn-0 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 2.1.0-amzn-0 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 2.1.0-amzn-0 | Hive command line client. | | hive-metastore-server | 2.1.0-amzn-0 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server | 2.1.0-amzn-0 | Service for accepting Hive queries as web requests. | | hue-server | 3.10.0-amzn-0 | Web application for analyzing data using Hadoop ecosystem applications | | mahout-client | 0.12.2 | Library for machine learning. | | mysql-server | 5.5.52 | MySQL database server. | | oozie-client | 4.2.0 | Oozie command-line client. | | oozie-server | 4.2.0 | Service for accepting Oozie workflow requests. | | phoenix-library | 4.7.0-HBase-1.2 | The phoenix libraries for server and client | | phoenix-query-server | 4.7.0-HBase-1.2 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.157.1 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.157.1 | Service for executing pieces of a query. | | pig-client | 0.16.0-amzn-0 | Pig command-line client. | | spark-client | 2.0.2 | Spark command-line clients. | | spark-history-server | 2.0.2 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 2.0.2 | In-memory execution engine for YARN. | | spark-yarn-slave | 2.0.2 | Apache Spark libraries needed by YARN slaves. | | sqoop-client | 1.4.6 | Apache Sqoop command-line client. | | tez-on-yarn | 0.8.4 | The tez YARN application and libraries. | | webserver | 2.4.23 | Apache HTTP server. | | zeppelin-server | 0.6.2 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.4.9 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.4.9 | ZooKeeper command line client. | ## 5.2.3 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). **emr-5.2.3 classifications** | Classifications | Description | | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | | core-site | Change values in Hadoop's core-site.xml file. | | emrfs-site | Change EMRFS settings. | | flink-conf | Change flink-conf.yaml settings. | | flink-log4j | Change Flink log4j.properties settings. | | flink-log4j-yarn-session | Change Flink log4j-yarn-session.properties settings. | | flink-log4j-cli | Change Flink log4j-cli.properties settings. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | | hadoop-ssl-server | Change hadoop ssl server configuration | | hadoop-ssl-client | Change hadoop ssl client configuration | | hbase | Amazon EMR-curated settings for Apache HBase. | | hbase-env | Change values in HBase's environment. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbaase.properties file. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | | hbase-site | Change values in HBase's hbase-site.xml file. | | hdfs-encryption-zones | Configure HDFS encryption zones. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | | hcatalog-env | Change values in HCatalog's environment. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | | hcatalog-webhcat-log4j2 | Change values in HCatalog WebHCat's log4j2.properties. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | | hive-beeline-log4j2 | Change values in Hive's beeline-log4j2.properties file. | | hive-env | Change values in the Hive environment. | | hive-exec-log4j2 | Change values in Hive's hive-exec-log4j2.properties file. | | hive-llap-daemon-log4j2 | Change values in Hive's llap-daemon-log4j2.properties file. | | hive-log4j2 | Change values in Hive's hive-log4j2.properties file. | | hive-site | Change values in Hive's hive-site.xml file | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | | hue-ini | Change values in Hue's ini file | | httpfs-env | Change values in the HTTPFS environment. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | | mapred-env | Change values in the MapReduce application's environment. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | | oozie-env | Change values in Oozie's environment. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | | oozie-site | Change values in Oozie's oozie-site.xml file. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | | pig-properties | Change values in Pig's pig.properties file. | | pig-log4j | Change values in Pig's log4j.properties file. | | presto-log | Change values in Presto's log.properties file. | | presto-config | Change values in Presto's config.properties file. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | | presto-connector-hive | Change values in Presto's hive.properties file. | | presto-connector-jmx | Change values in Presto's jmx.properties file. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | | presto-connector-localfile | Change values in Presto's localfile.properties file. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | | presto-connector-redis | Change values in Presto's redis.properties file. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | | spark | Amazon EMR-curated settings for Apache Spark. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | | spark-env | Change values in the Spark environment. | | spark-hive-site | Change values in Spark's hive-site.xml file | | spark-log4j | Change values in Spark's log4j.properties file. | | spark-metrics | Change values in Spark's metrics.properties file. | | sqoop-env | Change values in Sqoop's environment. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | | tez-site | Change values in Tez's tez-site.xml file. | | yarn-env | Change values in the YARN environment. | | yarn-site | Change values in YARN's yarn-site.xml file. | | zeppelin-env | Change values in the Zeppelin environment. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | # Amazon EMR release 5.2.2 ## 5.2.2 application versions This release includes the following applications: [https://flink.apache.org/](https://flink.apache.org/), [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [http://gethue.com/](http://gethue.com/), [http://mahout.apache.org/](http://mahout.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-5.2.2 | emr-5.2.1 | emr-5.2.0 | emr-5.1.1 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.10.75 | 1.10.75 | 1.10.75 | 1.10.75 | | Python | Not tracked | Not tracked | Not tracked | Not tracked | | Scala | 2.11.8 | 2.11.8 | 2.11.8 | 2.11.8 | | AmazonCloudWatchAgent | - | - | - | - | | Delta | - | - | - | - | | Flink | 1.1.3 | 1.1.3 | 1.1.3 | 1.1.3 | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 1.2.3 | 1.2.3 | 1.2.3 | 1.2.3 | | HCatalog | 2.1.0-amzn-0 | 2.1.0-amzn-0 | 2.1.0-amzn-0 | 2.1.0-amzn-0 | | Hadoop | 2.7.3-amzn-1 | 2.7.3-amzn-1 | 2.7.3-amzn-0 | 2.7.3-amzn-0 | | Hive | 2.1.0-amzn-0 | 2.1.0-amzn-0 | 2.1.0-amzn-0 | 2.1.0-amzn-0 | | Hudi | - | - | - | - | | Hue | 3.10.0-amzn-0 | 3.10.0-amzn-0 | 3.10.0-amzn-0 | 3.10.0-amzn-0 | | Iceberg | - | - | - | - | | JupyterEnterpriseGateway | - | - | - | - | | JupyterHub | - | - | - | - | | Livy | - | - | - | - | | MXNet | - | - | - | - | | Mahout | 0.12.2 | 0.12.2 | 0.12.2 | 0.12.2 | | Oozie | 4.2.0 | 4.2.0 | 4.2.0 | 4.2.0 | | Phoenix | 4.7.0-HBase-1.2 | 4.7.0-HBase-1.2 | 4.7.0-HBase-1.2 | 4.7.0-HBase-1.2 | | Pig | 0.16.0-amzn-0 | 0.16.0-amzn-0 | 0.16.0-amzn-0 | 0.16.0-amzn-0 | | Presto | 0.157.1 | 0.157.1 | 0.152.3 | 0.152.3 | | Spark | 2.0.2 | 2.0.2 | 2.0.2 | 2.0.1 | | Sqoop | 1.4.6 | 1.4.6 | 1.4.6 | 1.4.6 | | TensorFlow | - | - | - | - | | Tez | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | | Trino (PrestoSQL) | - | - | - | - | | Zeppelin | 0.6.2 | 0.6.2 | 0.6.2 | 0.6.2 | | ZooKeeper | 3.4.9 | 3.4.9 | 3.4.8 | 3.4.8 | ## 5.2.2 release notes The following release notes include information for the Amazon EMR 5.2.2 release. Changes are relative to the Amazon EMR 5.2.1 release. Release date: May 2, 2017 **Known issues resolved from previous releases** + Backported [SPARK-194459](https://issues.apache.org/jira/browse/SPARK-19459), which addresses an issue where reading from an ORC table with char/varchar columns can fail. ## 5.2.2 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | emr-ddb | 4.2.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 2.2.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.2.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-s3-dist-cp | 2.4.0 | Distributed copy application optimized for Amazon S3. | | emrfs | 2.13.0 | Amazon S3 connector for Hadoop ecosystem applications. | | flink-client | 1.1.3 | Apache Flink command line client scripts and applications. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 2.7.3-amzn-1 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 2.7.3-amzn-1 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 2.7.3-amzn-1 | HDFS command-line client and library | | hadoop-hdfs-namenode | 2.7.3-amzn-1 | HDFS service for tracking file names and block locations. | | hadoop-httpfs-server | 2.7.3-amzn-1 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 2.7.3-amzn-1 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 2.7.3-amzn-1 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 2.7.3-amzn-1 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 2.7.3-amzn-1 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 2.7.3-amzn-1 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 1.2.3 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 1.2.3 | Service for serving one or more HBase regions. | | hbase-client | 1.2.3 | HBase command-line client. | | hbase-rest-server | 1.2.3 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 1.2.3 | Service providing a Thrift endpoint to HBase. | | hcatalog-client | 2.1.0-amzn-0 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 2.1.0-amzn-0 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 2.1.0-amzn-0 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 2.1.0-amzn-0 | Hive command line client. | | hive-metastore-server | 2.1.0-amzn-0 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server | 2.1.0-amzn-0 | Service for accepting Hive queries as web requests. | | hue-server | 3.10.0-amzn-0 | Web application for analyzing data using Hadoop ecosystem applications | | mahout-client | 0.12.2 | Library for machine learning. | | mysql-server | 5.5.52 | MySQL database server. | | oozie-client | 4.2.0 | Oozie command-line client. | | oozie-server | 4.2.0 | Service for accepting Oozie workflow requests. | | phoenix-library | 4.7.0-HBase-1.2 | The phoenix libraries for server and client | | phoenix-query-server | 4.7.0-HBase-1.2 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.157.1 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.157.1 | Service for executing pieces of a query. | | pig-client | 0.16.0-amzn-0 | Pig command-line client. | | spark-client | 2.0.2 | Spark command-line clients. | | spark-history-server | 2.0.2 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 2.0.2 | In-memory execution engine for YARN. | | spark-yarn-slave | 2.0.2 | Apache Spark libraries needed by YARN slaves. | | sqoop-client | 1.4.6 | Apache Sqoop command-line client. | | tez-on-yarn | 0.8.4 | The tez YARN application and libraries. | | webserver | 2.4.23 | Apache HTTP server. | | zeppelin-server | 0.6.2 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.4.9 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.4.9 | ZooKeeper command line client. | ## 5.2.2 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). **emr-5.2.2 classifications** | Classifications | Description | | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | | core-site | Change values in Hadoop's core-site.xml file. | | emrfs-site | Change EMRFS settings. | | flink-conf | Change flink-conf.yaml settings. | | flink-log4j | Change Flink log4j.properties settings. | | flink-log4j-yarn-session | Change Flink log4j-yarn-session.properties settings. | | flink-log4j-cli | Change Flink log4j-cli.properties settings. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | | hadoop-ssl-server | Change hadoop ssl server configuration | | hadoop-ssl-client | Change hadoop ssl client configuration | | hbase | Amazon EMR-curated settings for Apache HBase. | | hbase-env | Change values in HBase's environment. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbaase.properties file. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | | hbase-site | Change values in HBase's hbase-site.xml file. | | hdfs-encryption-zones | Configure HDFS encryption zones. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | | hcatalog-env | Change values in HCatalog's environment. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | | hcatalog-webhcat-log4j2 | Change values in HCatalog WebHCat's log4j2.properties. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | | hive-beeline-log4j2 | Change values in Hive's beeline-log4j2.properties file. | | hive-env | Change values in the Hive environment. | | hive-exec-log4j2 | Change values in Hive's hive-exec-log4j2.properties file. | | hive-llap-daemon-log4j2 | Change values in Hive's llap-daemon-log4j2.properties file. | | hive-log4j2 | Change values in Hive's hive-log4j2.properties file. | | hive-site | Change values in Hive's hive-site.xml file | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | | hue-ini | Change values in Hue's ini file | | httpfs-env | Change values in the HTTPFS environment. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | | mapred-env | Change values in the MapReduce application's environment. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | | oozie-env | Change values in Oozie's environment. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | | oozie-site | Change values in Oozie's oozie-site.xml file. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | | pig-properties | Change values in Pig's pig.properties file. | | pig-log4j | Change values in Pig's log4j.properties file. | | presto-log | Change values in Presto's log.properties file. | | presto-config | Change values in Presto's config.properties file. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | | presto-connector-hive | Change values in Presto's hive.properties file. | | presto-connector-jmx | Change values in Presto's jmx.properties file. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | | presto-connector-localfile | Change values in Presto's localfile.properties file. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | | presto-connector-redis | Change values in Presto's redis.properties file. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | | spark | Amazon EMR-curated settings for Apache Spark. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | | spark-env | Change values in the Spark environment. | | spark-hive-site | Change values in Spark's hive-site.xml file | | spark-log4j | Change values in Spark's log4j.properties file. | | spark-metrics | Change values in Spark's metrics.properties file. | | sqoop-env | Change values in Sqoop's environment. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | | tez-site | Change values in Tez's tez-site.xml file. | | yarn-env | Change values in the YARN environment. | | yarn-site | Change values in YARN's yarn-site.xml file. | | zeppelin-env | Change values in the Zeppelin environment. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | # Amazon EMR release 5.2.1 ## 5.2.1 application versions This release includes the following applications: [https://flink.apache.org/](https://flink.apache.org/), [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [http://gethue.com/](http://gethue.com/), [http://mahout.apache.org/](http://mahout.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-5.2.1 | emr-5.2.0 | emr-5.1.1 | emr-5.1.0 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.10.75 | 1.10.75 | 1.10.75 | 1.10.75 | | Python | Not tracked | Not tracked | Not tracked | Not tracked | | Scala | 2.11.8 | 2.11.8 | 2.11.8 | 2.11.8 | | AmazonCloudWatchAgent | - | - | - | - | | Delta | - | - | - | - | | Flink | 1.1.3 | 1.1.3 | 1.1.3 | 1.1.3 | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 1.2.3 | 1.2.3 | 1.2.3 | 1.2.3 | | HCatalog | 2.1.0-amzn-0 | 2.1.0-amzn-0 | 2.1.0-amzn-0 | 2.1.0-amzn-0 | | Hadoop | 2.7.3-amzn-1 | 2.7.3-amzn-0 | 2.7.3-amzn-0 | 2.7.3-amzn-0 | | Hive | 2.1.0-amzn-0 | 2.1.0-amzn-0 | 2.1.0-amzn-0 | 2.1.0-amzn-0 | | Hudi | - | - | - | - | | Hue | 3.10.0-amzn-0 | 3.10.0-amzn-0 | 3.10.0-amzn-0 | 3.10.0-amzn-0 | | Iceberg | - | - | - | - | | JupyterEnterpriseGateway | - | - | - | - | | JupyterHub | - | - | - | - | | Livy | - | - | - | - | | MXNet | - | - | - | - | | Mahout | 0.12.2 | 0.12.2 | 0.12.2 | 0.12.2 | | Oozie | 4.2.0 | 4.2.0 | 4.2.0 | 4.2.0 | | Phoenix | 4.7.0-HBase-1.2 | 4.7.0-HBase-1.2 | 4.7.0-HBase-1.2 | 4.7.0-HBase-1.2 | | Pig | 0.16.0-amzn-0 | 0.16.0-amzn-0 | 0.16.0-amzn-0 | 0.16.0-amzn-0 | | Presto | 0.157.1 | 0.152.3 | 0.152.3 | 0.152.3 | | Spark | 2.0.2 | 2.0.2 | 2.0.1 | 2.0.1 | | Sqoop | 1.4.6 | 1.4.6 | 1.4.6 | 1.4.6 | | TensorFlow | - | - | - | - | | Tez | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | | Trino (PrestoSQL) | - | - | - | - | | Zeppelin | 0.6.2 | 0.6.2 | 0.6.2 | 0.6.2 | | ZooKeeper | 3.4.9 | 3.4.8 | 3.4.8 | 3.4.8 | ## 5.2.1 release notes The following release notes include information for the Amazon EMR 5.2.1 release. Changes are relative to the Amazon EMR 5.2.0 release. Release date: December 29, 2016 **Upgrades** + Upgraded to Presto 0.157.1. For more information, see [Presto Release Notes](https://prestodb.io/docs/current/release/release-0.157.1.html) in the Presto documentation. + Upgraded to Zookeeper 3.4.9. For more information, see [ZooKeeper Release Notes](https://zookeeper.apache.org/doc/r3.4.9/releasenotes.html) in the Apache ZooKeeper documentation. **Changes and enhancements** + Added support for the Amazon EC2 m4.16xlarge instance type in Amazon EMR version 4.8.3 and later, excluding 5.0.0, 5.0.3, and 5.2.0. + Amazon EMR releases are now based on Amazon Linux 2016.09. For more information, see [https://aws.amazon.com/amazon-linux-ami/2016.09-release-notes/](https://aws.amazon.com/amazon-linux-ami/2016.09-release-notes/). + The location of Flink and YARN configuration paths are now set by default in `/etc/default/flink` that you do not need to set the environment variables `FLINK_CONF_DIR` and `HADOOP_CONF_DIR` when running the `flink` or `yarn-session.sh` driver scripts to launch Flink jobs. + Added support for FlinkKinesisConsumer class. **Known issues resolved from previous releases** + Fixed an issue in Hadoop where the ReplicationMonitor thread could get stuck for a long time because of a race between replication and deletion of the same file in a large cluster. + Fixed an issue where ControlledJob\$1toString failed with a null pointer exception (NPE) when job status was not successfully updated. ## 5.2.1 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | emr-ddb | 4.2.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 2.2.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.2.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-s3-dist-cp | 2.4.0 | Distributed copy application optimized for Amazon S3. | | emrfs | 2.13.0 | Amazon S3 connector for Hadoop ecosystem applications. | | flink-client | 1.1.3 | Apache Flink command line client scripts and applications. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 2.7.3-amzn-1 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 2.7.3-amzn-1 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 2.7.3-amzn-1 | HDFS command-line client and library | | hadoop-hdfs-namenode | 2.7.3-amzn-1 | HDFS service for tracking file names and block locations. | | hadoop-httpfs-server | 2.7.3-amzn-1 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 2.7.3-amzn-1 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 2.7.3-amzn-1 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 2.7.3-amzn-1 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 2.7.3-amzn-1 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 2.7.3-amzn-1 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 1.2.3 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 1.2.3 | Service for serving one or more HBase regions. | | hbase-client | 1.2.3 | HBase command-line client. | | hbase-rest-server | 1.2.3 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 1.2.3 | Service providing a Thrift endpoint to HBase. | | hcatalog-client | 2.1.0-amzn-0 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 2.1.0-amzn-0 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 2.1.0-amzn-0 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 2.1.0-amzn-0 | Hive command line client. | | hive-metastore-server | 2.1.0-amzn-0 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server | 2.1.0-amzn-0 | Service for accepting Hive queries as web requests. | | hue-server | 3.10.0-amzn-0 | Web application for analyzing data using Hadoop ecosystem applications | | mahout-client | 0.12.2 | Library for machine learning. | | mysql-server | 5.5.52 | MySQL database server. | | oozie-client | 4.2.0 | Oozie command-line client. | | oozie-server | 4.2.0 | Service for accepting Oozie workflow requests. | | phoenix-library | 4.7.0-HBase-1.2 | The phoenix libraries for server and client | | phoenix-query-server | 4.7.0-HBase-1.2 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.157.1 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.157.1 | Service for executing pieces of a query. | | pig-client | 0.16.0-amzn-0 | Pig command-line client. | | spark-client | 2.0.2 | Spark command-line clients. | | spark-history-server | 2.0.2 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 2.0.2 | In-memory execution engine for YARN. | | spark-yarn-slave | 2.0.2 | Apache Spark libraries needed by YARN slaves. | | sqoop-client | 1.4.6 | Apache Sqoop command-line client. | | tez-on-yarn | 0.8.4 | The tez YARN application and libraries. | | webserver | 2.4.23 | Apache HTTP server. | | zeppelin-server | 0.6.2 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.4.9 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.4.9 | ZooKeeper command line client. | ## 5.2.1 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). **emr-5.2.1 classifications** | Classifications | Description | | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | | core-site | Change values in Hadoop's core-site.xml file. | | emrfs-site | Change EMRFS settings. | | flink-conf | Change flink-conf.yaml settings. | | flink-log4j | Change Flink log4j.properties settings. | | flink-log4j-yarn-session | Change Flink log4j-yarn-session.properties settings. | | flink-log4j-cli | Change Flink log4j-cli.properties settings. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | | hadoop-ssl-server | Change hadoop ssl server configuration | | hadoop-ssl-client | Change hadoop ssl client configuration | | hbase | Amazon EMR-curated settings for Apache HBase. | | hbase-env | Change values in HBase's environment. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbaase.properties file. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | | hbase-site | Change values in HBase's hbase-site.xml file. | | hdfs-encryption-zones | Configure HDFS encryption zones. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | | hcatalog-env | Change values in HCatalog's environment. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | | hcatalog-webhcat-log4j2 | Change values in HCatalog WebHCat's log4j2.properties. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | | hive-beeline-log4j2 | Change values in Hive's beeline-log4j2.properties file. | | hive-env | Change values in the Hive environment. | | hive-exec-log4j2 | Change values in Hive's hive-exec-log4j2.properties file. | | hive-llap-daemon-log4j2 | Change values in Hive's llap-daemon-log4j2.properties file. | | hive-log4j2 | Change values in Hive's hive-log4j2.properties file. | | hive-site | Change values in Hive's hive-site.xml file | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | | hue-ini | Change values in Hue's ini file | | httpfs-env | Change values in the HTTPFS environment. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | | mapred-env | Change values in the MapReduce application's environment. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | | oozie-env | Change values in Oozie's environment. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | | oozie-site | Change values in Oozie's oozie-site.xml file. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | | pig-properties | Change values in Pig's pig.properties file. | | pig-log4j | Change values in Pig's log4j.properties file. | | presto-log | Change values in Presto's log.properties file. | | presto-config | Change values in Presto's config.properties file. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | | presto-connector-hive | Change values in Presto's hive.properties file. | | presto-connector-jmx | Change values in Presto's jmx.properties file. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | | presto-connector-localfile | Change values in Presto's localfile.properties file. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | | presto-connector-redis | Change values in Presto's redis.properties file. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | | spark | Amazon EMR-curated settings for Apache Spark. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | | spark-env | Change values in the Spark environment. | | spark-hive-site | Change values in Spark's hive-site.xml file | | spark-log4j | Change values in Spark's log4j.properties file. | | spark-metrics | Change values in Spark's metrics.properties file. | | sqoop-env | Change values in Sqoop's environment. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | | tez-site | Change values in Tez's tez-site.xml file. | | yarn-env | Change values in the YARN environment. | | yarn-site | Change values in YARN's yarn-site.xml file. | | zeppelin-env | Change values in the Zeppelin environment. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | # Amazon EMR release 5.2.0 ## 5.2.0 application versions This release includes the following applications: [https://flink.apache.org/](https://flink.apache.org/), [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [http://gethue.com/](http://gethue.com/), [http://mahout.apache.org/](http://mahout.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-5.2.0 | emr-5.1.1 | emr-5.1.0 | emr-5.0.3 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.10.75 | 1.10.75 | 1.10.75 | 1.10.75 | | Python | Not tracked | Not tracked | Not tracked | Not tracked | | Scala | 2.11.8 | 2.11.8 | 2.11.8 | 2.11.8 | | AmazonCloudWatchAgent | - | - | - | - | | Delta | - | - | - | - | | Flink | 1.1.3 | 1.1.3 | 1.1.3 | - | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 1.2.3 | 1.2.3 | 1.2.3 | 1.2.2 | | HCatalog | 2.1.0-amzn-0 | 2.1.0-amzn-0 | 2.1.0-amzn-0 | 2.1.0-amzn-0 | | Hadoop | 2.7.3-amzn-0 | 2.7.3-amzn-0 | 2.7.3-amzn-0 | 2.7.3-amzn-0 | | Hive | 2.1.0-amzn-0 | 2.1.0-amzn-0 | 2.1.0-amzn-0 | 2.1.0-amzn-0 | | Hudi | - | - | - | - | | Hue | 3.10.0-amzn-0 | 3.10.0-amzn-0 | 3.10.0-amzn-0 | 3.10.0-amzn-0 | | Iceberg | - | - | - | - | | JupyterEnterpriseGateway | - | - | - | - | | JupyterHub | - | - | - | - | | Livy | - | - | - | - | | MXNet | - | - | - | - | | Mahout | 0.12.2 | 0.12.2 | 0.12.2 | 0.12.2 | | Oozie | 4.2.0 | 4.2.0 | 4.2.0 | 4.2.0 | | Phoenix | 4.7.0-HBase-1.2 | 4.7.0-HBase-1.2 | 4.7.0-HBase-1.2 | 4.7.0-HBase-1.2 | | Pig | 0.16.0-amzn-0 | 0.16.0-amzn-0 | 0.16.0-amzn-0 | 0.16.0-amzn-0 | | Presto | 0.152.3 | 0.152.3 | 0.152.3 | 0.152.3 | | Spark | 2.0.2 | 2.0.1 | 2.0.1 | 2.0.1 | | Sqoop | 1.4.6 | 1.4.6 | 1.4.6 | 1.4.6 | | TensorFlow | - | - | - | - | | Tez | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | | Trino (PrestoSQL) | - | - | - | - | | Zeppelin | 0.6.2 | 0.6.2 | 0.6.2 | 0.6.1 | | ZooKeeper | 3.4.8 | 3.4.8 | 3.4.8 | 3.4.8 | ## 5.2.0 release notes The following release notes include information for the Amazon EMR 5.2.0 release. Changes are relative to the Amazon EMR 5.1.0 release. Release date: November 21, 2016 **Changes and enhancements** + Added Amazon S3 storage mode for HBase. + Enables you to specify an Amazon S3 location for the HBase rootdir. For more information, see [HBase on Amazon S3](https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-hbase-s3.html). **Upgrades** + Upgraded to Spark 2.0.2 **Known issues resolved from previous releases** + Fixed an issue with /mnt being constrained to 2 TB on EBS-only instance types. + Fixed an issue with instance-controller and logpusher logs being output to their corresponding .out files instead of to their normal log4j-configured .log files, which rotate hourly. The .out files do not rotate, so this would eventually fill up the /emr partition. This issue only affects hardware virtual machine (HVM) instance types. ## 5.2.0 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | emr-ddb | 4.1.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 2.1.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.2.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-s3-dist-cp | 2.4.0 | Distributed copy application optimized for Amazon S3. | | emrfs | 2.12.0 | Amazon S3 connector for Hadoop ecosystem applications. | | flink-client | 1.1.3 | Apache Flink command line client scripts and applications. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 2.7.3-amzn-0 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 2.7.3-amzn-0 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 2.7.3-amzn-0 | HDFS command-line client and library | | hadoop-hdfs-namenode | 2.7.3-amzn-0 | HDFS service for tracking file names and block locations. | | hadoop-httpfs-server | 2.7.3-amzn-0 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 2.7.3-amzn-0 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 2.7.3-amzn-0 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 2.7.3-amzn-0 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 2.7.3-amzn-0 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 2.7.3-amzn-0 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 1.2.3 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 1.2.3 | Service for serving one or more HBase regions. | | hbase-client | 1.2.3 | HBase command-line client. | | hbase-rest-server | 1.2.3 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 1.2.3 | Service providing a Thrift endpoint to HBase. | | hcatalog-client | 2.1.0-amzn-0 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 2.1.0-amzn-0 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 2.1.0-amzn-0 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 2.1.0-amzn-0 | Hive command line client. | | hive-metastore-server | 2.1.0-amzn-0 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server | 2.1.0-amzn-0 | Service for accepting Hive queries as web requests. | | hue-server | 3.10.0-amzn-0 | Web application for analyzing data using Hadoop ecosystem applications | | mahout-client | 0.12.2 | Library for machine learning. | | mysql-server | 5.5.52 | MySQL database server. | | oozie-client | 4.2.0 | Oozie command-line client. | | oozie-server | 4.2.0 | Service for accepting Oozie workflow requests. | | phoenix-library | 4.7.0-HBase-1.2 | The phoenix libraries for server and client | | phoenix-query-server | 4.7.0-HBase-1.2 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.152.3 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.152.3 | Service for executing pieces of a query. | | pig-client | 0.16.0-amzn-0 | Pig command-line client. | | spark-client | 2.0.2 | Spark command-line clients. | | spark-history-server | 2.0.2 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 2.0.2 | In-memory execution engine for YARN. | | spark-yarn-slave | 2.0.2 | Apache Spark libraries needed by YARN slaves. | | sqoop-client | 1.4.6 | Apache Sqoop command-line client. | | tez-on-yarn | 0.8.4 | The tez YARN application and libraries. | | webserver | 2.4.23 | Apache HTTP server. | | zeppelin-server | 0.6.2 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.4.8 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.4.8 | ZooKeeper command line client. | ## 5.2.0 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). **emr-5.2.0 classifications** | Classifications | Description | | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | | core-site | Change values in Hadoop's core-site.xml file. | | emrfs-site | Change EMRFS settings. | | flink-conf | Change flink-conf.yaml settings. | | flink-log4j | Change Flink log4j.properties settings. | | flink-log4j-yarn-session | Change Flink log4j-yarn-session.properties settings. | | flink-log4j-cli | Change Flink log4j-cli.properties settings. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | | hadoop-ssl-server | Change hadoop ssl server configuration | | hadoop-ssl-client | Change hadoop ssl client configuration | | hbase | Amazon EMR-curated settings for Apache HBase. | | hbase-env | Change values in HBase's environment. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbaase.properties file. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | | hbase-site | Change values in HBase's hbase-site.xml file. | | hdfs-encryption-zones | Configure HDFS encryption zones. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | | hcatalog-env | Change values in HCatalog's environment. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | | hcatalog-webhcat-log4j2 | Change values in HCatalog WebHCat's log4j2.properties. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | | hive-beeline-log4j2 | Change values in Hive's beeline-log4j2.properties file. | | hive-env | Change values in the Hive environment. | | hive-exec-log4j2 | Change values in Hive's hive-exec-log4j2.properties file. | | hive-llap-daemon-log4j2 | Change values in Hive's llap-daemon-log4j2.properties file. | | hive-log4j2 | Change values in Hive's hive-log4j2.properties file. | | hive-site | Change values in Hive's hive-site.xml file | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | | hue-ini | Change values in Hue's ini file | | httpfs-env | Change values in the HTTPFS environment. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | | mapred-env | Change values in the MapReduce application's environment. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | | oozie-env | Change values in Oozie's environment. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | | oozie-site | Change values in Oozie's oozie-site.xml file. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | | pig-properties | Change values in Pig's pig.properties file. | | pig-log4j | Change values in Pig's log4j.properties file. | | presto-log | Change values in Presto's log.properties file. | | presto-config | Change values in Presto's config.properties file. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | | presto-connector-hive | Change values in Presto's hive.properties file. | | presto-connector-jmx | Change values in Presto's jmx.properties file. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | | presto-connector-localfile | Change values in Presto's localfile.properties file. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | | presto-connector-redis | Change values in Presto's redis.properties file. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | | spark | Amazon EMR-curated settings for Apache Spark. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | | spark-env | Change values in the Spark environment. | | spark-hive-site | Change values in Spark's hive-site.xml file | | spark-log4j | Change values in Spark's log4j.properties file. | | spark-metrics | Change values in Spark's metrics.properties file. | | sqoop-env | Change values in Sqoop's environment. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | | tez-site | Change values in Tez's tez-site.xml file. | | yarn-env | Change values in the YARN environment. | | yarn-site | Change values in YARN's yarn-site.xml file. | | zeppelin-env | Change values in the Zeppelin environment. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | # Amazon EMR release 5.1.1 ## 5.1.1 application versions This release includes the following applications: [https://flink.apache.org/](https://flink.apache.org/), [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [http://gethue.com/](http://gethue.com/), [http://mahout.apache.org/](http://mahout.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-5.1.1 | emr-5.1.0 | emr-5.0.3 | emr-5.0.2 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.10.75 | 1.10.75 | 1.10.75 | 1.10.75 | | Python | Not tracked | Not tracked | Not tracked | Not tracked | | Scala | 2.11.8 | 2.11.8 | 2.11.8 | 2.11.8 | | AmazonCloudWatchAgent | - | - | - | - | | Delta | - | - | - | - | | Flink | 1.1.3 | 1.1.3 | - | - | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 1.2.3 | 1.2.3 | 1.2.2 | 1.2.2 | | HCatalog | 2.1.0-amzn-0 | 2.1.0-amzn-0 | 2.1.0-amzn-0 | 2.1.0-amzn-0 | | Hadoop | 2.7.3-amzn-0 | 2.7.3-amzn-0 | 2.7.3-amzn-0 | 2.7.2-amzn-3 | | Hive | 2.1.0-amzn-0 | 2.1.0-amzn-0 | 2.1.0-amzn-0 | 2.1.0-amzn-0 | | Hudi | - | - | - | - | | Hue | 3.10.0-amzn-0 | 3.10.0-amzn-0 | 3.10.0-amzn-0 | 3.10.0-amzn-0 | | Iceberg | - | - | - | - | | JupyterEnterpriseGateway | - | - | - | - | | JupyterHub | - | - | - | - | | Livy | - | - | - | - | | MXNet | - | - | - | - | | Mahout | 0.12.2 | 0.12.2 | 0.12.2 | 0.12.2 | | Oozie | 4.2.0 | 4.2.0 | 4.2.0 | 4.2.0 | | Phoenix | 4.7.0-HBase-1.2 | 4.7.0-HBase-1.2 | 4.7.0-HBase-1.2 | 4.7.0-HBase-1.2 | | Pig | 0.16.0-amzn-0 | 0.16.0-amzn-0 | 0.16.0-amzn-0 | 0.16.0-amzn-0 | | Presto | 0.152.3 | 0.152.3 | 0.152.3 | 0.150 | | Spark | 2.0.1 | 2.0.1 | 2.0.1 | 2.0.0 | | Sqoop | 1.4.6 | 1.4.6 | 1.4.6 | 1.4.6 | | TensorFlow | - | - | - | - | | Tez | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | | Trino (PrestoSQL) | - | - | - | - | | Zeppelin | 0.6.2 | 0.6.2 | 0.6.1 | 0.6.1-SNAPSHOT | | ZooKeeper | 3.4.8 | 3.4.8 | 3.4.8 | 3.4.8 | ## 5.1.1 release notes This is a patch release to add AWS Signature Version 4 authentication for requests to Amazon S3. All applications and components are the same as the previous Amazon EMR release. **Important** In this release version, Amazon EMR uses AWS Signature Version 4 exclusively to authenticate requests to Amazon S3. For more information, see [Whats New](https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-whatsnew.html). ## 5.1.1 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | emr-ddb | 4.1.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 2.1.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.2.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-s3-dist-cp | 2.4.0 | Distributed copy application optimized for Amazon S3. | | emrfs | 2.11.0 | Amazon S3 connector for Hadoop ecosystem applications. | | flink-client | 1.1.3 | Apache Flink command line client scripts and applications. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 2.7.3-amzn-0 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 2.7.3-amzn-0 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 2.7.3-amzn-0 | HDFS command-line client and library | | hadoop-hdfs-namenode | 2.7.3-amzn-0 | HDFS service for tracking file names and block locations. | | hadoop-httpfs-server | 2.7.3-amzn-0 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 2.7.3-amzn-0 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 2.7.3-amzn-0 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 2.7.3-amzn-0 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 2.7.3-amzn-0 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 2.7.3-amzn-0 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 1.2.3 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 1.2.3 | Service for serving one or more HBase regions. | | hbase-client | 1.2.3 | HBase command-line client. | | hbase-rest-server | 1.2.3 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 1.2.3 | Service providing a Thrift endpoint to HBase. | | hcatalog-client | 2.1.0-amzn-0 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 2.1.0-amzn-0 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 2.1.0-amzn-0 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 2.1.0-amzn-0 | Hive command line client. | | hive-metastore-server | 2.1.0-amzn-0 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server | 2.1.0-amzn-0 | Service for accepting Hive queries as web requests. | | hue-server | 3.10.0-amzn-0 | Web application for analyzing data using Hadoop ecosystem applications | | mahout-client | 0.12.2 | Library for machine learning. | | mysql-server | 5.5.52 | MySQL database server. | | oozie-client | 4.2.0 | Oozie command-line client. | | oozie-server | 4.2.0 | Service for accepting Oozie workflow requests. | | phoenix-library | 4.7.0-HBase-1.2 | The phoenix libraries for server and client | | phoenix-query-server | 4.7.0-HBase-1.2 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.152.3 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.152.3 | Service for executing pieces of a query. | | pig-client | 0.16.0-amzn-0 | Pig command-line client. | | spark-client | 2.0.1 | Spark command-line clients. | | spark-history-server | 2.0.1 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 2.0.1 | In-memory execution engine for YARN. | | spark-yarn-slave | 2.0.1 | Apache Spark libraries needed by YARN slaves. | | sqoop-client | 1.4.6 | Apache Sqoop command-line client. | | tez-on-yarn | 0.8.4 | The tez YARN application and libraries. | | webserver | 2.4.23 | Apache HTTP server. | | zeppelin-server | 0.6.2 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.4.8 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.4.8 | ZooKeeper command line client. | ## 5.1.1 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). **emr-5.1.1 classifications** | Classifications | Description | | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | | core-site | Change values in Hadoop's core-site.xml file. | | emrfs-site | Change EMRFS settings. | | flink-conf | Change flink-conf.yaml settings. | | flink-log4j | Change Flink log4j.properties settings. | | flink-log4j-yarn-session | Change Flink log4j-yarn-session.properties settings. | | flink-log4j-cli | Change Flink log4j-cli.properties settings. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | | hadoop-ssl-server | Change hadoop ssl server configuration | | hadoop-ssl-client | Change hadoop ssl client configuration | | hbase-env | Change values in HBase's environment. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbaase.properties file. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | | hbase-site | Change values in HBase's hbase-site.xml file. | | hdfs-encryption-zones | Configure HDFS encryption zones. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | | hcatalog-env | Change values in HCatalog's environment. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | | hcatalog-webhcat-log4j2 | Change values in HCatalog WebHCat's log4j2.properties. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | | hive-beeline-log4j2 | Change values in Hive's beeline-log4j2.properties file. | | hive-env | Change values in the Hive environment. | | hive-exec-log4j2 | Change values in Hive's hive-exec-log4j2.properties file. | | hive-llap-daemon-log4j2 | Change values in Hive's llap-daemon-log4j2.properties file. | | hive-log4j2 | Change values in Hive's hive-log4j2.properties file. | | hive-site | Change values in Hive's hive-site.xml file | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | | hue-ini | Change values in Hue's ini file | | httpfs-env | Change values in the HTTPFS environment. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | | mapred-env | Change values in the MapReduce application's environment. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | | oozie-env | Change values in Oozie's environment. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | | oozie-site | Change values in Oozie's oozie-site.xml file. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | | pig-properties | Change values in Pig's pig.properties file. | | pig-log4j | Change values in Pig's log4j.properties file. | | presto-log | Change values in Presto's log.properties file. | | presto-config | Change values in Presto's config.properties file. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | | presto-connector-hive | Change values in Presto's hive.properties file. | | presto-connector-jmx | Change values in Presto's jmx.properties file. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | | presto-connector-localfile | Change values in Presto's localfile.properties file. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | | presto-connector-redis | Change values in Presto's redis.properties file. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | | spark | Amazon EMR-curated settings for Apache Spark. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | | spark-env | Change values in the Spark environment. | | spark-hive-site | Change values in Spark's hive-site.xml file | | spark-log4j | Change values in Spark's log4j.properties file. | | spark-metrics | Change values in Spark's metrics.properties file. | | sqoop-env | Change values in Sqoop's environment. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | | tez-site | Change values in Tez's tez-site.xml file. | | yarn-env | Change values in the YARN environment. | | yarn-site | Change values in YARN's yarn-site.xml file. | | zeppelin-env | Change values in the Zeppelin environment. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | # Amazon EMR release 5.1.0 ## 5.1.0 application versions This release includes the following applications: [https://flink.apache.org/](https://flink.apache.org/), [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [http://gethue.com/](http://gethue.com/), [http://mahout.apache.org/](http://mahout.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-5.1.0 | emr-5.0.3 | emr-5.0.2 | emr-5.0.1 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.10.75 | 1.10.75 | 1.10.75 | 1.10.75 | | Python | Not tracked | Not tracked | Not tracked | Not tracked | | Scala | 2.11.8 | 2.11.8 | 2.11.8 | 2.11.8 | | AmazonCloudWatchAgent | - | - | - | - | | Delta | - | - | - | - | | Flink | 1.1.3 | - | - | - | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 1.2.3 | 1.2.2 | 1.2.2 | 1.2.2 | | HCatalog | 2.1.0-amzn-0 | 2.1.0-amzn-0 | 2.1.0-amzn-0 | 2.1.0-amzn-0 | | Hadoop | 2.7.3-amzn-0 | 2.7.3-amzn-0 | 2.7.2-amzn-3 | 2.7.2-amzn-3 | | Hive | 2.1.0-amzn-0 | 2.1.0-amzn-0 | 2.1.0-amzn-0 | 2.1.0-amzn-0 | | Hudi | - | - | - | - | | Hue | 3.10.0-amzn-0 | 3.10.0-amzn-0 | 3.10.0-amzn-0 | 3.10.0-amzn-0 | | Iceberg | - | - | - | - | | JupyterEnterpriseGateway | - | - | - | - | | JupyterHub | - | - | - | - | | Livy | - | - | - | - | | MXNet | - | - | - | - | | Mahout | 0.12.2 | 0.12.2 | 0.12.2 | 0.12.2 | | Oozie | 4.2.0 | 4.2.0 | 4.2.0 | 4.2.0 | | Phoenix | 4.7.0-HBase-1.2 | 4.7.0-HBase-1.2 | 4.7.0-HBase-1.2 | 4.7.0-HBase-1.2 | | Pig | 0.16.0-amzn-0 | 0.16.0-amzn-0 | 0.16.0-amzn-0 | 0.16.0-amzn-0 | | Presto | 0.152.3 | 0.152.3 | 0.150 | 0.150 | | Spark | 2.0.1 | 2.0.1 | 2.0.0 | 2.0.0 | | Sqoop | 1.4.6 | 1.4.6 | 1.4.6 | 1.4.6 | | TensorFlow | - | - | - | - | | Tez | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | | Trino (PrestoSQL) | - | - | - | - | | Zeppelin | 0.6.2 | 0.6.1 | 0.6.1-SNAPSHOT | 0.6.1-SNAPSHOT | | ZooKeeper | 3.4.8 | 3.4.8 | 3.4.8 | 3.4.8 | ## 5.1.0 release notes The following release notes include information for the Amazon EMR 5.1.0 release. Changes are relative to the Amazon EMR 5.0.3 release. Release date: November 03, 2016 **Changes and enhancements** + Added support for Flink 1.1.3. + Presto has been added as an option in the notebook section of Hue. **Upgrades** + Upgraded to HBase 1.2.3 + Upgraded to Zeppelin 0.6.2 **Known issues resolved from previous releases** + Fixed an issue with Tez queries on Amazon S3 with ORC files did not perform as well as earlier Amazon EMR 4.x versions. ## 5.1.0 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | emr-ddb | 4.1.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 2.1.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.2.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-s3-dist-cp | 2.4.0 | Distributed copy application optimized for Amazon S3. | | emrfs | 2.11.0 | Amazon S3 connector for Hadoop ecosystem applications. | | flink-client | 1.1.3 | Apache Flink command line client scripts and applications. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 2.7.3-amzn-0 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 2.7.3-amzn-0 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 2.7.3-amzn-0 | HDFS command-line client and library | | hadoop-hdfs-namenode | 2.7.3-amzn-0 | HDFS service for tracking file names and block locations. | | hadoop-httpfs-server | 2.7.3-amzn-0 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 2.7.3-amzn-0 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 2.7.3-amzn-0 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 2.7.3-amzn-0 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 2.7.3-amzn-0 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 2.7.3-amzn-0 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 1.2.3 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 1.2.3 | Service for serving one or more HBase regions. | | hbase-client | 1.2.3 | HBase command-line client. | | hbase-rest-server | 1.2.3 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 1.2.3 | Service providing a Thrift endpoint to HBase. | | hcatalog-client | 2.1.0-amzn-0 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 2.1.0-amzn-0 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 2.1.0-amzn-0 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 2.1.0-amzn-0 | Hive command line client. | | hive-metastore-server | 2.1.0-amzn-0 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server | 2.1.0-amzn-0 | Service for accepting Hive queries as web requests. | | hue-server | 3.10.0-amzn-0 | Web application for analyzing data using Hadoop ecosystem applications | | mahout-client | 0.12.2 | Library for machine learning. | | mysql-server | 5.5.52 | MySQL database server. | | oozie-client | 4.2.0 | Oozie command-line client. | | oozie-server | 4.2.0 | Service for accepting Oozie workflow requests. | | phoenix-library | 4.7.0-HBase-1.2 | The phoenix libraries for server and client | | phoenix-query-server | 4.7.0-HBase-1.2 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.152.3 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.152.3 | Service for executing pieces of a query. | | pig-client | 0.16.0-amzn-0 | Pig command-line client. | | spark-client | 2.0.1 | Spark command-line clients. | | spark-history-server | 2.0.1 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 2.0.1 | In-memory execution engine for YARN. | | spark-yarn-slave | 2.0.1 | Apache Spark libraries needed by YARN slaves. | | sqoop-client | 1.4.6 | Apache Sqoop command-line client. | | tez-on-yarn | 0.8.4 | The tez YARN application and libraries. | | webserver | 2.4.23 | Apache HTTP server. | | zeppelin-server | 0.6.2 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.4.8 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.4.8 | ZooKeeper command line client. | ## 5.1.0 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). **emr-5.1.0 classifications** | Classifications | Description | | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | | core-site | Change values in Hadoop's core-site.xml file. | | emrfs-site | Change EMRFS settings. | | flink-conf | Change flink-conf.yaml settings. | | flink-log4j | Change Flink log4j.properties settings. | | flink-log4j-yarn-session | Change Flink log4j-yarn-session.properties settings. | | flink-log4j-cli | Change Flink log4j-cli.properties settings. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | | hadoop-ssl-server | Change hadoop ssl server configuration | | hadoop-ssl-client | Change hadoop ssl client configuration | | hbase-env | Change values in HBase's environment. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbaase.properties file. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | | hbase-site | Change values in HBase's hbase-site.xml file. | | hdfs-encryption-zones | Configure HDFS encryption zones. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | | hcatalog-env | Change values in HCatalog's environment. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | | hcatalog-webhcat-log4j2 | Change values in HCatalog WebHCat's log4j2.properties. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | | hive-beeline-log4j2 | Change values in Hive's beeline-log4j2.properties file. | | hive-env | Change values in the Hive environment. | | hive-exec-log4j2 | Change values in Hive's hive-exec-log4j2.properties file. | | hive-llap-daemon-log4j2 | Change values in Hive's llap-daemon-log4j2.properties file. | | hive-log4j2 | Change values in Hive's hive-log4j2.properties file. | | hive-site | Change values in Hive's hive-site.xml file | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | | hue-ini | Change values in Hue's ini file | | httpfs-env | Change values in the HTTPFS environment. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | | mapred-env | Change values in the MapReduce application's environment. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | | oozie-env | Change values in Oozie's environment. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | | oozie-site | Change values in Oozie's oozie-site.xml file. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | | pig-properties | Change values in Pig's pig.properties file. | | pig-log4j | Change values in Pig's log4j.properties file. | | presto-log | Change values in Presto's log.properties file. | | presto-config | Change values in Presto's config.properties file. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | | presto-connector-hive | Change values in Presto's hive.properties file. | | presto-connector-jmx | Change values in Presto's jmx.properties file. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | | presto-connector-localfile | Change values in Presto's localfile.properties file. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | | presto-connector-redis | Change values in Presto's redis.properties file. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | | spark | Amazon EMR-curated settings for Apache Spark. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | | spark-env | Change values in the Spark environment. | | spark-hive-site | Change values in Spark's hive-site.xml file | | spark-log4j | Change values in Spark's log4j.properties file. | | spark-metrics | Change values in Spark's metrics.properties file. | | sqoop-env | Change values in Sqoop's environment. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | | tez-site | Change values in Tez's tez-site.xml file. | | yarn-env | Change values in the YARN environment. | | yarn-site | Change values in YARN's yarn-site.xml file. | | zeppelin-env | Change values in the Zeppelin environment. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | # Amazon EMR release 5.0.3 ## 5.0.3 application versions This release includes the following applications: [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [http://gethue.com/](http://gethue.com/), [http://mahout.apache.org/](http://mahout.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-5.0.3 | emr-5.0.2 | emr-5.0.1 | emr-5.0.0 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.10.75 | 1.10.75 | 1.10.75 | 1.10.75 | | Python | Not tracked | Not tracked | Not tracked | Not tracked | | Scala | 2.11.8 | 2.11.8 | 2.11.8 | 2.11.8 | | AmazonCloudWatchAgent | - | - | - | - | | Delta | - | - | - | - | | Flink | - | - | - | - | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 1.2.2 | 1.2.2 | 1.2.2 | 1.2.2 | | HCatalog | 2.1.0-amzn-0 | 2.1.0-amzn-0 | 2.1.0-amzn-0 | 2.1.0-amzn-0 | | Hadoop | 2.7.3-amzn-0 | 2.7.2-amzn-3 | 2.7.2-amzn-3 | 2.7.2-amzn-3 | | Hive | 2.1.0-amzn-0 | 2.1.0-amzn-0 | 2.1.0-amzn-0 | 2.1.0-amzn-0 | | Hudi | - | - | - | - | | Hue | 3.10.0-amzn-0 | 3.10.0-amzn-0 | 3.10.0-amzn-0 | 3.10.0-amzn-0 | | Iceberg | - | - | - | - | | JupyterEnterpriseGateway | - | - | - | - | | JupyterHub | - | - | - | - | | Livy | - | - | - | - | | MXNet | - | - | - | - | | Mahout | 0.12.2 | 0.12.2 | 0.12.2 | 0.12.2 | | Oozie | 4.2.0 | 4.2.0 | 4.2.0 | 4.2.0 | | Phoenix | 4.7.0-HBase-1.2 | 4.7.0-HBase-1.2 | 4.7.0-HBase-1.2 | 4.7.0-HBase-1.2 | | Pig | 0.16.0-amzn-0 | 0.16.0-amzn-0 | 0.16.0-amzn-0 | 0.16.0-amzn-0 | | Presto | 0.152.3 | 0.150 | 0.150 | 0.150 | | Spark | 2.0.1 | 2.0.0 | 2.0.0 | 2.0.0 | | Sqoop | 1.4.6 | 1.4.6 | 1.4.6 | 1.4.6 | | TensorFlow | - | - | - | - | | Tez | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | | Trino (PrestoSQL) | - | - | - | - | | Zeppelin | 0.6.1 | 0.6.1-SNAPSHOT | 0.6.1-SNAPSHOT | 0.6.1-SNAPSHOT | | ZooKeeper | 3.4.8 | 3.4.8 | 3.4.8 | 3.4.8 | ## 5.0.3 release notes The following release notes include information for the Amazon EMR 5.0.3 release. Changes are relative to the Amazon EMR 5.0.0 release. Release date: October 24, 2016 **Upgrades** + Upgraded to Hadoop 2.7.3 + Upgraded to Presto 0.152.3, which includes support for the Presto web interface. You can access the Presto web interface on the Presto coordinator using port 8889. For more information about the Presto web interface, see [Web Interface](https://prestodb.io/docs/current/admin/web-interface.html) in the Presto documentation. + Upgraded to Spark 2.0.1 + Amazon EMR releases are now based on Amazon Linux 2016.09. For more information, see [https://aws.amazon.com/amazon-linux-ami/2016.09-release-notes/](https://aws.amazon.com/amazon-linux-ami/2016.09-release-notes/). ## 5.0.3 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | emr-ddb | 4.1.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 2.1.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.2.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-s3-dist-cp | 2.4.0 | Distributed copy application optimized for Amazon S3. | | emrfs | 2.10.0 | Amazon S3 connector for Hadoop ecosystem applications. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 2.7.3-amzn-0 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 2.7.3-amzn-0 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 2.7.3-amzn-0 | HDFS command-line client and library | | hadoop-hdfs-namenode | 2.7.3-amzn-0 | HDFS service for tracking file names and block locations. | | hadoop-httpfs-server | 2.7.3-amzn-0 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 2.7.3-amzn-0 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 2.7.3-amzn-0 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 2.7.3-amzn-0 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 2.7.3-amzn-0 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 2.7.3-amzn-0 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 1.2.2 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 1.2.2 | Service for serving one or more HBase regions. | | hbase-client | 1.2.2 | HBase command-line client. | | hbase-rest-server | 1.2.2 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 1.2.2 | Service providing a Thrift endpoint to HBase. | | hcatalog-client | 2.1.0-amzn-0 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 2.1.0-amzn-0 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 2.1.0-amzn-0 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 2.1.0-amzn-0 | Hive command line client. | | hive-metastore-server | 2.1.0-amzn-0 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server | 2.1.0-amzn-0 | Service for accepting Hive queries as web requests. | | hue-server | 3.10.0-amzn-0 | Web application for analyzing data using Hadoop ecosystem applications | | mahout-client | 0.12.2 | Library for machine learning. | | mysql-server | 5.5.52 | MySQL database server. | | oozie-client | 4.2.0 | Oozie command-line client. | | oozie-server | 4.2.0 | Service for accepting Oozie workflow requests. | | phoenix-library | 4.7.0-HBase-1.2 | The phoenix libraries for server and client | | phoenix-query-server | 4.7.0-HBase-1.2 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.152.3 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.152.3 | Service for executing pieces of a query. | | pig-client | 0.16.0-amzn-0 | Pig command-line client. | | spark-client | 2.0.1 | Spark command-line clients. | | spark-history-server | 2.0.1 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 2.0.1 | In-memory execution engine for YARN. | | spark-yarn-slave | 2.0.1 | Apache Spark libraries needed by YARN slaves. | | sqoop-client | 1.4.6 | Apache Sqoop command-line client. | | tez-on-yarn | 0.8.4 | The tez YARN application and libraries. | | webserver | 2.4.23 | Apache HTTP server. | | zeppelin-server | 0.6.1 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.4.8 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.4.8 | ZooKeeper command line client. | ## 5.0.3 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). **emr-5.0.3 classifications** | Classifications | Description | | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | | core-site | Change values in Hadoop's core-site.xml file. | | emrfs-site | Change EMRFS settings. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | | hadoop-ssl-server | Change hadoop ssl server configuration | | hadoop-ssl-client | Change hadoop ssl client configuration | | hbase-env | Change values in HBase's environment. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbaase.properties file. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | | hbase-site | Change values in HBase's hbase-site.xml file. | | hdfs-encryption-zones | Configure HDFS encryption zones. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | | hcatalog-env | Change values in HCatalog's environment. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | | hcatalog-webhcat-log4j2 | Change values in HCatalog WebHCat's log4j2.properties. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | | hive-beeline-log4j2 | Change values in Hive's beeline-log4j2.properties file. | | hive-env | Change values in the Hive environment. | | hive-exec-log4j2 | Change values in Hive's hive-exec-log4j2.properties file. | | hive-llap-daemon-log4j2 | Change values in Hive's llap-daemon-log4j2.properties file. | | hive-log4j2 | Change values in Hive's hive-log4j2.properties file. | | hive-site | Change values in Hive's hive-site.xml file | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | | hue-ini | Change values in Hue's ini file | | httpfs-env | Change values in the HTTPFS environment. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | | mapred-env | Change values in the MapReduce application's environment. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | | oozie-env | Change values in Oozie's environment. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | | oozie-site | Change values in Oozie's oozie-site.xml file. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | | pig-properties | Change values in Pig's pig.properties file. | | pig-log4j | Change values in Pig's log4j.properties file. | | presto-log | Change values in Presto's log.properties file. | | presto-config | Change values in Presto's config.properties file. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | | presto-connector-hive | Change values in Presto's hive.properties file. | | presto-connector-jmx | Change values in Presto's jmx.properties file. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | | presto-connector-localfile | Change values in Presto's localfile.properties file. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | | presto-connector-redis | Change values in Presto's redis.properties file. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | | spark | Amazon EMR-curated settings for Apache Spark. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | | spark-env | Change values in the Spark environment. | | spark-hive-site | Change values in Spark's hive-site.xml file | | spark-log4j | Change values in Spark's log4j.properties file. | | spark-metrics | Change values in Spark's metrics.properties file. | | sqoop-env | Change values in Sqoop's environment. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | | tez-site | Change values in Tez's tez-site.xml file. | | yarn-env | Change values in the YARN environment. | | yarn-site | Change values in YARN's yarn-site.xml file. | | zeppelin-env | Change values in the Zeppelin environment. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | # Amazon EMR release 5.0.2 ## 5.0.2 application versions This release includes the following applications: [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [http://gethue.com/](http://gethue.com/), [http://mahout.apache.org/](http://mahout.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-5.0.3 | emr-5.0.2 | emr-5.0.1 | emr-5.0.0 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.10.75 | 1.10.75 | 1.10.75 | 1.10.75 | | Python | Not tracked | Not tracked | Not tracked | Not tracked | | Scala | 2.11.8 | 2.11.8 | 2.11.8 | 2.11.8 | | AmazonCloudWatchAgent | - | - | - | - | | Delta | - | - | - | - | | Flink | - | - | - | - | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 1.2.2 | 1.2.2 | 1.2.2 | 1.2.2 | | HCatalog | 2.1.0-amzn-0 | 2.1.0-amzn-0 | 2.1.0-amzn-0 | 2.1.0-amzn-0 | | Hadoop | 2.7.3-amzn-0 | 2.7.2-amzn-3 | 2.7.2-amzn-3 | 2.7.2-amzn-3 | | Hive | 2.1.0-amzn-0 | 2.1.0-amzn-0 | 2.1.0-amzn-0 | 2.1.0-amzn-0 | | Hudi | - | - | - | - | | Hue | 3.10.0-amzn-0 | 3.10.0-amzn-0 | 3.10.0-amzn-0 | 3.10.0-amzn-0 | | Iceberg | - | - | - | - | | JupyterEnterpriseGateway | - | - | - | - | | JupyterHub | - | - | - | - | | Livy | - | - | - | - | | MXNet | - | - | - | - | | Mahout | 0.12.2 | 0.12.2 | 0.12.2 | 0.12.2 | | Oozie | 4.2.0 | 4.2.0 | 4.2.0 | 4.2.0 | | Phoenix | 4.7.0-HBase-1.2 | 4.7.0-HBase-1.2 | 4.7.0-HBase-1.2 | 4.7.0-HBase-1.2 | | Pig | 0.16.0-amzn-0 | 0.16.0-amzn-0 | 0.16.0-amzn-0 | 0.16.0-amzn-0 | | Presto | 0.152.3 | 0.150 | 0.150 | 0.150 | | Spark | 2.0.1 | 2.0.0 | 2.0.0 | 2.0.0 | | Sqoop | 1.4.6 | 1.4.6 | 1.4.6 | 1.4.6 | | TensorFlow | - | - | - | - | | Tez | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | | Trino (PrestoSQL) | - | - | - | - | | Zeppelin | 0.6.1 | 0.6.1-SNAPSHOT | 0.6.1-SNAPSHOT | 0.6.1-SNAPSHOT | | ZooKeeper | 3.4.8 | 3.4.8 | 3.4.8 | 3.4.8 | ## 5.0.2 release notes ## 5.0.2 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | emr-ddb | 4.0.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 2.1.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.2.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-s3-dist-cp | 2.4.0 | Distributed copy application optimized for Amazon S3. | | emrfs | 2.9.0 | Amazon S3 connector for Hadoop ecosystem applications. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 2.7.2-amzn-3 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 2.7.2-amzn-3 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 2.7.2-amzn-3 | HDFS command-line client and library | | hadoop-hdfs-namenode | 2.7.2-amzn-3 | HDFS service for tracking file names and block locations. | | hadoop-httpfs-server | 2.7.2-amzn-3 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 2.7.2-amzn-3 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 2.7.2-amzn-3 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 2.7.2-amzn-3 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 2.7.2-amzn-3 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 2.7.2-amzn-3 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 1.2.2 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 1.2.2 | Service for serving one or more HBase regions. | | hbase-client | 1.2.2 | HBase command-line client. | | hbase-rest-server | 1.2.2 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 1.2.2 | Service providing a Thrift endpoint to HBase. | | hcatalog-client | 2.1.0-amzn-0 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 2.1.0-amzn-0 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 2.1.0-amzn-0 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 2.1.0-amzn-0 | Hive command line client. | | hive-metastore-server | 2.1.0-amzn-0 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server | 2.1.0-amzn-0 | Service for accepting Hive queries as web requests. | | hue-server | 3.10.0-amzn-0 | Web application for analyzing data using Hadoop ecosystem applications | | mahout-client | 0.12.2 | Library for machine learning. | | mysql-server | 5.5.46 | MySQL database server. | | oozie-client | 4.2.0 | Oozie command-line client. | | oozie-server | 4.2.0 | Service for accepting Oozie workflow requests. | | phoenix-library | 4.7.0-HBase-1.2 | The phoenix libraries for server and client | | phoenix-query-server | 4.7.0-HBase-1.2 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.150 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.150 | Service for executing pieces of a query. | | pig-client | 0.16.0-amzn-0 | Pig command-line client. | | spark-client | 2.0.0 | Spark command-line clients. | | spark-history-server | 2.0.0 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 2.0.0 | In-memory execution engine for YARN. | | spark-yarn-slave | 2.0.0 | Apache Spark libraries needed by YARN slaves. | | sqoop-client | 1.4.6 | Apache Sqoop command-line client. | | tez-on-yarn | 0.8.4 | The tez YARN application and libraries. | | webserver | 2.4.23 | Apache HTTP server. | | zeppelin-server | 0.6.1-SNAPSHOT | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.4.8 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.4.8 | ZooKeeper command line client. | ## 5.0.2 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). **emr-5.0.2 classifications** | Classifications | Description | | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | | core-site | Change values in Hadoop's core-site.xml file. | | emrfs-site | Change EMRFS settings. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | | hadoop-ssl-server | Change hadoop ssl server configuration | | hadoop-ssl-client | Change hadoop ssl client configuration | | hbase-env | Change values in HBase's environment. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbaase.properties file. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | | hbase-site | Change values in HBase's hbase-site.xml file. | | hdfs-encryption-zones | Configure HDFS encryption zones. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | | hcatalog-env | Change values in HCatalog's environment. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | | hcatalog-webhcat-log4j2 | Change values in HCatalog WebHCat's log4j2.properties. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | | hive-beeline-log4j2 | Change values in Hive's beeline-log4j2.properties file. | | hive-env | Change values in the Hive environment. | | hive-exec-log4j2 | Change values in Hive's hive-exec-log4j2.properties file. | | hive-llap-daemon-log4j2 | Change values in Hive's llap-daemon-log4j2.properties file. | | hive-log4j2 | Change values in Hive's hive-log4j2.properties file. | | hive-site | Change values in Hive's hive-site.xml file | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | | hue-ini | Change values in Hue's ini file | | httpfs-env | Change values in the HTTPFS environment. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | | mapred-env | Change values in the MapReduce application's environment. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | | oozie-env | Change values in Oozie's environment. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | | oozie-site | Change values in Oozie's oozie-site.xml file. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | | pig-properties | Change values in Pig's pig.properties file. | | pig-log4j | Change values in Pig's log4j.properties file. | | presto-log | Change values in Presto's log.properties file. | | presto-config | Change values in Presto's config.properties file. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | | presto-connector-hive | Change values in Presto's hive.properties file. | | presto-connector-jmx | Change values in Presto's jmx.properties file. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | | presto-connector-localfile | Change values in Presto's localfile.properties file. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | | presto-connector-redis | Change values in Presto's redis.properties file. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | | spark | Amazon EMR-curated settings for Apache Spark. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | | spark-env | Change values in the Spark environment. | | spark-hive-site | Change values in Spark's hive-site.xml file | | spark-log4j | Change values in Spark's log4j.properties file. | | spark-metrics | Change values in Spark's metrics.properties file. | | sqoop-env | Change values in Sqoop's environment. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | | tez-site | Change values in Tez's tez-site.xml file. | | yarn-env | Change values in the YARN environment. | | yarn-site | Change values in YARN's yarn-site.xml file. | | zeppelin-env | Change values in the Zeppelin environment. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | # Amazon EMR release 5.0.1 ## 5.0.1 application versions This release includes the following applications: [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [http://gethue.com/](http://gethue.com/), [http://mahout.apache.org/](http://mahout.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-5.0.3 | emr-5.0.2 | emr-5.0.1 | emr-5.0.0 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.10.75 | 1.10.75 | 1.10.75 | 1.10.75 | | Python | Not tracked | Not tracked | Not tracked | Not tracked | | Scala | 2.11.8 | 2.11.8 | 2.11.8 | 2.11.8 | | AmazonCloudWatchAgent | - | - | - | - | | Delta | - | - | - | - | | Flink | - | - | - | - | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 1.2.2 | 1.2.2 | 1.2.2 | 1.2.2 | | HCatalog | 2.1.0-amzn-0 | 2.1.0-amzn-0 | 2.1.0-amzn-0 | 2.1.0-amzn-0 | | Hadoop | 2.7.3-amzn-0 | 2.7.2-amzn-3 | 2.7.2-amzn-3 | 2.7.2-amzn-3 | | Hive | 2.1.0-amzn-0 | 2.1.0-amzn-0 | 2.1.0-amzn-0 | 2.1.0-amzn-0 | | Hudi | - | - | - | - | | Hue | 3.10.0-amzn-0 | 3.10.0-amzn-0 | 3.10.0-amzn-0 | 3.10.0-amzn-0 | | Iceberg | - | - | - | - | | JupyterEnterpriseGateway | - | - | - | - | | JupyterHub | - | - | - | - | | Livy | - | - | - | - | | MXNet | - | - | - | - | | Mahout | 0.12.2 | 0.12.2 | 0.12.2 | 0.12.2 | | Oozie | 4.2.0 | 4.2.0 | 4.2.0 | 4.2.0 | | Phoenix | 4.7.0-HBase-1.2 | 4.7.0-HBase-1.2 | 4.7.0-HBase-1.2 | 4.7.0-HBase-1.2 | | Pig | 0.16.0-amzn-0 | 0.16.0-amzn-0 | 0.16.0-amzn-0 | 0.16.0-amzn-0 | | Presto | 0.152.3 | 0.150 | 0.150 | 0.150 | | Spark | 2.0.1 | 2.0.0 | 2.0.0 | 2.0.0 | | Sqoop | 1.4.6 | 1.4.6 | 1.4.6 | 1.4.6 | | TensorFlow | - | - | - | - | | Tez | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | | Trino (PrestoSQL) | - | - | - | - | | Zeppelin | 0.6.1 | 0.6.1-SNAPSHOT | 0.6.1-SNAPSHOT | 0.6.1-SNAPSHOT | | ZooKeeper | 3.4.8 | 3.4.8 | 3.4.8 | 3.4.8 | ## 5.0.1 release notes ## 5.0.1 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | emr-ddb | 4.0.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 2.1.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.2.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-s3-dist-cp | 2.4.0 | Distributed copy application optimized for Amazon S3. | | emrfs | 2.9.0 | Amazon S3 connector for Hadoop ecosystem applications. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 2.7.2-amzn-3 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 2.7.2-amzn-3 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 2.7.2-amzn-3 | HDFS command-line client and library | | hadoop-hdfs-namenode | 2.7.2-amzn-3 | HDFS service for tracking file names and block locations. | | hadoop-httpfs-server | 2.7.2-amzn-3 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 2.7.2-amzn-3 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 2.7.2-amzn-3 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 2.7.2-amzn-3 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 2.7.2-amzn-3 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 2.7.2-amzn-3 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 1.2.2 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 1.2.2 | Service for serving one or more HBase regions. | | hbase-client | 1.2.2 | HBase command-line client. | | hbase-rest-server | 1.2.2 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 1.2.2 | Service providing a Thrift endpoint to HBase. | | hcatalog-client | 2.1.0-amzn-0 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 2.1.0-amzn-0 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 2.1.0-amzn-0 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 2.1.0-amzn-0 | Hive command line client. | | hive-metastore-server | 2.1.0-amzn-0 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server | 2.1.0-amzn-0 | Service for accepting Hive queries as web requests. | | hue-server | 3.10.0-amzn-0 | Web application for analyzing data using Hadoop ecosystem applications | | mahout-client | 0.12.2 | Library for machine learning. | | mysql-server | 5.5.46 | MySQL database server. | | oozie-client | 4.2.0 | Oozie command-line client. | | oozie-server | 4.2.0 | Service for accepting Oozie workflow requests. | | phoenix-library | 4.7.0-HBase-1.2 | The phoenix libraries for server and client | | phoenix-query-server | 4.7.0-HBase-1.2 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.150 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.150 | Service for executing pieces of a query. | | pig-client | 0.16.0-amzn-0 | Pig command-line client. | | spark-client | 2.0.0 | Spark command-line clients. | | spark-history-server | 2.0.0 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 2.0.0 | In-memory execution engine for YARN. | | spark-yarn-slave | 2.0.0 | Apache Spark libraries needed by YARN slaves. | | sqoop-client | 1.4.6 | Apache Sqoop command-line client. | | tez-on-yarn | 0.8.4 | The tez YARN application and libraries. | | webserver | 2.4.23 | Apache HTTP server. | | zeppelin-server | 0.6.1-SNAPSHOT | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.4.8 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.4.8 | ZooKeeper command line client. | ## 5.0.1 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). **emr-5.0.1 classifications** | Classifications | Description | | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | | core-site | Change values in Hadoop's core-site.xml file. | | emrfs-site | Change EMRFS settings. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | | hadoop-ssl-server | Change hadoop ssl server configuration | | hadoop-ssl-client | Change hadoop ssl client configuration | | hbase-env | Change values in HBase's environment. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbaase.properties file. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | | hbase-site | Change values in HBase's hbase-site.xml file. | | hdfs-encryption-zones | Configure HDFS encryption zones. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | | hcatalog-env | Change values in HCatalog's environment. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | | hcatalog-webhcat-log4j2 | Change values in HCatalog WebHCat's log4j2.properties. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | | hive-beeline-log4j2 | Change values in Hive's beeline-log4j2.properties file. | | hive-env | Change values in the Hive environment. | | hive-exec-log4j2 | Change values in Hive's hive-exec-log4j2.properties file. | | hive-llap-daemon-log4j2 | Change values in Hive's llap-daemon-log4j2.properties file. | | hive-log4j2 | Change values in Hive's hive-log4j2.properties file. | | hive-site | Change values in Hive's hive-site.xml file | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | | hue-ini | Change values in Hue's ini file | | httpfs-env | Change values in the HTTPFS environment. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | | mapred-env | Change values in the MapReduce application's environment. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | | oozie-env | Change values in Oozie's environment. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | | oozie-site | Change values in Oozie's oozie-site.xml file. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | | pig-properties | Change values in Pig's pig.properties file. | | pig-log4j | Change values in Pig's log4j.properties file. | | presto-log | Change values in Presto's log.properties file. | | presto-config | Change values in Presto's config.properties file. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | | presto-connector-hive | Change values in Presto's hive.properties file. | | presto-connector-jmx | Change values in Presto's jmx.properties file. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | | presto-connector-localfile | Change values in Presto's localfile.properties file. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | | presto-connector-redis | Change values in Presto's redis.properties file. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | | spark | Amazon EMR-curated settings for Apache Spark. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | | spark-env | Change values in the Spark environment. | | spark-hive-site | Change values in Spark's hive-site.xml file | | spark-log4j | Change values in Spark's log4j.properties file. | | spark-metrics | Change values in Spark's metrics.properties file. | | sqoop-env | Change values in Sqoop's environment. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | | tez-site | Change values in Tez's tez-site.xml file. | | yarn-env | Change values in the YARN environment. | | yarn-site | Change values in YARN's yarn-site.xml file. | | zeppelin-env | Change values in the Zeppelin environment. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | # Amazon EMR release 5.0.0 ## 5.0.0 application versions This release includes the following applications: [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [http://gethue.com/](http://gethue.com/), [http://mahout.apache.org/](http://mahout.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-5.0.3 | emr-5.0.2 | emr-5.0.1 | emr-5.0.0 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.10.75 | 1.10.75 | 1.10.75 | 1.10.75 | | Python | Not tracked | Not tracked | Not tracked | Not tracked | | Scala | 2.11.8 | 2.11.8 | 2.11.8 | 2.11.8 | | AmazonCloudWatchAgent | - | - | - | - | | Delta | - | - | - | - | | Flink | - | - | - | - | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 1.2.2 | 1.2.2 | 1.2.2 | 1.2.2 | | HCatalog | 2.1.0-amzn-0 | 2.1.0-amzn-0 | 2.1.0-amzn-0 | 2.1.0-amzn-0 | | Hadoop | 2.7.3-amzn-0 | 2.7.2-amzn-3 | 2.7.2-amzn-3 | 2.7.2-amzn-3 | | Hive | 2.1.0-amzn-0 | 2.1.0-amzn-0 | 2.1.0-amzn-0 | 2.1.0-amzn-0 | | Hudi | - | - | - | - | | Hue | 3.10.0-amzn-0 | 3.10.0-amzn-0 | 3.10.0-amzn-0 | 3.10.0-amzn-0 | | Iceberg | - | - | - | - | | JupyterEnterpriseGateway | - | - | - | - | | JupyterHub | - | - | - | - | | Livy | - | - | - | - | | MXNet | - | - | - | - | | Mahout | 0.12.2 | 0.12.2 | 0.12.2 | 0.12.2 | | Oozie | 4.2.0 | 4.2.0 | 4.2.0 | 4.2.0 | | Phoenix | 4.7.0-HBase-1.2 | 4.7.0-HBase-1.2 | 4.7.0-HBase-1.2 | 4.7.0-HBase-1.2 | | Pig | 0.16.0-amzn-0 | 0.16.0-amzn-0 | 0.16.0-amzn-0 | 0.16.0-amzn-0 | | Presto | 0.152.3 | 0.150 | 0.150 | 0.150 | | Spark | 2.0.1 | 2.0.0 | 2.0.0 | 2.0.0 | | Sqoop | 1.4.6 | 1.4.6 | 1.4.6 | 1.4.6 | | TensorFlow | - | - | - | - | | Tez | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | | Trino (PrestoSQL) | - | - | - | - | | Zeppelin | 0.6.1 | 0.6.1-SNAPSHOT | 0.6.1-SNAPSHOT | 0.6.1-SNAPSHOT | | ZooKeeper | 3.4.8 | 3.4.8 | 3.4.8 | 3.4.8 | ## 5.0.0 release notes Release date: July 27, 2016 **Upgrades** + Upgraded to Hive 2.1 + Upgraded to Presto 0.150 + Upgraded to Spark 2.0 + Upgraded to Hue 3.10.0 + Upgraded to Pig 0.16.0 + Upgraded to Tez 0.8.4 + Upgraded to Zeppelin 0.6.1 **Changes and enhancements** + Amazon EMR supports the latest open-source versions of Hive (version 2.1) and Pig (version 0.16.0). If you have used Hive or Pig on Amazon EMR in the past, this may affect some use cases. For more information, see [Hive](https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-hive.html) and [Pig](https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-pig.html). + The default execution engine for Hive and Pig is now Tez. To change this, you would edit the appropriate values in the `hive-site` and `pig-properties` configuration classifications, respectively. + An enhanced step debugging feature was added, which allows you to see the root cause of step failures if the service can determine the cause. For more information, see [ Enhanced Step Debugging](https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-enhanced-step-debugging.html) in the Amazon EMR Management Guide. + Applications that previously ended with "-Sandbox" no longer have that suffix. This may break your automation, for example, if you are using scripts to launch clusters with these applications. The following table shows application names in Amazon EMR 4.7.2 versus Amazon EMR 5.0.0. **Application name changes** [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-500-release.html) + Spark is now compiled for Scala 2.11. + Java 8 is now the default JVM. All applications run using the Java 8 runtime. There are no changes to any application’s byte code target. Most applications continue to target Java 7. + Zeppelin now includes authentication features. For more information, see [Zeppelin](https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-zeppelin.html). + Added support for security configurations, which allow you to create and apply encryption options more easily. For more information, see [Data Encryption](https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-data-encryption.html). ## 5.0.0 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | emr-ddb | 4.0.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 2.1.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.2.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-s3-dist-cp | 2.4.0 | Distributed copy application optimized for Amazon S3. | | emrfs | 2.9.0 | Amazon S3 connector for Hadoop ecosystem applications. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 2.7.2-amzn-3 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 2.7.2-amzn-3 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 2.7.2-amzn-3 | HDFS command-line client and library | | hadoop-hdfs-namenode | 2.7.2-amzn-3 | HDFS service for tracking file names and block locations. | | hadoop-httpfs-server | 2.7.2-amzn-3 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 2.7.2-amzn-3 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 2.7.2-amzn-3 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 2.7.2-amzn-3 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 2.7.2-amzn-3 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 2.7.2-amzn-3 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 1.2.2 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 1.2.2 | Service for serving one or more HBase regions. | | hbase-client | 1.2.2 | HBase command-line client. | | hbase-rest-server | 1.2.2 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 1.2.2 | Service providing a Thrift endpoint to HBase. | | hcatalog-client | 2.1.0-amzn-0 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 2.1.0-amzn-0 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 2.1.0-amzn-0 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 2.1.0-amzn-0 | Hive command line client. | | hive-metastore-server | 2.1.0-amzn-0 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server | 2.1.0-amzn-0 | Service for accepting Hive queries as web requests. | | hue-server | 3.10.0-amzn-0 | Web application for analyzing data using Hadoop ecosystem applications | | mahout-client | 0.12.2 | Library for machine learning. | | mysql-server | 5.5.46 | MySQL database server. | | oozie-client | 4.2.0 | Oozie command-line client. | | oozie-server | 4.2.0 | Service for accepting Oozie workflow requests. | | phoenix-library | 4.7.0-HBase-1.2 | The phoenix libraries for server and client | | phoenix-query-server | 4.7.0-HBase-1.2 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.150 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.150 | Service for executing pieces of a query. | | pig-client | 0.16.0-amzn-0 | Pig command-line client. | | spark-client | 2.0.0 | Spark command-line clients. | | spark-history-server | 2.0.0 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 2.0.0 | In-memory execution engine for YARN. | | spark-yarn-slave | 2.0.0 | Apache Spark libraries needed by YARN slaves. | | sqoop-client | 1.4.6 | Apache Sqoop command-line client. | | tez-on-yarn | 0.8.4 | The tez YARN application and libraries. | | webserver | 2.4.23 | Apache HTTP server. | | zeppelin-server | 0.6.1-SNAPSHOT | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.4.8 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.4.8 | ZooKeeper command line client. | ## 5.0.0 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). **emr-5.0.0 classifications** | Classifications | Description | | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | | core-site | Change values in Hadoop's core-site.xml file. | | emrfs-site | Change EMRFS settings. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | | hadoop-ssl-server | Change hadoop ssl server configuration | | hadoop-ssl-client | Change hadoop ssl client configuration | | hbase-env | Change values in HBase's environment. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbaase.properties file. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | | hbase-site | Change values in HBase's hbase-site.xml file. | | hdfs-encryption-zones | Configure HDFS encryption zones. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | | hcatalog-env | Change values in HCatalog's environment. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | | hcatalog-webhcat-log4j2 | Change values in HCatalog WebHCat's log4j2.properties. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | | hive-beeline-log4j2 | Change values in Hive's beeline-log4j2.properties file. | | hive-env | Change values in the Hive environment. | | hive-exec-log4j2 | Change values in Hive's hive-exec-log4j2.properties file. | | hive-llap-daemon-log4j2 | Change values in Hive's llap-daemon-log4j2.properties file. | | hive-log4j2 | Change values in Hive's hive-log4j2.properties file. | | hive-site | Change values in Hive's hive-site.xml file | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | | hue-ini | Change values in Hue's ini file | | httpfs-env | Change values in the HTTPFS environment. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | | mapred-env | Change values in the MapReduce application's environment. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | | oozie-env | Change values in Oozie's environment. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | | oozie-site | Change values in Oozie's oozie-site.xml file. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | | pig-properties | Change values in Pig's pig.properties file. | | pig-log4j | Change values in Pig's log4j.properties file. | | presto-log | Change values in Presto's log.properties file. | | presto-config | Change values in Presto's config.properties file. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | | presto-connector-hive | Change values in Presto's hive.properties file. | | presto-connector-jmx | Change values in Presto's jmx.properties file. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | | presto-connector-localfile | Change values in Presto's localfile.properties file. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | | presto-connector-redis | Change values in Presto's redis.properties file. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | | spark | Amazon EMR-curated settings for Apache Spark. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | | spark-env | Change values in the Spark environment. | | spark-hive-site | Change values in Spark's hive-site.xml file | | spark-log4j | Change values in Spark's log4j.properties file. | | spark-metrics | Change values in Spark's metrics.properties file. | | sqoop-env | Change values in Sqoop's environment. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | | tez-site | Change values in Tez's tez-site.xml file. | | yarn-env | Change values in the YARN environment. | | yarn-site | Change values in YARN's yarn-site.xml file. | | zeppelin-env | Change values in the Zeppelin environment. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | # Amazon EMR 4.x release versions This section contains application versions, release notes, component versions, and configuration classifications available in each Amazon EMR 4.x release version. When you launch a cluster, you can choose from multiple releases of Amazon EMR. This allows you to test and use application versions that fit your compatibility requirements. You specify the release number with the *release label*. Release labels are in the form `emr-x.x.x`. For example, `emr-7.12.0`. New Amazon EMR releases are made available in different Regions over a period of several days, beginning with the first Region on the initial release date. The latest release version may not be available in your Region during this period. For a comprehensive table of application versions in every Amazon EMR 4.x release, see [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md). **Topics** + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) + [Differences in Amazon EMR 4.x release versions](emr-release-differences-4x.md) + [Amazon EMR release 4.9.6](emr-496-release.md) + [Amazon EMR release 4.9.5](emr-495-release.md) + [Amazon EMR release 4.9.4](emr-494-release.md) + [Amazon EMR release 4.9.3](emr-493-release.md) + [Amazon EMR release 4.9.2](emr-492-release.md) + [Amazon EMR release 4.9.1](emr-491-release.md) + [Amazon EMR release 4.8.5](emr-485-release.md) + [Amazon EMR release 4.8.4](emr-484-release.md) + [Amazon EMR release 4.8.3](emr-483-release.md) + [Amazon EMR release 4.8.2](emr-482-release.md) + [Amazon EMR release 4.8.1](emr-481-release.md) + [Amazon EMR release 4.8.0](emr-480-release.md) + [Amazon EMR release 4.7.4](emr-474-release.md) + [Amazon EMR release 4.7.3](emr-473-release.md) + [Amazon EMR release 4.7.2](emr-472-release.md) + [Amazon EMR release 4.7.1](emr-471-release.md) + [Amazon EMR release 4.7.0](emr-470-release.md) + [Amazon EMR release 4.6.1](emr-461-release.md) + [Amazon EMR release 4.6.0](emr-460-release.md) + [Amazon EMR release 4.5.0](emr-450-release.md) + [Amazon EMR release 4.4.0](emr-440-release.md) + [Amazon EMR release 4.3.0](emr-430-release.md) + [Amazon EMR release 4.2.0](emr-420-release.md) + [Amazon EMR release 4.1.0](emr-410-release.md) + [Amazon EMR release 4.0.0](emr-400-release.md) # Application versions in Amazon EMR 4.x releases The following table lists the application versions that are available in each Amazon EMR 4.x release. **Application version information** | | emr-4.9.6 | emr-4.9.5 | emr-4.9.4 | emr-4.9.3 | emr-4.9.2 | emr-4.9.1 | emr-4.8.5 | emr-4.8.4 | emr-4.8.3 | emr-4.8.2 | emr-4.8.1 | emr-4.8.0 | emr-4.7.4 | emr-4.7.3 | emr-4.7.2 | emr-4.7.1 | emr-4.7.0 | emr-4.6.1 | emr-4.6.0 | emr-4.5.0 | emr-4.4.0 | emr-4.3.0 | emr-4.2.0 | emr-4.1.0 | emr-4.0.0 | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.10.75 | 1.10.75 | 1.10.75 | 1.10.75 | 1.10.75 | 1.10.75 | 1.10.75 | 1.10.75 | 1.10.75 | 1.10.75 | 1.10.75 | 1.10.75 | 1.10.75 | 1.10.75 | 1.10.75 | 1.10.75 | 1.10.75 | 1.10.27 | 1.10.27 | 1.10.27 | 1.10.27 | 1.10.27 | 1.10.27 | Not tracked | Not tracked | | Python | Not tracked | Not tracked | Not tracked | Not tracked | Not tracked | Not tracked | Not tracked | Not tracked | Not tracked | Not tracked | Not tracked | Not tracked | Not tracked | Not tracked | Not tracked | Not tracked | Not tracked | Not tracked | Not tracked | Not tracked | Not tracked | Not tracked | Not tracked | Not tracked | Not tracked | | Scala | Not tracked | Not tracked | Not tracked | Not tracked | Not tracked | Not tracked | Not tracked | Not tracked | Not tracked | Not tracked | Not tracked | Not tracked | Not tracked | Not tracked | Not tracked | Not tracked | Not tracked | Not tracked | Not tracked | Not tracked | Not tracked | Not tracked | Not tracked | Not tracked | Not tracked | | AmazonCloudWatchAgent | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | Delta | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | Flink | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | 3.6.0 | - | - | | HBase | 1.2.2 | 1.2.2 | 1.2.2 | 1.2.2 | 1.2.2 | 1.2.2 | 1.2.2 | 1.2.2 | 1.2.2 | 1.2.2 | 1.2.2 | 1.2.2 | 1.2.1 | 1.2.1 | 1.2.1 | 1.2.1 | 1.2.1 | 1.2.0 | 1.2.0 | - | - | - | - | - | - | | HCatalog | 1.0.0-amzn-9 | 1.0.0-amzn-9 | 1.0.0-amzn-9 | 1.0.0-amzn-9 | 1.0.0-amzn-9 | 1.0.0-amzn-9 | 1.0.0-amzn-8 | 1.0.0-amzn-8 | 1.0.0-amzn-8 | 1.0.0-amzn-7 | 1.0.0-amzn-7 | 1.0.0-amzn-7 | 1.0.0-amzn-6 | 1.0.0-amzn-6 | 1.0.0-amzn-6 | 1.0.0-amzn-5 | 1.0.0-amzn-5 | 1.0.0-amzn-4 | 1.0.0-amzn-4 | 1.0.0-amzn-4 | 1.0.0-amzn-3 | - | - | - | - | | Hadoop | 2.7.3-amzn-2 | 2.7.3-amzn-2 | 2.7.3-amzn-2 | 2.7.3-amzn-2 | 2.7.3-amzn-2 | 2.7.3-amzn-2 | 2.7.3-amzn-1 | 2.7.3-amzn-1 | 2.7.3-amzn-1 | 2.7.3-amzn-0 | 2.7.2-amzn-4 | 2.7.2-amzn-4 | 2.7.2-amzn-3 | 2.7.2-amzn-3 | 2.7.2-amzn-3 | 2.7.2-amzn-2 | 2.7.2-amzn-2 | 2.7.2-amzn-1 | 2.7.2-amzn-1 | 2.7.2-amzn-0 | 2.7.1-amzn-1 | 2.7.1-amzn-0 | 2.6.0-amzn-2 | 2.6.0-amzn-1 | 2.6.0-amzn-0 | | Hive | 1.0.0-amzn-9 | 1.0.0-amzn-9 | 1.0.0-amzn-9 | 1.0.0-amzn-9 | 1.0.0-amzn-9 | 1.0.0-amzn-9 | 1.0.0-amzn-8 | 1.0.0-amzn-8 | 1.0.0-amzn-8 | 1.0.0-amzn-7 | 1.0.0-amzn-7 | 1.0.0-amzn-7 | 1.0.0-amzn-6 | 1.0.0-amzn-6 | 1.0.0-amzn-6 | 1.0.0-amzn-5 | 1.0.0-amzn-5 | 1.0.0-amzn-4 | 1.0.0-amzn-4 | 1.0.0-amzn-4 | 1.0.0-amzn-3 | 1.0.0-amzn-2 | 1.0.0-amzn-1 | 1.0.0-amzn-1 | 1.0.0-amzn-0 | | Hudi | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | Hue | 3.7.1-amzn-7 | 3.7.1-amzn-7 | 3.7.1-amzn-7 | 3.7.1-amzn-7 | 3.7.1-amzn-7 | 3.7.1-amzn-7 | 3.7.1-amzn-7 | 3.7.1-amzn-7 | 3.7.1-amzn-7 | 3.7.1-amzn-7 | 3.7.1-amzn-7 | 3.7.1-amzn-7 | 3.7.1-amzn-7 | 3.7.1-amzn-7 | 3.7.1-amzn-7 | 3.7.1-amzn-7 | 3.7.1-amzn-7 | 3.7.1-amzn-6 | 3.7.1-amzn-6 | 3.7.1-amzn-5 | 3.7.1-amzn-5 | 3.7.1-amzn-5 | 3.7.1-amzn-5 | 3.7.1-amzn-4 | - | | Iceberg | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | JupyterEnterpriseGateway | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | JupyterHub | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | Livy | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | MXNet | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | Mahout | 0.12.2 | 0.12.2 | 0.12.2 | 0.12.2 | 0.12.2 | 0.12.2 | 0.12.2 | 0.12.2 | 0.12.2 | 0.12.2 | 0.12.2 | 0.12.2 | 0.12.2 | 0.12.2 | 0.12.2 | 0.12.0 | 0.12.0 | 0.11.1 | 0.11.1 | 0.11.1 | 0.11.1 | 0.11.0 | 0.11.0 | 0.11.0 | 0.10.0 | | Oozie | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | Oozie-Sandbox | 4.2.0 | 4.2.0 | 4.2.0 | 4.2.0 | 4.2.0 | 4.2.0 | 4.2.0 | 4.2.0 | 4.2.0 | 4.2.0 | 4.2.0 | 4.2.0 | 4.2.0 | 4.2.0 | 4.2.0 | 4.2.0 | 4.2.0 | 4.2.0 | 4.2.0 | 4.2.0 | 4.2.0 | 4.2.0 | 4.2.0 | 4.0.1 | - | | Phoenix | 4.7.0-HBase-1.2 | 4.7.0-HBase-1.2 | 4.7.0-HBase-1.2 | 4.7.0-HBase-1.2 | 4.7.0-HBase-1.2 | 4.7.0-HBase-1.2 | 4.7.0-HBase-1.2 | 4.7.0-HBase-1.2 | 4.7.0-HBase-1.2 | 4.7.0-HBase-1.2 | 4.7.0-HBase-1.2 | 4.7.0-HBase-1.2 | 4.7.0-HBase-1.2 | 4.7.0-HBase-1.2 | 4.7.0-HBase-1.2 | 4.7.0-HBase-1.2 | 4.7.0-HBase-1.2 | - | - | - | - | - | - | - | - | | Pig | 0.14.0-amzn-0 | 0.14.0-amzn-0 | 0.14.0-amzn-0 | 0.14.0-amzn-0 | 0.14.0-amzn-0 | 0.14.0-amzn-0 | 0.14.0-amzn-0 | 0.14.0-amzn-0 | 0.14.0-amzn-0 | 0.14.0-amzn-0 | 0.14.0-amzn-0 | 0.14.0-amzn-0 | 0.14.0-amzn-0 | 0.14.0-amzn-0 | 0.14.0-amzn-0 | 0.14.0-amzn-0 | 0.14.0-amzn-0 | 0.14.0-amzn-0 | 0.14.0-amzn-0 | 0.14.0-amzn-0 | 0.14.0-amzn-0 | 0.14.0-amzn-0 | 0.14.0-amzn-0 | 0.14.0-amzn-0 | 0.14.0-amzn-0 | | Presto | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | Presto-Sandbox | 0.157.1 | 0.157.1 | 0.157.1 | 0.157.1 | 0.157.1 | 0.157.1 | 0.157.1 | 0.157.1 | 0.157.1 | 0.152.3 | 0.151 | 0.151 | 0.148 | 0.148 | 0.148 | 0.147 | 0.147 | 0.143 | 0.143 | 0.140 | 0.136 | 0.130 | 0.125 | 0.119 | - | | Spark | 1.6.3 | 1.6.3 | 1.6.3 | 1.6.3 | 1.6.3 | 1.6.3 | 1.6.3 | 1.6.3 | 1.6.3 | 1.6.2 | 1.6.2 | 1.6.2 | 1.6.2 | 1.6.2 | 1.6.2 | 1.6.1 | 1.6.1 | 1.6.1 | 1.6.1 | 1.6.1 | 1.6.0 | 1.6.0 | 1.5.2 | 1.5.0 | 1.4.1 | | Sqoop | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | Sqoop-Sandbox | 1.4.6 | 1.4.6 | 1.4.6 | 1.4.6 | 1.4.6 | 1.4.6 | 1.4.6 | 1.4.6 | 1.4.6 | 1.4.6 | 1.4.6 | 1.4.6 | 1.4.6 | 1.4.6 | 1.4.6 | 1.4.6 | 1.4.6 | 1.4.6 | 1.4.6 | 1.4.6 | 1.4.6 | - | - | - | - | | TensorFlow | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | Tez | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.3 | 0.8.3 | 0.8.3 | 0.8.3 | 0.8.3 | - | - | - | - | - | - | - | - | | Trino (PrestoSQL) | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | Zeppelin | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | Zeppelin-Sandbox | 0.6.1 | 0.6.1 | 0.6.1 | 0.6.1 | 0.6.1 | 0.6.1 | 0.6.1 | 0.6.1 | 0.6.1 | 0.6.1 | 0.6.1 | 0.6.1 | 0.5.6-incubating | 0.5.6-incubating | 0.5.6-incubating | 0.5.6-incubating | 0.5.6-incubating | 0.5.6-incubating | 0.5.6-incubating | 0.5.6-incubating | 0.5.6-incubating | 0.5.5-incubating-amzn-1 | 0.5.5-incubating-amzn-0 | 0.6.0-incubating-SNAPSHOT | - | | ZooKeeper | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | ZooKeeper-Sandbox | 3.4.9 | 3.4.9 | 3.4.9 | 3.4.9 | 3.4.9 | 3.4.9 | 3.4.9 | 3.4.9 | 3.4.9 | 3.4.8 | 3.4.8 | 3.4.8 | 3.4.8 | 3.4.8 | 3.4.8 | 3.4.8 | 3.4.8 | 3.4.8 | 3.4.8 | - | - | - | - | - | - | # Differences in Amazon EMR 4.x release versions Documentation for Amazon EMR features in the *Amazon EMR Management Guide* specify the Amazon EMR release version that a feature became available, as well as applicable differences between Amazon EMR features dating back to 4.0.0. Beginning with Amazon EMR release version 5.0.0, some applications got a significant version upgrade that changed installation or operational details, and others were promoted from sandbox applications to native applications. Each topic in this section provides notable application-specific differences when using Amazon EMR 4.x release versions. **Topics** + [Sandbox applications](emr-sandbox-apps-4x.md) + [Considerations for using Hive on Amazon EMR 4.x](emr-Hive-4x.md) + [Considerations for using Pig on Amazon EMR 4.x](emr-Pig-4x.md) # Sandbox applications When using Amazon EMR 4.x release versions, some applications are considered *sandbox* applications. Sandbox applications are early versions of the application that we made available at the time of the initial Amazon EMR release because of demand. You can use the console, AWS CLI, or API to have Amazon EMR install sandbox applications in the same way as native applications, but sandbox applications have limited support and documentation. Sandbox applications became native, fully supported applications in Amazon EMR release version 5.0.0 and later. The following are sandbox applications in Amazon EMR 4.x release versions: + Oozie + Presto + Sqoop + Zeppelin + ZooKeeper When you install sandbox applications, the application names are denoted with the suffix `-sandbox`. For example, to install the sandbox version of *Presto*, use `Presto-sandbox`. Installation may take longer than it does for a fully supported application. The version numbers listed for each application in this section correspond to the community version of the application. ## Oozie (sandbox versions) Oozie is available as a sandbox application beginning with Amazon EMR release version 4.1.0. Oozie examples are not installed by default using the sandbox versions. To install the examples, SSH to the master node and run `install-oozie-examples`. **Oozie-Sandbox version information** | Amazon EMR Release Label | Oozie-Sandbox Version | Components Installed With Oozie-Sandbox | | --- | --- | --- | | emr-4.9.6 | 4.2.0 | emrfs, emr-ddb, emr-goodies, emr-kinesis, emr-s3-dist-cp, hadoop-client, hadoop-mapred, hadoop-hdfs-datanode, hadoop-hdfs-library, hadoop-hdfs-namenode, hadoop-httpfs-server, hadoop-kms-server, hadoop-yarn-nodemanager, hadoop-yarn-resourcemanager, oozie-client, oozie-server | | emr-4.9.5 | 4.2.0 | emrfs, emr-ddb, emr-goodies, emr-kinesis, emr-s3-dist-cp, hadoop-client, hadoop-mapred, hadoop-hdfs-datanode, hadoop-hdfs-library, hadoop-hdfs-namenode, hadoop-httpfs-server, hadoop-kms-server, hadoop-yarn-nodemanager, hadoop-yarn-resourcemanager, oozie-client, oozie-server | | emr-4.9.4 | 4.2.0 | emrfs, emr-ddb, emr-goodies, emr-kinesis, emr-s3-dist-cp, hadoop-client, hadoop-mapred, hadoop-hdfs-datanode, hadoop-hdfs-library, hadoop-hdfs-namenode, hadoop-httpfs-server, hadoop-kms-server, hadoop-yarn-nodemanager, hadoop-yarn-resourcemanager, oozie-client, oozie-server | | emr-4.9.3 | 4.2.0 | emrfs, emr-ddb, emr-goodies, emr-kinesis, emr-s3-dist-cp, hadoop-client, hadoop-mapred, hadoop-hdfs-datanode, hadoop-hdfs-library, hadoop-hdfs-namenode, hadoop-httpfs-server, hadoop-kms-server, hadoop-yarn-nodemanager, hadoop-yarn-resourcemanager, oozie-client, oozie-server | | emr-4.9.2 | 4.2.0 | emrfs, emr-ddb, emr-goodies, emr-kinesis, emr-s3-dist-cp, hadoop-client, hadoop-mapred, hadoop-hdfs-datanode, hadoop-hdfs-library, hadoop-hdfs-namenode, hadoop-httpfs-server, hadoop-kms-server, hadoop-yarn-nodemanager, hadoop-yarn-resourcemanager, oozie-client, oozie-server | | emr-4.9.1 | 4.2.0 | emrfs, emr-ddb, emr-goodies, emr-kinesis, emr-s3-dist-cp, hadoop-client, hadoop-mapred, hadoop-hdfs-datanode, hadoop-hdfs-library, hadoop-hdfs-namenode, hadoop-httpfs-server, hadoop-kms-server, hadoop-yarn-nodemanager, hadoop-yarn-resourcemanager, oozie-client, oozie-server | | emr-4.8.5 | 4.2.0 | emrfs, emr-ddb, emr-goodies, emr-kinesis, emr-s3-dist-cp, hadoop-client, hadoop-mapred, hadoop-hdfs-datanode, hadoop-hdfs-library, hadoop-hdfs-namenode, hadoop-httpfs-server, hadoop-kms-server, hadoop-yarn-nodemanager, hadoop-yarn-resourcemanager, oozie-client, oozie-server | | emr-4.8.4 | 4.2.0 | emrfs, emr-ddb, emr-goodies, emr-kinesis, emr-s3-dist-cp, hadoop-client, hadoop-mapred, hadoop-hdfs-datanode, hadoop-hdfs-library, hadoop-hdfs-namenode, hadoop-httpfs-server, hadoop-kms-server, hadoop-yarn-nodemanager, hadoop-yarn-resourcemanager, oozie-client, oozie-server | | emr-4.8.3 | 4.2.0 | emrfs, emr-ddb, emr-goodies, emr-kinesis, emr-s3-dist-cp, hadoop-client, hadoop-mapred, hadoop-hdfs-datanode, hadoop-hdfs-library, hadoop-hdfs-namenode, hadoop-httpfs-server, hadoop-kms-server, hadoop-yarn-nodemanager, hadoop-yarn-resourcemanager, oozie-client, oozie-server | | emr-4.8.2 | 4.2.0 | emrfs, emr-ddb, emr-goodies, emr-kinesis, emr-s3-dist-cp, hadoop-client, hadoop-mapred, hadoop-hdfs-datanode, hadoop-hdfs-library, hadoop-hdfs-namenode, hadoop-httpfs-server, hadoop-kms-server, hadoop-yarn-nodemanager, hadoop-yarn-resourcemanager, oozie-client, oozie-server | | emr-4.8.1 | 4.2.0 | emrfs, emr-ddb, emr-goodies, emr-kinesis, emr-s3-dist-cp, hadoop-client, hadoop-mapred, hadoop-hdfs-datanode, hadoop-hdfs-library, hadoop-hdfs-namenode, hadoop-httpfs-server, hadoop-kms-server, hadoop-yarn-nodemanager, hadoop-yarn-resourcemanager, oozie-client, oozie-server | | emr-4.8.0 | 4.2.0 | emrfs, emr-ddb, emr-goodies, emr-kinesis, emr-s3-dist-cp, hadoop-client, hadoop-mapred, hadoop-hdfs-datanode, hadoop-hdfs-library, hadoop-hdfs-namenode, hadoop-httpfs-server, hadoop-kms-server, hadoop-yarn-nodemanager, hadoop-yarn-resourcemanager, oozie-client, oozie-server | | emr-4.7.4 | 4.2.0 | emrfs, emr-ddb, emr-goodies, emr-kinesis, emr-s3-dist-cp, hadoop-client, hadoop-mapred, hadoop-hdfs-datanode, hadoop-hdfs-library, hadoop-hdfs-namenode, hadoop-httpfs-server, hadoop-kms-server, hadoop-yarn-nodemanager, hadoop-yarn-resourcemanager, oozie-client, oozie-server | | emr-4.7.3 | 4.2.0 | emrfs, emr-ddb, emr-goodies, emr-kinesis, emr-s3-dist-cp, hadoop-client, hadoop-mapred, hadoop-hdfs-datanode, hadoop-hdfs-library, hadoop-hdfs-namenode, hadoop-httpfs-server, hadoop-kms-server, hadoop-yarn-nodemanager, hadoop-yarn-resourcemanager, oozie-client, oozie-server | | emr-4.7.2 | 4.2.0 | emrfs, emr-ddb, emr-goodies, emr-kinesis, emr-s3-dist-cp, hadoop-client, hadoop-mapred, hadoop-hdfs-datanode, hadoop-hdfs-library, hadoop-hdfs-namenode, hadoop-httpfs-server, hadoop-kms-server, hadoop-yarn-nodemanager, hadoop-yarn-resourcemanager, oozie-client, oozie-server | | emr-4.7.1 | 4.2.0 | emrfs, emr-ddb, emr-goodies, emr-kinesis, emr-s3-dist-cp, hadoop-client, hadoop-mapred, hadoop-hdfs-datanode, hadoop-hdfs-library, hadoop-hdfs-namenode, hadoop-httpfs-server, hadoop-kms-server, hadoop-yarn-nodemanager, hadoop-yarn-resourcemanager, oozie-client, oozie-server | | emr-4.7.0 | 4.2.0 | emrfs, emr-ddb, emr-goodies, emr-kinesis, emr-s3-dist-cp, hadoop-client, hadoop-mapred, hadoop-hdfs-datanode, hadoop-hdfs-library, hadoop-hdfs-namenode, hadoop-httpfs-server, hadoop-kms-server, hadoop-yarn-nodemanager, hadoop-yarn-resourcemanager, oozie-client, oozie-server | | emr-4.6.1 | 4.2.0 | emrfs, emr-ddb, emr-goodies, emr-kinesis, emr-s3-dist-cp, hadoop-client, hadoop-mapred, hadoop-hdfs-datanode, hadoop-hdfs-library, hadoop-hdfs-namenode, hadoop-httpfs-server, hadoop-yarn-nodemanager, hadoop-yarn-resourcemanager, oozie-client, oozie-server | | emr-4.6.0 | 4.2.0 | emrfs, emr-ddb, emr-goodies, emr-kinesis, emr-s3-dist-cp, hadoop-client, hadoop-mapred, hadoop-hdfs-datanode, hadoop-hdfs-library, hadoop-hdfs-namenode, hadoop-httpfs-server, hadoop-yarn-nodemanager, hadoop-yarn-resourcemanager, oozie-client, oozie-server | | emr-4.5.0 | 4.2.0 | emrfs, emr-ddb, emr-goodies, emr-kinesis, emr-s3-dist-cp, hadoop-client, hadoop-mapred, hadoop-hdfs-datanode, hadoop-hdfs-library, hadoop-hdfs-namenode, hadoop-httpfs-server, hadoop-yarn-nodemanager, hadoop-yarn-resourcemanager, oozie-client, oozie-server | | emr-4.4.0 | 4.2.0 | emrfs, emr-ddb, emr-goodies, emr-kinesis, emr-s3-dist-cp, hadoop-client, hadoop-mapred, hadoop-hdfs-datanode, hadoop-hdfs-library, hadoop-hdfs-namenode, hadoop-httpfs-server, hadoop-yarn-nodemanager, hadoop-yarn-resourcemanager, oozie-client, oozie-server | | emr-4.3.0 | 4.2.0 | emrfs, emr-ddb, emr-goodies, emr-kinesis, emr-s3-dist-cp, hadoop-client, hadoop-mapred, hadoop-hdfs-datanode, hadoop-hdfs-library, hadoop-hdfs-namenode, hadoop-httpfs-server, hadoop-yarn-nodemanager, hadoop-yarn-resourcemanager, oozie-client, oozie-server | | emr-4.2.0 | 4.2.0 | emrfs, emr-ddb, emr-goodies, emr-kinesis, emr-s3-dist-cp, hadoop-client, hadoop-mapred, hadoop-hdfs-datanode, hadoop-hdfs-library, hadoop-hdfs-namenode, hadoop-httpfs-server, hadoop-yarn-nodemanager, hadoop-yarn-resourcemanager, oozie-client, oozie-server | | emr-4.1.0 | 4.0.1 | emrfs, emr-ddb, emr-goodies, emr-kinesis, emr-s3-dist-cp, hadoop-client, hadoop-mapred, hadoop-hdfs-datanode, hadoop-hdfs-library, hadoop-hdfs-namenode, hadoop-httpfs-server, hadoop-yarn-nodemanager, hadoop-yarn-resourcemanager, oozie-client, oozie-server | ## Presto (sandbox versions) Presto is available as a sandbox application beginning with Amazon EMR release version 4.1.0. **Presto-Sandbox version information** | Amazon EMR Release Label | Presto-Sandbox Version | Components Installed With Presto-Sandbox | | --- | --- | --- | | emr-4.9.6 | 0.157.1 | emrfs, emr-goodies, hadoop-client, hadoop-hdfs-datanode, hadoop-hdfs-library, hadoop-hdfs-namenode, hadoop-kms-server, hadoop-yarn-nodemanager, hadoop-yarn-resourcemanager, hive-client, hcatalog-server, mysql-server, presto-coordinator, presto-worker | | emr-4.9.5 | 0.157.1 | emrfs, emr-goodies, hadoop-client, hadoop-hdfs-datanode, hadoop-hdfs-library, hadoop-hdfs-namenode, hadoop-kms-server, hadoop-yarn-nodemanager, hadoop-yarn-resourcemanager, hive-client, hcatalog-server, mysql-server, presto-coordinator, presto-worker | | emr-4.9.4 | 0.157.1 | emrfs, emr-goodies, hadoop-client, hadoop-hdfs-datanode, hadoop-hdfs-library, hadoop-hdfs-namenode, hadoop-kms-server, hadoop-yarn-nodemanager, hadoop-yarn-resourcemanager, hive-client, hcatalog-server, mysql-server, presto-coordinator, presto-worker | | emr-4.9.3 | 0.157.1 | emrfs, emr-goodies, hadoop-client, hadoop-hdfs-datanode, hadoop-hdfs-library, hadoop-hdfs-namenode, hadoop-kms-server, hadoop-yarn-nodemanager, hadoop-yarn-resourcemanager, hive-client, hcatalog-server, mysql-server, presto-coordinator, presto-worker | | emr-4.9.2 | 0.157.1 | emrfs, emr-goodies, hadoop-client, hadoop-hdfs-datanode, hadoop-hdfs-library, hadoop-hdfs-namenode, hadoop-kms-server, hadoop-yarn-nodemanager, hadoop-yarn-resourcemanager, hive-client, hcatalog-server, mysql-server, presto-coordinator, presto-worker | | emr-4.9.1 | 0.157.1 | emrfs, emr-goodies, hadoop-client, hadoop-hdfs-datanode, hadoop-hdfs-library, hadoop-hdfs-namenode, hadoop-kms-server, hadoop-yarn-nodemanager, hadoop-yarn-resourcemanager, hive-client, hcatalog-server, mysql-server, presto-coordinator, presto-worker | | emr-4.8.5 | 0.157.1 | emrfs, emr-goodies, hadoop-client, hadoop-hdfs-datanode, hadoop-hdfs-library, hadoop-hdfs-namenode, hadoop-kms-server, hadoop-yarn-nodemanager, hadoop-yarn-resourcemanager, hive-client, hcatalog-server, mysql-server, presto-coordinator, presto-worker | | emr-4.8.4 | 0.157.1 | emrfs, emr-goodies, hadoop-client, hadoop-hdfs-datanode, hadoop-hdfs-library, hadoop-hdfs-namenode, hadoop-kms-server, hadoop-yarn-nodemanager, hadoop-yarn-resourcemanager, hive-client, hcatalog-server, mysql-server, presto-coordinator, presto-worker | | emr-4.8.3 | 0.157.1 | emrfs, emr-goodies, hadoop-client, hadoop-hdfs-datanode, hadoop-hdfs-library, hadoop-hdfs-namenode, hadoop-kms-server, hadoop-yarn-nodemanager, hadoop-yarn-resourcemanager, hive-client, hcatalog-server, mysql-server, presto-coordinator, presto-worker | | emr-4.8.2 | 0.152.3 | emrfs, emr-goodies, hadoop-client, hadoop-hdfs-datanode, hadoop-hdfs-library, hadoop-hdfs-namenode, hadoop-kms-server, hadoop-yarn-nodemanager, hadoop-yarn-resourcemanager, hive-client, hcatalog-server, mysql-server, presto-coordinator, presto-worker | | emr-4.8.1 | 0.151 | emrfs, emr-goodies, hadoop-client, hadoop-hdfs-datanode, hadoop-hdfs-library, hadoop-hdfs-namenode, hadoop-kms-server, hadoop-yarn-nodemanager, hadoop-yarn-resourcemanager, hive-client, hcatalog-server, mysql-server, presto-coordinator, presto-worker | | emr-4.8.0 | 0.151 | emrfs, emr-goodies, hadoop-client, hadoop-hdfs-datanode, hadoop-hdfs-library, hadoop-hdfs-namenode, hadoop-kms-server, hadoop-yarn-nodemanager, hadoop-yarn-resourcemanager, hive-client, hcatalog-server, mysql-server, presto-coordinator, presto-worker | | emr-4.7.4 | 0.148 | emrfs, emr-goodies, hadoop-client, hadoop-hdfs-datanode, hadoop-hdfs-library, hadoop-hdfs-namenode, hadoop-kms-server, hadoop-yarn-nodemanager, hadoop-yarn-resourcemanager, hive-client, hcatalog-server, mysql-server, presto-coordinator, presto-worker | | emr-4.7.3 | 0.148 | emrfs, emr-goodies, hadoop-client, hadoop-hdfs-datanode, hadoop-hdfs-library, hadoop-hdfs-namenode, hadoop-kms-server, hadoop-yarn-nodemanager, hadoop-yarn-resourcemanager, hive-client, hcatalog-server, mysql-server, presto-coordinator, presto-worker | | emr-4.7.2 | 0.148 | emrfs, emr-goodies, hadoop-client, hadoop-hdfs-datanode, hadoop-hdfs-library, hadoop-hdfs-namenode, hadoop-kms-server, hadoop-yarn-nodemanager, hadoop-yarn-resourcemanager, hive-client, hcatalog-server, mysql-server, presto-coordinator, presto-worker | | emr-4.7.1 | 0.147 | emrfs, emr-goodies, hadoop-client, hadoop-hdfs-datanode, hadoop-hdfs-library, hadoop-hdfs-namenode, hadoop-kms-server, hadoop-yarn-nodemanager, hadoop-yarn-resourcemanager, hive-client, hcatalog-server, mysql-server, presto-coordinator, presto-worker | | emr-4.7.0 | 0.147 | emrfs, emr-goodies, hadoop-client, hadoop-hdfs-datanode, hadoop-hdfs-library, hadoop-hdfs-namenode, hadoop-kms-server, hadoop-yarn-nodemanager, hadoop-yarn-resourcemanager, hive-client, hcatalog-server, mysql-server, presto-coordinator, presto-worker | | emr-4.6.1 | 0.143 | emrfs, emr-goodies, hadoop-client, hadoop-hdfs-datanode, hadoop-hdfs-library, hadoop-hdfs-namenode, hadoop-yarn-nodemanager, hadoop-yarn-resourcemanager, hive-client, hive-metastore-server, mysql-server, presto-coordinator, presto-worker | | emr-4.6.0 | 0.143 | emrfs, emr-goodies, hadoop-client, hadoop-hdfs-datanode, hadoop-hdfs-library, hadoop-hdfs-namenode, hadoop-yarn-nodemanager, hadoop-yarn-resourcemanager, hive-client, hive-metastore-server, mysql-server, presto-coordinator, presto-worker | | emr-4.5.0 | 0.140 | emrfs, emr-goodies, hadoop-client, hadoop-hdfs-datanode, hadoop-hdfs-library, hadoop-hdfs-namenode, hadoop-yarn-nodemanager, hadoop-yarn-resourcemanager, hive-client, hive-metastore-server, mysql-server, presto-coordinator, presto-worker | | emr-4.4.0 | 0.136 | emrfs, emr-goodies, hadoop-client, hadoop-hdfs-datanode, hadoop-hdfs-library, hadoop-hdfs-namenode, hadoop-yarn-nodemanager, hadoop-yarn-resourcemanager, hive-client, hive-metastore-server, mysql-server, presto-coordinator, presto-worker | | emr-4.3.0 | 0.130 | emrfs, emr-goodies, hadoop-client, hadoop-hdfs-datanode, hadoop-hdfs-library, hadoop-hdfs-namenode, hadoop-yarn-nodemanager, hadoop-yarn-resourcemanager, hive-client, hive-metastore-server, mysql-server, presto-coordinator, presto-worker | | emr-4.2.0 | 0.125 | emrfs, emr-goodies, hadoop-client, hadoop-hdfs-datanode, hadoop-hdfs-library, hadoop-hdfs-namenode, hadoop-yarn-nodemanager, hadoop-yarn-resourcemanager, hive-client, hive-metastore-server, mysql-server, presto-coordinator, presto-worker | | emr-4.1.0 | 0.119 | emrfs, emr-goodies, hadoop-client, hadoop-hdfs-datanode, hadoop-hdfs-library, hadoop-hdfs-namenode, hadoop-yarn-nodemanager, hadoop-yarn-resourcemanager, hive-client, hive-metastore-server, mysql-server, presto-coordinator, presto-worker | ## Sqoop (sandbox versions) Sqoop is available as a sandbox application beginning with Amazon EMR release version 4.4.0. **Sqoop-Sandbox version information** | Amazon EMR Release Label | Sqoop-Sandbox Version | Components Installed With Sqoop-Sandbox | | --- | --- | --- | | emr-4.9.6 | 1.4.6 | emrfs, emr-ddb, emr-goodies, hadoop-client, hadoop-mapred, hadoop-hdfs-datanode, hadoop-hdfs-library, hadoop-hdfs-namenode, hadoop-httpfs-server, hadoop-kms-server, hadoop-yarn-nodemanager, hadoop-yarn-resourcemanager, mysql-server, sqoop-client | | emr-4.9.5 | 1.4.6 | emrfs, emr-ddb, emr-goodies, hadoop-client, hadoop-mapred, hadoop-hdfs-datanode, hadoop-hdfs-library, hadoop-hdfs-namenode, hadoop-httpfs-server, hadoop-kms-server, hadoop-yarn-nodemanager, hadoop-yarn-resourcemanager, mysql-server, sqoop-client | | emr-4.9.4 | 1.4.6 | emrfs, emr-ddb, emr-goodies, hadoop-client, hadoop-mapred, hadoop-hdfs-datanode, hadoop-hdfs-library, hadoop-hdfs-namenode, hadoop-httpfs-server, hadoop-kms-server, hadoop-yarn-nodemanager, hadoop-yarn-resourcemanager, mysql-server, sqoop-client | | emr-4.9.3 | 1.4.6 | emrfs, emr-ddb, emr-goodies, hadoop-client, hadoop-mapred, hadoop-hdfs-datanode, hadoop-hdfs-library, hadoop-hdfs-namenode, hadoop-httpfs-server, hadoop-kms-server, hadoop-yarn-nodemanager, hadoop-yarn-resourcemanager, mysql-server, sqoop-client | | emr-4.9.2 | 1.4.6 | emrfs, emr-ddb, emr-goodies, hadoop-client, hadoop-mapred, hadoop-hdfs-datanode, hadoop-hdfs-library, hadoop-hdfs-namenode, hadoop-httpfs-server, hadoop-kms-server, hadoop-yarn-nodemanager, hadoop-yarn-resourcemanager, mysql-server, sqoop-client | | emr-4.9.1 | 1.4.6 | emrfs, emr-ddb, emr-goodies, hadoop-client, hadoop-mapred, hadoop-hdfs-datanode, hadoop-hdfs-library, hadoop-hdfs-namenode, hadoop-httpfs-server, hadoop-kms-server, hadoop-yarn-nodemanager, hadoop-yarn-resourcemanager, mysql-server, sqoop-client | | emr-4.8.5 | 1.4.6 | emrfs, emr-ddb, emr-goodies, hadoop-client, hadoop-mapred, hadoop-hdfs-datanode, hadoop-hdfs-library, hadoop-hdfs-namenode, hadoop-httpfs-server, hadoop-kms-server, hadoop-yarn-nodemanager, hadoop-yarn-resourcemanager, mysql-server, sqoop-client | | emr-4.8.4 | 1.4.6 | emrfs, emr-ddb, emr-goodies, hadoop-client, hadoop-mapred, hadoop-hdfs-datanode, hadoop-hdfs-library, hadoop-hdfs-namenode, hadoop-httpfs-server, hadoop-kms-server, hadoop-yarn-nodemanager, hadoop-yarn-resourcemanager, mysql-server, sqoop-client | | emr-4.8.3 | 1.4.6 | emrfs, emr-ddb, emr-goodies, hadoop-client, hadoop-mapred, hadoop-hdfs-datanode, hadoop-hdfs-library, hadoop-hdfs-namenode, hadoop-httpfs-server, hadoop-kms-server, hadoop-yarn-nodemanager, hadoop-yarn-resourcemanager, mysql-server, sqoop-client | | emr-4.8.2 | 1.4.6 | emrfs, emr-ddb, emr-goodies, hadoop-client, hadoop-mapred, hadoop-hdfs-datanode, hadoop-hdfs-library, hadoop-hdfs-namenode, hadoop-httpfs-server, hadoop-kms-server, hadoop-yarn-nodemanager, hadoop-yarn-resourcemanager, mysql-server, sqoop-client | | emr-4.8.1 | 1.4.6 | emrfs, emr-ddb, emr-goodies, hadoop-client, hadoop-mapred, hadoop-hdfs-datanode, hadoop-hdfs-library, hadoop-hdfs-namenode, hadoop-httpfs-server, hadoop-kms-server, hadoop-yarn-nodemanager, hadoop-yarn-resourcemanager, mysql-server, sqoop-client | | emr-4.8.0 | 1.4.6 | emrfs, emr-ddb, emr-goodies, hadoop-client, hadoop-mapred, hadoop-hdfs-datanode, hadoop-hdfs-library, hadoop-hdfs-namenode, hadoop-httpfs-server, hadoop-kms-server, hadoop-yarn-nodemanager, hadoop-yarn-resourcemanager, mysql-server, sqoop-client | | emr-4.7.4 | 1.4.6 | emrfs, emr-ddb, emr-goodies, hadoop-client, hadoop-mapred, hadoop-hdfs-datanode, hadoop-hdfs-library, hadoop-hdfs-namenode, hadoop-httpfs-server, hadoop-kms-server, hadoop-yarn-nodemanager, hadoop-yarn-resourcemanager, mysql-server, sqoop-client | | emr-4.7.3 | 1.4.6 | emrfs, emr-ddb, emr-goodies, hadoop-client, hadoop-mapred, hadoop-hdfs-datanode, hadoop-hdfs-library, hadoop-hdfs-namenode, hadoop-httpfs-server, hadoop-kms-server, hadoop-yarn-nodemanager, hadoop-yarn-resourcemanager, mysql-server, sqoop-client | | emr-4.7.2 | 1.4.6 | emrfs, emr-ddb, emr-goodies, hadoop-client, hadoop-mapred, hadoop-hdfs-datanode, hadoop-hdfs-library, hadoop-hdfs-namenode, hadoop-httpfs-server, hadoop-kms-server, hadoop-yarn-nodemanager, hadoop-yarn-resourcemanager, mysql-server, sqoop-client | | emr-4.7.1 | 1.4.6 | emrfs, emr-ddb, emr-goodies, hadoop-client, hadoop-mapred, hadoop-hdfs-datanode, hadoop-hdfs-library, hadoop-hdfs-namenode, hadoop-httpfs-server, hadoop-kms-server, hadoop-yarn-nodemanager, hadoop-yarn-resourcemanager, mysql-server, sqoop-client | | emr-4.7.0 | 1.4.6 | emrfs, emr-ddb, emr-goodies, hadoop-client, hadoop-mapred, hadoop-hdfs-datanode, hadoop-hdfs-library, hadoop-hdfs-namenode, hadoop-httpfs-server, hadoop-kms-server, hadoop-yarn-nodemanager, hadoop-yarn-resourcemanager, mysql-server, sqoop-client | | emr-4.6.1 | 1.4.6 | emrfs, emr-ddb, emr-goodies, hadoop-client, hadoop-mapred, hadoop-hdfs-datanode, hadoop-hdfs-library, hadoop-hdfs-namenode, hadoop-httpfs-server, hadoop-yarn-nodemanager, hadoop-yarn-resourcemanager, mysql-server, sqoop-client | | emr-4.6.0 | 1.4.6 | emrfs, emr-ddb, emr-goodies, hadoop-client, hadoop-mapred, hadoop-hdfs-datanode, hadoop-hdfs-library, hadoop-hdfs-namenode, hadoop-httpfs-server, hadoop-yarn-nodemanager, hadoop-yarn-resourcemanager, mysql-server, sqoop-client | | emr-4.5.0 | 1.4.6 | emrfs, emr-ddb, emr-goodies, hadoop-client, hadoop-mapred, hadoop-hdfs-datanode, hadoop-hdfs-library, hadoop-hdfs-namenode, hadoop-httpfs-server, hadoop-yarn-nodemanager, hadoop-yarn-resourcemanager, sqoop-client | | emr-4.4.0 | 1.4.6 | emrfs, emr-ddb, emr-goodies, hadoop-client, hadoop-mapred, hadoop-hdfs-datanode, hadoop-hdfs-library, hadoop-hdfs-namenode, hadoop-httpfs-server, hadoop-yarn-nodemanager, hadoop-yarn-resourcemanager, sqoop-client | ## Zeppelin (sandbox versions) Zeppelin is available as a sandbox application beginning with Amazon EMR release version 4.1.0. **Zeppelin-Sandbox version information** | Amazon EMR Release Label | Zeppelin-Sandbox Version | Components Installed With Zeppelin-Sandbox | | --- | --- | --- | | emr-4.9.6 | 0.6.1 | emrfs, emr-goodies, hadoop-client, hadoop-hdfs-datanode, hadoop-hdfs-library, hadoop-hdfs-namenode, hadoop-httpfs-server, hadoop-kms-server, hadoop-yarn-nodemanager, hadoop-yarn-resourcemanager, spark-client, spark-history-server, spark-on-yarn, spark-yarn-slave, zeppelin-server | | emr-4.9.5 | 0.6.1 | emrfs, emr-goodies, hadoop-client, hadoop-hdfs-datanode, hadoop-hdfs-library, hadoop-hdfs-namenode, hadoop-httpfs-server, hadoop-kms-server, hadoop-yarn-nodemanager, hadoop-yarn-resourcemanager, spark-client, spark-history-server, spark-on-yarn, spark-yarn-slave, zeppelin-server | | emr-4.9.4 | 0.6.1 | emrfs, emr-goodies, hadoop-client, hadoop-hdfs-datanode, hadoop-hdfs-library, hadoop-hdfs-namenode, hadoop-httpfs-server, hadoop-kms-server, hadoop-yarn-nodemanager, hadoop-yarn-resourcemanager, spark-client, spark-history-server, spark-on-yarn, spark-yarn-slave, zeppelin-server | | emr-4.9.3 | 0.6.1 | emrfs, emr-goodies, hadoop-client, hadoop-hdfs-datanode, hadoop-hdfs-library, hadoop-hdfs-namenode, hadoop-httpfs-server, hadoop-kms-server, hadoop-yarn-nodemanager, hadoop-yarn-resourcemanager, spark-client, spark-history-server, spark-on-yarn, spark-yarn-slave, zeppelin-server | | emr-4.9.2 | 0.6.1 | emrfs, emr-goodies, hadoop-client, hadoop-hdfs-datanode, hadoop-hdfs-library, hadoop-hdfs-namenode, hadoop-httpfs-server, hadoop-kms-server, hadoop-yarn-nodemanager, hadoop-yarn-resourcemanager, spark-client, spark-history-server, spark-on-yarn, spark-yarn-slave, zeppelin-server | | emr-4.9.1 | 0.6.1 | emrfs, emr-goodies, hadoop-client, hadoop-hdfs-datanode, hadoop-hdfs-library, hadoop-hdfs-namenode, hadoop-httpfs-server, hadoop-kms-server, hadoop-yarn-nodemanager, hadoop-yarn-resourcemanager, spark-client, spark-history-server, spark-on-yarn, spark-yarn-slave, zeppelin-server | | emr-4.8.5 | 0.6.1 | emrfs, emr-goodies, hadoop-client, hadoop-hdfs-datanode, hadoop-hdfs-library, hadoop-hdfs-namenode, hadoop-httpfs-server, hadoop-kms-server, hadoop-yarn-nodemanager, hadoop-yarn-resourcemanager, spark-client, spark-history-server, spark-on-yarn, spark-yarn-slave, zeppelin-server | | emr-4.8.4 | 0.6.1 | emrfs, emr-goodies, hadoop-client, hadoop-hdfs-datanode, hadoop-hdfs-library, hadoop-hdfs-namenode, hadoop-httpfs-server, hadoop-kms-server, hadoop-yarn-nodemanager, hadoop-yarn-resourcemanager, spark-client, spark-history-server, spark-on-yarn, spark-yarn-slave, zeppelin-server | | emr-4.8.3 | 0.6.1 | emrfs, emr-goodies, hadoop-client, hadoop-hdfs-datanode, hadoop-hdfs-library, hadoop-hdfs-namenode, hadoop-httpfs-server, hadoop-kms-server, hadoop-yarn-nodemanager, hadoop-yarn-resourcemanager, spark-client, spark-history-server, spark-on-yarn, spark-yarn-slave, zeppelin-server | | emr-4.8.2 | 0.6.1 | emrfs, emr-goodies, hadoop-client, hadoop-hdfs-datanode, hadoop-hdfs-library, hadoop-hdfs-namenode, hadoop-httpfs-server, hadoop-kms-server, hadoop-yarn-nodemanager, hadoop-yarn-resourcemanager, spark-client, spark-history-server, spark-on-yarn, spark-yarn-slave, zeppelin-server | | emr-4.8.1 | 0.6.1 | emrfs, emr-goodies, hadoop-client, hadoop-hdfs-datanode, hadoop-hdfs-library, hadoop-hdfs-namenode, hadoop-httpfs-server, hadoop-kms-server, hadoop-yarn-nodemanager, hadoop-yarn-resourcemanager, spark-client, spark-history-server, spark-on-yarn, spark-yarn-slave, zeppelin-server | | emr-4.8.0 | 0.6.1 | emrfs, emr-goodies, hadoop-client, hadoop-hdfs-datanode, hadoop-hdfs-library, hadoop-hdfs-namenode, hadoop-httpfs-server, hadoop-kms-server, hadoop-yarn-nodemanager, hadoop-yarn-resourcemanager, spark-client, spark-history-server, spark-on-yarn, spark-yarn-slave, zeppelin-server | | emr-4.7.4 | 0.5.6-incubating | emrfs, emr-goodies, hadoop-client, hadoop-hdfs-datanode, hadoop-hdfs-library, hadoop-hdfs-namenode, hadoop-httpfs-server, hadoop-kms-server, hadoop-yarn-nodemanager, hadoop-yarn-resourcemanager, spark-client, spark-history-server, spark-on-yarn, spark-yarn-slave, zeppelin-server | | emr-4.7.3 | 0.5.6-incubating | emrfs, emr-goodies, hadoop-client, hadoop-hdfs-datanode, hadoop-hdfs-library, hadoop-hdfs-namenode, hadoop-httpfs-server, hadoop-kms-server, hadoop-yarn-nodemanager, hadoop-yarn-resourcemanager, spark-client, spark-history-server, spark-on-yarn, spark-yarn-slave, zeppelin-server | | emr-4.7.2 | 0.5.6-incubating | emrfs, emr-goodies, hadoop-client, hadoop-hdfs-datanode, hadoop-hdfs-library, hadoop-hdfs-namenode, hadoop-httpfs-server, hadoop-kms-server, hadoop-yarn-nodemanager, hadoop-yarn-resourcemanager, spark-client, spark-history-server, spark-on-yarn, spark-yarn-slave, zeppelin-server | | emr-4.7.1 | 0.5.6-incubating | emrfs, emr-goodies, hadoop-client, hadoop-hdfs-datanode, hadoop-hdfs-library, hadoop-hdfs-namenode, hadoop-httpfs-server, hadoop-kms-server, hadoop-yarn-nodemanager, hadoop-yarn-resourcemanager, spark-client, spark-history-server, spark-on-yarn, spark-yarn-slave, zeppelin-server | | emr-4.7.0 | 0.5.6-incubating | emrfs, emr-goodies, hadoop-client, hadoop-hdfs-datanode, hadoop-hdfs-library, hadoop-hdfs-namenode, hadoop-httpfs-server, hadoop-kms-server, hadoop-yarn-nodemanager, hadoop-yarn-resourcemanager, spark-client, spark-history-server, spark-on-yarn, spark-yarn-slave, zeppelin-server | | emr-4.6.1 | 0.5.6-incubating | emrfs, emr-goodies, hadoop-client, hadoop-hdfs-datanode, hadoop-hdfs-library, hadoop-hdfs-namenode, hadoop-httpfs-server, hadoop-yarn-nodemanager, hadoop-yarn-resourcemanager, spark-client, spark-history-server, spark-on-yarn, spark-yarn-slave, zeppelin-server | | emr-4.6.0 | 0.5.6-incubating | emrfs, emr-goodies, hadoop-client, hadoop-hdfs-datanode, hadoop-hdfs-library, hadoop-hdfs-namenode, hadoop-httpfs-server, hadoop-yarn-nodemanager, hadoop-yarn-resourcemanager, spark-client, spark-history-server, spark-on-yarn, spark-yarn-slave, zeppelin-server | | emr-4.5.0 | 0.5.6-incubating | emrfs, emr-goodies, hadoop-client, hadoop-hdfs-datanode, hadoop-hdfs-library, hadoop-hdfs-namenode, hadoop-httpfs-server, hadoop-yarn-nodemanager, hadoop-yarn-resourcemanager, spark-client, spark-history-server, spark-on-yarn, spark-yarn-slave, zeppelin-server | | emr-4.4.0 | 0.5.6-incubating | emrfs, emr-goodies, hadoop-client, hadoop-hdfs-datanode, hadoop-hdfs-library, hadoop-hdfs-namenode, hadoop-httpfs-server, hadoop-yarn-nodemanager, hadoop-yarn-resourcemanager, spark-client, spark-history-server, spark-on-yarn, spark-yarn-slave, zeppelin-server | | emr-4.3.0 | 0.5.5-incubating-amzn-1 | emrfs, emr-goodies, hadoop-client, hadoop-hdfs-datanode, hadoop-hdfs-library, hadoop-hdfs-namenode, hadoop-httpfs-server, hadoop-yarn-nodemanager, hadoop-yarn-resourcemanager, spark-client, spark-history-server, spark-on-yarn, spark-yarn-slave, zeppelin-server | | emr-4.2.0 | 0.5.5-incubating-amzn-0 | emrfs, emr-goodies, hadoop-client, hadoop-hdfs-datanode, hadoop-hdfs-library, hadoop-hdfs-namenode, hadoop-httpfs-server, hadoop-yarn-nodemanager, hadoop-yarn-resourcemanager, spark-client, spark-history-server, spark-on-yarn, spark-yarn-slave, zeppelin-server | | emr-4.1.0 | 0.6.0-incubating-SNAPSHOT | emrfs, emr-goodies, hadoop-client, hadoop-hdfs-datanode, hadoop-hdfs-library, hadoop-hdfs-namenode, hadoop-httpfs-server, hadoop-yarn-nodemanager, hadoop-yarn-resourcemanager, spark-client, spark-history-server, spark-on-yarn, spark-yarn-slave, zeppelin-server | ## ZooKeeper (sandbox versions) Zookeeper is available as a sandbox application beginning with Amazon EMR release version 4.6.0. **ZooKeeper-Sandbox version information** | Amazon EMR Release Label | ZooKeeper-Sandbox Version | Components Installed With ZooKeeper-Sandbox | | --- | --- | --- | | emr-4.9.6 | 3.4.9 | emrfs, emr-goodies, hadoop-client, hadoop-hdfs-datanode, hadoop-hdfs-library, hadoop-hdfs-namenode, hadoop-httpfs-server, hadoop-kms-server, hadoop-yarn-nodemanager, hadoop-yarn-resourcemanager, zookeeper-client, zookeeper-server | | emr-4.9.5 | 3.4.9 | emrfs, emr-goodies, hadoop-client, hadoop-hdfs-datanode, hadoop-hdfs-library, hadoop-hdfs-namenode, hadoop-httpfs-server, hadoop-kms-server, hadoop-yarn-nodemanager, hadoop-yarn-resourcemanager, zookeeper-client, zookeeper-server | | emr-4.9.4 | 3.4.9 | emrfs, emr-goodies, hadoop-client, hadoop-hdfs-datanode, hadoop-hdfs-library, hadoop-hdfs-namenode, hadoop-httpfs-server, hadoop-kms-server, hadoop-yarn-nodemanager, hadoop-yarn-resourcemanager, zookeeper-client, zookeeper-server | | emr-4.9.3 | 3.4.9 | emrfs, emr-goodies, hadoop-client, hadoop-hdfs-datanode, hadoop-hdfs-library, hadoop-hdfs-namenode, hadoop-httpfs-server, hadoop-kms-server, hadoop-yarn-nodemanager, hadoop-yarn-resourcemanager, zookeeper-client, zookeeper-server | | emr-4.9.2 | 3.4.9 | emrfs, emr-goodies, hadoop-client, hadoop-hdfs-datanode, hadoop-hdfs-library, hadoop-hdfs-namenode, hadoop-httpfs-server, hadoop-kms-server, hadoop-yarn-nodemanager, hadoop-yarn-resourcemanager, zookeeper-client, zookeeper-server | | emr-4.9.1 | 3.4.9 | emrfs, emr-goodies, hadoop-client, hadoop-hdfs-datanode, hadoop-hdfs-library, hadoop-hdfs-namenode, hadoop-httpfs-server, hadoop-kms-server, hadoop-yarn-nodemanager, hadoop-yarn-resourcemanager, zookeeper-client, zookeeper-server | | emr-4.8.5 | 3.4.9 | emrfs, emr-goodies, hadoop-client, hadoop-hdfs-datanode, hadoop-hdfs-library, hadoop-hdfs-namenode, hadoop-httpfs-server, hadoop-kms-server, hadoop-yarn-nodemanager, hadoop-yarn-resourcemanager, zookeeper-client, zookeeper-server | | emr-4.8.4 | 3.4.9 | emrfs, emr-goodies, hadoop-client, hadoop-hdfs-datanode, hadoop-hdfs-library, hadoop-hdfs-namenode, hadoop-httpfs-server, hadoop-kms-server, hadoop-yarn-nodemanager, hadoop-yarn-resourcemanager, zookeeper-client, zookeeper-server | | emr-4.8.3 | 3.4.9 | emrfs, emr-goodies, hadoop-client, hadoop-hdfs-datanode, hadoop-hdfs-library, hadoop-hdfs-namenode, hadoop-httpfs-server, hadoop-kms-server, hadoop-yarn-nodemanager, hadoop-yarn-resourcemanager, zookeeper-client, zookeeper-server | | emr-4.8.2 | 3.4.8 | emrfs, emr-goodies, hadoop-client, hadoop-hdfs-datanode, hadoop-hdfs-library, hadoop-hdfs-namenode, hadoop-httpfs-server, hadoop-kms-server, hadoop-yarn-nodemanager, hadoop-yarn-resourcemanager, zookeeper-client, zookeeper-server | | emr-4.8.1 | 3.4.8 | emrfs, emr-goodies, hadoop-client, hadoop-hdfs-datanode, hadoop-hdfs-library, hadoop-hdfs-namenode, hadoop-httpfs-server, hadoop-kms-server, hadoop-yarn-nodemanager, hadoop-yarn-resourcemanager, zookeeper-client, zookeeper-server | | emr-4.8.0 | 3.4.8 | emrfs, emr-goodies, hadoop-client, hadoop-hdfs-datanode, hadoop-hdfs-library, hadoop-hdfs-namenode, hadoop-httpfs-server, hadoop-kms-server, hadoop-yarn-nodemanager, hadoop-yarn-resourcemanager, zookeeper-client, zookeeper-server | | emr-4.7.4 | 3.4.8 | emrfs, emr-goodies, hadoop-client, hadoop-hdfs-datanode, hadoop-hdfs-library, hadoop-hdfs-namenode, hadoop-httpfs-server, hadoop-kms-server, hadoop-yarn-nodemanager, hadoop-yarn-resourcemanager, zookeeper-client, zookeeper-server | | emr-4.7.3 | 3.4.8 | emrfs, emr-goodies, hadoop-client, hadoop-hdfs-datanode, hadoop-hdfs-library, hadoop-hdfs-namenode, hadoop-httpfs-server, hadoop-kms-server, hadoop-yarn-nodemanager, hadoop-yarn-resourcemanager, zookeeper-client, zookeeper-server | | emr-4.7.2 | 3.4.8 | emrfs, emr-goodies, hadoop-client, hadoop-hdfs-datanode, hadoop-hdfs-library, hadoop-hdfs-namenode, hadoop-httpfs-server, hadoop-kms-server, hadoop-yarn-nodemanager, hadoop-yarn-resourcemanager, zookeeper-client, zookeeper-server | | emr-4.7.1 | 3.4.8 | emrfs, emr-goodies, hadoop-client, hadoop-hdfs-datanode, hadoop-hdfs-library, hadoop-hdfs-namenode, hadoop-httpfs-server, hadoop-kms-server, hadoop-yarn-nodemanager, hadoop-yarn-resourcemanager, zookeeper-client, zookeeper-server | | emr-4.7.0 | 3.4.8 | emrfs, emr-goodies, hadoop-client, hadoop-hdfs-datanode, hadoop-hdfs-library, hadoop-hdfs-namenode, hadoop-httpfs-server, hadoop-kms-server, hadoop-yarn-nodemanager, hadoop-yarn-resourcemanager, zookeeper-client, zookeeper-server | | emr-4.6.1 | 3.4.8 | emrfs, emr-goodies, hadoop-client, hadoop-hdfs-datanode, hadoop-hdfs-library, hadoop-hdfs-namenode, hadoop-httpfs-server, hadoop-yarn-nodemanager, hadoop-yarn-resourcemanager, zookeeper-client, zookeeper-server | | emr-4.6.0 | 3.4.8 | emrfs, emr-goodies, hadoop-client, hadoop-hdfs-datanode, hadoop-hdfs-library, hadoop-hdfs-namenode, hadoop-httpfs-server, hadoop-yarn-nodemanager, hadoop-yarn-resourcemanager, zookeeper-client, zookeeper-server | # Considerations for using Hive on Amazon EMR 4.x This section covers differences to consider when using Hive version 1.0.0 on Amazon EMR 4.x release versions, as compared to Hive 2.x on Amazon EMR 5.x release versions. ## ACID transactions not supported Hive on Amazon EMR 4.x release versions does not support ACID transactions with Hive data stored in Amazon S3 when using 4.x release versions. If you try to create a transactional table in Amazon S3, an exception occurs. ## Reading and writing to tables in Amazon S3 Hive on Amazon EMR 4.x release versions can write directly to Amazon S3 without using temporary files. This improves performance, but a consequence is that you cannot read and write to the same table in Amazon S3 within the same Hive statement. A workaround is to create and use a temporary table in HDFS. The following example shows how to use multiple Hive statements to update a table in Amazon S3. The statements create a temporary table in HDFS named `tmp` based on a table in Amazon S3 named `my_s3_table`. The table in Amazon S3 is then updated with the contents of the temporary table. ``` CREATE TEMPORARY TABLE tmp LIKE my_s3_table; INSERT OVERWRITE TABLE tmp SELECT ....; INSERT OVERWRITE TABLE my_s3_table SELECT * FROM tmp; ``` ## Log4j vs. Log4j 2 Hive on Amazon EMR 4.x release versions uses Log4j. Beginning with version 5.0.0, Log4j 2 is the default. These versions may require different logging configurations. See [Apache Log4j 2](http://logging.apache.org/log4j/2.x/) for details. ## MapReduce is the default execution engine Hive on Amazon EMR 4.x release versions uses MapReduce as the default execution engine. Beginning with Amazon EMR version 5.0.0, Tez is the default, which provides improved performance for most workflows. ## Hive authorization Hive on Amazon EMR 4.x release versions supports [Hive authorization](https://cwiki.apache.org/confluence/display/Hive/LanguageManual+Authorization) for HDFS but not for EMRFS and Amazon S3. Amazon EMR clusters run with authorization disabled by default. ## Hive file merge behavior with Amazon S3 Hive on Amazon EMR 4.x release versions merges small files at the end of a map-only job if `hive.merge.mapfiles` is `true`. A merge is triggered only if the average output size of the job is less than the `hive.merge.smallfiles.avgsize` setting. Amazon EMR Hive has exactly the same behavior if the final output path is in HDFS. If the output path is in Amazon S3, however, the `hive.merge.smallfiles.avgsize` parameter is ignored. In that situation, the merge task is always triggered if `hive.merge.mapfiles` is set to `true`. # Considerations for using Pig on Amazon EMR 4.x Pig version 0.14.0 is installed on clusters created using Amazon EMR 4.x release versions. Pig was upgraded to version 0.16.0 in Amazon EMR 5.0.0. Significant differences are covered below. ## Different default execution engine Pig version 0.14.0 on Amazon EMR 4.x release versions uses MapReduce as the default execution engine. Pig 0.16.0 and later use Apache Tez. You can explicitly set `exectype=mapreduce` in the `pig-properties` configuration classification to use MapReduce. ## Dropped Pig user-defined functions (UDFs) Custom UDFs that were available in Pig on Amazon EMR 4.x release versions were dropped beginning with Pig 0.16.0. Most of the UDFs have equivalent functions you can use instead. The following table lists dropped UDFs and equivalent functions. For more information, see [Built-in functions](https://pig.apache.org/docs/r0.16.0/func.html) on the Apache Pig site. | Dropped UDF | Equivalent function | | --- | --- | | FORMAT\$1DT(dtformat, date) | GetHour(date), GetMinute(date), GetMonth(date), GetSecond(date), GetWeek(date), GetYear(date), GetDay(date) | | EXTRACT(string, pattern) | REGEX\$1EXTRACT\$1ALL(string, pattern) | | REPLACE(string, pattern, replacement) | REPLACE(string, pattern, replacement) | | DATE\$1TIME() | ToDate() | | DURATION(dt, dt2) | WeeksBetween(dt, dt2), YearsBetween(dt, dt2), SecondsBetween(dt, dt2), MonthsBetween(dt, dt2), MinutesBetween(dt, dt2), HoursBetween(dt, dt2) | | EXTRACT\$1DT(format, date) | GetHour(date), GetMinute(date), GetMonth(date), GetSecond(date), GetWeek(date), GetYear(date), GetDay(date) | | OFFSET\$1DT(date, duration) | AddDuration(date, duration), SubtractDuration(date, duration) | | PERIOD(dt, dt2) | WeeksBetween(dt, dt2), YearsBetween(dt, dt2), SecondsBetween(dt, dt2), MonthsBetween(dt, dt2), MinutesBetween(dt, dt2), HoursBetween(dt, dt2) | | CAPITALIZE(string) | UCFIRST(string) | | CONCAT\$1WITH() | CONCAT() | | INDEX\$1OF() | INDEXOF() | | LAST\$1INDEX\$1OF() | LAST\$1INDEXOF() | | SPLIT\$1ON\$1REGEX() | STRSPLT() | | UNCAPITALIZE() | LCFIRST() | The following UDFs were dropped with no equivalent: FORMAT(), LOCAL\$1DATE(), LOCAL\$1TIME(), CENTER(), LEFT\$1PAD(), REPEAT(), REPLACE\$1ONCE(), RIGHT\$1PAD(), STRIP(), STRIP\$1END(), STRIP\$1START(), SWAP\$1CASE(). ## Discontinued Grunt commands Some Grunt commands were discontinued beginning with Pig 0.16.0. The following table lists Grunt commands in Pig 0.14.0 and the equivalent commands in the current version, where applicable. **Pig 0.14.0 and equivalent current Grunt commands** | Pig 0.14.0 Grunt command | Pig Grunt command in 0.16.0 and later | | --- | --- | | cat ) | fs -cat ; | | cd ; | No equivalent | | ls ; | fs -ls ; | | move ; | fs -mv ; | | copy ; | fs -cp ; | | copyToLocal ; | fs -copyToLocal ; | | copyFromLocal ; | fs -copyFromLocal ; | | mkdir ; | fs -mkdir ; | | rm ; | fs -rm -r -skipTrash ; | | rmf ; | fs -rm -r -skipTrash ; | ## Capability removed for non-HDFS home directories Pig 0.14.0 on Amazon EMR 4.x release versions has two mechanisms to allow users other than the `hadoop` user, who don't have home directories, to run Pig scripts. The first mechanism is an automatic fallback that sets the initial working directory to the root directory if the home directory doesn't exist. The second is a `pig.initial.fs.name` property that allows you to change the initial working directory. These mechanisms are not available beginning with Amazon EMR version 5.0.0, and users must have a home directory on HDFS. This doesn't apply to the `hadoop` user because a home directory is provisioned at launch. Scripts run using Hadoop jar steps default to the Hadoop user unless another user is explicitly specified using `command-runner.jar`. # Amazon EMR release 4.9.6 ## 4.9.6 application versions This release includes the following applications: [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [http://gethue.com/](http://gethue.com/), [http://mahout.apache.org/](http://mahout.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-4.9.6 | emr-4.9.5 | emr-4.9.4 | emr-4.9.3 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.10.75 | 1.10.75 | 1.10.75 | 1.10.75 | | Python | Not tracked | Not tracked | Not tracked | Not tracked | | Scala | Not tracked | Not tracked | Not tracked | Not tracked | | AmazonCloudWatchAgent | - | - | - | - | | Delta | - | - | - | - | | Flink | - | - | - | - | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 1.2.2 | 1.2.2 | 1.2.2 | 1.2.2 | | HCatalog | 1.0.0-amzn-9 | 1.0.0-amzn-9 | 1.0.0-amzn-9 | 1.0.0-amzn-9 | | Hadoop | 2.7.3-amzn-2 | 2.7.3-amzn-2 | 2.7.3-amzn-2 | 2.7.3-amzn-2 | | Hive | 1.0.0-amzn-9 | 1.0.0-amzn-9 | 1.0.0-amzn-9 | 1.0.0-amzn-9 | | Hudi | - | - | - | - | | Hue | 3.7.1-amzn-7 | 3.7.1-amzn-7 | 3.7.1-amzn-7 | 3.7.1-amzn-7 | | Iceberg | - | - | - | - | | JupyterEnterpriseGateway | - | - | - | - | | JupyterHub | - | - | - | - | | Livy | - | - | - | - | | MXNet | - | - | - | - | | Mahout | 0.12.2 | 0.12.2 | 0.12.2 | 0.12.2 | | Oozie | - | - | - | - | | Oozie-Sandbox | 4.2.0 | 4.2.0 | 4.2.0 | 4.2.0 | | Phoenix | 4.7.0-HBase-1.2 | 4.7.0-HBase-1.2 | 4.7.0-HBase-1.2 | 4.7.0-HBase-1.2 | | Pig | 0.14.0-amzn-0 | 0.14.0-amzn-0 | 0.14.0-amzn-0 | 0.14.0-amzn-0 | | Presto | - | - | - | - | | Presto-Sandbox | 0.157.1 | 0.157.1 | 0.157.1 | 0.157.1 | | Spark | 1.6.3 | 1.6.3 | 1.6.3 | 1.6.3 | | Sqoop | - | - | - | - | | Sqoop-Sandbox | 1.4.6 | 1.4.6 | 1.4.6 | 1.4.6 | | TensorFlow | - | - | - | - | | Tez | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | | Trino (PrestoSQL) | - | - | - | - | | Zeppelin | - | - | - | - | | Zeppelin-Sandbox | 0.6.1 | 0.6.1 | 0.6.1 | 0.6.1 | | ZooKeeper | - | - | - | - | | ZooKeeper-Sandbox | 3.4.9 | 3.4.9 | 3.4.9 | 3.4.9 | ## 4.9.6 release notes This is a patch release to add AWS Signature Version 4 authentication for requests to Amazon S3. All applications and components are the same as the previous Amazon EMR release. **Important** In this release version, Amazon EMR uses AWS Signature Version 4 exclusively to authenticate requests to Amazon S3. For more information, see [Whats New](https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-whatsnew.html). ## 4.9.6 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | emr-ddb | 4.3.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 2.2.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.3.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-s3-dist-cp | 2.4.0 | Distributed copy application optimized for Amazon S3. | | emrfs | 2.17.0 | Amazon S3 connector for Hadoop ecosystem applications. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 2.7.3-amzn-2 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 2.7.3-amzn-2 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 2.7.3-amzn-2 | HDFS command-line client and library | | hadoop-hdfs-namenode | 2.7.3-amzn-2 | HDFS service for tracking file names and block locations. | | hadoop-httpfs-server | 2.7.3-amzn-2 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 2.7.3-amzn-2 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 2.7.3-amzn-2 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 2.7.3-amzn-2 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 2.7.3-amzn-2 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 2.7.3-amzn-2 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 1.2.2 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 1.2.2 | Service for serving one or more HBase regions. | | hbase-client | 1.2.2 | HBase command-line client. | | hbase-rest-server | 1.2.2 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 1.2.2 | Service providing a Thrift endpoint to HBase. | | hcatalog-client | 1.0.0-amzn-9 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 1.0.0-amzn-9 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 1.0.0-amzn-9 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 1.0.0-amzn-9 | Hive command line client. | | hive-metastore-server | 1.0.0-amzn-9 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server | 1.0.0-amzn-9 | Service for accepting Hive queries as web requests. | | hue-server | 3.7.1-amzn-7 | Web application for analyzing data using Hadoop ecosystem applications | | mahout-client | 0.12.2 | Library for machine learning. | | mysql-server | 5.5.54\$1 | MySQL database server. | | oozie-client | 4.2.0 | Oozie command-line client. | | oozie-server | 4.2.0 | Service for accepting Oozie workflow requests. | | phoenix-library | 4.7.0-HBase-1.2 | The phoenix libraries for server and client | | phoenix-query-server | 4.7.0-HBase-1.2 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.157.1 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.157.1 | Service for executing pieces of a query. | | pig-client | 0.14.0-amzn-0 | Pig command-line client. | | spark-client | 1.6.3 | Spark command-line clients. | | spark-history-server | 1.6.3 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 1.6.3 | In-memory execution engine for YARN. | | spark-yarn-slave | 1.6.3 | Apache Spark libraries needed by YARN slaves. | | sqoop-client | 1.4.6 | Apache Sqoop command-line client. | | tez-on-yarn | 0.8.4 | The tez YARN application and libraries. | | webserver | 2.4.25\$1 | Apache HTTP server. | | zeppelin-server | 0.6.1 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.4.9 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.4.9 | ZooKeeper command line client. | ## 4.9.6 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). **emr-4.9.6 classifications** | Classifications | Description | | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | | core-site | Change values in Hadoop's core-site.xml file. | | emrfs-site | Change EMRFS settings. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | | hadoop-ssl-server | Change hadoop ssl server configuration | | hadoop-ssl-client | Change hadoop ssl client configuration | | hbase-env | Change values in HBase's environment. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbaase.properties file. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | | hbase-site | Change values in HBase's hbase-site.xml file. | | hdfs-encryption-zones | Configure HDFS encryption zones. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | | hcatalog-env | Change values in HCatalog's environment. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | | hcatalog-webhcat-log4j | Change values in HCatalog WebHCat's log4j.properties. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | | hive-env | Change values in the Hive environment. | | hive-exec-log4j | Change values in Hive's hive-exec-log4j.properties file. | | hive-log4j | Change values in Hive's hive-log4j.properties file. | | hive-site | Change values in Hive's hive-site.xml file | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | | hue-ini | Change values in Hue's ini file | | httpfs-env | Change values in the HTTPFS environment. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | | mapred-env | Change values in the MapReduce application's environment. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | | oozie-env | Change values in Oozie's environment. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | | oozie-site | Change values in Oozie's oozie-site.xml file. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | | pig-properties | Change values in Pig's pig.properties file. | | pig-log4j | Change values in Pig's log4j.properties file. | | presto-log | Change values in Presto's log.properties file. | | presto-config | Change values in Presto's config.properties file. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | | presto-connector-hive | Change values in Presto's hive.properties file. | | presto-connector-jmx | Change values in Presto's jmx.properties file. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | | presto-connector-localfile | Change values in Presto's localfile.properties file. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | | presto-connector-redis | Change values in Presto's redis.properties file. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | | spark | Amazon EMR-curated settings for Apache Spark. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | | spark-env | Change values in the Spark environment. | | spark-log4j | Change values in Spark's log4j.properties file. | | spark-metrics | Change values in Spark's metrics.properties file. | | sqoop-env | Change values in Sqoop's environment. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | | tez-site | Change values in Tez's tez-site.xml file. | | yarn-env | Change values in the YARN environment. | | yarn-site | Change values in YARN's yarn-site.xml file. | | zeppelin-env | Change values in the Zeppelin environment. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | # Amazon EMR release 4.9.5 ## 4.9.5 application versions This release includes the following applications: [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [http://gethue.com/](http://gethue.com/), [http://mahout.apache.org/](http://mahout.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-4.9.5 | emr-4.9.4 | emr-4.9.3 | emr-4.9.2 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.10.75 | 1.10.75 | 1.10.75 | 1.10.75 | | Python | Not tracked | Not tracked | Not tracked | Not tracked | | Scala | Not tracked | Not tracked | Not tracked | Not tracked | | AmazonCloudWatchAgent | - | - | - | - | | Delta | - | - | - | - | | Flink | - | - | - | - | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 1.2.2 | 1.2.2 | 1.2.2 | 1.2.2 | | HCatalog | 1.0.0-amzn-9 | 1.0.0-amzn-9 | 1.0.0-amzn-9 | 1.0.0-amzn-9 | | Hadoop | 2.7.3-amzn-2 | 2.7.3-amzn-2 | 2.7.3-amzn-2 | 2.7.3-amzn-2 | | Hive | 1.0.0-amzn-9 | 1.0.0-amzn-9 | 1.0.0-amzn-9 | 1.0.0-amzn-9 | | Hudi | - | - | - | - | | Hue | 3.7.1-amzn-7 | 3.7.1-amzn-7 | 3.7.1-amzn-7 | 3.7.1-amzn-7 | | Iceberg | - | - | - | - | | JupyterEnterpriseGateway | - | - | - | - | | JupyterHub | - | - | - | - | | Livy | - | - | - | - | | MXNet | - | - | - | - | | Mahout | 0.12.2 | 0.12.2 | 0.12.2 | 0.12.2 | | Oozie | - | - | - | - | | Oozie-Sandbox | 4.2.0 | 4.2.0 | 4.2.0 | 4.2.0 | | Phoenix | 4.7.0-HBase-1.2 | 4.7.0-HBase-1.2 | 4.7.0-HBase-1.2 | 4.7.0-HBase-1.2 | | Pig | 0.14.0-amzn-0 | 0.14.0-amzn-0 | 0.14.0-amzn-0 | 0.14.0-amzn-0 | | Presto | - | - | - | - | | Presto-Sandbox | 0.157.1 | 0.157.1 | 0.157.1 | 0.157.1 | | Spark | 1.6.3 | 1.6.3 | 1.6.3 | 1.6.3 | | Sqoop | - | - | - | - | | Sqoop-Sandbox | 1.4.6 | 1.4.6 | 1.4.6 | 1.4.6 | | TensorFlow | - | - | - | - | | Tez | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | | Trino (PrestoSQL) | - | - | - | - | | Zeppelin | - | - | - | - | | Zeppelin-Sandbox | 0.6.1 | 0.6.1 | 0.6.1 | 0.6.1 | | ZooKeeper | - | - | - | - | | ZooKeeper-Sandbox | 3.4.9 | 3.4.9 | 3.4.9 | 3.4.9 | ## 4.9.5 release notes The following release notes include information for Amazon EMR release 4.9.5. Changes are relative to 4.9.4. Initial release date: August 29, 2018 **Changes, enhancements, and resolved issues** + HBase + This release addresses a potential security vulnerability. ## 4.9.5 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | emr-ddb | 4.3.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 2.2.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.3.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-s3-dist-cp | 2.4.0 | Distributed copy application optimized for Amazon S3. | | emrfs | 2.17.0 | Amazon S3 connector for Hadoop ecosystem applications. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 2.7.3-amzn-2 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 2.7.3-amzn-2 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 2.7.3-amzn-2 | HDFS command-line client and library | | hadoop-hdfs-namenode | 2.7.3-amzn-2 | HDFS service for tracking file names and block locations. | | hadoop-httpfs-server | 2.7.3-amzn-2 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 2.7.3-amzn-2 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 2.7.3-amzn-2 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 2.7.3-amzn-2 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 2.7.3-amzn-2 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 2.7.3-amzn-2 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 1.2.2 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 1.2.2 | Service for serving one or more HBase regions. | | hbase-client | 1.2.2 | HBase command-line client. | | hbase-rest-server | 1.2.2 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 1.2.2 | Service providing a Thrift endpoint to HBase. | | hcatalog-client | 1.0.0-amzn-9 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 1.0.0-amzn-9 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 1.0.0-amzn-9 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 1.0.0-amzn-9 | Hive command line client. | | hive-metastore-server | 1.0.0-amzn-9 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server | 1.0.0-amzn-9 | Service for accepting Hive queries as web requests. | | hue-server | 3.7.1-amzn-7 | Web application for analyzing data using Hadoop ecosystem applications | | mahout-client | 0.12.2 | Library for machine learning. | | mysql-server | 5.5.54\$1 | MySQL database server. | | oozie-client | 4.2.0 | Oozie command-line client. | | oozie-server | 4.2.0 | Service for accepting Oozie workflow requests. | | phoenix-library | 4.7.0-HBase-1.2 | The phoenix libraries for server and client | | phoenix-query-server | 4.7.0-HBase-1.2 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.157.1 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.157.1 | Service for executing pieces of a query. | | pig-client | 0.14.0-amzn-0 | Pig command-line client. | | spark-client | 1.6.3 | Spark command-line clients. | | spark-history-server | 1.6.3 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 1.6.3 | In-memory execution engine for YARN. | | spark-yarn-slave | 1.6.3 | Apache Spark libraries needed by YARN slaves. | | sqoop-client | 1.4.6 | Apache Sqoop command-line client. | | tez-on-yarn | 0.8.4 | The tez YARN application and libraries. | | webserver | 2.4.25\$1 | Apache HTTP server. | | zeppelin-server | 0.6.1 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.4.9 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.4.9 | ZooKeeper command line client. | ## 4.9.5 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). **emr-4.9.5 classifications** | Classifications | Description | | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | | core-site | Change values in Hadoop's core-site.xml file. | | emrfs-site | Change EMRFS settings. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | | hadoop-ssl-server | Change hadoop ssl server configuration | | hadoop-ssl-client | Change hadoop ssl client configuration | | hbase-env | Change values in HBase's environment. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbaase.properties file. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | | hbase-site | Change values in HBase's hbase-site.xml file. | | hdfs-encryption-zones | Configure HDFS encryption zones. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | | hcatalog-env | Change values in HCatalog's environment. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | | hcatalog-webhcat-log4j | Change values in HCatalog WebHCat's log4j.properties. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | | hive-env | Change values in the Hive environment. | | hive-exec-log4j | Change values in Hive's hive-exec-log4j.properties file. | | hive-log4j | Change values in Hive's hive-log4j.properties file. | | hive-site | Change values in Hive's hive-site.xml file | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | | hue-ini | Change values in Hue's ini file | | httpfs-env | Change values in the HTTPFS environment. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | | mapred-env | Change values in the MapReduce application's environment. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | | oozie-env | Change values in Oozie's environment. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | | oozie-site | Change values in Oozie's oozie-site.xml file. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | | pig-properties | Change values in Pig's pig.properties file. | | pig-log4j | Change values in Pig's log4j.properties file. | | presto-log | Change values in Presto's log.properties file. | | presto-config | Change values in Presto's config.properties file. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | | presto-connector-hive | Change values in Presto's hive.properties file. | | presto-connector-jmx | Change values in Presto's jmx.properties file. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | | presto-connector-localfile | Change values in Presto's localfile.properties file. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | | presto-connector-redis | Change values in Presto's redis.properties file. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | | spark | Amazon EMR-curated settings for Apache Spark. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | | spark-env | Change values in the Spark environment. | | spark-log4j | Change values in Spark's log4j.properties file. | | spark-metrics | Change values in Spark's metrics.properties file. | | sqoop-env | Change values in Sqoop's environment. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | | tez-site | Change values in Tez's tez-site.xml file. | | yarn-env | Change values in the YARN environment. | | yarn-site | Change values in YARN's yarn-site.xml file. | | zeppelin-env | Change values in the Zeppelin environment. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | # Amazon EMR release 4.9.4 ## 4.9.4 application versions This release includes the following applications: [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [http://gethue.com/](http://gethue.com/), [http://mahout.apache.org/](http://mahout.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-4.9.4 | emr-4.9.3 | emr-4.9.2 | emr-4.9.1 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.10.75 | 1.10.75 | 1.10.75 | 1.10.75 | | Python | Not tracked | Not tracked | Not tracked | Not tracked | | Scala | Not tracked | Not tracked | Not tracked | Not tracked | | AmazonCloudWatchAgent | - | - | - | - | | Delta | - | - | - | - | | Flink | - | - | - | - | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 1.2.2 | 1.2.2 | 1.2.2 | 1.2.2 | | HCatalog | 1.0.0-amzn-9 | 1.0.0-amzn-9 | 1.0.0-amzn-9 | 1.0.0-amzn-9 | | Hadoop | 2.7.3-amzn-2 | 2.7.3-amzn-2 | 2.7.3-amzn-2 | 2.7.3-amzn-2 | | Hive | 1.0.0-amzn-9 | 1.0.0-amzn-9 | 1.0.0-amzn-9 | 1.0.0-amzn-9 | | Hudi | - | - | - | - | | Hue | 3.7.1-amzn-7 | 3.7.1-amzn-7 | 3.7.1-amzn-7 | 3.7.1-amzn-7 | | Iceberg | - | - | - | - | | JupyterEnterpriseGateway | - | - | - | - | | JupyterHub | - | - | - | - | | Livy | - | - | - | - | | MXNet | - | - | - | - | | Mahout | 0.12.2 | 0.12.2 | 0.12.2 | 0.12.2 | | Oozie | - | - | - | - | | Oozie-Sandbox | 4.2.0 | 4.2.0 | 4.2.0 | 4.2.0 | | Phoenix | 4.7.0-HBase-1.2 | 4.7.0-HBase-1.2 | 4.7.0-HBase-1.2 | 4.7.0-HBase-1.2 | | Pig | 0.14.0-amzn-0 | 0.14.0-amzn-0 | 0.14.0-amzn-0 | 0.14.0-amzn-0 | | Presto | - | - | - | - | | Presto-Sandbox | 0.157.1 | 0.157.1 | 0.157.1 | 0.157.1 | | Spark | 1.6.3 | 1.6.3 | 1.6.3 | 1.6.3 | | Sqoop | - | - | - | - | | Sqoop-Sandbox | 1.4.6 | 1.4.6 | 1.4.6 | 1.4.6 | | TensorFlow | - | - | - | - | | Tez | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | | Trino (PrestoSQL) | - | - | - | - | | Zeppelin | - | - | - | - | | Zeppelin-Sandbox | 0.6.1 | 0.6.1 | 0.6.1 | 0.6.1 | | ZooKeeper | - | - | - | - | | ZooKeeper-Sandbox | 3.4.9 | 3.4.9 | 3.4.9 | 3.4.9 | ## 4.9.4 release notes The following release notes include information for Amazon EMR release 4.9.4. Changes are relative to 4.9.3. Initial release date: March 29, 2018 **Changes, enhancements, and resolved issues** + Updated the Amazon Linux kernel of the defaultAmazon Linux AMI for Amazon EMR to address potential vulnerabilities. ## 4.9.4 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | emr-ddb | 4.3.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 2.2.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.3.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-s3-dist-cp | 2.4.0 | Distributed copy application optimized for Amazon S3. | | emrfs | 2.17.0 | Amazon S3 connector for Hadoop ecosystem applications. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 2.7.3-amzn-2 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 2.7.3-amzn-2 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 2.7.3-amzn-2 | HDFS command-line client and library | | hadoop-hdfs-namenode | 2.7.3-amzn-2 | HDFS service for tracking file names and block locations. | | hadoop-httpfs-server | 2.7.3-amzn-2 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 2.7.3-amzn-2 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 2.7.3-amzn-2 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 2.7.3-amzn-2 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 2.7.3-amzn-2 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 2.7.3-amzn-2 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 1.2.2 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 1.2.2 | Service for serving one or more HBase regions. | | hbase-client | 1.2.2 | HBase command-line client. | | hbase-rest-server | 1.2.2 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 1.2.2 | Service providing a Thrift endpoint to HBase. | | hcatalog-client | 1.0.0-amzn-9 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 1.0.0-amzn-9 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 1.0.0-amzn-9 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 1.0.0-amzn-9 | Hive command line client. | | hive-metastore-server | 1.0.0-amzn-9 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server | 1.0.0-amzn-9 | Service for accepting Hive queries as web requests. | | hue-server | 3.7.1-amzn-7 | Web application for analyzing data using Hadoop ecosystem applications | | mahout-client | 0.12.2 | Library for machine learning. | | mysql-server | 5.5.54\$1 | MySQL database server. | | oozie-client | 4.2.0 | Oozie command-line client. | | oozie-server | 4.2.0 | Service for accepting Oozie workflow requests. | | phoenix-library | 4.7.0-HBase-1.2 | The phoenix libraries for server and client | | phoenix-query-server | 4.7.0-HBase-1.2 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.157.1 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.157.1 | Service for executing pieces of a query. | | pig-client | 0.14.0-amzn-0 | Pig command-line client. | | spark-client | 1.6.3 | Spark command-line clients. | | spark-history-server | 1.6.3 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 1.6.3 | In-memory execution engine for YARN. | | spark-yarn-slave | 1.6.3 | Apache Spark libraries needed by YARN slaves. | | sqoop-client | 1.4.6 | Apache Sqoop command-line client. | | tez-on-yarn | 0.8.4 | The tez YARN application and libraries. | | webserver | 2.4.25\$1 | Apache HTTP server. | | zeppelin-server | 0.6.1 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.4.9 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.4.9 | ZooKeeper command line client. | ## 4.9.4 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). **emr-4.9.4 classifications** | Classifications | Description | | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | | core-site | Change values in Hadoop's core-site.xml file. | | emrfs-site | Change EMRFS settings. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | | hadoop-ssl-server | Change hadoop ssl server configuration | | hadoop-ssl-client | Change hadoop ssl client configuration | | hbase-env | Change values in HBase's environment. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbaase.properties file. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | | hbase-site | Change values in HBase's hbase-site.xml file. | | hdfs-encryption-zones | Configure HDFS encryption zones. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | | hcatalog-env | Change values in HCatalog's environment. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | | hcatalog-webhcat-log4j | Change values in HCatalog WebHCat's log4j.properties. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | | hive-env | Change values in the Hive environment. | | hive-exec-log4j | Change values in Hive's hive-exec-log4j.properties file. | | hive-log4j | Change values in Hive's hive-log4j.properties file. | | hive-site | Change values in Hive's hive-site.xml file | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | | hue-ini | Change values in Hue's ini file | | httpfs-env | Change values in the HTTPFS environment. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | | mapred-env | Change values in the MapReduce application's environment. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | | oozie-env | Change values in Oozie's environment. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | | oozie-site | Change values in Oozie's oozie-site.xml file. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | | pig-properties | Change values in Pig's pig.properties file. | | pig-log4j | Change values in Pig's log4j.properties file. | | presto-log | Change values in Presto's log.properties file. | | presto-config | Change values in Presto's config.properties file. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | | presto-connector-hive | Change values in Presto's hive.properties file. | | presto-connector-jmx | Change values in Presto's jmx.properties file. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | | presto-connector-localfile | Change values in Presto's localfile.properties file. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | | presto-connector-redis | Change values in Presto's redis.properties file. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | | spark | Amazon EMR-curated settings for Apache Spark. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | | spark-env | Change values in the Spark environment. | | spark-log4j | Change values in Spark's log4j.properties file. | | spark-metrics | Change values in Spark's metrics.properties file. | | sqoop-env | Change values in Sqoop's environment. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | | tez-site | Change values in Tez's tez-site.xml file. | | yarn-env | Change values in the YARN environment. | | yarn-site | Change values in YARN's yarn-site.xml file. | | zeppelin-env | Change values in the Zeppelin environment. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | # Amazon EMR release 4.9.3 ## 4.9.3 application versions This release includes the following applications: [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [http://gethue.com/](http://gethue.com/), [http://mahout.apache.org/](http://mahout.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-4.9.3 | emr-4.9.2 | emr-4.9.1 | emr-4.8.5 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.10.75 | 1.10.75 | 1.10.75 | 1.10.75 | | Python | Not tracked | Not tracked | Not tracked | Not tracked | | Scala | Not tracked | Not tracked | Not tracked | Not tracked | | AmazonCloudWatchAgent | - | - | - | - | | Delta | - | - | - | - | | Flink | - | - | - | - | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 1.2.2 | 1.2.2 | 1.2.2 | 1.2.2 | | HCatalog | 1.0.0-amzn-9 | 1.0.0-amzn-9 | 1.0.0-amzn-9 | 1.0.0-amzn-8 | | Hadoop | 2.7.3-amzn-2 | 2.7.3-amzn-2 | 2.7.3-amzn-2 | 2.7.3-amzn-1 | | Hive | 1.0.0-amzn-9 | 1.0.0-amzn-9 | 1.0.0-amzn-9 | 1.0.0-amzn-8 | | Hudi | - | - | - | - | | Hue | 3.7.1-amzn-7 | 3.7.1-amzn-7 | 3.7.1-amzn-7 | 3.7.1-amzn-7 | | Iceberg | - | - | - | - | | JupyterEnterpriseGateway | - | - | - | - | | JupyterHub | - | - | - | - | | Livy | - | - | - | - | | MXNet | - | - | - | - | | Mahout | 0.12.2 | 0.12.2 | 0.12.2 | 0.12.2 | | Oozie | - | - | - | - | | Oozie-Sandbox | 4.2.0 | 4.2.0 | 4.2.0 | 4.2.0 | | Phoenix | 4.7.0-HBase-1.2 | 4.7.0-HBase-1.2 | 4.7.0-HBase-1.2 | 4.7.0-HBase-1.2 | | Pig | 0.14.0-amzn-0 | 0.14.0-amzn-0 | 0.14.0-amzn-0 | 0.14.0-amzn-0 | | Presto | - | - | - | - | | Presto-Sandbox | 0.157.1 | 0.157.1 | 0.157.1 | 0.157.1 | | Spark | 1.6.3 | 1.6.3 | 1.6.3 | 1.6.3 | | Sqoop | - | - | - | - | | Sqoop-Sandbox | 1.4.6 | 1.4.6 | 1.4.6 | 1.4.6 | | TensorFlow | - | - | - | - | | Tez | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | | Trino (PrestoSQL) | - | - | - | - | | Zeppelin | - | - | - | - | | Zeppelin-Sandbox | 0.6.1 | 0.6.1 | 0.6.1 | 0.6.1 | | ZooKeeper | - | - | - | - | | ZooKeeper-Sandbox | 3.4.9 | 3.4.9 | 3.4.9 | 3.4.9 | ## 4.9.3 release notes The following release notes include information for the Amazon EMR 4.9.3 release. Changes are relative to the Amazon EMR 4.9.2 release. Initial release date: January 22, 2018 **Changes, enhancements, and resolved issues** + Updated the Amazon Linux kernel of the defaultAmazon Linux AMI for Amazon EMR to address vulnerabilities associated with speculative execution (CVE-2017-5715, CVE-2017-5753, and CVE-2017-5754). For more information, see [https://aws.amazon.com/security/security-bulletins/AWS-2018-013/](https://aws.amazon.com/security/security-bulletins/AWS-2018-013/). ## 4.9.3 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | emr-ddb | 4.3.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 2.2.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.3.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-s3-dist-cp | 2.4.0 | Distributed copy application optimized for Amazon S3. | | emrfs | 2.17.0 | Amazon S3 connector for Hadoop ecosystem applications. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 2.7.3-amzn-2 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 2.7.3-amzn-2 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 2.7.3-amzn-2 | HDFS command-line client and library | | hadoop-hdfs-namenode | 2.7.3-amzn-2 | HDFS service for tracking file names and block locations. | | hadoop-httpfs-server | 2.7.3-amzn-2 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 2.7.3-amzn-2 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 2.7.3-amzn-2 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 2.7.3-amzn-2 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 2.7.3-amzn-2 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 2.7.3-amzn-2 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 1.2.2 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 1.2.2 | Service for serving one or more HBase regions. | | hbase-client | 1.2.2 | HBase command-line client. | | hbase-rest-server | 1.2.2 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 1.2.2 | Service providing a Thrift endpoint to HBase. | | hcatalog-client | 1.0.0-amzn-9 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 1.0.0-amzn-9 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 1.0.0-amzn-9 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 1.0.0-amzn-9 | Hive command line client. | | hive-metastore-server | 1.0.0-amzn-9 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server | 1.0.0-amzn-9 | Service for accepting Hive queries as web requests. | | hue-server | 3.7.1-amzn-7 | Web application for analyzing data using Hadoop ecosystem applications | | mahout-client | 0.12.2 | Library for machine learning. | | mysql-server | 5.5.54\$1 | MySQL database server. | | oozie-client | 4.2.0 | Oozie command-line client. | | oozie-server | 4.2.0 | Service for accepting Oozie workflow requests. | | phoenix-library | 4.7.0-HBase-1.2 | The phoenix libraries for server and client | | phoenix-query-server | 4.7.0-HBase-1.2 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.157.1 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.157.1 | Service for executing pieces of a query. | | pig-client | 0.14.0-amzn-0 | Pig command-line client. | | spark-client | 1.6.3 | Spark command-line clients. | | spark-history-server | 1.6.3 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 1.6.3 | In-memory execution engine for YARN. | | spark-yarn-slave | 1.6.3 | Apache Spark libraries needed by YARN slaves. | | sqoop-client | 1.4.6 | Apache Sqoop command-line client. | | tez-on-yarn | 0.8.4 | The tez YARN application and libraries. | | webserver | 2.4.25\$1 | Apache HTTP server. | | zeppelin-server | 0.6.1 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.4.9 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.4.9 | ZooKeeper command line client. | ## 4.9.3 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). **emr-4.9.3 classifications** | Classifications | Description | | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | | core-site | Change values in Hadoop's core-site.xml file. | | emrfs-site | Change EMRFS settings. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | | hadoop-ssl-server | Change hadoop ssl server configuration | | hadoop-ssl-client | Change hadoop ssl client configuration | | hbase-env | Change values in HBase's environment. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbaase.properties file. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | | hbase-site | Change values in HBase's hbase-site.xml file. | | hdfs-encryption-zones | Configure HDFS encryption zones. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | | hcatalog-env | Change values in HCatalog's environment. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | | hcatalog-webhcat-log4j | Change values in HCatalog WebHCat's log4j.properties. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | | hive-env | Change values in the Hive environment. | | hive-exec-log4j | Change values in Hive's hive-exec-log4j.properties file. | | hive-log4j | Change values in Hive's hive-log4j.properties file. | | hive-site | Change values in Hive's hive-site.xml file | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | | hue-ini | Change values in Hue's ini file | | httpfs-env | Change values in the HTTPFS environment. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | | mapred-env | Change values in the MapReduce application's environment. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | | oozie-env | Change values in Oozie's environment. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | | oozie-site | Change values in Oozie's oozie-site.xml file. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | | pig-properties | Change values in Pig's pig.properties file. | | pig-log4j | Change values in Pig's log4j.properties file. | | presto-log | Change values in Presto's log.properties file. | | presto-config | Change values in Presto's config.properties file. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | | presto-connector-hive | Change values in Presto's hive.properties file. | | presto-connector-jmx | Change values in Presto's jmx.properties file. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | | presto-connector-localfile | Change values in Presto's localfile.properties file. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | | presto-connector-redis | Change values in Presto's redis.properties file. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | | spark | Amazon EMR-curated settings for Apache Spark. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | | spark-env | Change values in the Spark environment. | | spark-log4j | Change values in Spark's log4j.properties file. | | spark-metrics | Change values in Spark's metrics.properties file. | | sqoop-env | Change values in Sqoop's environment. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | | tez-site | Change values in Tez's tez-site.xml file. | | yarn-env | Change values in the YARN environment. | | yarn-site | Change values in YARN's yarn-site.xml file. | | zeppelin-env | Change values in the Zeppelin environment. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | # Amazon EMR release 4.9.2 ## 4.9.2 application versions This release includes the following applications: [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [http://gethue.com/](http://gethue.com/), [http://mahout.apache.org/](http://mahout.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-4.9.2 | emr-4.9.1 | emr-4.8.5 | emr-4.8.4 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.10.75 | 1.10.75 | 1.10.75 | 1.10.75 | | Python | Not tracked | Not tracked | Not tracked | Not tracked | | Scala | Not tracked | Not tracked | Not tracked | Not tracked | | AmazonCloudWatchAgent | - | - | - | - | | Delta | - | - | - | - | | Flink | - | - | - | - | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 1.2.2 | 1.2.2 | 1.2.2 | 1.2.2 | | HCatalog | 1.0.0-amzn-9 | 1.0.0-amzn-9 | 1.0.0-amzn-8 | 1.0.0-amzn-8 | | Hadoop | 2.7.3-amzn-2 | 2.7.3-amzn-2 | 2.7.3-amzn-1 | 2.7.3-amzn-1 | | Hive | 1.0.0-amzn-9 | 1.0.0-amzn-9 | 1.0.0-amzn-8 | 1.0.0-amzn-8 | | Hudi | - | - | - | - | | Hue | 3.7.1-amzn-7 | 3.7.1-amzn-7 | 3.7.1-amzn-7 | 3.7.1-amzn-7 | | Iceberg | - | - | - | - | | JupyterEnterpriseGateway | - | - | - | - | | JupyterHub | - | - | - | - | | Livy | - | - | - | - | | MXNet | - | - | - | - | | Mahout | 0.12.2 | 0.12.2 | 0.12.2 | 0.12.2 | | Oozie | - | - | - | - | | Oozie-Sandbox | 4.2.0 | 4.2.0 | 4.2.0 | 4.2.0 | | Phoenix | 4.7.0-HBase-1.2 | 4.7.0-HBase-1.2 | 4.7.0-HBase-1.2 | 4.7.0-HBase-1.2 | | Pig | 0.14.0-amzn-0 | 0.14.0-amzn-0 | 0.14.0-amzn-0 | 0.14.0-amzn-0 | | Presto | - | - | - | - | | Presto-Sandbox | 0.157.1 | 0.157.1 | 0.157.1 | 0.157.1 | | Spark | 1.6.3 | 1.6.3 | 1.6.3 | 1.6.3 | | Sqoop | - | - | - | - | | Sqoop-Sandbox | 1.4.6 | 1.4.6 | 1.4.6 | 1.4.6 | | TensorFlow | - | - | - | - | | Tez | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | | Trino (PrestoSQL) | - | - | - | - | | Zeppelin | - | - | - | - | | Zeppelin-Sandbox | 0.6.1 | 0.6.1 | 0.6.1 | 0.6.1 | | ZooKeeper | - | - | - | - | | ZooKeeper-Sandbox | 3.4.9 | 3.4.9 | 3.4.9 | 3.4.9 | ## 4.9.2 release notes The following release notes include information for the Amazon EMR 4.9.2 release. Changes are relative to the Amazon EMR 4.9.1 release. Release date: July 13, 2017 Minor changes, bug fixes, and enhancements were made in this release. ## 4.9.2 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | emr-ddb | 4.3.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 2.2.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.3.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-s3-dist-cp | 2.4.0 | Distributed copy application optimized for Amazon S3. | | emrfs | 2.17.0 | Amazon S3 connector for Hadoop ecosystem applications. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 2.7.3-amzn-2 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 2.7.3-amzn-2 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 2.7.3-amzn-2 | HDFS command-line client and library | | hadoop-hdfs-namenode | 2.7.3-amzn-2 | HDFS service for tracking file names and block locations. | | hadoop-httpfs-server | 2.7.3-amzn-2 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 2.7.3-amzn-2 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 2.7.3-amzn-2 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 2.7.3-amzn-2 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 2.7.3-amzn-2 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 2.7.3-amzn-2 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 1.2.2 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 1.2.2 | Service for serving one or more HBase regions. | | hbase-client | 1.2.2 | HBase command-line client. | | hbase-rest-server | 1.2.2 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 1.2.2 | Service providing a Thrift endpoint to HBase. | | hcatalog-client | 1.0.0-amzn-9 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 1.0.0-amzn-9 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 1.0.0-amzn-9 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 1.0.0-amzn-9 | Hive command line client. | | hive-metastore-server | 1.0.0-amzn-9 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server | 1.0.0-amzn-9 | Service for accepting Hive queries as web requests. | | hue-server | 3.7.1-amzn-7 | Web application for analyzing data using Hadoop ecosystem applications | | mahout-client | 0.12.2 | Library for machine learning. | | mysql-server | 5.5.54\$1 | MySQL database server. | | oozie-client | 4.2.0 | Oozie command-line client. | | oozie-server | 4.2.0 | Service for accepting Oozie workflow requests. | | phoenix-library | 4.7.0-HBase-1.2 | The phoenix libraries for server and client | | phoenix-query-server | 4.7.0-HBase-1.2 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.157.1 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.157.1 | Service for executing pieces of a query. | | pig-client | 0.14.0-amzn-0 | Pig command-line client. | | spark-client | 1.6.3 | Spark command-line clients. | | spark-history-server | 1.6.3 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 1.6.3 | In-memory execution engine for YARN. | | spark-yarn-slave | 1.6.3 | Apache Spark libraries needed by YARN slaves. | | sqoop-client | 1.4.6 | Apache Sqoop command-line client. | | tez-on-yarn | 0.8.4 | The tez YARN application and libraries. | | webserver | 2.4.25\$1 | Apache HTTP server. | | zeppelin-server | 0.6.1 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.4.9 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.4.9 | ZooKeeper command line client. | ## 4.9.2 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). **emr-4.9.2 classifications** | Classifications | Description | | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | | core-site | Change values in Hadoop's core-site.xml file. | | emrfs-site | Change EMRFS settings. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | | hadoop-ssl-server | Change hadoop ssl server configuration | | hadoop-ssl-client | Change hadoop ssl client configuration | | hbase-env | Change values in HBase's environment. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbaase.properties file. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | | hbase-site | Change values in HBase's hbase-site.xml file. | | hdfs-encryption-zones | Configure HDFS encryption zones. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | | hcatalog-env | Change values in HCatalog's environment. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | | hcatalog-webhcat-log4j | Change values in HCatalog WebHCat's log4j.properties. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | | hive-env | Change values in the Hive environment. | | hive-exec-log4j | Change values in Hive's hive-exec-log4j.properties file. | | hive-log4j | Change values in Hive's hive-log4j.properties file. | | hive-site | Change values in Hive's hive-site.xml file | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | | hue-ini | Change values in Hue's ini file | | httpfs-env | Change values in the HTTPFS environment. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | | mapred-env | Change values in the MapReduce application's environment. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | | oozie-env | Change values in Oozie's environment. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | | oozie-site | Change values in Oozie's oozie-site.xml file. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | | pig-properties | Change values in Pig's pig.properties file. | | pig-log4j | Change values in Pig's log4j.properties file. | | presto-log | Change values in Presto's log.properties file. | | presto-config | Change values in Presto's config.properties file. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | | presto-connector-hive | Change values in Presto's hive.properties file. | | presto-connector-jmx | Change values in Presto's jmx.properties file. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | | presto-connector-localfile | Change values in Presto's localfile.properties file. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | | presto-connector-redis | Change values in Presto's redis.properties file. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | | spark | Amazon EMR-curated settings for Apache Spark. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | | spark-env | Change values in the Spark environment. | | spark-log4j | Change values in Spark's log4j.properties file. | | spark-metrics | Change values in Spark's metrics.properties file. | | sqoop-env | Change values in Sqoop's environment. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | | tez-site | Change values in Tez's tez-site.xml file. | | yarn-env | Change values in the YARN environment. | | yarn-site | Change values in YARN's yarn-site.xml file. | | zeppelin-env | Change values in the Zeppelin environment. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | # Amazon EMR release 4.9.1 ## 4.9.1 application versions This release includes the following applications: [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [http://gethue.com/](http://gethue.com/), [http://mahout.apache.org/](http://mahout.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-4.9.1 | emr-4.8.5 | emr-4.8.4 | emr-4.8.3 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.10.75 | 1.10.75 | 1.10.75 | 1.10.75 | | Python | Not tracked | Not tracked | Not tracked | Not tracked | | Scala | Not tracked | Not tracked | Not tracked | Not tracked | | AmazonCloudWatchAgent | - | - | - | - | | Delta | - | - | - | - | | Flink | - | - | - | - | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 1.2.2 | 1.2.2 | 1.2.2 | 1.2.2 | | HCatalog | 1.0.0-amzn-9 | 1.0.0-amzn-8 | 1.0.0-amzn-8 | 1.0.0-amzn-8 | | Hadoop | 2.7.3-amzn-2 | 2.7.3-amzn-1 | 2.7.3-amzn-1 | 2.7.3-amzn-1 | | Hive | 1.0.0-amzn-9 | 1.0.0-amzn-8 | 1.0.0-amzn-8 | 1.0.0-amzn-8 | | Hudi | - | - | - | - | | Hue | 3.7.1-amzn-7 | 3.7.1-amzn-7 | 3.7.1-amzn-7 | 3.7.1-amzn-7 | | Iceberg | - | - | - | - | | JupyterEnterpriseGateway | - | - | - | - | | JupyterHub | - | - | - | - | | Livy | - | - | - | - | | MXNet | - | - | - | - | | Mahout | 0.12.2 | 0.12.2 | 0.12.2 | 0.12.2 | | Oozie | - | - | - | - | | Oozie-Sandbox | 4.2.0 | 4.2.0 | 4.2.0 | 4.2.0 | | Phoenix | 4.7.0-HBase-1.2 | 4.7.0-HBase-1.2 | 4.7.0-HBase-1.2 | 4.7.0-HBase-1.2 | | Pig | 0.14.0-amzn-0 | 0.14.0-amzn-0 | 0.14.0-amzn-0 | 0.14.0-amzn-0 | | Presto | - | - | - | - | | Presto-Sandbox | 0.157.1 | 0.157.1 | 0.157.1 | 0.157.1 | | Spark | 1.6.3 | 1.6.3 | 1.6.3 | 1.6.3 | | Sqoop | - | - | - | - | | Sqoop-Sandbox | 1.4.6 | 1.4.6 | 1.4.6 | 1.4.6 | | TensorFlow | - | - | - | - | | Tez | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | | Trino (PrestoSQL) | - | - | - | - | | Zeppelin | - | - | - | - | | Zeppelin-Sandbox | 0.6.1 | 0.6.1 | 0.6.1 | 0.6.1 | | ZooKeeper | - | - | - | - | | ZooKeeper-Sandbox | 3.4.9 | 3.4.9 | 3.4.9 | 3.4.9 | ## 4.9.1 release notes The following release notes include information for the Amazon EMR 4.9.1 release. Changes are relative to the Amazon EMR 4.8.4 release. Release date: April 10, 2017 **Known issues resolved from previous releases** + Backports of [HIVE-9976](https://issues.apache.org/jira/browse/HIVE-9976) and [HIVE-10106](https://issues.apache.org/jira/browse/HIVE-10106) + Fixed an issue in YARN where a large number of nodes (greater than 2,000) and containers (greater than 5,000) would cause an out-of-memory error, for example: `"Exception in thread main java.lang.OutOfMemoryError"`. **Changes and enhancements** + Amazon EMR releases are now based on Amazon Linux 2017.03. For more information, see [https://aws.amazon.com/amazon-linux-ami/2017.03-release-notes/](https://aws.amazon.com/amazon-linux-ami/2017.03-release-notes/). + Removed Python 2.6 from the Amazon EMR base Linux image. You can install Python 2.6 manually if necessary. ## 4.9.1 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | emr-ddb | 4.2.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 2.2.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.3.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-s3-dist-cp | 2.4.0 | Distributed copy application optimized for Amazon S3. | | emrfs | 2.15.0 | Amazon S3 connector for Hadoop ecosystem applications. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 2.7.3-amzn-2 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 2.7.3-amzn-2 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 2.7.3-amzn-2 | HDFS command-line client and library | | hadoop-hdfs-namenode | 2.7.3-amzn-2 | HDFS service for tracking file names and block locations. | | hadoop-httpfs-server | 2.7.3-amzn-2 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 2.7.3-amzn-2 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 2.7.3-amzn-2 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 2.7.3-amzn-2 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 2.7.3-amzn-2 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 2.7.3-amzn-2 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 1.2.2 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 1.2.2 | Service for serving one or more HBase regions. | | hbase-client | 1.2.2 | HBase command-line client. | | hbase-rest-server | 1.2.2 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 1.2.2 | Service providing a Thrift endpoint to HBase. | | hcatalog-client | 1.0.0-amzn-9 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 1.0.0-amzn-9 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 1.0.0-amzn-9 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 1.0.0-amzn-9 | Hive command line client. | | hive-metastore-server | 1.0.0-amzn-9 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server | 1.0.0-amzn-9 | Service for accepting Hive queries as web requests. | | hue-server | 3.7.1-amzn-7 | Web application for analyzing data using Hadoop ecosystem applications | | mahout-client | 0.12.2 | Library for machine learning. | | mysql-server | 5.5.54\$1 | MySQL database server. | | oozie-client | 4.2.0 | Oozie command-line client. | | oozie-server | 4.2.0 | Service for accepting Oozie workflow requests. | | phoenix-library | 4.7.0-HBase-1.2 | The phoenix libraries for server and client | | phoenix-query-server | 4.7.0-HBase-1.2 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.157.1 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.157.1 | Service for executing pieces of a query. | | pig-client | 0.14.0-amzn-0 | Pig command-line client. | | spark-client | 1.6.3 | Spark command-line clients. | | spark-history-server | 1.6.3 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 1.6.3 | In-memory execution engine for YARN. | | spark-yarn-slave | 1.6.3 | Apache Spark libraries needed by YARN slaves. | | sqoop-client | 1.4.6 | Apache Sqoop command-line client. | | tez-on-yarn | 0.8.4 | The tez YARN application and libraries. | | webserver | 2.4.25\$1 | Apache HTTP server. | | zeppelin-server | 0.6.1 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.4.9 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.4.9 | ZooKeeper command line client. | ## 4.9.1 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). **emr-4.9.1 classifications** | Classifications | Description | | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | | core-site | Change values in Hadoop's core-site.xml file. | | emrfs-site | Change EMRFS settings. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | | hadoop-ssl-server | Change hadoop ssl server configuration | | hadoop-ssl-client | Change hadoop ssl client configuration | | hbase-env | Change values in HBase's environment. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbaase.properties file. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | | hbase-site | Change values in HBase's hbase-site.xml file. | | hdfs-encryption-zones | Configure HDFS encryption zones. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | | hcatalog-env | Change values in HCatalog's environment. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | | hcatalog-webhcat-log4j | Change values in HCatalog WebHCat's log4j.properties. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | | hive-env | Change values in the Hive environment. | | hive-exec-log4j | Change values in Hive's hive-exec-log4j.properties file. | | hive-log4j | Change values in Hive's hive-log4j.properties file. | | hive-site | Change values in Hive's hive-site.xml file | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | | hue-ini | Change values in Hue's ini file | | httpfs-env | Change values in the HTTPFS environment. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | | mapred-env | Change values in the MapReduce application's environment. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | | oozie-env | Change values in Oozie's environment. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | | oozie-site | Change values in Oozie's oozie-site.xml file. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | | pig-properties | Change values in Pig's pig.properties file. | | pig-log4j | Change values in Pig's log4j.properties file. | | presto-log | Change values in Presto's log.properties file. | | presto-config | Change values in Presto's config.properties file. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | | presto-connector-hive | Change values in Presto's hive.properties file. | | presto-connector-jmx | Change values in Presto's jmx.properties file. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | | presto-connector-localfile | Change values in Presto's localfile.properties file. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | | presto-connector-redis | Change values in Presto's redis.properties file. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | | spark | Amazon EMR-curated settings for Apache Spark. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | | spark-env | Change values in the Spark environment. | | spark-log4j | Change values in Spark's log4j.properties file. | | spark-metrics | Change values in Spark's metrics.properties file. | | sqoop-env | Change values in Sqoop's environment. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | | tez-site | Change values in Tez's tez-site.xml file. | | yarn-env | Change values in the YARN environment. | | yarn-site | Change values in YARN's yarn-site.xml file. | | zeppelin-env | Change values in the Zeppelin environment. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | # Amazon EMR release 4.8.5 ## 4.8.5 application versions This release includes the following applications: [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [http://gethue.com/](http://gethue.com/), [http://mahout.apache.org/](http://mahout.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-4.8.5 | emr-4.8.4 | emr-4.8.3 | emr-4.8.2 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.10.75 | 1.10.75 | 1.10.75 | 1.10.75 | | Python | Not tracked | Not tracked | Not tracked | Not tracked | | Scala | Not tracked | Not tracked | Not tracked | Not tracked | | AmazonCloudWatchAgent | - | - | - | - | | Delta | - | - | - | - | | Flink | - | - | - | - | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 1.2.2 | 1.2.2 | 1.2.2 | 1.2.2 | | HCatalog | 1.0.0-amzn-8 | 1.0.0-amzn-8 | 1.0.0-amzn-8 | 1.0.0-amzn-7 | | Hadoop | 2.7.3-amzn-1 | 2.7.3-amzn-1 | 2.7.3-amzn-1 | 2.7.3-amzn-0 | | Hive | 1.0.0-amzn-8 | 1.0.0-amzn-8 | 1.0.0-amzn-8 | 1.0.0-amzn-7 | | Hudi | - | - | - | - | | Hue | 3.7.1-amzn-7 | 3.7.1-amzn-7 | 3.7.1-amzn-7 | 3.7.1-amzn-7 | | Iceberg | - | - | - | - | | JupyterEnterpriseGateway | - | - | - | - | | JupyterHub | - | - | - | - | | Livy | - | - | - | - | | MXNet | - | - | - | - | | Mahout | 0.12.2 | 0.12.2 | 0.12.2 | 0.12.2 | | Oozie | - | - | - | - | | Oozie-Sandbox | 4.2.0 | 4.2.0 | 4.2.0 | 4.2.0 | | Phoenix | 4.7.0-HBase-1.2 | 4.7.0-HBase-1.2 | 4.7.0-HBase-1.2 | 4.7.0-HBase-1.2 | | Pig | 0.14.0-amzn-0 | 0.14.0-amzn-0 | 0.14.0-amzn-0 | 0.14.0-amzn-0 | | Presto | - | - | - | - | | Presto-Sandbox | 0.157.1 | 0.157.1 | 0.157.1 | 0.152.3 | | Spark | 1.6.3 | 1.6.3 | 1.6.3 | 1.6.2 | | Sqoop | - | - | - | - | | Sqoop-Sandbox | 1.4.6 | 1.4.6 | 1.4.6 | 1.4.6 | | TensorFlow | - | - | - | - | | Tez | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | | Trino (PrestoSQL) | - | - | - | - | | Zeppelin | - | - | - | - | | Zeppelin-Sandbox | 0.6.1 | 0.6.1 | 0.6.1 | 0.6.1 | | ZooKeeper | - | - | - | - | | ZooKeeper-Sandbox | 3.4.9 | 3.4.9 | 3.4.9 | 3.4.8 | ## 4.8.5 release notes This is a patch release to add AWS Signature Version 4 authentication for requests to Amazon S3. All applications and components are the same as the previous Amazon EMR release. **Important** In this release version, Amazon EMR uses AWS Signature Version 4 exclusively to authenticate requests to Amazon S3. For more information, see [Whats New](https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-whatsnew.html). ## 4.8.5 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | emr-ddb | 4.2.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 2.2.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.2.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-s3-dist-cp | 2.4.0 | Distributed copy application optimized for Amazon S3. | | emrfs | 2.14.0 | Amazon S3 connector for Hadoop ecosystem applications. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 2.7.3-amzn-1 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 2.7.3-amzn-1 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 2.7.3-amzn-1 | HDFS command-line client and library | | hadoop-hdfs-namenode | 2.7.3-amzn-1 | HDFS service for tracking file names and block locations. | | hadoop-httpfs-server | 2.7.3-amzn-1 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 2.7.3-amzn-1 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 2.7.3-amzn-1 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 2.7.3-amzn-1 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 2.7.3-amzn-1 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 2.7.3-amzn-1 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 1.2.2 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 1.2.2 | Service for serving one or more HBase regions. | | hbase-client | 1.2.2 | HBase command-line client. | | hbase-rest-server | 1.2.2 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 1.2.2 | Service providing a Thrift endpoint to HBase. | | hcatalog-client | 1.0.0-amzn-8 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 1.0.0-amzn-8 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 1.0.0-amzn-8 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 1.0.0-amzn-8 | Hive command line client. | | hive-metastore-server | 1.0.0-amzn-8 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server | 1.0.0-amzn-8 | Service for accepting Hive queries as web requests. | | hue-server | 3.7.1-amzn-7 | Web application for analyzing data using Hadoop ecosystem applications | | mahout-client | 0.12.2 | Library for machine learning. | | mysql-server | 5.5.54\$1 | MySQL database server. | | oozie-client | 4.2.0 | Oozie command-line client. | | oozie-server | 4.2.0 | Service for accepting Oozie workflow requests. | | phoenix-library | 4.7.0-HBase-1.2 | The phoenix libraries for server and client | | phoenix-query-server | 4.7.0-HBase-1.2 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.157.1 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.157.1 | Service for executing pieces of a query. | | pig-client | 0.14.0-amzn-0 | Pig command-line client. | | spark-client | 1.6.3 | Spark command-line clients. | | spark-history-server | 1.6.3 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 1.6.3 | In-memory execution engine for YARN. | | spark-yarn-slave | 1.6.3 | Apache Spark libraries needed by YARN slaves. | | sqoop-client | 1.4.6 | Apache Sqoop command-line client. | | tez-on-yarn | 0.8.4 | The tez YARN application and libraries. | | webserver | 2.4.25\$1 | Apache HTTP server. | | zeppelin-server | 0.6.1 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.4.9 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.4.9 | ZooKeeper command line client. | ## 4.8.5 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). **emr-4.8.5 classifications** | Classifications | Description | | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | | core-site | Change values in Hadoop's core-site.xml file. | | emrfs-site | Change EMRFS settings. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | | hadoop-ssl-server | Change hadoop ssl server configuration | | hadoop-ssl-client | Change hadoop ssl client configuration | | hbase-env | Change values in HBase's environment. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbaase.properties file. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | | hbase-site | Change values in HBase's hbase-site.xml file. | | hdfs-encryption-zones | Configure HDFS encryption zones. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | | hcatalog-env | Change values in HCatalog's environment. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | | hcatalog-webhcat-log4j | Change values in HCatalog WebHCat's log4j.properties. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | | hive-env | Change values in the Hive environment. | | hive-exec-log4j | Change values in Hive's hive-exec-log4j.properties file. | | hive-log4j | Change values in Hive's hive-log4j.properties file. | | hive-site | Change values in Hive's hive-site.xml file | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | | hue-ini | Change values in Hue's ini file | | httpfs-env | Change values in the HTTPFS environment. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | | mapred-env | Change values in the MapReduce application's environment. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | | oozie-env | Change values in Oozie's environment. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | | oozie-site | Change values in Oozie's oozie-site.xml file. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | | pig-properties | Change values in Pig's pig.properties file. | | pig-log4j | Change values in Pig's log4j.properties file. | | presto-log | Change values in Presto's log.properties file. | | presto-config | Change values in Presto's config.properties file. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | | presto-connector-hive | Change values in Presto's hive.properties file. | | presto-connector-jmx | Change values in Presto's jmx.properties file. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | | presto-connector-localfile | Change values in Presto's localfile.properties file. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | | presto-connector-redis | Change values in Presto's redis.properties file. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | | spark | Amazon EMR-curated settings for Apache Spark. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | | spark-env | Change values in the Spark environment. | | spark-log4j | Change values in Spark's log4j.properties file. | | spark-metrics | Change values in Spark's metrics.properties file. | | sqoop-env | Change values in Sqoop's environment. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | | tez-site | Change values in Tez's tez-site.xml file. | | yarn-env | Change values in the YARN environment. | | yarn-site | Change values in YARN's yarn-site.xml file. | | zeppelin-env | Change values in the Zeppelin environment. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | # Amazon EMR release 4.8.4 ## 4.8.4 application versions This release includes the following applications: [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [http://gethue.com/](http://gethue.com/), [http://mahout.apache.org/](http://mahout.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-4.8.4 | emr-4.8.3 | emr-4.8.2 | emr-4.8.1 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.10.75 | 1.10.75 | 1.10.75 | 1.10.75 | | Python | Not tracked | Not tracked | Not tracked | Not tracked | | Scala | Not tracked | Not tracked | Not tracked | Not tracked | | AmazonCloudWatchAgent | - | - | - | - | | Delta | - | - | - | - | | Flink | - | - | - | - | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 1.2.2 | 1.2.2 | 1.2.2 | 1.2.2 | | HCatalog | 1.0.0-amzn-8 | 1.0.0-amzn-8 | 1.0.0-amzn-7 | 1.0.0-amzn-7 | | Hadoop | 2.7.3-amzn-1 | 2.7.3-amzn-1 | 2.7.3-amzn-0 | 2.7.2-amzn-4 | | Hive | 1.0.0-amzn-8 | 1.0.0-amzn-8 | 1.0.0-amzn-7 | 1.0.0-amzn-7 | | Hudi | - | - | - | - | | Hue | 3.7.1-amzn-7 | 3.7.1-amzn-7 | 3.7.1-amzn-7 | 3.7.1-amzn-7 | | Iceberg | - | - | - | - | | JupyterEnterpriseGateway | - | - | - | - | | JupyterHub | - | - | - | - | | Livy | - | - | - | - | | MXNet | - | - | - | - | | Mahout | 0.12.2 | 0.12.2 | 0.12.2 | 0.12.2 | | Oozie | - | - | - | - | | Oozie-Sandbox | 4.2.0 | 4.2.0 | 4.2.0 | 4.2.0 | | Phoenix | 4.7.0-HBase-1.2 | 4.7.0-HBase-1.2 | 4.7.0-HBase-1.2 | 4.7.0-HBase-1.2 | | Pig | 0.14.0-amzn-0 | 0.14.0-amzn-0 | 0.14.0-amzn-0 | 0.14.0-amzn-0 | | Presto | - | - | - | - | | Presto-Sandbox | 0.157.1 | 0.157.1 | 0.152.3 | 0.151 | | Spark | 1.6.3 | 1.6.3 | 1.6.2 | 1.6.2 | | Sqoop | - | - | - | - | | Sqoop-Sandbox | 1.4.6 | 1.4.6 | 1.4.6 | 1.4.6 | | TensorFlow | - | - | - | - | | Tez | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | | Trino (PrestoSQL) | - | - | - | - | | Zeppelin | - | - | - | - | | Zeppelin-Sandbox | 0.6.1 | 0.6.1 | 0.6.1 | 0.6.1 | | ZooKeeper | - | - | - | - | | ZooKeeper-Sandbox | 3.4.9 | 3.4.9 | 3.4.8 | 3.4.8 | ## 4.8.4 release notes The following release notes include information for the Amazon EMR 4.8.4 release. Changes are relative to the Amazon EMR 4.8.3 release. Release date: February 7, 2017 Minor changes, bug fixes, and enhancements were made in this release. ## 4.8.4 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | emr-ddb | 4.2.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 2.2.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.2.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-s3-dist-cp | 2.4.0 | Distributed copy application optimized for Amazon S3. | | emrfs | 2.14.0 | Amazon S3 connector for Hadoop ecosystem applications. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 2.7.3-amzn-1 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 2.7.3-amzn-1 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 2.7.3-amzn-1 | HDFS command-line client and library | | hadoop-hdfs-namenode | 2.7.3-amzn-1 | HDFS service for tracking file names and block locations. | | hadoop-httpfs-server | 2.7.3-amzn-1 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 2.7.3-amzn-1 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 2.7.3-amzn-1 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 2.7.3-amzn-1 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 2.7.3-amzn-1 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 2.7.3-amzn-1 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 1.2.2 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 1.2.2 | Service for serving one or more HBase regions. | | hbase-client | 1.2.2 | HBase command-line client. | | hbase-rest-server | 1.2.2 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 1.2.2 | Service providing a Thrift endpoint to HBase. | | hcatalog-client | 1.0.0-amzn-8 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 1.0.0-amzn-8 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 1.0.0-amzn-8 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 1.0.0-amzn-8 | Hive command line client. | | hive-metastore-server | 1.0.0-amzn-8 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server | 1.0.0-amzn-8 | Service for accepting Hive queries as web requests. | | hue-server | 3.7.1-amzn-7 | Web application for analyzing data using Hadoop ecosystem applications | | mahout-client | 0.12.2 | Library for machine learning. | | mysql-server | 5.5.54\$1 | MySQL database server. | | oozie-client | 4.2.0 | Oozie command-line client. | | oozie-server | 4.2.0 | Service for accepting Oozie workflow requests. | | phoenix-library | 4.7.0-HBase-1.2 | The phoenix libraries for server and client | | phoenix-query-server | 4.7.0-HBase-1.2 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.157.1 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.157.1 | Service for executing pieces of a query. | | pig-client | 0.14.0-amzn-0 | Pig command-line client. | | spark-client | 1.6.3 | Spark command-line clients. | | spark-history-server | 1.6.3 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 1.6.3 | In-memory execution engine for YARN. | | spark-yarn-slave | 1.6.3 | Apache Spark libraries needed by YARN slaves. | | sqoop-client | 1.4.6 | Apache Sqoop command-line client. | | tez-on-yarn | 0.8.4 | The tez YARN application and libraries. | | webserver | 2.4.25\$1 | Apache HTTP server. | | zeppelin-server | 0.6.1 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.4.9 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.4.9 | ZooKeeper command line client. | ## 4.8.4 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). **emr-4.8.4 classifications** | Classifications | Description | | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | | core-site | Change values in Hadoop's core-site.xml file. | | emrfs-site | Change EMRFS settings. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | | hadoop-ssl-server | Change hadoop ssl server configuration | | hadoop-ssl-client | Change hadoop ssl client configuration | | hbase-env | Change values in HBase's environment. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbaase.properties file. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | | hbase-site | Change values in HBase's hbase-site.xml file. | | hdfs-encryption-zones | Configure HDFS encryption zones. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | | hcatalog-env | Change values in HCatalog's environment. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | | hcatalog-webhcat-log4j | Change values in HCatalog WebHCat's log4j.properties. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | | hive-env | Change values in the Hive environment. | | hive-exec-log4j | Change values in Hive's hive-exec-log4j.properties file. | | hive-log4j | Change values in Hive's hive-log4j.properties file. | | hive-site | Change values in Hive's hive-site.xml file | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | | hue-ini | Change values in Hue's ini file | | httpfs-env | Change values in the HTTPFS environment. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | | mapred-env | Change values in the MapReduce application's environment. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | | oozie-env | Change values in Oozie's environment. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | | oozie-site | Change values in Oozie's oozie-site.xml file. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | | pig-properties | Change values in Pig's pig.properties file. | | pig-log4j | Change values in Pig's log4j.properties file. | | presto-log | Change values in Presto's log.properties file. | | presto-config | Change values in Presto's config.properties file. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | | presto-connector-hive | Change values in Presto's hive.properties file. | | presto-connector-jmx | Change values in Presto's jmx.properties file. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | | presto-connector-localfile | Change values in Presto's localfile.properties file. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | | presto-connector-redis | Change values in Presto's redis.properties file. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | | spark | Amazon EMR-curated settings for Apache Spark. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | | spark-env | Change values in the Spark environment. | | spark-log4j | Change values in Spark's log4j.properties file. | | spark-metrics | Change values in Spark's metrics.properties file. | | sqoop-env | Change values in Sqoop's environment. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | | tez-site | Change values in Tez's tez-site.xml file. | | yarn-env | Change values in the YARN environment. | | yarn-site | Change values in YARN's yarn-site.xml file. | | zeppelin-env | Change values in the Zeppelin environment. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | # Amazon EMR release 4.8.3 ## 4.8.3 application versions This release includes the following applications: [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [http://gethue.com/](http://gethue.com/), [http://mahout.apache.org/](http://mahout.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-4.8.3 | emr-4.8.2 | emr-4.8.1 | emr-4.8.0 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.10.75 | 1.10.75 | 1.10.75 | 1.10.75 | | Python | Not tracked | Not tracked | Not tracked | Not tracked | | Scala | Not tracked | Not tracked | Not tracked | Not tracked | | AmazonCloudWatchAgent | - | - | - | - | | Delta | - | - | - | - | | Flink | - | - | - | - | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 1.2.2 | 1.2.2 | 1.2.2 | 1.2.2 | | HCatalog | 1.0.0-amzn-8 | 1.0.0-amzn-7 | 1.0.0-amzn-7 | 1.0.0-amzn-7 | | Hadoop | 2.7.3-amzn-1 | 2.7.3-amzn-0 | 2.7.2-amzn-4 | 2.7.2-amzn-4 | | Hive | 1.0.0-amzn-8 | 1.0.0-amzn-7 | 1.0.0-amzn-7 | 1.0.0-amzn-7 | | Hudi | - | - | - | - | | Hue | 3.7.1-amzn-7 | 3.7.1-amzn-7 | 3.7.1-amzn-7 | 3.7.1-amzn-7 | | Iceberg | - | - | - | - | | JupyterEnterpriseGateway | - | - | - | - | | JupyterHub | - | - | - | - | | Livy | - | - | - | - | | MXNet | - | - | - | - | | Mahout | 0.12.2 | 0.12.2 | 0.12.2 | 0.12.2 | | Oozie | - | - | - | - | | Oozie-Sandbox | 4.2.0 | 4.2.0 | 4.2.0 | 4.2.0 | | Phoenix | 4.7.0-HBase-1.2 | 4.7.0-HBase-1.2 | 4.7.0-HBase-1.2 | 4.7.0-HBase-1.2 | | Pig | 0.14.0-amzn-0 | 0.14.0-amzn-0 | 0.14.0-amzn-0 | 0.14.0-amzn-0 | | Presto | - | - | - | - | | Presto-Sandbox | 0.157.1 | 0.152.3 | 0.151 | 0.151 | | Spark | 1.6.3 | 1.6.2 | 1.6.2 | 1.6.2 | | Sqoop | - | - | - | - | | Sqoop-Sandbox | 1.4.6 | 1.4.6 | 1.4.6 | 1.4.6 | | TensorFlow | - | - | - | - | | Tez | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.4 | | Trino (PrestoSQL) | - | - | - | - | | Zeppelin | - | - | - | - | | Zeppelin-Sandbox | 0.6.1 | 0.6.1 | 0.6.1 | 0.6.1 | | ZooKeeper | - | - | - | - | | ZooKeeper-Sandbox | 3.4.9 | 3.4.8 | 3.4.8 | 3.4.8 | ## 4.8.3 release notes The following release notes include information for the Amazon EMR 4.8.3 release. Changes are relative to the Amazon EMR 4.8.2 release. Release date: December 29, 2016 **Upgrades** + Upgraded to Presto 0.157.1. For more information, see [Presto Release Notes](https://prestodb.io/docs/current/release/release-0.157.1.html) in the Presto documentation. + Upgraded to Spark 1.6.3. For more information, see [Spark Release Notes](http://spark.apache.org/releases/spark-release-1-6-3.html) in the Apache Spark documentation. + Upgraded to ZooKeeper 3.4.9. For more information, see [ZooKeeper Release Notes](https://zookeeper.apache.org/doc/r3.4.9/releasenotes.html) in the Apache ZooKeeper documentation. **Changes and enhancements** + Added support for the Amazon EC2 m4.16xlarge instance type in Amazon EMR version 4.8.3 and later, excluding 5.0.0, 5.0.3, and 5.2.0. + Amazon EMR releases are now based on Amazon Linux 2016.09. For more information, see [https://aws.amazon.com/amazon-linux-ami/2016.09-release-notes/](https://aws.amazon.com/amazon-linux-ami/2016.09-release-notes/). **Known issues resolved from previous releases** + Fixed an issue in Hadoop where the ReplicationMonitor thread could get stuck for a long time because of a race between replication and deletion of the same file in a large cluster. + Fixed an issue where ControlledJob\$1toString failed with a null pointer exception (NPE) when job status was not successfully updated. ## 4.8.3 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | emr-ddb | 4.2.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 2.2.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.2.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-s3-dist-cp | 2.4.0 | Distributed copy application optimized for Amazon S3. | | emrfs | 2.13.0 | Amazon S3 connector for Hadoop ecosystem applications. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 2.7.3-amzn-1 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 2.7.3-amzn-1 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 2.7.3-amzn-1 | HDFS command-line client and library | | hadoop-hdfs-namenode | 2.7.3-amzn-1 | HDFS service for tracking file names and block locations. | | hadoop-httpfs-server | 2.7.3-amzn-1 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 2.7.3-amzn-1 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 2.7.3-amzn-1 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 2.7.3-amzn-1 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 2.7.3-amzn-1 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 2.7.3-amzn-1 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 1.2.2 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 1.2.2 | Service for serving one or more HBase regions. | | hbase-client | 1.2.2 | HBase command-line client. | | hbase-rest-server | 1.2.2 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 1.2.2 | Service providing a Thrift endpoint to HBase. | | hcatalog-client | 1.0.0-amzn-8 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 1.0.0-amzn-8 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 1.0.0-amzn-8 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 1.0.0-amzn-8 | Hive command line client. | | hive-metastore-server | 1.0.0-amzn-8 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server | 1.0.0-amzn-8 | Service for accepting Hive queries as web requests. | | hue-server | 3.7.1-amzn-7 | Web application for analyzing data using Hadoop ecosystem applications | | mahout-client | 0.12.2 | Library for machine learning. | | mysql-server | 5.5.52 | MySQL database server. | | oozie-client | 4.2.0 | Oozie command-line client. | | oozie-server | 4.2.0 | Service for accepting Oozie workflow requests. | | phoenix-library | 4.7.0-HBase-1.2 | The phoenix libraries for server and client | | phoenix-query-server | 4.7.0-HBase-1.2 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.157.1 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.157.1 | Service for executing pieces of a query. | | pig-client | 0.14.0-amzn-0 | Pig command-line client. | | spark-client | 1.6.3 | Spark command-line clients. | | spark-history-server | 1.6.3 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 1.6.3 | In-memory execution engine for YARN. | | spark-yarn-slave | 1.6.3 | Apache Spark libraries needed by YARN slaves. | | sqoop-client | 1.4.6 | Apache Sqoop command-line client. | | tez-on-yarn | 0.8.4 | The tez YARN application and libraries. | | webserver | 2.4.23 | Apache HTTP server. | | zeppelin-server | 0.6.1 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.4.9 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.4.9 | ZooKeeper command line client. | ## 4.8.3 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). **emr-4.8.3 classifications** | Classifications | Description | | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | | core-site | Change values in Hadoop's core-site.xml file. | | emrfs-site | Change EMRFS settings. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | | hadoop-ssl-server | Change hadoop ssl server configuration | | hadoop-ssl-client | Change hadoop ssl client configuration | | hbase-env | Change values in HBase's environment. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbaase.properties file. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | | hbase-site | Change values in HBase's hbase-site.xml file. | | hdfs-encryption-zones | Configure HDFS encryption zones. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | | hcatalog-env | Change values in HCatalog's environment. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | | hcatalog-webhcat-log4j | Change values in HCatalog WebHCat's log4j.properties. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | | hive-env | Change values in the Hive environment. | | hive-exec-log4j | Change values in Hive's hive-exec-log4j.properties file. | | hive-log4j | Change values in Hive's hive-log4j.properties file. | | hive-site | Change values in Hive's hive-site.xml file | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | | hue-ini | Change values in Hue's ini file | | httpfs-env | Change values in the HTTPFS environment. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | | mapred-env | Change values in the MapReduce application's environment. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | | oozie-env | Change values in Oozie's environment. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | | oozie-site | Change values in Oozie's oozie-site.xml file. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | | pig-properties | Change values in Pig's pig.properties file. | | pig-log4j | Change values in Pig's log4j.properties file. | | presto-log | Change values in Presto's log.properties file. | | presto-config | Change values in Presto's config.properties file. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | | presto-connector-hive | Change values in Presto's hive.properties file. | | presto-connector-jmx | Change values in Presto's jmx.properties file. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | | presto-connector-localfile | Change values in Presto's localfile.properties file. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | | presto-connector-redis | Change values in Presto's redis.properties file. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | | spark | Amazon EMR-curated settings for Apache Spark. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | | spark-env | Change values in the Spark environment. | | spark-log4j | Change values in Spark's log4j.properties file. | | spark-metrics | Change values in Spark's metrics.properties file. | | sqoop-env | Change values in Sqoop's environment. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | | tez-site | Change values in Tez's tez-site.xml file. | | yarn-env | Change values in the YARN environment. | | yarn-site | Change values in YARN's yarn-site.xml file. | | zeppelin-env | Change values in the Zeppelin environment. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | # Amazon EMR release 4.8.2 ## 4.8.2 application versions This release includes the following applications: [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [http://gethue.com/](http://gethue.com/), [http://mahout.apache.org/](http://mahout.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-4.8.2 | emr-4.8.1 | emr-4.8.0 | emr-4.7.4 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.10.75 | 1.10.75 | 1.10.75 | 1.10.75 | | Python | Not tracked | Not tracked | Not tracked | Not tracked | | Scala | Not tracked | Not tracked | Not tracked | Not tracked | | AmazonCloudWatchAgent | - | - | - | - | | Delta | - | - | - | - | | Flink | - | - | - | - | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 1.2.2 | 1.2.2 | 1.2.2 | 1.2.1 | | HCatalog | 1.0.0-amzn-7 | 1.0.0-amzn-7 | 1.0.0-amzn-7 | 1.0.0-amzn-6 | | Hadoop | 2.7.3-amzn-0 | 2.7.2-amzn-4 | 2.7.2-amzn-4 | 2.7.2-amzn-3 | | Hive | 1.0.0-amzn-7 | 1.0.0-amzn-7 | 1.0.0-amzn-7 | 1.0.0-amzn-6 | | Hudi | - | - | - | - | | Hue | 3.7.1-amzn-7 | 3.7.1-amzn-7 | 3.7.1-amzn-7 | 3.7.1-amzn-7 | | Iceberg | - | - | - | - | | JupyterEnterpriseGateway | - | - | - | - | | JupyterHub | - | - | - | - | | Livy | - | - | - | - | | MXNet | - | - | - | - | | Mahout | 0.12.2 | 0.12.2 | 0.12.2 | 0.12.2 | | Oozie | - | - | - | - | | Oozie-Sandbox | 4.2.0 | 4.2.0 | 4.2.0 | 4.2.0 | | Phoenix | 4.7.0-HBase-1.2 | 4.7.0-HBase-1.2 | 4.7.0-HBase-1.2 | 4.7.0-HBase-1.2 | | Pig | 0.14.0-amzn-0 | 0.14.0-amzn-0 | 0.14.0-amzn-0 | 0.14.0-amzn-0 | | Presto | - | - | - | - | | Presto-Sandbox | 0.152.3 | 0.151 | 0.151 | 0.148 | | Spark | 1.6.2 | 1.6.2 | 1.6.2 | 1.6.2 | | Sqoop | - | - | - | - | | Sqoop-Sandbox | 1.4.6 | 1.4.6 | 1.4.6 | 1.4.6 | | TensorFlow | - | - | - | - | | Tez | 0.8.4 | 0.8.4 | 0.8.4 | 0.8.3 | | Trino (PrestoSQL) | - | - | - | - | | Zeppelin | - | - | - | - | | Zeppelin-Sandbox | 0.6.1 | 0.6.1 | 0.6.1 | 0.5.6-incubating | | ZooKeeper | - | - | - | - | | ZooKeeper-Sandbox | 3.4.8 | 3.4.8 | 3.4.8 | 3.4.8 | ## 4.8.2 release notes The following release notes include information for the Amazon EMR 4.8.2 release. Changes are relative to the Amazon EMR 4.8.0 release. Release date: October 24, 2016 **Upgrades** + Upgraded to Hadoop 2.7.3 + Upgraded to Presto 0.152.3, which includes support for the Presto web interface. You can access the Presto web interface on the Presto coordinator using port 8889. For more information about the Presto web interface, see [Web Interface](https://prestodb.io/docs/current/admin/web-interface.html) in the Presto documentation. + Amazon EMR releases are now based on Amazon Linux 2016.09. For more information, see [https://aws.amazon.com/amazon-linux-ami/2016.09-release-notes/](https://aws.amazon.com/amazon-linux-ami/2016.09-release-notes/). ## 4.8.2 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | emr-ddb | 4.1.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 2.1.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.2.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-s3-dist-cp | 2.4.0 | Distributed copy application optimized for Amazon S3. | | emrfs | 2.10.0 | Amazon S3 connector for Hadoop ecosystem applications. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 2.7.3-amzn-0 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 2.7.3-amzn-0 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 2.7.3-amzn-0 | HDFS command-line client and library | | hadoop-hdfs-namenode | 2.7.3-amzn-0 | HDFS service for tracking file names and block locations. | | hadoop-httpfs-server | 2.7.3-amzn-0 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 2.7.3-amzn-0 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 2.7.3-amzn-0 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 2.7.3-amzn-0 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 2.7.3-amzn-0 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 2.7.3-amzn-0 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 1.2.2 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 1.2.2 | Service for serving one or more HBase regions. | | hbase-client | 1.2.2 | HBase command-line client. | | hbase-rest-server | 1.2.2 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 1.2.2 | Service providing a Thrift endpoint to HBase. | | hcatalog-client | 1.0.0-amzn-7 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 1.0.0-amzn-7 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 1.0.0-amzn-7 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 1.0.0-amzn-7 | Hive command line client. | | hive-metastore-server | 1.0.0-amzn-7 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server | 1.0.0-amzn-7 | Service for accepting Hive queries as web requests. | | hue-server | 3.7.1-amzn-7 | Web application for analyzing data using Hadoop ecosystem applications | | mahout-client | 0.12.2 | Library for machine learning. | | mysql-server | 5.5.52 | MySQL database server. | | oozie-client | 4.2.0 | Oozie command-line client. | | oozie-server | 4.2.0 | Service for accepting Oozie workflow requests. | | phoenix-library | 4.7.0-HBase-1.2 | The phoenix libraries for server and client | | phoenix-query-server | 4.7.0-HBase-1.2 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.152.3 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.152.3 | Service for executing pieces of a query. | | pig-client | 0.14.0-amzn-0 | Pig command-line client. | | spark-client | 1.6.2 | Spark command-line clients. | | spark-history-server | 1.6.2 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 1.6.2 | In-memory execution engine for YARN. | | spark-yarn-slave | 1.6.2 | Apache Spark libraries needed by YARN slaves. | | sqoop-client | 1.4.6 | Apache Sqoop command-line client. | | tez-on-yarn | 0.8.4 | The tez YARN application and libraries. | | webserver | 2.4.23 | Apache HTTP server. | | zeppelin-server | 0.6.1 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.4.8 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.4.8 | ZooKeeper command line client. | ## 4.8.2 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). **emr-4.8.2 classifications** | Classifications | Description | | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | | core-site | Change values in Hadoop's core-site.xml file. | | emrfs-site | Change EMRFS settings. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | | hadoop-ssl-server | Change hadoop ssl server configuration | | hadoop-ssl-client | Change hadoop ssl client configuration | | hbase-env | Change values in HBase's environment. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbaase.properties file. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | | hbase-site | Change values in HBase's hbase-site.xml file. | | hdfs-encryption-zones | Configure HDFS encryption zones. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | | hcatalog-env | Change values in HCatalog's environment. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | | hcatalog-webhcat-log4j | Change values in HCatalog WebHCat's log4j.properties. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | | hive-env | Change values in the Hive environment. | | hive-exec-log4j | Change values in Hive's hive-exec-log4j.properties file. | | hive-log4j | Change values in Hive's hive-log4j.properties file. | | hive-site | Change values in Hive's hive-site.xml file | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | | hue-ini | Change values in Hue's ini file | | httpfs-env | Change values in the HTTPFS environment. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | | mapred-env | Change values in the MapReduce application's environment. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | | oozie-env | Change values in Oozie's environment. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | | oozie-site | Change values in Oozie's oozie-site.xml file. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | | pig-properties | Change values in Pig's pig.properties file. | | pig-log4j | Change values in Pig's log4j.properties file. | | presto-log | Change values in Presto's log.properties file. | | presto-config | Change values in Presto's config.properties file. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | | presto-connector-hive | Change values in Presto's hive.properties file. | | presto-connector-jmx | Change values in Presto's jmx.properties file. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | | presto-connector-localfile | Change values in Presto's localfile.properties file. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | | presto-connector-redis | Change values in Presto's redis.properties file. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | | spark | Amazon EMR-curated settings for Apache Spark. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | | spark-env | Change values in the Spark environment. | | spark-log4j | Change values in Spark's log4j.properties file. | | spark-metrics | Change values in Spark's metrics.properties file. | | sqoop-env | Change values in Sqoop's environment. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | | tez-site | Change values in Tez's tez-site.xml file. | | yarn-env | Change values in the YARN environment. | | yarn-site | Change values in YARN's yarn-site.xml file. | | zeppelin-env | Change values in the Zeppelin environment. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | # Amazon EMR release 4.8.1 ## 4.8.1 application versions This release includes the following applications: [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [http://gethue.com/](http://gethue.com/), [http://mahout.apache.org/](http://mahout.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-4.8.1 | emr-4.8.0 | emr-4.7.4 | emr-4.7.3 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.10.75 | 1.10.75 | 1.10.75 | 1.10.75 | | Python | Not tracked | Not tracked | Not tracked | Not tracked | | Scala | Not tracked | Not tracked | Not tracked | Not tracked | | AmazonCloudWatchAgent | - | - | - | - | | Delta | - | - | - | - | | Flink | - | - | - | - | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 1.2.2 | 1.2.2 | 1.2.1 | 1.2.1 | | HCatalog | 1.0.0-amzn-7 | 1.0.0-amzn-7 | 1.0.0-amzn-6 | 1.0.0-amzn-6 | | Hadoop | 2.7.2-amzn-4 | 2.7.2-amzn-4 | 2.7.2-amzn-3 | 2.7.2-amzn-3 | | Hive | 1.0.0-amzn-7 | 1.0.0-amzn-7 | 1.0.0-amzn-6 | 1.0.0-amzn-6 | | Hudi | - | - | - | - | | Hue | 3.7.1-amzn-7 | 3.7.1-amzn-7 | 3.7.1-amzn-7 | 3.7.1-amzn-7 | | Iceberg | - | - | - | - | | JupyterEnterpriseGateway | - | - | - | - | | JupyterHub | - | - | - | - | | Livy | - | - | - | - | | MXNet | - | - | - | - | | Mahout | 0.12.2 | 0.12.2 | 0.12.2 | 0.12.2 | | Oozie | - | - | - | - | | Oozie-Sandbox | 4.2.0 | 4.2.0 | 4.2.0 | 4.2.0 | | Phoenix | 4.7.0-HBase-1.2 | 4.7.0-HBase-1.2 | 4.7.0-HBase-1.2 | 4.7.0-HBase-1.2 | | Pig | 0.14.0-amzn-0 | 0.14.0-amzn-0 | 0.14.0-amzn-0 | 0.14.0-amzn-0 | | Presto | - | - | - | - | | Presto-Sandbox | 0.151 | 0.151 | 0.148 | 0.148 | | Spark | 1.6.2 | 1.6.2 | 1.6.2 | 1.6.2 | | Sqoop | - | - | - | - | | Sqoop-Sandbox | 1.4.6 | 1.4.6 | 1.4.6 | 1.4.6 | | TensorFlow | - | - | - | - | | Tez | 0.8.4 | 0.8.4 | 0.8.3 | 0.8.3 | | Trino (PrestoSQL) | - | - | - | - | | Zeppelin | - | - | - | - | | Zeppelin-Sandbox | 0.6.1 | 0.6.1 | 0.5.6-incubating | 0.5.6-incubating | | ZooKeeper | - | - | - | - | | ZooKeeper-Sandbox | 3.4.8 | 3.4.8 | 3.4.8 | 3.4.8 | ## 4.8.1 release notes ## 4.8.1 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | emr-ddb | 3.2.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 2.1.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.2.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-s3-dist-cp | 2.4.0 | Distributed copy application optimized for Amazon S3. | | emrfs | 2.9.0 | Amazon S3 connector for Hadoop ecosystem applications. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 2.7.2-amzn-4 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 2.7.2-amzn-4 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 2.7.2-amzn-4 | HDFS command-line client and library | | hadoop-hdfs-namenode | 2.7.2-amzn-4 | HDFS service for tracking file names and block locations. | | hadoop-httpfs-server | 2.7.2-amzn-4 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 2.7.2-amzn-4 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 2.7.2-amzn-4 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 2.7.2-amzn-4 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 2.7.2-amzn-4 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 2.7.2-amzn-4 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 1.2.2 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 1.2.2 | Service for serving one or more HBase regions. | | hbase-client | 1.2.2 | HBase command-line client. | | hbase-rest-server | 1.2.2 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 1.2.2 | Service providing a Thrift endpoint to HBase. | | hcatalog-client | 1.0.0-amzn-7 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 1.0.0-amzn-7 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 1.0.0-amzn-7 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 1.0.0-amzn-7 | Hive command line client. | | hive-metastore-server | 1.0.0-amzn-7 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server | 1.0.0-amzn-7 | Service for accepting Hive queries as web requests. | | hue-server | 3.7.1-amzn-7 | Web application for analyzing data using Hadoop ecosystem applications | | mahout-client | 0.12.2 | Library for machine learning. | | mysql-server | 5.5.51 | MySQL database server. | | oozie-client | 4.2.0 | Oozie command-line client. | | oozie-server | 4.2.0 | Service for accepting Oozie workflow requests. | | phoenix-library | 4.7.0-HBase-1.2 | The phoenix libraries for server and client | | phoenix-query-server | 4.7.0-HBase-1.2 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.151 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.151 | Service for executing pieces of a query. | | pig-client | 0.14.0-amzn-0 | Pig command-line client. | | spark-client | 1.6.2 | Spark command-line clients. | | spark-history-server | 1.6.2 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 1.6.2 | In-memory execution engine for YARN. | | spark-yarn-slave | 1.6.2 | Apache Spark libraries needed by YARN slaves. | | sqoop-client | 1.4.6 | Apache Sqoop command-line client. | | tez-on-yarn | 0.8.4 | The tez YARN application and libraries. | | webserver | 2.4.23 | Apache HTTP server. | | zeppelin-server | 0.6.1 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.4.8 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.4.8 | ZooKeeper command line client. | ## 4.8.1 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). **emr-4.8.1 classifications** | Classifications | Description | | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | | core-site | Change values in Hadoop's core-site.xml file. | | emrfs-site | Change EMRFS settings. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | | hadoop-ssl-server | Change hadoop ssl server configuration | | hadoop-ssl-client | Change hadoop ssl client configuration | | hbase-env | Change values in HBase's environment. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbaase.properties file. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | | hbase-site | Change values in HBase's hbase-site.xml file. | | hdfs-encryption-zones | Configure HDFS encryption zones. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | | hcatalog-env | Change values in HCatalog's environment. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | | hcatalog-webhcat-log4j | Change values in HCatalog WebHCat's log4j.properties. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | | hive-env | Change values in the Hive environment. | | hive-exec-log4j | Change values in Hive's hive-exec-log4j.properties file. | | hive-log4j | Change values in Hive's hive-log4j.properties file. | | hive-site | Change values in Hive's hive-site.xml file | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | | hue-ini | Change values in Hue's ini file | | httpfs-env | Change values in the HTTPFS environment. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | | mapred-env | Change values in the MapReduce application's environment. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | | oozie-env | Change values in Oozie's environment. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | | oozie-site | Change values in Oozie's oozie-site.xml file. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | | pig-properties | Change values in Pig's pig.properties file. | | pig-log4j | Change values in Pig's log4j.properties file. | | presto-log | Change values in Presto's log.properties file. | | presto-config | Change values in Presto's config.properties file. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | | presto-connector-hive | Change values in Presto's hive.properties file. | | presto-connector-jmx | Change values in Presto's jmx.properties file. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | | presto-connector-localfile | Change values in Presto's localfile.properties file. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | | presto-connector-redis | Change values in Presto's redis.properties file. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | | spark | Amazon EMR-curated settings for Apache Spark. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | | spark-env | Change values in the Spark environment. | | spark-log4j | Change values in Spark's log4j.properties file. | | spark-metrics | Change values in Spark's metrics.properties file. | | sqoop-env | Change values in Sqoop's environment. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | | tez-site | Change values in Tez's tez-site.xml file. | | yarn-env | Change values in the YARN environment. | | yarn-site | Change values in YARN's yarn-site.xml file. | | zeppelin-env | Change values in the Zeppelin environment. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | # Amazon EMR release 4.8.0 ## 4.8.0 application versions This release includes the following applications: [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [http://gethue.com/](http://gethue.com/), [http://mahout.apache.org/](http://mahout.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-4.8.0 | emr-4.7.4 | emr-4.7.3 | emr-4.7.2 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.10.75 | 1.10.75 | 1.10.75 | 1.10.75 | | Python | Not tracked | Not tracked | Not tracked | Not tracked | | Scala | Not tracked | Not tracked | Not tracked | Not tracked | | AmazonCloudWatchAgent | - | - | - | - | | Delta | - | - | - | - | | Flink | - | - | - | - | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 1.2.2 | 1.2.1 | 1.2.1 | 1.2.1 | | HCatalog | 1.0.0-amzn-7 | 1.0.0-amzn-6 | 1.0.0-amzn-6 | 1.0.0-amzn-6 | | Hadoop | 2.7.2-amzn-4 | 2.7.2-amzn-3 | 2.7.2-amzn-3 | 2.7.2-amzn-3 | | Hive | 1.0.0-amzn-7 | 1.0.0-amzn-6 | 1.0.0-amzn-6 | 1.0.0-amzn-6 | | Hudi | - | - | - | - | | Hue | 3.7.1-amzn-7 | 3.7.1-amzn-7 | 3.7.1-amzn-7 | 3.7.1-amzn-7 | | Iceberg | - | - | - | - | | JupyterEnterpriseGateway | - | - | - | - | | JupyterHub | - | - | - | - | | Livy | - | - | - | - | | MXNet | - | - | - | - | | Mahout | 0.12.2 | 0.12.2 | 0.12.2 | 0.12.2 | | Oozie | - | - | - | - | | Oozie-Sandbox | 4.2.0 | 4.2.0 | 4.2.0 | 4.2.0 | | Phoenix | 4.7.0-HBase-1.2 | 4.7.0-HBase-1.2 | 4.7.0-HBase-1.2 | 4.7.0-HBase-1.2 | | Pig | 0.14.0-amzn-0 | 0.14.0-amzn-0 | 0.14.0-amzn-0 | 0.14.0-amzn-0 | | Presto | - | - | - | - | | Presto-Sandbox | 0.151 | 0.148 | 0.148 | 0.148 | | Spark | 1.6.2 | 1.6.2 | 1.6.2 | 1.6.2 | | Sqoop | - | - | - | - | | Sqoop-Sandbox | 1.4.6 | 1.4.6 | 1.4.6 | 1.4.6 | | TensorFlow | - | - | - | - | | Tez | 0.8.4 | 0.8.3 | 0.8.3 | 0.8.3 | | Trino (PrestoSQL) | - | - | - | - | | Zeppelin | - | - | - | - | | Zeppelin-Sandbox | 0.6.1 | 0.5.6-incubating | 0.5.6-incubating | 0.5.6-incubating | | ZooKeeper | - | - | - | - | | ZooKeeper-Sandbox | 3.4.8 | 3.4.8 | 3.4.8 | 3.4.8 | ## 4.8.0 release notes The following release notes include information for the Amazon EMR 4.8.0 release. Changes are relative to the Amazon EMR 4.7.2 release. Release date: September 7, 2016 **Upgrades** + Upgraded to HBase 1.2.2 + Upgraded to Presto-Sandbox 0.151 + Upgraded to Tez 0.8.4 + Upgraded to Zeppelin-Sandbox 0.6.1 **Changes and enhancements** + Fixed an issue in YARN where the ApplicationMaster would attempt to clean up containers that no longer exist because their instances have been terminated. + Corrected the hive-server2 URL for Hive2 actions in the Oozie examples. + Added support for additional Presto catalogs. + Backported patches: [HIVE-8948](https://issues.apache.org/jira/browse/HIVE-8948), [HIVE-12679](https://issues.apache.org/jira/browse/HIVE-12679), [HIVE-13405](https://issues.apache.org/jira/browse/HIVE-13405), [PHOENIX-3116](https://issues.apache.org/jira/browse/PHOENIX-3116), [HADOOP-12689](https://issues.apache.org/jira/browse/HADOOP-12689) + Added support for security configurations, which allow you to create and apply encryption options more easily. For more information, see [Data Encryption](https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-data-encryption.html). ## 4.8.0 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | emr-ddb | 3.2.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 2.1.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.2.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-s3-dist-cp | 2.4.0 | Distributed copy application optimized for Amazon S3. | | emrfs | 2.9.0 | Amazon S3 connector for Hadoop ecosystem applications. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 2.7.2-amzn-4 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 2.7.2-amzn-4 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 2.7.2-amzn-4 | HDFS command-line client and library | | hadoop-hdfs-namenode | 2.7.2-amzn-4 | HDFS service for tracking file names and block locations. | | hadoop-httpfs-server | 2.7.2-amzn-4 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 2.7.2-amzn-4 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 2.7.2-amzn-4 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 2.7.2-amzn-4 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 2.7.2-amzn-4 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 2.7.2-amzn-4 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 1.2.2 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 1.2.2 | Service for serving one or more HBase regions. | | hbase-client | 1.2.2 | HBase command-line client. | | hbase-rest-server | 1.2.2 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 1.2.2 | Service providing a Thrift endpoint to HBase. | | hcatalog-client | 1.0.0-amzn-7 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 1.0.0-amzn-7 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 1.0.0-amzn-7 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 1.0.0-amzn-7 | Hive command line client. | | hive-metastore-server | 1.0.0-amzn-7 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server | 1.0.0-amzn-7 | Service for accepting Hive queries as web requests. | | hue-server | 3.7.1-amzn-7 | Web application for analyzing data using Hadoop ecosystem applications | | mahout-client | 0.12.2 | Library for machine learning. | | mysql-server | 5.5.51 | MySQL database server. | | oozie-client | 4.2.0 | Oozie command-line client. | | oozie-server | 4.2.0 | Service for accepting Oozie workflow requests. | | phoenix-library | 4.7.0-HBase-1.2 | The phoenix libraries for server and client | | phoenix-query-server | 4.7.0-HBase-1.2 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.151 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.151 | Service for executing pieces of a query. | | pig-client | 0.14.0-amzn-0 | Pig command-line client. | | spark-client | 1.6.2 | Spark command-line clients. | | spark-history-server | 1.6.2 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 1.6.2 | In-memory execution engine for YARN. | | spark-yarn-slave | 1.6.2 | Apache Spark libraries needed by YARN slaves. | | sqoop-client | 1.4.6 | Apache Sqoop command-line client. | | tez-on-yarn | 0.8.4 | The tez YARN application and libraries. | | webserver | 2.4.23 | Apache HTTP server. | | zeppelin-server | 0.6.1 | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.4.8 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.4.8 | ZooKeeper command line client. | ## 4.8.0 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). **emr-4.8.0 classifications** | Classifications | Description | | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | | core-site | Change values in Hadoop's core-site.xml file. | | emrfs-site | Change EMRFS settings. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | | hadoop-ssl-server | Change hadoop ssl server configuration | | hadoop-ssl-client | Change hadoop ssl client configuration | | hbase-env | Change values in HBase's environment. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbaase.properties file. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | | hbase-site | Change values in HBase's hbase-site.xml file. | | hdfs-encryption-zones | Configure HDFS encryption zones. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | | hcatalog-env | Change values in HCatalog's environment. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | | hcatalog-webhcat-log4j | Change values in HCatalog WebHCat's log4j.properties. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | | hive-env | Change values in the Hive environment. | | hive-exec-log4j | Change values in Hive's hive-exec-log4j.properties file. | | hive-log4j | Change values in Hive's hive-log4j.properties file. | | hive-site | Change values in Hive's hive-site.xml file | | hiveserver2-site | Change values in Hive Server2's hiveserver2-site.xml file | | hue-ini | Change values in Hue's ini file | | httpfs-env | Change values in the HTTPFS environment. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | | mapred-env | Change values in the MapReduce application's environment. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | | oozie-env | Change values in Oozie's environment. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | | oozie-site | Change values in Oozie's oozie-site.xml file. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | | pig-properties | Change values in Pig's pig.properties file. | | pig-log4j | Change values in Pig's log4j.properties file. | | presto-log | Change values in Presto's log.properties file. | | presto-config | Change values in Presto's config.properties file. | | presto-connector-blackhole | Change values in Presto's blackhole.properties file. | | presto-connector-cassandra | Change values in Presto's cassandra.properties file. | | presto-connector-hive | Change values in Presto's hive.properties file. | | presto-connector-jmx | Change values in Presto's jmx.properties file. | | presto-connector-kafka | Change values in Presto's kafka.properties file. | | presto-connector-localfile | Change values in Presto's localfile.properties file. | | presto-connector-mongodb | Change values in Presto's mongodb.properties file. | | presto-connector-mysql | Change values in Presto's mysql.properties file. | | presto-connector-postgresql | Change values in Presto's postgresql.properties file. | | presto-connector-raptor | Change values in Presto's raptor.properties file. | | presto-connector-redis | Change values in Presto's redis.properties file. | | presto-connector-tpch | Change values in Presto's tpch.properties file. | | spark | Amazon EMR-curated settings for Apache Spark. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | | spark-env | Change values in the Spark environment. | | spark-log4j | Change values in Spark's log4j.properties file. | | spark-metrics | Change values in Spark's metrics.properties file. | | sqoop-env | Change values in Sqoop's environment. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | | tez-site | Change values in Tez's tez-site.xml file. | | yarn-env | Change values in the YARN environment. | | yarn-site | Change values in YARN's yarn-site.xml file. | | zeppelin-env | Change values in the Zeppelin environment. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | # Amazon EMR release 4.7.4 ## 4.7.4 application versions This release includes the following applications: [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [http://gethue.com/](http://gethue.com/), [http://mahout.apache.org/](http://mahout.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-4.7.4 | emr-4.7.3 | emr-4.7.2 | emr-4.7.1 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.10.75 | 1.10.75 | 1.10.75 | 1.10.75 | | Python | Not tracked | Not tracked | Not tracked | Not tracked | | Scala | Not tracked | Not tracked | Not tracked | Not tracked | | AmazonCloudWatchAgent | - | - | - | - | | Delta | - | - | - | - | | Flink | - | - | - | - | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 1.2.1 | 1.2.1 | 1.2.1 | 1.2.1 | | HCatalog | 1.0.0-amzn-6 | 1.0.0-amzn-6 | 1.0.0-amzn-6 | 1.0.0-amzn-5 | | Hadoop | 2.7.2-amzn-3 | 2.7.2-amzn-3 | 2.7.2-amzn-3 | 2.7.2-amzn-2 | | Hive | 1.0.0-amzn-6 | 1.0.0-amzn-6 | 1.0.0-amzn-6 | 1.0.0-amzn-5 | | Hudi | - | - | - | - | | Hue | 3.7.1-amzn-7 | 3.7.1-amzn-7 | 3.7.1-amzn-7 | 3.7.1-amzn-7 | | Iceberg | - | - | - | - | | JupyterEnterpriseGateway | - | - | - | - | | JupyterHub | - | - | - | - | | Livy | - | - | - | - | | MXNet | - | - | - | - | | Mahout | 0.12.2 | 0.12.2 | 0.12.2 | 0.12.0 | | Oozie | - | - | - | - | | Oozie-Sandbox | 4.2.0 | 4.2.0 | 4.2.0 | 4.2.0 | | Phoenix | 4.7.0-HBase-1.2 | 4.7.0-HBase-1.2 | 4.7.0-HBase-1.2 | 4.7.0-HBase-1.2 | | Pig | 0.14.0-amzn-0 | 0.14.0-amzn-0 | 0.14.0-amzn-0 | 0.14.0-amzn-0 | | Presto | - | - | - | - | | Presto-Sandbox | 0.148 | 0.148 | 0.148 | 0.147 | | Spark | 1.6.2 | 1.6.2 | 1.6.2 | 1.6.1 | | Sqoop | - | - | - | - | | Sqoop-Sandbox | 1.4.6 | 1.4.6 | 1.4.6 | 1.4.6 | | TensorFlow | - | - | - | - | | Tez | 0.8.3 | 0.8.3 | 0.8.3 | 0.8.3 | | Trino (PrestoSQL) | - | - | - | - | | Zeppelin | - | - | - | - | | Zeppelin-Sandbox | 0.5.6-incubating | 0.5.6-incubating | 0.5.6-incubating | 0.5.6-incubating | | ZooKeeper | - | - | - | - | | ZooKeeper-Sandbox | 3.4.8 | 3.4.8 | 3.4.8 | 3.4.8 | ## 4.7.4 release notes This is a patch release to add AWS Signature Version 4 authentication for requests to Amazon S3. All applications and components are the same as the previous Amazon EMR release. **Important** In this release version, Amazon EMR uses AWS Signature Version 4 exclusively to authenticate requests to Amazon S3. For more information, see [Whats New](https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-whatsnew.html). ## 4.7.4 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | emr-ddb | 3.2.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 2.1.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.2.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-s3-dist-cp | 2.4.0 | Distributed copy application optimized for Amazon S3. | | emrfs | 2.8.0 | Amazon S3 connector for Hadoop ecosystem applications. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 2.7.2-amzn-3 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 2.7.2-amzn-3 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 2.7.2-amzn-3 | HDFS command-line client and library | | hadoop-hdfs-namenode | 2.7.2-amzn-3 | HDFS service for tracking file names and block locations. | | hadoop-httpfs-server | 2.7.2-amzn-3 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 2.7.2-amzn-3 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 2.7.2-amzn-3 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 2.7.2-amzn-3 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 2.7.2-amzn-3 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 2.7.2-amzn-3 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 1.2.1 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 1.2.1 | Service for serving one or more HBase regions. | | hbase-client | 1.2.1 | HBase command-line client. | | hbase-rest-server | 1.2.1 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 1.2.1 | Service providing a Thrift endpoint to HBase. | | hcatalog-client | 1.0.0-amzn-6 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 1.0.0-amzn-6 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 1.0.0-amzn-6 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 1.0.0-amzn-6 | Hive command line client. | | hive-metastore-server | 1.0.0-amzn-6 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server | 1.0.0-amzn-6 | Service for accepting Hive queries as web requests. | | hue-server | 3.7.1-amzn-7 | Web application for analyzing data using Hadoop ecosystem applications | | mahout-client | 0.12.2 | Library for machine learning. | | mysql-server | 5.5.46 | MySQL database server. | | oozie-client | 4.2.0 | Oozie command-line client. | | oozie-server | 4.2.0 | Service for accepting Oozie workflow requests. | | phoenix-library | 4.7.0-HBase-1.2 | The phoenix libraries for server and client | | phoenix-query-server | 4.7.0-HBase-1.2 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.148 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.148 | Service for executing pieces of a query. | | pig-client | 0.14.0-amzn-0 | Pig command-line client. | | spark-client | 1.6.2 | Spark command-line clients. | | spark-history-server | 1.6.2 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 1.6.2 | In-memory execution engine for YARN. | | spark-yarn-slave | 1.6.2 | Apache Spark libraries needed by YARN slaves. | | sqoop-client | 1.4.6 | Apache Sqoop command-line client. | | tez-on-yarn | 0.8.3 | The tez YARN application and libraries. | | webserver | 2.4.23 | Apache HTTP server. | | zeppelin-server | 0.5.6-incubating | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.4.8 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.4.8 | ZooKeeper command line client. | ## 4.7.4 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). **emr-4.7.4 classifications** | Classifications | Description | | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | | core-site | Change values in Hadoop's core-site.xml file. | | emrfs-site | Change EMRFS settings. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | | hadoop-ssl-server | Change hadoop ssl server configuration | | hadoop-ssl-client | Change hadoop ssl client configuration | | hbase-env | Change values in HBase's environment. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbaase.properties file. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | | hbase-site | Change values in HBase's hbase-site.xml file. | | hdfs-encryption-zones | Configure HDFS encryption zones. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | | hcatalog-env | Change values in HCatalog's environment. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | | hcatalog-webhcat-log4j | Change values in HCatalog WebHCat's log4j.properties. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | | hive-env | Change values in the Hive environment. | | hive-exec-log4j | Change values in Hive's hive-exec-log4j.properties file. | | hive-log4j | Change values in Hive's hive-log4j.properties file. | | hive-site | Change values in Hive's hive-site.xml file | | hue-ini | Change values in Hue's ini file | | httpfs-env | Change values in the HTTPFS environment. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | | mapred-env | Change values in the MapReduce application's environment. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | | oozie-env | Change values in Oozie's environment. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | | oozie-site | Change values in Oozie's oozie-site.xml file. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | | pig-properties | Change values in Pig's pig.properties file. | | pig-log4j | Change values in Pig's log4j.properties file. | | presto-log | Change values in Presto's log.properties file. | | presto-config | Change values in Presto's config.properties file. | | presto-connector-hive | Change values in Presto's hive.properties file. | | spark | Amazon EMR-curated settings for Apache Spark. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | | spark-env | Change values in the Spark environment. | | spark-log4j | Change values in Spark's log4j.properties file. | | spark-metrics | Change values in Spark's metrics.properties file. | | sqoop-env | Change values in Sqoop's environment. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | | tez-site | Change values in Tez's tez-site.xml file. | | yarn-env | Change values in the YARN environment. | | yarn-site | Change values in YARN's yarn-site.xml file. | | zeppelin-env | Change values in the Zeppelin environment. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | # Amazon EMR release 4.7.3 ## 4.7.3 application versions This release includes the following applications: [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [http://gethue.com/](http://gethue.com/), [http://mahout.apache.org/](http://mahout.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-4.7.3 | emr-4.7.2 | emr-4.7.1 | emr-4.7.0 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.10.75 | 1.10.75 | 1.10.75 | 1.10.75 | | Python | Not tracked | Not tracked | Not tracked | Not tracked | | Scala | Not tracked | Not tracked | Not tracked | Not tracked | | AmazonCloudWatchAgent | - | - | - | - | | Delta | - | - | - | - | | Flink | - | - | - | - | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 1.2.1 | 1.2.1 | 1.2.1 | 1.2.1 | | HCatalog | 1.0.0-amzn-6 | 1.0.0-amzn-6 | 1.0.0-amzn-5 | 1.0.0-amzn-5 | | Hadoop | 2.7.2-amzn-3 | 2.7.2-amzn-3 | 2.7.2-amzn-2 | 2.7.2-amzn-2 | | Hive | 1.0.0-amzn-6 | 1.0.0-amzn-6 | 1.0.0-amzn-5 | 1.0.0-amzn-5 | | Hudi | - | - | - | - | | Hue | 3.7.1-amzn-7 | 3.7.1-amzn-7 | 3.7.1-amzn-7 | 3.7.1-amzn-7 | | Iceberg | - | - | - | - | | JupyterEnterpriseGateway | - | - | - | - | | JupyterHub | - | - | - | - | | Livy | - | - | - | - | | MXNet | - | - | - | - | | Mahout | 0.12.2 | 0.12.2 | 0.12.0 | 0.12.0 | | Oozie | - | - | - | - | | Oozie-Sandbox | 4.2.0 | 4.2.0 | 4.2.0 | 4.2.0 | | Phoenix | 4.7.0-HBase-1.2 | 4.7.0-HBase-1.2 | 4.7.0-HBase-1.2 | 4.7.0-HBase-1.2 | | Pig | 0.14.0-amzn-0 | 0.14.0-amzn-0 | 0.14.0-amzn-0 | 0.14.0-amzn-0 | | Presto | - | - | - | - | | Presto-Sandbox | 0.148 | 0.148 | 0.147 | 0.147 | | Spark | 1.6.2 | 1.6.2 | 1.6.1 | 1.6.1 | | Sqoop | - | - | - | - | | Sqoop-Sandbox | 1.4.6 | 1.4.6 | 1.4.6 | 1.4.6 | | TensorFlow | - | - | - | - | | Tez | 0.8.3 | 0.8.3 | 0.8.3 | 0.8.3 | | Trino (PrestoSQL) | - | - | - | - | | Zeppelin | - | - | - | - | | Zeppelin-Sandbox | 0.5.6-incubating | 0.5.6-incubating | 0.5.6-incubating | 0.5.6-incubating | | ZooKeeper | - | - | - | - | | ZooKeeper-Sandbox | 3.4.8 | 3.4.8 | 3.4.8 | 3.4.8 | ## 4.7.3 release notes ## 4.7.3 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | emr-ddb | 3.2.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 2.1.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.2.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-s3-dist-cp | 2.4.0 | Distributed copy application optimized for Amazon S3. | | emrfs | 2.8.0 | Amazon S3 connector for Hadoop ecosystem applications. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 2.7.2-amzn-3 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 2.7.2-amzn-3 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 2.7.2-amzn-3 | HDFS command-line client and library | | hadoop-hdfs-namenode | 2.7.2-amzn-3 | HDFS service for tracking file names and block locations. | | hadoop-httpfs-server | 2.7.2-amzn-3 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 2.7.2-amzn-3 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 2.7.2-amzn-3 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 2.7.2-amzn-3 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 2.7.2-amzn-3 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 2.7.2-amzn-3 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 1.2.1 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 1.2.1 | Service for serving one or more HBase regions. | | hbase-client | 1.2.1 | HBase command-line client. | | hbase-rest-server | 1.2.1 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 1.2.1 | Service providing a Thrift endpoint to HBase. | | hcatalog-client | 1.0.0-amzn-6 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 1.0.0-amzn-6 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 1.0.0-amzn-6 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 1.0.0-amzn-6 | Hive command line client. | | hive-metastore-server | 1.0.0-amzn-6 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server | 1.0.0-amzn-6 | Service for accepting Hive queries as web requests. | | hue-server | 3.7.1-amzn-7 | Web application for analyzing data using Hadoop ecosystem applications | | mahout-client | 0.12.2 | Library for machine learning. | | mysql-server | 5.5.46 | MySQL database server. | | oozie-client | 4.2.0 | Oozie command-line client. | | oozie-server | 4.2.0 | Service for accepting Oozie workflow requests. | | phoenix-library | 4.7.0-HBase-1.2 | The phoenix libraries for server and client | | phoenix-query-server | 4.7.0-HBase-1.2 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.148 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.148 | Service for executing pieces of a query. | | pig-client | 0.14.0-amzn-0 | Pig command-line client. | | spark-client | 1.6.2 | Spark command-line clients. | | spark-history-server | 1.6.2 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 1.6.2 | In-memory execution engine for YARN. | | spark-yarn-slave | 1.6.2 | Apache Spark libraries needed by YARN slaves. | | sqoop-client | 1.4.6 | Apache Sqoop command-line client. | | tez-on-yarn | 0.8.3 | The tez YARN application and libraries. | | webserver | 2.4.23 | Apache HTTP server. | | zeppelin-server | 0.5.6-incubating | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.4.8 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.4.8 | ZooKeeper command line client. | ## 4.7.3 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). **emr-4.7.3 classifications** | Classifications | Description | | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | | core-site | Change values in Hadoop's core-site.xml file. | | emrfs-site | Change EMRFS settings. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | | hadoop-ssl-server | Change hadoop ssl server configuration | | hadoop-ssl-client | Change hadoop ssl client configuration | | hbase-env | Change values in HBase's environment. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbaase.properties file. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | | hbase-site | Change values in HBase's hbase-site.xml file. | | hdfs-encryption-zones | Configure HDFS encryption zones. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | | hcatalog-env | Change values in HCatalog's environment. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | | hcatalog-webhcat-log4j | Change values in HCatalog WebHCat's log4j.properties. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | | hive-env | Change values in the Hive environment. | | hive-exec-log4j | Change values in Hive's hive-exec-log4j.properties file. | | hive-log4j | Change values in Hive's hive-log4j.properties file. | | hive-site | Change values in Hive's hive-site.xml file | | hue-ini | Change values in Hue's ini file | | httpfs-env | Change values in the HTTPFS environment. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | | mapred-env | Change values in the MapReduce application's environment. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | | oozie-env | Change values in Oozie's environment. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | | oozie-site | Change values in Oozie's oozie-site.xml file. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | | pig-properties | Change values in Pig's pig.properties file. | | pig-log4j | Change values in Pig's log4j.properties file. | | presto-log | Change values in Presto's log.properties file. | | presto-config | Change values in Presto's config.properties file. | | presto-connector-hive | Change values in Presto's hive.properties file. | | spark | Amazon EMR-curated settings for Apache Spark. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | | spark-env | Change values in the Spark environment. | | spark-log4j | Change values in Spark's log4j.properties file. | | spark-metrics | Change values in Spark's metrics.properties file. | | sqoop-env | Change values in Sqoop's environment. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | | tez-site | Change values in Tez's tez-site.xml file. | | yarn-env | Change values in the YARN environment. | | yarn-site | Change values in YARN's yarn-site.xml file. | | zeppelin-env | Change values in the Zeppelin environment. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | # Amazon EMR release 4.7.2 ## 4.7.2 application versions This release includes the following applications: [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [http://gethue.com/](http://gethue.com/), [http://mahout.apache.org/](http://mahout.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-4.7.2 | emr-4.7.1 | emr-4.7.0 | emr-4.6.1 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.10.75 | 1.10.75 | 1.10.75 | 1.10.27 | | Python | Not tracked | Not tracked | Not tracked | Not tracked | | Scala | Not tracked | Not tracked | Not tracked | Not tracked | | AmazonCloudWatchAgent | - | - | - | - | | Delta | - | - | - | - | | Flink | - | - | - | - | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 1.2.1 | 1.2.1 | 1.2.1 | 1.2.0 | | HCatalog | 1.0.0-amzn-6 | 1.0.0-amzn-5 | 1.0.0-amzn-5 | 1.0.0-amzn-4 | | Hadoop | 2.7.2-amzn-3 | 2.7.2-amzn-2 | 2.7.2-amzn-2 | 2.7.2-amzn-1 | | Hive | 1.0.0-amzn-6 | 1.0.0-amzn-5 | 1.0.0-amzn-5 | 1.0.0-amzn-4 | | Hudi | - | - | - | - | | Hue | 3.7.1-amzn-7 | 3.7.1-amzn-7 | 3.7.1-amzn-7 | 3.7.1-amzn-6 | | Iceberg | - | - | - | - | | JupyterEnterpriseGateway | - | - | - | - | | JupyterHub | - | - | - | - | | Livy | - | - | - | - | | MXNet | - | - | - | - | | Mahout | 0.12.2 | 0.12.0 | 0.12.0 | 0.11.1 | | Oozie | - | - | - | - | | Oozie-Sandbox | 4.2.0 | 4.2.0 | 4.2.0 | 4.2.0 | | Phoenix | 4.7.0-HBase-1.2 | 4.7.0-HBase-1.2 | 4.7.0-HBase-1.2 | - | | Pig | 0.14.0-amzn-0 | 0.14.0-amzn-0 | 0.14.0-amzn-0 | 0.14.0-amzn-0 | | Presto | - | - | - | - | | Presto-Sandbox | 0.148 | 0.147 | 0.147 | 0.143 | | Spark | 1.6.2 | 1.6.1 | 1.6.1 | 1.6.1 | | Sqoop | - | - | - | - | | Sqoop-Sandbox | 1.4.6 | 1.4.6 | 1.4.6 | 1.4.6 | | TensorFlow | - | - | - | - | | Tez | 0.8.3 | 0.8.3 | 0.8.3 | - | | Trino (PrestoSQL) | - | - | - | - | | Zeppelin | - | - | - | - | | Zeppelin-Sandbox | 0.5.6-incubating | 0.5.6-incubating | 0.5.6-incubating | 0.5.6-incubating | | ZooKeeper | - | - | - | - | | ZooKeeper-Sandbox | 3.4.8 | 3.4.8 | 3.4.8 | 3.4.8 | ## 4.7.2 release notes The following release notes include information for Amazon EMR 4.7.2. Release date: July 15, 2016 **Features** + Upgraded to Mahout 0.12.2 + Upgraded to Presto 0.148 + Upgraded to Spark 1.6.2 + You can now create an AWSCredentialsProvider for use with EMRFS using a URI as a parameter. For more information, see [Create an AWSCredentialsProvider for EMRFS](https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-plan-credentialsprovider.html). + EMRFS now allows users to configure a custom DynamoDB endpoint for their Consistent View metadata using the `fs.s3.consistent.dynamodb.endpoint` property in `emrfs-site.xml`. + Added a script in `/usr/bin` called `spark-example`, which wraps `/usr/lib/spark/spark/bin/run-example` so you can run examples directly. For instance, to run the SparkPi example that comes with the Spark distribution, you can run `spark-example SparkPi 100` from the command line or using `command-runner.jar` as a step in the API. **Known issues resolved from previous releases** + Fixed an issue where Oozie had the `spark-assembly.jar` was not in the correct location when Spark was also installed, which resulted in failure to launch Spark applications with Oozie. + Fixed an issue with Spark Log4j-based logging in YARN containers. ## 4.7.2 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | emr-ddb | 3.2.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 2.1.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.2.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-s3-dist-cp | 2.4.0 | Distributed copy application optimized for Amazon S3. | | emrfs | 2.8.0 | Amazon S3 connector for Hadoop ecosystem applications. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 2.7.2-amzn-3 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 2.7.2-amzn-3 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 2.7.2-amzn-3 | HDFS command-line client and library | | hadoop-hdfs-namenode | 2.7.2-amzn-3 | HDFS service for tracking file names and block locations. | | hadoop-httpfs-server | 2.7.2-amzn-3 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 2.7.2-amzn-3 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 2.7.2-amzn-3 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 2.7.2-amzn-3 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 2.7.2-amzn-3 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 2.7.2-amzn-3 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 1.2.1 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 1.2.1 | Service for serving one or more HBase regions. | | hbase-client | 1.2.1 | HBase command-line client. | | hbase-rest-server | 1.2.1 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 1.2.1 | Service providing a Thrift endpoint to HBase. | | hcatalog-client | 1.0.0-amzn-6 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 1.0.0-amzn-6 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 1.0.0-amzn-6 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 1.0.0-amzn-6 | Hive command line client. | | hive-metastore-server | 1.0.0-amzn-6 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server | 1.0.0-amzn-6 | Service for accepting Hive queries as web requests. | | hue-server | 3.7.1-amzn-7 | Web application for analyzing data using Hadoop ecosystem applications | | mahout-client | 0.12.2 | Library for machine learning. | | mysql-server | 5.5.46 | MySQL database server. | | oozie-client | 4.2.0 | Oozie command-line client. | | oozie-server | 4.2.0 | Service for accepting Oozie workflow requests. | | phoenix-library | 4.7.0-HBase-1.2 | The phoenix libraries for server and client | | phoenix-query-server | 4.7.0-HBase-1.2 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.148 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.148 | Service for executing pieces of a query. | | pig-client | 0.14.0-amzn-0 | Pig command-line client. | | spark-client | 1.6.2 | Spark command-line clients. | | spark-history-server | 1.6.2 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 1.6.2 | In-memory execution engine for YARN. | | spark-yarn-slave | 1.6.2 | Apache Spark libraries needed by YARN slaves. | | sqoop-client | 1.4.6 | Apache Sqoop command-line client. | | tez-on-yarn | 0.8.3 | The tez YARN application and libraries. | | webserver | 2.4.23 | Apache HTTP server. | | zeppelin-server | 0.5.6-incubating | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.4.8 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.4.8 | ZooKeeper command line client. | ## 4.7.2 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). **emr-4.7.2 classifications** | Classifications | Description | | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | | core-site | Change values in Hadoop's core-site.xml file. | | emrfs-site | Change EMRFS settings. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | | hadoop-ssl-server | Change hadoop ssl server configuration | | hadoop-ssl-client | Change hadoop ssl client configuration | | hbase-env | Change values in HBase's environment. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbaase.properties file. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | | hbase-site | Change values in HBase's hbase-site.xml file. | | hdfs-encryption-zones | Configure HDFS encryption zones. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | | hcatalog-env | Change values in HCatalog's environment. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | | hcatalog-webhcat-log4j | Change values in HCatalog WebHCat's log4j.properties. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | | hive-env | Change values in the Hive environment. | | hive-exec-log4j | Change values in Hive's hive-exec-log4j.properties file. | | hive-log4j | Change values in Hive's hive-log4j.properties file. | | hive-site | Change values in Hive's hive-site.xml file | | hue-ini | Change values in Hue's ini file | | httpfs-env | Change values in the HTTPFS environment. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | | mapred-env | Change values in the MapReduce application's environment. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | | oozie-env | Change values in Oozie's environment. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | | oozie-site | Change values in Oozie's oozie-site.xml file. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | | pig-properties | Change values in Pig's pig.properties file. | | pig-log4j | Change values in Pig's log4j.properties file. | | presto-log | Change values in Presto's log.properties file. | | presto-config | Change values in Presto's config.properties file. | | presto-connector-hive | Change values in Presto's hive.properties file. | | spark | Amazon EMR-curated settings for Apache Spark. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | | spark-env | Change values in the Spark environment. | | spark-log4j | Change values in Spark's log4j.properties file. | | spark-metrics | Change values in Spark's metrics.properties file. | | sqoop-env | Change values in Sqoop's environment. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | | tez-site | Change values in Tez's tez-site.xml file. | | yarn-env | Change values in the YARN environment. | | yarn-site | Change values in YARN's yarn-site.xml file. | | zeppelin-env | Change values in the Zeppelin environment. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | # Amazon EMR release 4.7.1 ## 4.7.1 application versions This release includes the following applications: [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [http://gethue.com/](http://gethue.com/), [http://mahout.apache.org/](http://mahout.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-4.7.1 | emr-4.7.0 | emr-4.6.1 | emr-4.6.0 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.10.75 | 1.10.75 | 1.10.27 | 1.10.27 | | Python | Not tracked | Not tracked | Not tracked | Not tracked | | Scala | Not tracked | Not tracked | Not tracked | Not tracked | | AmazonCloudWatchAgent | - | - | - | - | | Delta | - | - | - | - | | Flink | - | - | - | - | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 1.2.1 | 1.2.1 | 1.2.0 | 1.2.0 | | HCatalog | 1.0.0-amzn-5 | 1.0.0-amzn-5 | 1.0.0-amzn-4 | 1.0.0-amzn-4 | | Hadoop | 2.7.2-amzn-2 | 2.7.2-amzn-2 | 2.7.2-amzn-1 | 2.7.2-amzn-1 | | Hive | 1.0.0-amzn-5 | 1.0.0-amzn-5 | 1.0.0-amzn-4 | 1.0.0-amzn-4 | | Hudi | - | - | - | - | | Hue | 3.7.1-amzn-7 | 3.7.1-amzn-7 | 3.7.1-amzn-6 | 3.7.1-amzn-6 | | Iceberg | - | - | - | - | | JupyterEnterpriseGateway | - | - | - | - | | JupyterHub | - | - | - | - | | Livy | - | - | - | - | | MXNet | - | - | - | - | | Mahout | 0.12.0 | 0.12.0 | 0.11.1 | 0.11.1 | | Oozie | - | - | - | - | | Oozie-Sandbox | 4.2.0 | 4.2.0 | 4.2.0 | 4.2.0 | | Phoenix | 4.7.0-HBase-1.2 | 4.7.0-HBase-1.2 | - | - | | Pig | 0.14.0-amzn-0 | 0.14.0-amzn-0 | 0.14.0-amzn-0 | 0.14.0-amzn-0 | | Presto | - | - | - | - | | Presto-Sandbox | 0.147 | 0.147 | 0.143 | 0.143 | | Spark | 1.6.1 | 1.6.1 | 1.6.1 | 1.6.1 | | Sqoop | - | - | - | - | | Sqoop-Sandbox | 1.4.6 | 1.4.6 | 1.4.6 | 1.4.6 | | TensorFlow | - | - | - | - | | Tez | 0.8.3 | 0.8.3 | - | - | | Trino (PrestoSQL) | - | - | - | - | | Zeppelin | - | - | - | - | | Zeppelin-Sandbox | 0.5.6-incubating | 0.5.6-incubating | 0.5.6-incubating | 0.5.6-incubating | | ZooKeeper | - | - | - | - | | ZooKeeper-Sandbox | 3.4.8 | 3.4.8 | 3.4.8 | 3.4.8 | ## 4.7.1 release notes The following release notes include information for Amazon EMR 4.7.1. Release date: June 10, 2016 **Known issues resolved from previous releases** + Fixed an issue that extended the startup time of clusters launched in a VPC with private subnets. The bug only impacted clusters launched with the Amazon EMR 4.7.0 release. + Fixed an issue that improperly handled listing of files in Amazon EMR for clusters launched with the Amazon EMR 4.7.0 release. ## 4.7.1 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | emr-ddb | 3.1.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 2.0.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.2.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-s3-dist-cp | 2.4.0 | Distributed copy application optimized for Amazon S3. | | emrfs | 2.7.1 | Amazon S3 connector for Hadoop ecosystem applications. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 2.7.2-amzn-2 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 2.7.2-amzn-2 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 2.7.2-amzn-2 | HDFS command-line client and library | | hadoop-hdfs-namenode | 2.7.2-amzn-2 | HDFS service for tracking file names and block locations. | | hadoop-httpfs-server | 2.7.2-amzn-2 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 2.7.2-amzn-2 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 2.7.2-amzn-2 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 2.7.2-amzn-2 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 2.7.2-amzn-2 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 2.7.2-amzn-2 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 1.2.1 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 1.2.1 | Service for serving one or more HBase regions. | | hbase-client | 1.2.1 | HBase command-line client. | | hbase-rest-server | 1.2.1 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 1.2.1 | Service providing a Thrift endpoint to HBase. | | hcatalog-client | 1.0.0-amzn-5 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 1.0.0-amzn-5 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 1.0.0-amzn-5 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 1.0.0-amzn-5 | Hive command line client. | | hive-metastore-server | 1.0.0-amzn-5 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server | 1.0.0-amzn-5 | Service for accepting Hive queries as web requests. | | hue-server | 3.7.1-amzn-7 | Web application for analyzing data using Hadoop ecosystem applications | | mahout-client | 0.12.0 | Library for machine learning. | | mysql-server | 5.5.46 | MySQL database server. | | oozie-client | 4.2.0 | Oozie command-line client. | | oozie-server | 4.2.0 | Service for accepting Oozie workflow requests. | | phoenix-library | 4.7.0-HBase-1.2 | The phoenix libraries for server and client | | phoenix-query-server | 4.7.0-HBase-1.2 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.147 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.147 | Service for executing pieces of a query. | | pig-client | 0.14.0-amzn-0 | Pig command-line client. | | spark-client | 1.6.1 | Spark command-line clients. | | spark-history-server | 1.6.1 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 1.6.1 | In-memory execution engine for YARN. | | spark-yarn-slave | 1.6.1 | Apache Spark libraries needed by YARN slaves. | | sqoop-client | 1.4.6 | Apache Sqoop command-line client. | | tez-on-yarn | 0.8.3 | The tez YARN application and libraries. | | webserver | 2.4.18 | Apache HTTP server. | | zeppelin-server | 0.5.6-incubating | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.4.8 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.4.8 | ZooKeeper command line client. | ## 4.7.1 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). **emr-4.7.1 classifications** | Classifications | Description | | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | | core-site | Change values in Hadoop's core-site.xml file. | | emrfs-site | Change EMRFS settings. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | | hbase-env | Change values in HBase's environment. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbaase.properties file. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | | hbase-site | Change values in HBase's hbase-site.xml file. | | hdfs-encryption-zones | Configure HDFS encryption zones. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | | hcatalog-env | Change values in HCatalog's environment. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | | hcatalog-webhcat-log4j | Change values in HCatalog WebHCat's log4j.properties. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | | hive-env | Change values in the Hive environment. | | hive-exec-log4j | Change values in Hive's hive-exec-log4j.properties file. | | hive-log4j | Change values in Hive's hive-log4j.properties file. | | hive-site | Change values in Hive's hive-site.xml file | | hue-ini | Change values in Hue's ini file | | httpfs-env | Change values in the HTTPFS environment. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | | mapred-env | Change values in the MapReduce application's environment. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | | oozie-env | Change values in Oozie's environment. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | | oozie-site | Change values in Oozie's oozie-site.xml file. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | | pig-properties | Change values in Pig's pig.properties file. | | pig-log4j | Change values in Pig's log4j.properties file. | | presto-log | Change values in Presto's log.properties file. | | presto-config | Change values in Presto's config.properties file. | | presto-connector-hive | Change values in Presto's hive.properties file. | | spark | Amazon EMR-curated settings for Apache Spark. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | | spark-env | Change values in the Spark environment. | | spark-log4j | Change values in Spark's log4j.properties file. | | spark-metrics | Change values in Spark's metrics.properties file. | | sqoop-env | Change values in Sqoop's environment. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | | tez-site | Change values in Tez's tez-site.xml file. | | yarn-env | Change values in the YARN environment. | | yarn-site | Change values in YARN's yarn-site.xml file. | | zeppelin-env | Change values in the Zeppelin environment. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | # Amazon EMR release 4.7.0 ## 4.7.0 application versions This release includes the following applications: [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [http://gethue.com/](http://gethue.com/), [http://mahout.apache.org/](http://mahout.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [https://phoenix.apache.org/](https://phoenix.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://tez.apache.org/](https://tez.apache.org/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-4.7.0 | emr-4.6.1 | emr-4.6.0 | emr-4.5.0 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.10.75 | 1.10.27 | 1.10.27 | 1.10.27 | | Python | Not tracked | Not tracked | Not tracked | Not tracked | | Scala | Not tracked | Not tracked | Not tracked | Not tracked | | AmazonCloudWatchAgent | - | - | - | - | | Delta | - | - | - | - | | Flink | - | - | - | - | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 1.2.1 | 1.2.0 | 1.2.0 | - | | HCatalog | 1.0.0-amzn-5 | 1.0.0-amzn-4 | 1.0.0-amzn-4 | 1.0.0-amzn-4 | | Hadoop | 2.7.2-amzn-2 | 2.7.2-amzn-1 | 2.7.2-amzn-1 | 2.7.2-amzn-0 | | Hive | 1.0.0-amzn-5 | 1.0.0-amzn-4 | 1.0.0-amzn-4 | 1.0.0-amzn-4 | | Hudi | - | - | - | - | | Hue | 3.7.1-amzn-7 | 3.7.1-amzn-6 | 3.7.1-amzn-6 | 3.7.1-amzn-5 | | Iceberg | - | - | - | - | | JupyterEnterpriseGateway | - | - | - | - | | JupyterHub | - | - | - | - | | Livy | - | - | - | - | | MXNet | - | - | - | - | | Mahout | 0.12.0 | 0.11.1 | 0.11.1 | 0.11.1 | | Oozie | - | - | - | - | | Oozie-Sandbox | 4.2.0 | 4.2.0 | 4.2.0 | 4.2.0 | | Phoenix | 4.7.0-HBase-1.2 | - | - | - | | Pig | 0.14.0-amzn-0 | 0.14.0-amzn-0 | 0.14.0-amzn-0 | 0.14.0-amzn-0 | | Presto | - | - | - | - | | Presto-Sandbox | 0.147 | 0.143 | 0.143 | 0.140 | | Spark | 1.6.1 | 1.6.1 | 1.6.1 | 1.6.1 | | Sqoop | - | - | - | - | | Sqoop-Sandbox | 1.4.6 | 1.4.6 | 1.4.6 | 1.4.6 | | TensorFlow | - | - | - | - | | Tez | 0.8.3 | - | - | - | | Trino (PrestoSQL) | - | - | - | - | | Zeppelin | - | - | - | - | | Zeppelin-Sandbox | 0.5.6-incubating | 0.5.6-incubating | 0.5.6-incubating | 0.5.6-incubating | | ZooKeeper | - | - | - | - | | ZooKeeper-Sandbox | 3.4.8 | 3.4.8 | 3.4.8 | - | ## 4.7.0 release notes **Important** Amazon EMR 4.7.0 is deprecated. Use Amazon EMR 4.7.1 or later instead. Release date: June 2, 2016 **Features** + Added Apache Phoenix 4.7.0 + Added Apache Tez 0.8.3 + Upgraded to HBase 1.2.1 + Upgraded to Mahout 0.12.0 + Upgraded to Presto 0.147 + Upgraded the AWS SDK for Java to 1.10.75 + The final flag was removed from the `mapreduce.cluster.local.dir` property in `mapred-site.xml` to allow users to run Pig in local mode. + Amazon Redshift JDBC Drivers Available on Cluster Amazon Redshift JDBC drivers are now included at `/usr/share/aws/redshift/jdbc`. `/usr/share/aws/redshift/jdbc/RedshiftJDBC41.jar` is the JDBC 4.1-compatible Amazon Redshift driver and `/usr/share/aws/redshift/jdbc/RedshiftJDBC4.jar` is the JDBC 4.0-compatible Amazon Redshift driver. For more information, see [Configure a JDBC Connection](https://docs.aws.amazon.com/redshift/latest/mgmt/configure-jdbc-connection.html) in the *Amazon Redshift Management Guide*. + Java 8 Except for Presto, OpenJDK 1.7 is the default JDK used for all applications. However, both OpenJDK 1.7 and 1.8 are installed. For information about how to set `JAVA_HOME` for applications, see [Configuring Applications to Use Java 8](https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-configure-apps.html#configuring-java8). **Known issues resolved from previous releases** + Fixed a kernel issue that significantly affected performance on Throughput Optimized HDD (st1) EBS volumes for Amazon EMR in emr-4.6.0. + Fixed an issue where a cluster would fail if any HDFS encryption zone were specified without choosing Hadoop as an application. + Changed the default HDFS write policy from `RoundRobin` to `AvailableSpaceVolumeChoosingPolicy`. Some volumes were not properly utilized with the RoundRobin configuration, which resulted in failed core nodes and an unreliable HDFS. + Fixed an issue with the EMRFS CLI, which would cause an exception when creating the default DynamoDB metadata table for consistent views. + Fixed a deadlock issue in EMRFS that potentially occurred during multipart rename and copy operations. + Fixed an issue with EMRFS that caused the CopyPart size default to be 5 MB. The default is now properly set at 128 MB. + Fixed an issue with the Zeppelin upstart configuration that potentially prevented you from stopping the service. + Fixed an issue with Spark and Zeppelin, which prevented you from using the `s3a://` URI scheme because `/usr/lib/hadoop/hadoop-aws.jar` was not properly loaded in their respective classpath. + Backported [HUE-2484](https://issues.cloudera.org/browse/HUE-2484). + Backported a [commit](https://github.com/cloudera/hue/commit/c3c89f085e7a29c9fac7de016d881142d90af3eb) from Hue 3.9.0 (no JIRA exists) to fix an issue with the HBase browser sample. + Backported [HIVE-9073](https://issues.apache.org/jira/browse/HIVE-9073). ## 4.7.0 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | emr-ddb | 3.1.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 2.0.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.2.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-s3-dist-cp | 2.4.0 | Distributed copy application optimized for Amazon S3. | | emrfs | 2.7.1 | Amazon S3 connector for Hadoop ecosystem applications. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 2.7.2-amzn-2 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 2.7.2-amzn-2 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 2.7.2-amzn-2 | HDFS command-line client and library | | hadoop-hdfs-namenode | 2.7.2-amzn-2 | HDFS service for tracking file names and block locations. | | hadoop-httpfs-server | 2.7.2-amzn-2 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 2.7.2-amzn-2 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 2.7.2-amzn-2 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 2.7.2-amzn-2 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 2.7.2-amzn-2 | YARN service for allocating and managing cluster resources and distributed applications. | | hadoop-yarn-timeline-server | 2.7.2-amzn-2 | Service for retrieving current and historical information for YARN applications. | | hbase-hmaster | 1.2.1 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 1.2.1 | Service for serving one or more HBase regions. | | hbase-client | 1.2.1 | HBase command-line client. | | hbase-rest-server | 1.2.1 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 1.2.1 | Service providing a Thrift endpoint to HBase. | | hcatalog-client | 1.0.0-amzn-5 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 1.0.0-amzn-5 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 1.0.0-amzn-5 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 1.0.0-amzn-5 | Hive command line client. | | hive-metastore-server | 1.0.0-amzn-5 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server | 1.0.0-amzn-5 | Service for accepting Hive queries as web requests. | | hue-server | 3.7.1-amzn-7 | Web application for analyzing data using Hadoop ecosystem applications | | mahout-client | 0.12.0 | Library for machine learning. | | mysql-server | 5.5.46 | MySQL database server. | | oozie-client | 4.2.0 | Oozie command-line client. | | oozie-server | 4.2.0 | Service for accepting Oozie workflow requests. | | phoenix-library | 4.7.0-HBase-1.2 | The phoenix libraries for server and client | | phoenix-query-server | 4.7.0-HBase-1.2 | A light weight server providing JDBC access as well as Protocol Buffers and JSON format access to the Avatica API | | presto-coordinator | 0.147 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.147 | Service for executing pieces of a query. | | pig-client | 0.14.0-amzn-0 | Pig command-line client. | | spark-client | 1.6.1 | Spark command-line clients. | | spark-history-server | 1.6.1 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 1.6.1 | In-memory execution engine for YARN. | | spark-yarn-slave | 1.6.1 | Apache Spark libraries needed by YARN slaves. | | sqoop-client | 1.4.6 | Apache Sqoop command-line client. | | tez-on-yarn | 0.8.3 | The tez YARN application and libraries. | | webserver | 2.4.18 | Apache HTTP server. | | zeppelin-server | 0.5.6-incubating | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.4.8 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.4.8 | ZooKeeper command line client. | ## 4.7.0 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). **emr-4.7.0 classifications** | Classifications | Description | | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | | core-site | Change values in Hadoop's core-site.xml file. | | emrfs-site | Change EMRFS settings. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | | hbase-env | Change values in HBase's environment. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbaase.properties file. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | | hbase-site | Change values in HBase's hbase-site.xml file. | | hdfs-encryption-zones | Configure HDFS encryption zones. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | | hcatalog-env | Change values in HCatalog's environment. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | | hcatalog-webhcat-log4j | Change values in HCatalog WebHCat's log4j.properties. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | | hive-env | Change values in the Hive environment. | | hive-exec-log4j | Change values in Hive's hive-exec-log4j.properties file. | | hive-log4j | Change values in Hive's hive-log4j.properties file. | | hive-site | Change values in Hive's hive-site.xml file | | hue-ini | Change values in Hue's ini file | | httpfs-env | Change values in the HTTPFS environment. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | | mapred-env | Change values in the MapReduce application's environment. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | | oozie-env | Change values in Oozie's environment. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | | oozie-site | Change values in Oozie's oozie-site.xml file. | | phoenix-hbase-metrics | Change values in Phoenix's hadoop-metrics2-hbase.properties file. | | phoenix-hbase-site | Change values in Phoenix's hbase-site.xml file. | | phoenix-log4j | Change values in Phoenix's log4j.properties file. | | phoenix-metrics | Change values in Phoenix's hadoop-metrics2-phoenix.properties file. | | pig-properties | Change values in Pig's pig.properties file. | | pig-log4j | Change values in Pig's log4j.properties file. | | presto-log | Change values in Presto's log.properties file. | | presto-config | Change values in Presto's config.properties file. | | presto-connector-hive | Change values in Presto's hive.properties file. | | spark | Amazon EMR-curated settings for Apache Spark. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | | spark-env | Change values in the Spark environment. | | spark-log4j | Change values in Spark's log4j.properties file. | | spark-metrics | Change values in Spark's metrics.properties file. | | sqoop-env | Change values in Sqoop's environment. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | | tez-site | Change values in Tez's tez-site.xml file. | | yarn-env | Change values in the YARN environment. | | yarn-site | Change values in YARN's yarn-site.xml file. | | zeppelin-env | Change values in the Zeppelin environment. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | # Amazon EMR release 4.6.1 ## 4.6.1 application versions This release includes the following applications: [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [http://gethue.com/](http://gethue.com/), [http://mahout.apache.org/](http://mahout.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-4.6.1 | emr-4.6.0 | emr-4.5.0 | emr-4.4.0 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.10.27 | 1.10.27 | 1.10.27 | 1.10.27 | | Python | Not tracked | Not tracked | Not tracked | Not tracked | | Scala | Not tracked | Not tracked | Not tracked | Not tracked | | AmazonCloudWatchAgent | - | - | - | - | | Delta | - | - | - | - | | Flink | - | - | - | - | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 1.2.0 | 1.2.0 | - | - | | HCatalog | 1.0.0-amzn-4 | 1.0.0-amzn-4 | 1.0.0-amzn-4 | 1.0.0-amzn-3 | | Hadoop | 2.7.2-amzn-1 | 2.7.2-amzn-1 | 2.7.2-amzn-0 | 2.7.1-amzn-1 | | Hive | 1.0.0-amzn-4 | 1.0.0-amzn-4 | 1.0.0-amzn-4 | 1.0.0-amzn-3 | | Hudi | - | - | - | - | | Hue | 3.7.1-amzn-6 | 3.7.1-amzn-6 | 3.7.1-amzn-5 | 3.7.1-amzn-5 | | Iceberg | - | - | - | - | | JupyterEnterpriseGateway | - | - | - | - | | JupyterHub | - | - | - | - | | Livy | - | - | - | - | | MXNet | - | - | - | - | | Mahout | 0.11.1 | 0.11.1 | 0.11.1 | 0.11.1 | | Oozie | - | - | - | - | | Oozie-Sandbox | 4.2.0 | 4.2.0 | 4.2.0 | 4.2.0 | | Phoenix | - | - | - | - | | Pig | 0.14.0-amzn-0 | 0.14.0-amzn-0 | 0.14.0-amzn-0 | 0.14.0-amzn-0 | | Presto | - | - | - | - | | Presto-Sandbox | 0.143 | 0.143 | 0.140 | 0.136 | | Spark | 1.6.1 | 1.6.1 | 1.6.1 | 1.6.0 | | Sqoop | - | - | - | - | | Sqoop-Sandbox | 1.4.6 | 1.4.6 | 1.4.6 | 1.4.6 | | TensorFlow | - | - | - | - | | Tez | - | - | - | - | | Trino (PrestoSQL) | - | - | - | - | | Zeppelin | - | - | - | - | | Zeppelin-Sandbox | 0.5.6-incubating | 0.5.6-incubating | 0.5.6-incubating | 0.5.6-incubating | | ZooKeeper | - | - | - | - | | ZooKeeper-Sandbox | 3.4.8 | 3.4.8 | - | - | ## 4.6.1 release notes ## 4.6.1 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | emr-ddb | 3.0.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 2.0.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.1.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-s3-dist-cp | 2.3.0 | Distributed copy application optimized for Amazon S3. | | emrfs | 2.6.0 | Amazon S3 connector for Hadoop ecosystem applications. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 2.7.2-amzn-1 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 2.7.2-amzn-1 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 2.7.2-amzn-1 | HDFS command-line client and library | | hadoop-hdfs-namenode | 2.7.2-amzn-1 | HDFS service for tracking file names and block locations. | | hadoop-httpfs-server | 2.7.2-amzn-1 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 2.7.2-amzn-1 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 2.7.2-amzn-1 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 2.7.2-amzn-1 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 2.7.2-amzn-1 | YARN service for allocating and managing cluster resources and distributed applications. | | hbase-hmaster | 1.2.0 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 1.2.0 | Service for serving one or more HBase regions. | | hbase-client | 1.2.0 | HBase command-line client. | | hbase-rest-server | 1.2.0 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 1.2.0 | Service providing a Thrift endpoint to HBase. | | hcatalog-client | 1.0.0-amzn-4 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 1.0.0-amzn-4 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 1.0.0-amzn-4 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 1.0.0-amzn-4 | Hive command line client. | | hive-metastore-server | 1.0.0-amzn-4 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server | 1.0.0-amzn-4 | Service for accepting Hive queries as web requests. | | hue-server | 3.7.1-amzn-6 | Web application for analyzing data using Hadoop ecosystem applications | | mahout-client | 0.11.1 | Library for machine learning. | | mysql-server | 5.5 | MySQL database server. | | oozie-client | 4.2.0 | Oozie command-line client. | | oozie-server | 4.2.0 | Service for accepting Oozie workflow requests. | | presto-coordinator | 0.143 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.143 | Service for executing pieces of a query. | | pig-client | 0.14.0-amzn-0 | Pig command-line client. | | spark-client | 1.6.1 | Spark command-line clients. | | spark-history-server | 1.6.1 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 1.6.1 | In-memory execution engine for YARN. | | spark-yarn-slave | 1.6.1 | Apache Spark libraries needed by YARN slaves. | | sqoop-client | 1.4.6 | Apache Sqoop command-line client. | | webserver | 2.4 | Apache HTTP server. | | zeppelin-server | 0.5.6-incubating | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.4.8 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.4.8 | ZooKeeper command line client. | ## 4.6.1 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). **emr-4.6.1 classifications** | Classifications | Description | | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | | core-site | Change values in Hadoop's core-site.xml file. | | emrfs-site | Change EMRFS settings. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | | hbase-env | Change values in HBase's environment. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbaase.properties file. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | | hbase-site | Change values in HBase's hbase-site.xml file. | | hdfs-encryption-zones | Configure HDFS encryption zones. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | | hcatalog-env | Change values in HCatalog's environment. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | | hcatalog-webhcat-log4j | Change values in HCatalog WebHCat's log4j.properties. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | | hive-env | Change values in the Hive environment. | | hive-exec-log4j | Change values in Hive's hive-exec-log4j.properties file. | | hive-log4j | Change values in Hive's hive-log4j.properties file. | | hive-site | Change values in Hive's hive-site.xml file | | hue-ini | Change values in Hue's ini file | | httpfs-env | Change values in the HTTPFS environment. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | | mapred-env | Change values in the MapReduce application's environment. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | | oozie-env | Change values in Oozie's environment. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | | oozie-site | Change values in Oozie's oozie-site.xml file. | | pig-properties | Change values in Pig's pig.properties file. | | pig-log4j | Change values in Pig's log4j.properties file. | | presto-log | Change values in Presto's log.properties file. | | presto-config | Change values in Presto's config.properties file. | | presto-connector-hive | Change values in Presto's hive.properties file. | | spark | Amazon EMR-curated settings for Apache Spark. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | | spark-env | Change values in the Spark environment. | | spark-log4j | Change values in Spark's log4j.properties file. | | spark-metrics | Change values in Spark's metrics.properties file. | | sqoop-env | Change values in Sqoop's environment. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | | yarn-env | Change values in the YARN environment. | | yarn-site | Change values in YARN's yarn-site.xml file. | | zeppelin-env | Change values in the Zeppelin environment. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | # Amazon EMR release 4.6.0 ## 4.6.0 application versions This release includes the following applications: [http://ganglia.info](http://ganglia.info), [http://hbase.apache.org/](http://hbase.apache.org/), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [http://gethue.com/](http://gethue.com/), [http://mahout.apache.org/](http://mahout.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/), and [https://zookeeper.apache.org](https://zookeeper.apache.org). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-4.6.0 | emr-4.5.0 | emr-4.4.0 | emr-4.3.0 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.10.27 | 1.10.27 | 1.10.27 | 1.10.27 | | Python | Not tracked | Not tracked | Not tracked | Not tracked | | Scala | Not tracked | Not tracked | Not tracked | Not tracked | | AmazonCloudWatchAgent | - | - | - | - | | Delta | - | - | - | - | | Flink | - | - | - | - | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.7.2 | | HBase | 1.2.0 | - | - | - | | HCatalog | 1.0.0-amzn-4 | 1.0.0-amzn-4 | 1.0.0-amzn-3 | - | | Hadoop | 2.7.2-amzn-1 | 2.7.2-amzn-0 | 2.7.1-amzn-1 | 2.7.1-amzn-0 | | Hive | 1.0.0-amzn-4 | 1.0.0-amzn-4 | 1.0.0-amzn-3 | 1.0.0-amzn-2 | | Hudi | - | - | - | - | | Hue | 3.7.1-amzn-6 | 3.7.1-amzn-5 | 3.7.1-amzn-5 | 3.7.1-amzn-5 | | Iceberg | - | - | - | - | | JupyterEnterpriseGateway | - | - | - | - | | JupyterHub | - | - | - | - | | Livy | - | - | - | - | | MXNet | - | - | - | - | | Mahout | 0.11.1 | 0.11.1 | 0.11.1 | 0.11.0 | | Oozie | - | - | - | - | | Oozie-Sandbox | 4.2.0 | 4.2.0 | 4.2.0 | 4.2.0 | | Phoenix | - | - | - | - | | Pig | 0.14.0-amzn-0 | 0.14.0-amzn-0 | 0.14.0-amzn-0 | 0.14.0-amzn-0 | | Presto | - | - | - | - | | Presto-Sandbox | 0.143 | 0.140 | 0.136 | 0.130 | | Spark | 1.6.1 | 1.6.1 | 1.6.0 | 1.6.0 | | Sqoop | - | - | - | - | | Sqoop-Sandbox | 1.4.6 | 1.4.6 | 1.4.6 | - | | TensorFlow | - | - | - | - | | Tez | - | - | - | - | | Trino (PrestoSQL) | - | - | - | - | | Zeppelin | - | - | - | - | | Zeppelin-Sandbox | 0.5.6-incubating | 0.5.6-incubating | 0.5.6-incubating | 0.5.5-incubating-amzn-1 | | ZooKeeper | - | - | - | - | | ZooKeeper-Sandbox | 3.4.8 | - | - | - | ## 4.6.0 release notes The following release notes include information for the Amazon EMR 4.6.0 release. + Added HBase 1.2.0 + Added Zookeeper-Sandbox 3.4.8 + Upgraded to Presto-Sandbox 0.143 + Amazon EMR releases are now based on Amazon Linux 2016.03.0. For more information, see [https://aws.amazon.com/amazon-linux-ami/2016.03-release-notes/](https://aws.amazon.com/amazon-linux-ami/2016.03-release-notes/). + Issue Affecting Throughput Optimized HDD (st1) EBS Volume Types An issue in the Linux kernel versions 4.2 and above significantly affects performance on Throughput Optimized HDD (st1) EBS volumes for EMR. This release (emr-4.6.0) uses kernel version 4.4.5 and hence is impacted. Therefore, we recommend not using emr-4.6.0 if you want to use st1 EBS volumes. You can use emr-4.5.0 or prior Amazon EMR releases with st1 without impact. In addition, we provide the fix with future releases. + Python Defaults Python 3.4 is now installed by default, but Python 2.7 remains the system default. You may configure Python 3.4 as the system default using either a bootstrap action; you can use the configuration API to set PYSPARK\$1PYTHON export to `/usr/bin/python3.4` in the `spark-env` classification to affect the Python version used by PySpark. + Java 8 Except for Presto, OpenJDK 1.7 is the default JDK used for all applications. However, both OpenJDK 1.7 and 1.8 are installed. For information about how to set `JAVA_HOME` for applications, see [Configuring Applications to Use Java 8](https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-configure-apps.html#configuring-java8). **Known issues resolved from previous releases** + Fixed an issue where application provisioning would sometimes randomly fail due to a generated password. + Previously, `mysqld` was installed on all nodes. Now, it is only installed on the master instance and only if the chosen application includes `mysql-server` as a component. Currently, the following applications include the `mysql-server` component: HCatalog, Hive, Hue, Presto-Sandbox, and Sqoop-Sandbox. + Changed `yarn.scheduler.maximum-allocation-vcores` to 80 from the default of 32, which fixes an issue introduced in emr-4.4.0 that mainly occurs with Spark while using the `maximizeResourceAllocation` option in a cluster whose core instance type is one of a few large instance types that have the YARN vcores set higher than 32; namely c4.8xlarge, cc2.8xlarge, hs1.8xlarge, i2.8xlarge, m2.4xlarge, r3.8xlarge, d2.8xlarge, or m4.10xlarge were affected by this issue. + s3-dist-cp now uses EMRFS for all Amazon S3 nominations and no longer stages to a temporary HDFS directory. + Fixed an issue with exception handling for client-side encryption multipart uploads. + Added an option to allow users to change the Amazon S3 storage class. By default this setting is `STANDARD`. The `emrfs-site` configuration classification setting is `fs.s3.storageClass` and the possible values are `STANDARD`, `STANDARD_IA`, and `REDUCED_REDUNDANCY`. For more information about storage classes, see [Storage Classes](https://docs.aws.amazon.com/AmazonS3/latest/userguide/storage-class-intro.html) in the *Amazon Simple Storage Service User Guide*. ## 4.6.0 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | emr-ddb | 3.0.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 2.0.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.1.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-s3-dist-cp | 2.3.0 | Distributed copy application optimized for Amazon S3. | | emrfs | 2.6.0 | Amazon S3 connector for Hadoop ecosystem applications. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 2.7.2-amzn-1 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 2.7.2-amzn-1 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 2.7.2-amzn-1 | HDFS command-line client and library | | hadoop-hdfs-namenode | 2.7.2-amzn-1 | HDFS service for tracking file names and block locations. | | hadoop-httpfs-server | 2.7.2-amzn-1 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 2.7.2-amzn-1 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 2.7.2-amzn-1 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 2.7.2-amzn-1 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 2.7.2-amzn-1 | YARN service for allocating and managing cluster resources and distributed applications. | | hbase-hmaster | 1.2.0 | Service for an HBase cluster responsible for coordination of Regions and execution of administrative commands. | | hbase-region-server | 1.2.0 | Service for serving one or more HBase regions. | | hbase-client | 1.2.0 | HBase command-line client. | | hbase-rest-server | 1.2.0 | Service providing a RESTful HTTP endpoint for HBase. | | hbase-thrift-server | 1.2.0 | Service providing a Thrift endpoint to HBase. | | hcatalog-client | 1.0.0-amzn-4 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 1.0.0-amzn-4 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 1.0.0-amzn-4 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 1.0.0-amzn-4 | Hive command line client. | | hive-metastore-server | 1.0.0-amzn-4 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server | 1.0.0-amzn-4 | Service for accepting Hive queries as web requests. | | hue-server | 3.7.1-amzn-6 | Web application for analyzing data using Hadoop ecosystem applications | | mahout-client | 0.11.1 | Library for machine learning. | | mysql-server | 5.5 | MySQL database server. | | oozie-client | 4.2.0 | Oozie command-line client. | | oozie-server | 4.2.0 | Service for accepting Oozie workflow requests. | | presto-coordinator | 0.143 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.143 | Service for executing pieces of a query. | | pig-client | 0.14.0-amzn-0 | Pig command-line client. | | spark-client | 1.6.1 | Spark command-line clients. | | spark-history-server | 1.6.1 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 1.6.1 | In-memory execution engine for YARN. | | spark-yarn-slave | 1.6.1 | Apache Spark libraries needed by YARN slaves. | | sqoop-client | 1.4.6 | Apache Sqoop command-line client. | | webserver | 2.4 | Apache HTTP server. | | zeppelin-server | 0.5.6-incubating | Web-based notebook that enables interactive data analytics. | | zookeeper-server | 3.4.8 | Centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. | | zookeeper-client | 3.4.8 | ZooKeeper command line client. | ## 4.6.0 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). **emr-4.6.0 classifications** | Classifications | Description | | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | | core-site | Change values in Hadoop's core-site.xml file. | | emrfs-site | Change EMRFS settings. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | | hbase-env | Change values in HBase's environment. | | hbase-log4j | Change values in HBase's hbase-log4j.properties file. | | hbase-metrics | Change values in HBase's hadoop-metrics2-hbaase.properties file. | | hbase-policy | Change values in HBase's hbase-policy.xml file. | | hbase-site | Change values in HBase's hbase-site.xml file. | | hdfs-encryption-zones | Configure HDFS encryption zones. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | | hcatalog-env | Change values in HCatalog's environment. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | | hcatalog-webhcat-log4j | Change values in HCatalog WebHCat's log4j.properties. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | | hive-env | Change values in the Hive environment. | | hive-exec-log4j | Change values in Hive's hive-exec-log4j.properties file. | | hive-log4j | Change values in Hive's hive-log4j.properties file. | | hive-site | Change values in Hive's hive-site.xml file | | hue-ini | Change values in Hue's ini file | | httpfs-env | Change values in the HTTPFS environment. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | | mapred-env | Change values in the MapReduce application's environment. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | | oozie-env | Change values in Oozie's environment. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | | oozie-site | Change values in Oozie's oozie-site.xml file. | | pig-properties | Change values in Pig's pig.properties file. | | pig-log4j | Change values in Pig's log4j.properties file. | | presto-log | Change values in Presto's log.properties file. | | presto-config | Change values in Presto's config.properties file. | | presto-connector-hive | Change values in Presto's hive.properties file. | | spark | Amazon EMR-curated settings for Apache Spark. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | | spark-env | Change values in the Spark environment. | | spark-log4j | Change values in Spark's log4j.properties file. | | spark-metrics | Change values in Spark's metrics.properties file. | | sqoop-env | Change values in Sqoop's environment. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | | yarn-env | Change values in the YARN environment. | | yarn-site | Change values in YARN's yarn-site.xml file. | | zeppelin-env | Change values in the Zeppelin environment. | | zookeeper-config | Change values in ZooKeeper's zoo.cfg file. | | zookeeper-log4j | Change values in ZooKeeper's log4j.properties file. | # Amazon EMR release 4.5.0 ## 4.5.0 application versions This release includes the following applications: [http://ganglia.info](http://ganglia.info), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [http://gethue.com/](http://gethue.com/), [http://mahout.apache.org/](http://mahout.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), and [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-4.5.0 | emr-4.4.0 | emr-4.3.0 | emr-4.2.0 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.10.27 | 1.10.27 | 1.10.27 | 1.10.27 | | Python | Not tracked | Not tracked | Not tracked | Not tracked | | Scala | Not tracked | Not tracked | Not tracked | Not tracked | | AmazonCloudWatchAgent | - | - | - | - | | Delta | - | - | - | - | | Flink | - | - | - | - | | Ganglia | 3.7.2 | 3.7.2 | 3.7.2 | 3.6.0 | | HBase | - | - | - | - | | HCatalog | 1.0.0-amzn-4 | 1.0.0-amzn-3 | - | - | | Hadoop | 2.7.2-amzn-0 | 2.7.1-amzn-1 | 2.7.1-amzn-0 | 2.6.0-amzn-2 | | Hive | 1.0.0-amzn-4 | 1.0.0-amzn-3 | 1.0.0-amzn-2 | 1.0.0-amzn-1 | | Hudi | - | - | - | - | | Hue | 3.7.1-amzn-5 | 3.7.1-amzn-5 | 3.7.1-amzn-5 | 3.7.1-amzn-5 | | Iceberg | - | - | - | - | | JupyterEnterpriseGateway | - | - | - | - | | JupyterHub | - | - | - | - | | Livy | - | - | - | - | | MXNet | - | - | - | - | | Mahout | 0.11.1 | 0.11.1 | 0.11.0 | 0.11.0 | | Oozie | - | - | - | - | | Oozie-Sandbox | 4.2.0 | 4.2.0 | 4.2.0 | 4.2.0 | | Phoenix | - | - | - | - | | Pig | 0.14.0-amzn-0 | 0.14.0-amzn-0 | 0.14.0-amzn-0 | 0.14.0-amzn-0 | | Presto | - | - | - | - | | Presto-Sandbox | 0.140 | 0.136 | 0.130 | 0.125 | | Spark | 1.6.1 | 1.6.0 | 1.6.0 | 1.5.2 | | Sqoop | - | - | - | - | | Sqoop-Sandbox | 1.4.6 | 1.4.6 | - | - | | TensorFlow | - | - | - | - | | Tez | - | - | - | - | | Trino (PrestoSQL) | - | - | - | - | | Zeppelin | - | - | - | - | | Zeppelin-Sandbox | 0.5.6-incubating | 0.5.6-incubating | 0.5.5-incubating-amzn-1 | 0.5.5-incubating-amzn-0 | | ZooKeeper | - | - | - | - | | ZooKeeper-Sandbox | - | - | - | - | ## 4.5.0 release notes The following release notes include information for the Amazon EMR 4.5.0 release. Release date: April 4, 2016 **Features** + Upgraded to Spark 1.6.1 + Upgraded to Hadoop 2.7.2 + Upgraded to Presto 0.140 + Added AWS KMS support for Amazon S3 server-side encryption. **Known issues resolved from previous releases** + Fixed an issue where MySQL and Apache servers would not start after a node was rebooted. + Fixed an issue where IMPORT did not work correctly with non-partitioned tables stored in Amazon S3 + Fixed an issue with Presto where it requires the staging directory to be `/mnt/tmp` rather than `/tmp` when writing to Hive tables. ## 4.5.0 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | emr-ddb | 3.0.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 2.0.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.1.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-s3-dist-cp | 2.2.0 | Distributed copy application optimized for Amazon S3. | | emrfs | 2.5.0 | Amazon S3 connector for Hadoop ecosystem applications. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 2.7.2-amzn-0 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 2.7.2-amzn-0 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 2.7.2-amzn-0 | HDFS command-line client and library | | hadoop-hdfs-namenode | 2.7.2-amzn-0 | HDFS service for tracking file names and block locations. | | hadoop-httpfs-server | 2.7.2-amzn-0 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 2.7.2-amzn-0 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 2.7.2-amzn-0 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 2.7.2-amzn-0 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 2.7.2-amzn-0 | YARN service for allocating and managing cluster resources and distributed applications. | | hcatalog-client | 1.0.0-amzn-4 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 1.0.0-amzn-4 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 1.0.0-amzn-4 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 1.0.0-amzn-4 | Hive command line client. | | hive-metastore-server | 1.0.0-amzn-4 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server | 1.0.0-amzn-4 | Service for accepting Hive queries as web requests. | | hue-server | 3.7.1-amzn-5 | Web application for analyzing data using Hadoop ecosystem applications | | mahout-client | 0.11.1 | Library for machine learning. | | mysql-server | 5.5 | MySQL database server. | | oozie-client | 4.2.0 | Oozie command-line client. | | oozie-server | 4.2.0 | Service for accepting Oozie workflow requests. | | presto-coordinator | 0.140 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.140 | Service for executing pieces of a query. | | pig-client | 0.14.0-amzn-0 | Pig command-line client. | | spark-client | 1.6.1 | Spark command-line clients. | | spark-history-server | 1.6.1 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 1.6.1 | In-memory execution engine for YARN. | | spark-yarn-slave | 1.6.1 | Apache Spark libraries needed by YARN slaves. | | sqoop-client | 1.4.6 | Apache Sqoop command-line client. | | webserver | 2.4 | Apache HTTP server. | | zeppelin-server | 0.5.6-incubating | Web-based notebook that enables interactive data analytics. | ## 4.5.0 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). **emr-4.5.0 classifications** | Classifications | Description | | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | | core-site | Change values in Hadoop's core-site.xml file. | | emrfs-site | Change EMRFS settings. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | | hdfs-encryption-zones | Configure HDFS encryption zones. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | | hcatalog-env | Change values in HCatalog's environment. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | | hcatalog-webhcat-log4j | Change values in HCatalog WebHCat's log4j.properties. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | | hive-env | Change values in the Hive environment. | | hive-exec-log4j | Change values in Hive's hive-exec-log4j.properties file. | | hive-log4j | Change values in Hive's hive-log4j.properties file. | | hive-site | Change values in Hive's hive-site.xml file | | hue-ini | Change values in Hue's ini file | | httpfs-env | Change values in the HTTPFS environment. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | | mapred-env | Change values in the MapReduce application's environment. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | | oozie-env | Change values in Oozie's environment. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | | oozie-site | Change values in Oozie's oozie-site.xml file. | | pig-properties | Change values in Pig's pig.properties file. | | pig-log4j | Change values in Pig's log4j.properties file. | | presto-log | Change values in Presto's log.properties file. | | presto-config | Change values in Presto's config.properties file. | | presto-connector-hive | Change values in Presto's hive.properties file. | | spark | Amazon EMR-curated settings for Apache Spark. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | | spark-env | Change values in the Spark environment. | | spark-log4j | Change values in Spark's log4j.properties file. | | spark-metrics | Change values in Spark's metrics.properties file. | | sqoop-env | Change values in Sqoop's environment. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | | yarn-env | Change values in the YARN environment. | | yarn-site | Change values in YARN's yarn-site.xml file. | | zeppelin-env | Change values in the Zeppelin environment. | # Amazon EMR release 4.4.0 ## 4.4.0 application versions This release includes the following applications: [http://ganglia.info](http://ganglia.info), [https://cwiki.apache.org/confluence/display/Hive/HCatalog](https://cwiki.apache.org/confluence/display/Hive/HCatalog), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [http://gethue.com/](http://gethue.com/), [http://mahout.apache.org/](http://mahout.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), [http://sqoop.apache.org/](http://sqoop.apache.org/), and [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-4.4.0 | emr-4.3.0 | emr-4.2.0 | emr-4.1.0 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.10.27 | 1.10.27 | 1.10.27 | Not tracked | | Python | Not tracked | Not tracked | Not tracked | Not tracked | | Scala | Not tracked | Not tracked | Not tracked | Not tracked | | AmazonCloudWatchAgent | - | - | - | - | | Delta | - | - | - | - | | Flink | - | - | - | - | | Ganglia | 3.7.2 | 3.7.2 | 3.6.0 | - | | HBase | - | - | - | - | | HCatalog | 1.0.0-amzn-3 | - | - | - | | Hadoop | 2.7.1-amzn-1 | 2.7.1-amzn-0 | 2.6.0-amzn-2 | 2.6.0-amzn-1 | | Hive | 1.0.0-amzn-3 | 1.0.0-amzn-2 | 1.0.0-amzn-1 | 1.0.0-amzn-1 | | Hudi | - | - | - | - | | Hue | 3.7.1-amzn-5 | 3.7.1-amzn-5 | 3.7.1-amzn-5 | 3.7.1-amzn-4 | | Iceberg | - | - | - | - | | JupyterEnterpriseGateway | - | - | - | - | | JupyterHub | - | - | - | - | | Livy | - | - | - | - | | MXNet | - | - | - | - | | Mahout | 0.11.1 | 0.11.0 | 0.11.0 | 0.11.0 | | Oozie | - | - | - | - | | Oozie-Sandbox | 4.2.0 | 4.2.0 | 4.2.0 | 4.0.1 | | Phoenix | - | - | - | - | | Pig | 0.14.0-amzn-0 | 0.14.0-amzn-0 | 0.14.0-amzn-0 | 0.14.0-amzn-0 | | Presto | - | - | - | - | | Presto-Sandbox | 0.136 | 0.130 | 0.125 | 0.119 | | Spark | 1.6.0 | 1.6.0 | 1.5.2 | 1.5.0 | | Sqoop | - | - | - | - | | Sqoop-Sandbox | 1.4.6 | - | - | - | | TensorFlow | - | - | - | - | | Tez | - | - | - | - | | Trino (PrestoSQL) | - | - | - | - | | Zeppelin | - | - | - | - | | Zeppelin-Sandbox | 0.5.6-incubating | 0.5.5-incubating-amzn-1 | 0.5.5-incubating-amzn-0 | 0.6.0-incubating-SNAPSHOT | | ZooKeeper | - | - | - | - | | ZooKeeper-Sandbox | - | - | - | - | ## 4.4.0 release notes The following release notes include information for the Amazon EMR 4.4.0 release. Release date: March 14, 2016 **Features** + Added HCatalog 1.0.0 + Added Sqoop-Sandbox 1.4.6 + Upgraded to Presto 0.136 + Upgraded to Zeppelin 0.5.6 + Upgraded to Mahout 0.11.1 + Enabled `dynamicResourceAllocation` by default. + Added a table of all configuration classifications for the release. For more information, see the Configuration Classifications table in [Configuring Applications](https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-configure-apps.html). **Known issues resolved from previous releases** + Fixed an issue where the `maximizeResourceAllocation` setting would not reserve enough memory for YARN ApplicationMaster daemons. + Fixed an issue encountered with a custom DNS. If any entries in `resolve.conf` precede the custom entries provided, then the custom entries are not resolvable. This behavior was affected by clusters in a VPC where the default VPC name server is inserted as the top entry in `resolve.conf`. + Fixed an issue where the default Python moved to version 2.7 and boto was not installed for that version. + Fixed an issue where YARN containers and Spark applications would generate a unique Ganglia round robin database (rrd) file, which resulted in the first disk attached to the instance filling up. Because of this fix, YARN container level metrics have been disabled and Spark application level metrics have been disabled. + Fixed an issue in log pusher where it would delete all empty log folders. The effect was that the Hive CLI was not able to log because log pusher was removing the empty `user` folder under `/var/log/hive`. + Fixed an issue affecting Hive imports, which affected partitioning and resulted in an error during import. + Fixed an issue where EMRFS and s3-dist-cp did not properly handle bucket names that contain periods. + Changed a behavior in EMRFS so that in versioning-enabled buckets the `_$folder$` marker file is not continuously created, which may contribute to improved performance for versioning-enabled buckets. + Changed the behavior in EMRFS such that it does not use instruction files except for cases where client-side encryption is enabled. If you want to delete instruction files while using client-side encryption, you can set the emrfs-site.xml property, `fs.s3.cse.cryptoStorageMode.deleteInstructionFiles.enabled`, to true. + Changed YARN log aggregation to retain logs at the aggregation destination for two days. The default destination is your cluster HDFS storage. If you want to change this duration, change the value of `yarn.log-aggregation.retain-seconds` using the `yarn-site` configuration classification when you create your cluster. As always, you can save your application logs to Amazon S3 using the `log-uri` parameter when you create your cluster. **Patches Applied** + [HIVE-9655](https://issues.apache.org/jira/browse/HIVE-9655) + [HIVE-9183](https://issues.apache.org/jira/browse/HIVE-9183) + [HADOOP-12810](https://issues.apache.org/jira/browse/HADOOP-12810) ## 4.4.0 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | emr-ddb | 3.0.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 2.0.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.1.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-s3-dist-cp | 2.2.0 | Distributed copy application optimized for Amazon S3. | | emrfs | 2.4.0 | Amazon S3 connector for Hadoop ecosystem applications. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 2.7.1-amzn-1 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 2.7.1-amzn-1 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 2.7.1-amzn-1 | HDFS command-line client and library | | hadoop-hdfs-namenode | 2.7.1-amzn-1 | HDFS service for tracking file names and block locations. | | hadoop-httpfs-server | 2.7.1-amzn-1 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 2.7.1-amzn-1 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 2.7.1-amzn-1 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 2.7.1-amzn-1 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 2.7.1-amzn-1 | YARN service for allocating and managing cluster resources and distributed applications. | | hcatalog-client | 1.0.0-amzn-3 | The 'hcat' command line client for manipulating hcatalog-server. | | hcatalog-server | 1.0.0-amzn-3 | Service providing HCatalog, a table and storage management layer for distributed applications. | | hcatalog-webhcat-server | 1.0.0-amzn-3 | HTTP endpoint providing a REST interface to HCatalog. | | hive-client | 1.0.0-amzn-3 | Hive command line client. | | hive-metastore-server | 1.0.0-amzn-3 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server | 1.0.0-amzn-3 | Service for accepting Hive queries as web requests. | | hue-server | 3.7.1-amzn-5 | Web application for analyzing data using Hadoop ecosystem applications | | mahout-client | 0.11.1 | Library for machine learning. | | mysql-server | 5.5 | MySQL database server. | | oozie-client | 4.2.0 | Oozie command-line client. | | oozie-server | 4.2.0 | Service for accepting Oozie workflow requests. | | presto-coordinator | 0.136 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.136 | Service for executing pieces of a query. | | pig-client | 0.14.0-amzn-0 | Pig command-line client. | | spark-client | 1.6.0 | Spark command-line clients. | | spark-history-server | 1.6.0 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 1.6.0 | In-memory execution engine for YARN. | | spark-yarn-slave | 1.6.0 | Apache Spark libraries needed by YARN slaves. | | sqoop-client | 1.4.6 | Apache Sqoop command-line client. | | webserver | 2.4 | Apache HTTP server. | | zeppelin-server | 0.5.6-incubating | Web-based notebook that enables interactive data analytics. | ## 4.4.0 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). **emr-4.4.0 classifications** | Classifications | Description | | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | | core-site | Change values in Hadoop's core-site.xml file. | | emrfs-site | Change EMRFS settings. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | | hdfs-encryption-zones | Configure HDFS encryption zones. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | | hcatalog-env | Change values in HCatalog's environment. | | hcatalog-server-jndi | Change values in HCatalog's jndi.properties. | | hcatalog-server-proto-hive-site | Change values in HCatalog's proto-hive-site.xml. | | hcatalog-webhcat-env | Change values in HCatalog WebHCat's environment. | | hcatalog-webhcat-log4j | Change values in HCatalog WebHCat's log4j.properties. | | hcatalog-webhcat-site | Change values in HCatalog WebHCat's webhcat-site.xml file. | | hive-env | Change values in the Hive environment. | | hive-exec-log4j | Change values in Hive's hive-exec-log4j.properties file. | | hive-log4j | Change values in Hive's hive-log4j.properties file. | | hive-site | Change values in Hive's hive-site.xml file | | hue-ini | Change values in Hue's ini file | | httpfs-env | Change values in the HTTPFS environment. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | | mapred-env | Change values in the MapReduce application's environment. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | | oozie-env | Change values in Oozie's environment. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | | oozie-site | Change values in Oozie's oozie-site.xml file. | | pig-properties | Change values in Pig's pig.properties file. | | pig-log4j | Change values in Pig's log4j.properties file. | | presto-log | Change values in Presto's log.properties file. | | presto-config | Change values in Presto's config.properties file. | | presto-connector-hive | Change values in Presto's hive.properties file. | | spark | Amazon EMR-curated settings for Apache Spark. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | | spark-env | Change values in the Spark environment. | | spark-log4j | Change values in Spark's log4j.properties file. | | spark-metrics | Change values in Spark's metrics.properties file. | | sqoop-env | Change values in Sqoop's environment. | | sqoop-oraoop-site | Change values in Sqoop OraOop's oraoop-site.xml file. | | sqoop-site | Change values in Sqoop's sqoop-site.xml file. | | yarn-env | Change values in the YARN environment. | | yarn-site | Change values in YARN's yarn-site.xml file. | | zeppelin-env | Change values in the Zeppelin environment. | # Amazon EMR release 4.3.0 ## 4.3.0 application versions This release includes the following applications: [http://ganglia.info](http://ganglia.info), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [http://gethue.com/](http://gethue.com/), [http://mahout.apache.org/](http://mahout.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), and [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-4.3.0 | emr-4.2.0 | emr-4.1.0 | emr-4.0.0 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.10.27 | 1.10.27 | Not tracked | Not tracked | | Python | Not tracked | Not tracked | Not tracked | Not tracked | | Scala | Not tracked | Not tracked | Not tracked | Not tracked | | AmazonCloudWatchAgent | - | - | - | - | | Delta | - | - | - | - | | Flink | - | - | - | - | | Ganglia | 3.7.2 | 3.6.0 | - | - | | HBase | - | - | - | - | | HCatalog | - | - | - | - | | Hadoop | 2.7.1-amzn-0 | 2.6.0-amzn-2 | 2.6.0-amzn-1 | 2.6.0-amzn-0 | | Hive | 1.0.0-amzn-2 | 1.0.0-amzn-1 | 1.0.0-amzn-1 | 1.0.0-amzn-0 | | Hudi | - | - | - | - | | Hue | 3.7.1-amzn-5 | 3.7.1-amzn-5 | 3.7.1-amzn-4 | - | | Iceberg | - | - | - | - | | JupyterEnterpriseGateway | - | - | - | - | | JupyterHub | - | - | - | - | | Livy | - | - | - | - | | MXNet | - | - | - | - | | Mahout | 0.11.0 | 0.11.0 | 0.11.0 | 0.10.0 | | Oozie | - | - | - | - | | Oozie-Sandbox | 4.2.0 | 4.2.0 | 4.0.1 | - | | Phoenix | - | - | - | - | | Pig | 0.14.0-amzn-0 | 0.14.0-amzn-0 | 0.14.0-amzn-0 | 0.14.0-amzn-0 | | Presto | - | - | - | - | | Presto-Sandbox | 0.130 | 0.125 | 0.119 | - | | Spark | 1.6.0 | 1.5.2 | 1.5.0 | 1.4.1 | | Sqoop | - | - | - | - | | Sqoop-Sandbox | - | - | - | - | | TensorFlow | - | - | - | - | | Tez | - | - | - | - | | Trino (PrestoSQL) | - | - | - | - | | Zeppelin | - | - | - | - | | Zeppelin-Sandbox | 0.5.5-incubating-amzn-1 | 0.5.5-incubating-amzn-0 | 0.6.0-incubating-SNAPSHOT | - | | ZooKeeper | - | - | - | - | | ZooKeeper-Sandbox | - | - | - | - | ## 4.3.0 release notes The following release notes include information for the Amazon EMR 4.3.0 release. Release date: January 19, 2016 **Features** + Upgraded to Hadoop 2.7.1 + Upgraded to Spark 1.6.0 + Upgraded Ganglia to 3.7.2 + Upgraded Presto to 0.130 + Amazon EMR made some changes to `spark.dynamicAllocation.enabled` when it is set to true; it is false by default. When set to true, this affects the defaults set by the `maximizeResourceAllocation` setting: + If `spark.dynamicAllocation.enabled` is set to true, `spark.executor.instances` is not set by `maximizeResourceAllocation`. + The `spark.driver.memory` setting is now configured based on the instance types in the cluster in a similar way to how `spark.executors.memory` is set. However, because the Spark driver application may run on either the master or one of the core instances (for example, in YARN client and cluster modes, respectively), the `spark.driver.memory` setting is set based on the instance type of the smaller instance type between these two instance groups. + The `spark.default.parallelism` setting is now set at twice the number of CPU cores available for YARN containers. In previous releases, this was half that value. + The calculations for the memory overhead reserved for Spark YARN processes were adjusted to be more accurate, resulting in a small increase in the total amount of memory available to Spark (that is, `spark.executor.memory`). **Known issues resolved from previous releases** + YARN log aggregation is now enabled by default. + Fixed an issue where logs would not be pushed to Amazon S3 logs bucket for the cluster when YARN log aggregation was enabled. + YARN container sizes now have a new minimum of 32 across all node types. + Fixed an issue with Ganglia that caused excessive disk I/O on the primary node in large clusters. + Fixed an issue that prevented applications logs from being pushed to Amazon S3 when a cluster is shutting down. + Fixed an issue in EMRFS CLI that caused certain commands to fail. + Fixed an issue with Zeppelin that prevented dependencies from being loaded in the underlying SparkContext. + Fixed an issue that resulted from issuing a resize attempting to add instances. + Fixed an issue in Hive where CREATE TABLE AS SELECT makes excessive list calls to Amazon S3. + Fixed an issue where large clusters would not provision properly when Hue, Oozie, and Ganglia are installed. + Fixed an issue in s3-dist-cp where it would return a zero exit code even if it failed with an error. ## 4.3.0 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | emr-ddb | 3.0.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 2.0.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.1.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-s3-dist-cp | 2.1.0 | Distributed copy application optimized for Amazon S3. | | emrfs | 2.3.0 | Amazon S3 connector for Hadoop ecosystem applications. | | ganglia-monitor | 3.7.2 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.7.2 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.7.1 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 2.7.1-amzn-0 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 2.7.1-amzn-0 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 2.7.1-amzn-0 | HDFS command-line client and library | | hadoop-hdfs-namenode | 2.7.1-amzn-0 | HDFS service for tracking file names and block locations. | | hadoop-httpfs-server | 2.7.1-amzn-0 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 2.7.1-amzn-0 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 2.7.1-amzn-0 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 2.7.1-amzn-0 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 2.7.1-amzn-0 | YARN service for allocating and managing cluster resources and distributed applications. | | hive-client | 1.0.0-amzn-2 | Hive command line client. | | hive-metastore-server | 1.0.0-amzn-2 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server | 1.0.0-amzn-2 | Service for accepting Hive queries as web requests. | | hue-server | 3.7.1-amzn-5 | Web application for analyzing data using Hadoop ecosystem applications | | mahout-client | 0.11.0 | Library for machine learning. | | mysql-server | 5.5 | MySQL database server. | | oozie-client | 4.2.0 | Oozie command-line client. | | oozie-server | 4.2.0 | Service for accepting Oozie workflow requests. | | presto-coordinator | 0.130 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.130 | Service for executing pieces of a query. | | pig-client | 0.14.0-amzn-0 | Pig command-line client. | | spark-client | 1.6.0 | Spark command-line clients. | | spark-history-server | 1.6.0 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 1.6.0 | In-memory execution engine for YARN. | | spark-yarn-slave | 1.6.0 | Apache Spark libraries needed by YARN slaves. | | webserver | 2.4 | Apache HTTP server. | | zeppelin-server | 0.5.5-incubating-amzn-1 | Web-based notebook that enables interactive data analytics. | ## 4.3.0 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). **emr-4.3.0 classifications** | Classifications | Description | | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | | core-site | Change values in Hadoop's core-site.xml file. | | emrfs-site | Change EMRFS settings. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | | hdfs-encryption-zones | Configure HDFS encryption zones. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | | hive-env | Change values in the Hive environment. | | hive-exec-log4j | Change values in Hive's hive-exec-log4j.properties file. | | hive-log4j | Change values in Hive's hive-log4j.properties file. | | hive-site | Change values in Hive's hive-site.xml file | | hue-ini | Change values in Hue's ini file | | httpfs-env | Change values in the HTTPFS environment. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | | mapred-env | Change values in the MapReduce application's environment. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | | oozie-env | Change values in Oozie's environment. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | | oozie-site | Change values in Oozie's oozie-site.xml file. | | pig-properties | Change values in Pig's pig.properties file. | | pig-log4j | Change values in Pig's log4j.properties file. | | presto-log | Change values in Presto's log.properties file. | | presto-config | Change values in Presto's config.properties file. | | presto-connector-hive | Change values in Presto's hive.properties file. | | spark | Amazon EMR-curated settings for Apache Spark. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | | spark-env | Change values in the Spark environment. | | spark-log4j | Change values in Spark's log4j.properties file. | | spark-metrics | Change values in Spark's metrics.properties file. | | yarn-env | Change values in the YARN environment. | | yarn-site | Change values in YARN's yarn-site.xml file. | | zeppelin-env | Change values in the Zeppelin environment. | # Amazon EMR release 4.2.0 ## 4.2.0 application versions This release includes the following applications: [http://ganglia.info](http://ganglia.info), [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [http://gethue.com/](http://gethue.com/), [http://mahout.apache.org/](http://mahout.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), and [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-4.3.0 | emr-4.2.0 | emr-4.1.0 | emr-4.0.0 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.10.27 | 1.10.27 | Not tracked | Not tracked | | Python | Not tracked | Not tracked | Not tracked | Not tracked | | Scala | Not tracked | Not tracked | Not tracked | Not tracked | | AmazonCloudWatchAgent | - | - | - | - | | Delta | - | - | - | - | | Flink | - | - | - | - | | Ganglia | 3.7.2 | 3.6.0 | - | - | | HBase | - | - | - | - | | HCatalog | - | - | - | - | | Hadoop | 2.7.1-amzn-0 | 2.6.0-amzn-2 | 2.6.0-amzn-1 | 2.6.0-amzn-0 | | Hive | 1.0.0-amzn-2 | 1.0.0-amzn-1 | 1.0.0-amzn-1 | 1.0.0-amzn-0 | | Hudi | - | - | - | - | | Hue | 3.7.1-amzn-5 | 3.7.1-amzn-5 | 3.7.1-amzn-4 | - | | Iceberg | - | - | - | - | | JupyterEnterpriseGateway | - | - | - | - | | JupyterHub | - | - | - | - | | Livy | - | - | - | - | | MXNet | - | - | - | - | | Mahout | 0.11.0 | 0.11.0 | 0.11.0 | 0.10.0 | | Oozie | - | - | - | - | | Oozie-Sandbox | 4.2.0 | 4.2.0 | 4.0.1 | - | | Phoenix | - | - | - | - | | Pig | 0.14.0-amzn-0 | 0.14.0-amzn-0 | 0.14.0-amzn-0 | 0.14.0-amzn-0 | | Presto | - | - | - | - | | Presto-Sandbox | 0.130 | 0.125 | 0.119 | - | | Spark | 1.6.0 | 1.5.2 | 1.5.0 | 1.4.1 | | Sqoop | - | - | - | - | | Sqoop-Sandbox | - | - | - | - | | TensorFlow | - | - | - | - | | Tez | - | - | - | - | | Trino (PrestoSQL) | - | - | - | - | | Zeppelin | - | - | - | - | | Zeppelin-Sandbox | 0.5.5-incubating-amzn-1 | 0.5.5-incubating-amzn-0 | 0.6.0-incubating-SNAPSHOT | - | | ZooKeeper | - | - | - | - | | ZooKeeper-Sandbox | - | - | - | - | ## 4.2.0 release notes The following release notes include information for the Amazon EMR 4.2.0 release. Release date: November 18, 2015 **Features** + Added Ganglia support + Upgraded to Spark 1.5.2 + Upgraded to Presto 0.125 + Upgraded Oozie to 4.2.0 + Upgraded Zeppelin to 0.5.5 + Upgraded the AWS SDK for Java to 1.10.27 **Known issues resolved from previous releases** + Fixed an issue with the EMRFS CLI where it did not use the default metadata table name. + Fixed an issue encountered when using ORC-backed tables in Amazon S3. + Fixed an issue encountered with a Python version mismatch in the Spark configuration. + Fixed an issue when a YARN node status fails to report because of DNS issues for clusters in a VPC. + Fixed an issue encountered when YARN decommissioned nodes, resulting in hung applications or the inability to schedule new applications. + Fixed an issue encountered when clusters terminated with status TIMED\$1OUT\$1STARTING. + Fixed an issue encountered when including the EMRFS Scala dependency in other builds. The Scala dependency has been removed. ## 4.2.0 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | emr-ddb | 3.0.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 2.0.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.1.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-s3-dist-cp | 2.0.0 | Distributed copy application optimized for Amazon S3. | | emrfs | 2.2.0 | Amazon S3 connector for Hadoop ecosystem applications. | | ganglia-monitor | 3.6.0 | Embedded Ganglia agent for Hadoop ecosystem applications along with the Ganglia monitoring agent. | | ganglia-metadata-collector | 3.6.0 | Ganglia metadata collector for aggregating metrics from Ganglia monitoring agents. | | ganglia-web | 3.5.10 | Web application for viewing metrics collected by the Ganglia metadata collector. | | hadoop-client | 2.6.0-amzn-2 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 2.6.0-amzn-2 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 2.6.0-amzn-2 | HDFS command-line client and library | | hadoop-hdfs-namenode | 2.6.0-amzn-2 | HDFS service for tracking file names and block locations. | | hadoop-httpfs-server | 2.6.0-amzn-2 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 2.6.0-amzn-2 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 2.6.0-amzn-2 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 2.6.0-amzn-2 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 2.6.0-amzn-2 | YARN service for allocating and managing cluster resources and distributed applications. | | hive-client | 1.0.0-amzn-1 | Hive command line client. | | hive-metastore-server | 1.0.0-amzn-1 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server | 1.0.0-amzn-1 | Service for accepting Hive queries as web requests. | | hue-server | 3.7.1-amzn-5 | Web application for analyzing data using Hadoop ecosystem applications | | mahout-client | 0.11.0 | Library for machine learning. | | mysql-server | 5.5 | MySQL database server. | | oozie-client | 4.2.0 | Oozie command-line client. | | oozie-server | 4.2.0 | Service for accepting Oozie workflow requests. | | presto-coordinator | 0.125 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.125 | Service for executing pieces of a query. | | pig-client | 0.14.0-amzn-0 | Pig command-line client. | | spark-client | 1.5.2 | Spark command-line clients. | | spark-history-server | 1.5.2 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 1.5.2 | In-memory execution engine for YARN. | | spark-yarn-slave | 1.5.2 | Apache Spark libraries needed by YARN slaves. | | webserver | 2.4 | Apache HTTP server. | | zeppelin-server | 0.5.5-incubating-amzn-0 | Web-based notebook that enables interactive data analytics. | ## 4.2.0 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). **emr-4.2.0 classifications** | Classifications | Description | | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | | core-site | Change values in Hadoop's core-site.xml file. | | emrfs-site | Change EMRFS settings. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | | hdfs-encryption-zones | Configure HDFS encryption zones. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | | hive-env | Change values in the Hive environment. | | hive-exec-log4j | Change values in Hive's hive-exec-log4j.properties file. | | hive-log4j | Change values in Hive's hive-log4j.properties file. | | hive-site | Change values in Hive's hive-site.xml file | | hue-ini | Change values in Hue's ini file | | httpfs-env | Change values in the HTTPFS environment. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | | mapred-env | Change values in the MapReduce application's environment. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | | oozie-env | Change values in Oozie's environment. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | | oozie-site | Change values in Oozie's oozie-site.xml file. | | pig-properties | Change values in Pig's pig.properties file. | | pig-log4j | Change values in Pig's log4j.properties file. | | presto-log | Change values in Presto's log.properties file. | | presto-config | Change values in Presto's config.properties file. | | presto-connector-hive | Change values in Presto's hive.properties file. | | spark | Amazon EMR-curated settings for Apache Spark. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | | spark-env | Change values in the Spark environment. | | spark-log4j | Change values in Spark's log4j.properties file. | | spark-metrics | Change values in Spark's metrics.properties file. | | yarn-env | Change values in the YARN environment. | | yarn-site | Change values in YARN's yarn-site.xml file. | | zeppelin-env | Change values in the Zeppelin environment. | # Amazon EMR release 4.1.0 ## 4.1.0 application versions This release includes the following applications: [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [http://gethue.com/](http://gethue.com/), [http://mahout.apache.org/](http://mahout.apache.org/), [http://oozie.apache.org/](http://oozie.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), [https://prestodb.io/](https://prestodb.io/), [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/), and [https://zeppelin.incubator.apache.org/](https://zeppelin.incubator.apache.org/). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-4.3.0 | emr-4.2.0 | emr-4.1.0 | emr-4.0.0 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.10.27 | 1.10.27 | Not tracked | Not tracked | | Python | Not tracked | Not tracked | Not tracked | Not tracked | | Scala | Not tracked | Not tracked | Not tracked | Not tracked | | AmazonCloudWatchAgent | - | - | - | - | | Delta | - | - | - | - | | Flink | - | - | - | - | | Ganglia | 3.7.2 | 3.6.0 | - | - | | HBase | - | - | - | - | | HCatalog | - | - | - | - | | Hadoop | 2.7.1-amzn-0 | 2.6.0-amzn-2 | 2.6.0-amzn-1 | 2.6.0-amzn-0 | | Hive | 1.0.0-amzn-2 | 1.0.0-amzn-1 | 1.0.0-amzn-1 | 1.0.0-amzn-0 | | Hudi | - | - | - | - | | Hue | 3.7.1-amzn-5 | 3.7.1-amzn-5 | 3.7.1-amzn-4 | - | | Iceberg | - | - | - | - | | JupyterEnterpriseGateway | - | - | - | - | | JupyterHub | - | - | - | - | | Livy | - | - | - | - | | MXNet | - | - | - | - | | Mahout | 0.11.0 | 0.11.0 | 0.11.0 | 0.10.0 | | Oozie | - | - | - | - | | Oozie-Sandbox | 4.2.0 | 4.2.0 | 4.0.1 | - | | Phoenix | - | - | - | - | | Pig | 0.14.0-amzn-0 | 0.14.0-amzn-0 | 0.14.0-amzn-0 | 0.14.0-amzn-0 | | Presto | - | - | - | - | | Presto-Sandbox | 0.130 | 0.125 | 0.119 | - | | Spark | 1.6.0 | 1.5.2 | 1.5.0 | 1.4.1 | | Sqoop | - | - | - | - | | Sqoop-Sandbox | - | - | - | - | | TensorFlow | - | - | - | - | | Tez | - | - | - | - | | Trino (PrestoSQL) | - | - | - | - | | Zeppelin | - | - | - | - | | Zeppelin-Sandbox | 0.5.5-incubating-amzn-1 | 0.5.5-incubating-amzn-0 | 0.6.0-incubating-SNAPSHOT | - | | ZooKeeper | - | - | - | - | | ZooKeeper-Sandbox | - | - | - | - | ## 4.1.0 release notes ## 4.1.0 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | emr-ddb | 3.0.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 2.0.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.1.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-s3-dist-cp | 2.0.0 | Distributed copy application optimized for Amazon S3. | | emrfs | 2.1.0 | Amazon S3 connector for Hadoop ecosystem applications. | | hadoop-client | 2.6.0-amzn-1 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 2.6.0-amzn-1 | HDFS node-level service for storing blocks. | | hadoop-hdfs-library | 2.6.0-amzn-1 | HDFS command-line client and library | | hadoop-hdfs-namenode | 2.6.0-amzn-1 | HDFS service for tracking file names and block locations. | | hadoop-httpfs-server | 2.6.0-amzn-1 | HTTP endpoint for HDFS operations. | | hadoop-kms-server | 2.6.0-amzn-1 | Cryptographic key management server based on Hadoop's KeyProvider API. | | hadoop-mapred | 2.6.0-amzn-1 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 2.6.0-amzn-1 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 2.6.0-amzn-1 | YARN service for allocating and managing cluster resources and distributed applications. | | hive-client | 1.0.0-amzn-1 | Hive command line client. | | hive-metastore-server | 1.0.0-amzn-1 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server | 1.0.0-amzn-1 | Service for accepting Hive queries as web requests. | | hue-server | 3.7.1-amzn-4 | Web application for analyzing data using Hadoop ecosystem applications | | mahout-client | 0.11.0 | Library for machine learning. | | mysql-server | 5.5 | MySQL database server. | | oozie-client | 4.0.1 | Oozie command-line client. | | oozie-server | 4.0.1 | Service for accepting Oozie workflow requests. | | presto-coordinator | 0.119 | Service for accepting queries and managing query execution among presto-workers. | | presto-worker | 0.119 | Service for executing pieces of a query. | | pig-client | 0.14.0-amzn-0 | Pig command-line client. | | spark-client | 1.5.0 | Spark command-line clients. | | spark-history-server | 1.5.0 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 1.5.0 | In-memory execution engine for YARN. | | spark-yarn-slave | 1.5.0 | Apache Spark libraries needed by YARN slaves. | | zeppelin-server | 0.6.0-incubating-SNAPSHOT | Web-based notebook that enables interactive data analytics. | ## 4.1.0 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). **emr-4.1.0 classifications** | Classifications | Description | | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | | core-site | Change values in Hadoop's core-site.xml file. | | emrfs-site | Change EMRFS settings. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | | hdfs-encryption-zones | Configure HDFS encryption zones. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | | hive-env | Change values in the Hive environment. | | hive-exec-log4j | Change values in Hive's hive-exec-log4j.properties file. | | hive-log4j | Change values in Hive's hive-log4j.properties file. | | hive-site | Change values in Hive's hive-site.xml file | | hue-ini | Change values in Hue's ini file | | httpfs-env | Change values in the HTTPFS environment. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | | hadoop-kms-acls | Change values in Hadoop's kms-acls.xml file. | | hadoop-kms-env | Change values in the Hadoop KMS environment. | | hadoop-kms-log4j | Change values in Hadoop's kms-log4j.properties file. | | hadoop-kms-site | Change values in Hadoop's kms-site.xml file. | | mapred-env | Change values in the MapReduce application's environment. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | | oozie-env | Change values in Oozie's environment. | | oozie-log4j | Change values in Oozie's oozie-log4j.properties file. | | oozie-site | Change values in Oozie's oozie-site.xml file. | | pig-properties | Change values in Pig's pig.properties file. | | pig-log4j | Change values in Pig's log4j.properties file. | | presto-log | Change values in Presto's log.properties file. | | presto-config | Change values in Presto's config.properties file. | | spark | Amazon EMR-curated settings for Apache Spark. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | | spark-env | Change values in the Spark environment. | | spark-log4j | Change values in Spark's log4j.properties file. | | yarn-env | Change values in the YARN environment. | | yarn-site | Change values in YARN's yarn-site.xml file. | | zeppelin-env | Change values in the Zeppelin environment. | # Amazon EMR release 4.0.0 ## 4.0.0 application versions This release includes the following applications: [http://hadoop.apache.org/docs/current/](http://hadoop.apache.org/docs/current/), [http://hive.apache.org/](http://hive.apache.org/), [http://mahout.apache.org/](http://mahout.apache.org/), [http://pig.apache.org/](http://pig.apache.org/), and [https://spark.apache.org/docs/latest/](https://spark.apache.org/docs/latest/). The table below lists the application versions available in this release of Amazon EMR and the application versions in the preceding three Amazon EMR releases (when applicable). For a comprehensive history of application versions for each release of Amazon EMR, see the following topics: + [Application versions in Amazon EMR 7.x releases](emr-release-app-versions-7.x.md) + [Application versions in Amazon EMR 6.x releases](emr-release-app-versions-6.x.md) + [Application versions in Amazon EMR 5.x releases](emr-release-app-versions-5.x.md) + [Application versions in Amazon EMR 4.x releases](emr-release-app-versions-4.x.md) **Application version information** | | emr-4.3.0 | emr-4.2.0 | emr-4.1.0 | emr-4.0.0 | | --- | --- | --- | --- | --- | | AWS SDK for Java | 1.10.27 | 1.10.27 | Not tracked | Not tracked | | Python | Not tracked | Not tracked | Not tracked | Not tracked | | Scala | Not tracked | Not tracked | Not tracked | Not tracked | | AmazonCloudWatchAgent | - | - | - | - | | Delta | - | - | - | - | | Flink | - | - | - | - | | Ganglia | 3.7.2 | 3.6.0 | - | - | | HBase | - | - | - | - | | HCatalog | - | - | - | - | | Hadoop | 2.7.1-amzn-0 | 2.6.0-amzn-2 | 2.6.0-amzn-1 | 2.6.0-amzn-0 | | Hive | 1.0.0-amzn-2 | 1.0.0-amzn-1 | 1.0.0-amzn-1 | 1.0.0-amzn-0 | | Hudi | - | - | - | - | | Hue | 3.7.1-amzn-5 | 3.7.1-amzn-5 | 3.7.1-amzn-4 | - | | Iceberg | - | - | - | - | | JupyterEnterpriseGateway | - | - | - | - | | JupyterHub | - | - | - | - | | Livy | - | - | - | - | | MXNet | - | - | - | - | | Mahout | 0.11.0 | 0.11.0 | 0.11.0 | 0.10.0 | | Oozie | - | - | - | - | | Oozie-Sandbox | 4.2.0 | 4.2.0 | 4.0.1 | - | | Phoenix | - | - | - | - | | Pig | 0.14.0-amzn-0 | 0.14.0-amzn-0 | 0.14.0-amzn-0 | 0.14.0-amzn-0 | | Presto | - | - | - | - | | Presto-Sandbox | 0.130 | 0.125 | 0.119 | - | | Spark | 1.6.0 | 1.5.2 | 1.5.0 | 1.4.1 | | Sqoop | - | - | - | - | | Sqoop-Sandbox | - | - | - | - | | TensorFlow | - | - | - | - | | Tez | - | - | - | - | | Trino (PrestoSQL) | - | - | - | - | | Zeppelin | - | - | - | - | | Zeppelin-Sandbox | 0.5.5-incubating-amzn-1 | 0.5.5-incubating-amzn-0 | 0.6.0-incubating-SNAPSHOT | - | | ZooKeeper | - | - | - | - | | ZooKeeper-Sandbox | - | - | - | - | ## 4.0.0 release notes ## 4.0.0 component versions The components that Amazon EMR installs with this release are listed below. Some are installed as part of big-data application packages. Others are unique to Amazon EMR and installed for system processes and features. These typically start with `emr` or `aws`. Big-data application packages in the most recent Amazon EMR release are usually the latest version found in the community. We make community releases available in Amazon EMR as quickly as possible. Some components in Amazon EMR differ from community versions. These components have a version label in the form `CommunityVersion-amzn-EmrVersion`. The `EmrVersion` starts at 0. For example, if open source community component named `myapp-component` with version 2.2 has been modified three times for inclusion in different Amazon EMR releases, its release version is listed as `2.2-amzn-2`. | Component | Version | Description | | --- | --- | --- | | emr-ddb | 3.0.0 | Amazon DynamoDB connector for Hadoop ecosystem applications. | | emr-goodies | 2.0.0 | Extra convenience libraries for the Hadoop ecosystem. | | emr-kinesis | 3.0.0 | Amazon Kinesis connector for Hadoop ecosystem applications. | | emr-s3-dist-cp | 2.0.0 | Distributed copy application optimized for Amazon S3. | | emrfs | 2.0.0 | Amazon S3 connector for Hadoop ecosystem applications. | | hadoop-client | 2.6.0-amzn-0 | Hadoop command-line clients such as 'hdfs', 'hadoop', or 'yarn'. | | hadoop-hdfs-datanode | 2.6.0-amzn-0 | HDFS node-level service for storing blocks. | | hadoop-hdfs-namenode | 2.6.0-amzn-0 | HDFS service for tracking file names and block locations. | | hadoop-httpfs-server | 2.6.0-amzn-0 | HTTP endpoint for HDFS operations. | | hadoop-mapred | 2.6.0-amzn-0 | MapReduce execution engine libraries for running a MapReduce application. | | hadoop-yarn-nodemanager | 2.6.0-amzn-0 | YARN service for managing containers on an individual node. | | hadoop-yarn-resourcemanager | 2.6.0-amzn-0 | YARN service for allocating and managing cluster resources and distributed applications. | | hive-client | 1.0.0-amzn-0 | Hive command line client. | | hive-metastore-server | 1.0.0-amzn-0 | Service for accessing the Hive metastore, a semantic repository storing metadata for SQL on Hadoop operations. | | hive-server | 1.0.0-amzn-0 | Service for accepting Hive queries as web requests. | | mahout-client | 0.10.0 | Library for machine learning. | | mysql-server | 5.5 | MySQL database server. | | pig-client | 0.14.0-amzn-0 | Pig command-line client. | | spark-client | 1.4.1 | Spark command-line clients. | | spark-history-server | 1.4.1 | Web UI for viewing logged events for the lifetime of a completed Spark application. | | spark-on-yarn | 1.4.1 | In-memory execution engine for YARN. | | spark-yarn-slave | 1.4.1 | Apache Spark libraries needed by YARN slaves. | ## 4.0.0 configuration classifications Configuration classifications allow you to customize applications. These often correspond to a configuration XML file for the application, such as `hive-site.xml`. For more information, see [Configure applications](emr-configure-apps.md). **emr-4.0.0 classifications** | Classifications | Description | | --- | --- | | capacity-scheduler | Change values in Hadoop's capacity-scheduler.xml file. | | core-site | Change values in Hadoop's core-site.xml file. | | emrfs-site | Change EMRFS settings. | | hadoop-env | Change values in the Hadoop environment for all Hadoop components. | | hadoop-log4j | Change values in Hadoop's log4j.properties file. | | hdfs-site | Change values in HDFS's hdfs-site.xml. | | hive-env | Change values in the Hive environment. | | hive-exec-log4j | Change values in Hive's hive-exec-log4j.properties file. | | hive-log4j | Change values in Hive's hive-log4j.properties file. | | hive-site | Change values in Hive's hive-site.xml file | | httpfs-env | Change values in the HTTPFS environment. | | httpfs-site | Change values in Hadoop's httpfs-site.xml file. | | mapred-env | Change values in the MapReduce application's environment. | | mapred-site | Change values in the MapReduce application's mapred-site.xml file. | | pig-properties | Change values in Pig's pig.properties file. | | pig-log4j | Change values in Pig's log4j.properties file. | | spark | Amazon EMR-curated settings for Apache Spark. | | spark-defaults | Change values in Spark's spark-defaults.conf file. | | spark-env | Change values in the Spark environment. | | spark-log4j | Change values in Spark's log4j.properties file. | | yarn-env | Change values in the YARN environment. | | yarn-site | Change values in YARN's yarn-site.xml file. | # Amazon EMR 2.x and 3.x AMI versions **Note** AWS is updating the TLS configuration for all AWS API endpoints to a minimum version of TLS 1.2. Amazon EMR releases 3.10 and lower only support TLS 1.0/1.1 connections. After December 4, 2023, you won't be able to create clusters with Amazon EMR 3.10 and lower. If you use Amazon EMR 3.10 or lower, we recommend that you immediately test and migrate your workloads to the latest Amazon EMR release. For more information, see the [AWS Security Blog](https://aws.amazon.com/blogs//security/tls-1-2-required-for-aws-endpoints/). Amazon EMR 2.x and 3.x releases, called *AMI versions*, are made available for pre-existing solutions that require them for compatibility reasons. We do not recommend creating new clusters or new solutions with these release versions. They lack features of newer releases and include outdated application packages. We recommend that you build solutions using the most recent Amazon EMR release version. The scope of differences between the 2.x and 3.x series release versions and recent Amazon EMR release versions is significant. Those differences range from how you create and configure a cluster to the ports and directory structure of applications on the cluster. This section attempts to cover the most significant differences for Amazon EMR, as well as specific application configuration and management differences. It is not comprehensive. If you create and use clusters in the 2.x or 3.x series, you may encounter differences not covered in this section. **Topics** + [Creating a cluster with earlier AMI versions of Amazon EMR](emr-3x-create.md) + [Installing applications with earlier AMI versions of Amazon EMR](emr-3x-install-apps.md) + [Customizing cluster and application configuration with earlier AMI versions of Amazon EMR](emr-3x-customizeappconfig.md) + [Hive application specifics for earlier AMI versions of Amazon EMR](emr-3x-hive.md) + [HBase application specifics for earlier AMI versions of Amazon EMR](emr-3x-hbase.md) + [Pig application specifics for earlier AMI versions of Amazon EMR](emr-3x-pig.md) + [Spark application specifics with earlier AMI versions of Amazon EMR](emr-3x-spark.md) + [S3DistCp utility differences with earlier AMI versions of Amazon EMR](emr-3x-s3distcp.md) # Creating a cluster with earlier AMI versions of Amazon EMR Amazon EMR 2.x and 3.x releases are referenced by AMI version. With Amazon EMR release 4.0.0 and later, releases are referenced by release version, using a release label such as `emr-5.11.0`. This change is most apparent when you create a cluster using the AWS CLI or programmatically. When you use the AWS CLI to create a cluster using an AMI release version, use the `--ami-version` option, for example, `--ami-version 3.11.0`. Many options, features, and applications introduced in Amazon EMR 4.0.0 and later are not available when you specify an `--ami-version`. For more information, see [create-cluster](https://docs.aws.amazon.com/cli/latest/reference/emr/create-cluster.html) in the *AWS CLI Command Reference*. The following example AWS CLI command launches a cluster using an AMI version. **Note** Linux line continuation characters (\$1) are included for readability. They can be removed or used in Linux commands. For Windows, remove them or replace with a caret (^). ``` aws emr create-cluster --name "Test cluster" --ami-version 3.11.0 \ --applications Name=Hue Name=Hive Name=Pig \ --use-default-roles --ec2-attributes KeyName=myKey \ --instance-groups InstanceGroupType=MASTER,InstanceCount=1,\ InstanceType=m3.xlarge InstanceGroupType=CORE,InstanceCount=2,\ InstanceType=m3.xlarge --bootstrap-actions Path=s3://elasticmapreduce/bootstrap-actions/configure-hadoop,\ Name="Configuring infinite JVM reuse",Args=["-m","mapred.job.reuse.jvm.num.tasks=-1"] ``` Programmatically, all Amazon EMR release versions use the `RunJobFlowRequest` action in the EMR API to create clusters. The following example Java code creates a cluster using AMI release version 3.11.0. ``` RunJobFlowRequest request = new RunJobFlowRequest() .withName("AmiVersion Cluster") .withAmiVersion("3.11.0") .withInstances(new JobFlowInstancesConfig() .withEc2KeyName("myKeyPair") .withInstanceCount(1) .withKeepJobFlowAliveWhenNoSteps(true) .withMasterInstanceType("m3.xlarge") .withSlaveInstanceType("m3.xlarge"); ``` The following `RunJobFlowRequest` call uses a release label instead: ``` RunJobFlowRequest request = new RunJobFlowRequest() .withName("ReleaseLabel Cluster") .withReleaseLabel("emr-7.12.0") .withInstances(new JobFlowInstancesConfig() .withEc2KeyName("myKeyPair") .withInstanceCount(1) .withKeepJobFlowAliveWhenNoSteps(true) .withMasterInstanceType("m3.xlarge") .withSlaveInstanceType("m3.xlarge"); ``` ## Configuring cluster size When your cluster runs, Hadoop determines the number of mapper and reducer tasks needed to process the data. Larger clusters should have more tasks for better resource use and shorter processing time. Typically, an EMR cluster remains the same size during the entire cluster; you set the number of tasks when you create the cluster. When you resize a running cluster, you can vary the processing during the cluster execution. Therefore, instead of using a fixed number of tasks, you can vary the number of tasks during the life of the cluster. There are two configuration options to help set the ideal number of tasks: + `mapred.map.tasksperslot` + `mapred.reduce.tasksperslot` You can set both options in the `mapred-conf.xml` file. When you submit a job to the cluster, the job client checks the current total number of map and reduce slots available clusterwide. The job client then uses the following equations to set the number of tasks: + `mapred.map.tasks` =` mapred.map.tasksperslot` \$1 map slots in cluster + `mapred.reduce.tasks` = `mapred.reduce.tasksperslot` \$1 reduce slots in cluster The job client only reads the `tasksperslot` parameter if the number of tasks is not configured. You can override the number of tasks at any time, either for all clusters via a bootstrap action or individually per job by adding a step to change the configuration. Amazon EMR withstands task node failures and continues cluster execution even if a task node becomes unavailable. Amazon EMR automatically provisions additional task nodes to replace those that fail. You can have a different number of task nodes for each cluster step. You can also add a step to a running cluster to modify the number of task nodes. Because all steps are guaranteed to run sequentially by default, you can specify the number of running task nodes for any step. # Installing applications with earlier AMI versions of Amazon EMR When using an AMI version, applications are installed in any number of ways, including using the `NewSupportedProducts` parameter for the [RunJobFlow](https://docs.aws.amazon.com/ElasticMapReduce/latest/API/API_RunJobFlow.html) action, using bootstrap actions, and using the [Step](https://docs.aws.amazon.com/ElasticMapReduce/latest/API/API_Step.html) action. # Customizing cluster and application configuration with earlier AMI versions of Amazon EMR Amazon EMR release version 4.0.0 introduced a simplified method of configuring applications using configuration classifications. For more information, see [Configure applications](emr-configure-apps.md). When using an AMI version, you configure applications using bootstrap actions along with arguments that you pass. For example, the `configure-hadoop` and `configure-daemons` bootstrap actions set Hadoop and YARN–specific environment properties like `--namenode-heap-size`. In more recent versions, these are configured using the `hadoop-env` and `yarn-env` configuration classifications. For bootstrap actions that perform common configurations, see the [emr-bootstrap-actions repository on Github](https://github.com/awslabs/emr-bootstrap-actions). The following tables map bootstrap actions to configuration classifications in more recent Amazon EMR release versions. **Hadoop** | Affected application file name | AMI version bootstrap action | Configuration classification | | --- | --- | --- | | core-site.xml | configure-hadoop -c | core-site | | log4j.properties | configure-hadoop -l | hadoop-log4j | | hdfs-site.xml | configure-hadoop -s | hdfs-site | | n/a | n/a | hdfs-encryption-zones | | mapred-site.xml | configure-hadoop -m | mapred-site | | yarn-site.xml | configure-hadoop -y | yarn-site | | httpfs-site.xml | configure-hadoop -t | httpfs-site | | capacity-scheduler.xml | configure-hadoop -z | capacity-scheduler | | yarn-env.sh | configure-daemons --resourcemanager-opts | yarn-env | **Hive** | Affected application file name | AMI version bootstrap action | Configuration classification | | --- | --- | --- | | hive-env.sh | n/a | hive-env | | hive-site.xml | hive-script --install-hive-site \$1\$1MY\$1HIVE\$1SITE\$1FILE\$1 | hive-site | | hive-exec-log4j.properties | n/a | hive-exec-log4j | | hive-log4j.properties | n/a | hive-log4j | **EMRFS** | Affected application file name | AMI version bootstrap action | Configuration classification | | --- | --- | --- | | emrfs-site.xml | configure-hadoop -e | emrfs-site | | n/a | s3get -s s3://custom-provider.jar -d /usr/share/aws/emr/auxlib/ | emrfs-site (with new setting fs.s3.cse.encryptionMaterialsProvider.uri) | For a list of all classifications, see [Configure applications](emr-configure-apps.md). ## Application environment variables When using an AMI version, a `hadoop-user-env.sh` script is used along with the `configure-daemons` bootstrap action to configure the Hadoop environment. The script includes the following actions: ``` #!/bin/bash export HADOOP_USER_CLASSPATH_FIRST=true; echo "HADOOP_CLASSPATH=/path/to/my.jar" >> /home/hadoop/conf/hadoop-user-env.sh ``` In Amazon EMR release 4.x, you do the same using the `hadoop-env` configuration classification, as shown in the following example: ``` [       {          "Classification":"hadoop-env",          "Properties":{          },          "Configurations":[             {                "Classification":"export",                "Properties":{                   "HADOOP_USER_CLASSPATH_FIRST":"true",                   "HADOOP_CLASSPATH":"/path/to/my.jar"                }             }          ]       }    ] ``` As another example, using `configure-daemons` and passing `--namenode-heap-size=2048` and `--namenode-opts=-XX:GCTimeRatio=19` is equivalent to the following configuration classifications. ``` [       {          "Classification":"hadoop-env",          "Properties":{          },          "Configurations":[             {                "Classification":"export",                "Properties":{                   "HADOOP_DATANODE_HEAPSIZE": "2048", "HADOOP_NAMENODE_OPTS": "-XX:GCTimeRatio=19"                }             }          ]       }    ] ``` Other application environment variables are no longer defined in `/home/hadoop/.bashrc`. Instead, they are primarily set in `/etc/default` files per component or application, such as `/etc/default/hadoop`. Wrapper scripts in `/usr/bin/` installed by application RPMs may also set additional environment variables before involving the actual bin script. ## Service ports When using an AMI version, some services use custom ports. **Changes in port settings** | Setting | AMI version 3.x | Open-source default | | --- | --- | --- | | fs.default.name | hdfs://emrDeterminedIP:9000 | default (hdfs://emrDeterminedIP:8020) | | dfs.datanode.address | 0.0.0.0:9200 | default (0.0.0.0:50010) | | dfs.datanode.http.address | 0.0.0.0:9102 | default (0.0.0.0:50075) | | dfs.datanode.https.address | 0.0.0.0:9402 | default (0.0.0.0:50475) | | dfs.datanode.ipc.address | 0.0.0.0:9201 | default (0.0.0.0:50020) | | dfs.http.address | 0.0.0.0:9101 | default (0.0.0.0:50070) | | dfs.https.address | 0.0.0.0:9202 | default (0.0.0.0:50470) | | dfs.secondary.http.address | 0.0.0.0:9104 | default (0.0.0.0:50090) | | yarn.nodemanager.address | 0.0.0.0:9103 | default (\$1\$1yarn.nodemanager.hostname\$1:0) | | yarn.nodemanager.localizer.address | 0.0.0.0:9033 | default (\$1\$1yarn.nodemanager.hostname\$1:8040) | | yarn.nodemanager.webapp.address | 0.0.0.0:9035 | default (\$1\$1yarn.nodemanager.hostname\$1:8042) | | yarn.resourcemanager.address | emrDeterminedIP:9022 | default (\$1\$1yarn.resourcemanager.hostname\$1:8032) | | yarn.resourcemanager.admin.address | emrDeterminedIP:9025 | default (\$1\$1yarn.resourcemanager.hostname\$1:8033) | | yarn.resourcemanager.resource-tracker.address | emrDeterminedIP:9023 | default (\$1\$1yarn.resourcemanager.hostname\$1:8031) | | yarn.resourcemanager.scheduler.address | emrDeterminedIP:9024 | default (\$1\$1yarn.resourcemanager.hostname\$1:8030) | | yarn.resourcemanager.webapp.address | 0.0.0.0:9026 | default (\$1\$1yarn.resourcemanager.hostname\$1:8088) | | yarn.web-proxy.address | emrDeterminedIP:9046 | default (no-value) | | yarn.resourcemanager.hostname | 0.0.0.0 (default) | emrDeterminedIP | **Note** The *emrDeterminedIP* is an IP address that is generated by Amazon EMR. ## Users When using an AMI version, the user `hadoop` runs all processes and owns all files. In Amazon EMR release version 4.0.0 and later, users exist at the application and component level. ## Installation sequence, installed artifacts, and log file locations When using an AMI version, application artifacts and their configuration directories are installed in the `/home/hadoop/application` directory. For example, if you installed Hive, the directory would be `/home/hadoop/hive`. In Amazon EMR release 4.0.0 and later, application artifacts are installed in the `/usr/lib/application` directory. When using an AMI version, log files are found in various places. The table below lists locations. **Changes in log locations on Amazon S3** | Daemon or application | Directory location | | --- | --- | | instance-state | node/instance-id/instance-state/ | | hadoop-hdfs-namenode | daemons/instance-id/hadoop-hadoop-namenode.log | | hadoop-hdfs-datanode | daemons/instance-id/hadoop-hadoop-datanode.log | | hadoop-yarn (ResourceManager) | daemons/instance-id/yarn-hadoop-resourcemanager | | hadoop-yarn (Proxy Server) | daemons/instance-id/yarn-hadoop-proxyserver | | mapred-historyserver | daemons/instance-id/ | | httpfs | daemons/instance-id/httpfs.log | | hive-server | node/instance-id/hive-server/hive-server.log | | hive-metastore | node/instance-id/apps/hive.log | | Hive CLI | node/instance-id/apps/hive.log | | YARN applications user logs and container logs | task-attempts/ | | Mahout | N/A | | Pig | N/A | | spark-historyserver | N/A | | mapreduce job history files | jobs/ | ## Command runner When using an AMI version, many scripts or programs, like `/home/hadoop/contrib/streaming/hadoop-streaming.jar`, are not placed on the shell login path environment, so you need to specify the full path when you use a jar file such as command-runner.jar or script-runner.jar to execute the scripts. The `command-runner.jar` is located on the AMI so there is no need to know a full URI as was the case with `script-runner.jar`. ## Replication factor The replication factor lets you configure when to start a Hadoop JVM. You can start a new Hadoop JVM for every task, which provides better task isolation, or you can share JVMs between tasks, providing lower framework overhead. If you are processing many small files, it makes sense to reuse the JVM many times to amortize the cost of start-up. However, if each task takes a long time or processes a large amount of data, then you might choose to not reuse the JVM to ensure that all memory is freed for subsequent tasks. When using an AMI version, you can customize the replication factor using the `configure-hadoop` bootstrap action to set the `mapred.job.reuse.jvm.num.tasks` property. The following example demonstrates setting the JVM reuse factor for infinite JVM reuse. **Note** Linux line continuation characters (\$1) are included for readability. They can be removed or used in Linux commands. For Windows, remove them or replace with a caret (^). ``` aws emr create-cluster --name "Test cluster" --ami-version 3.11.0 \ --applications Name=Hue Name=Hive Name=Pig \ --use-default-roles --ec2-attributes KeyName=myKey \ --instance-groups InstanceGroupType=MASTER,InstanceCount=1,InstanceType=m3.xlarge \ InstanceGroupType=CORE,InstanceCount=2,InstanceType=m3.xlarge \ --bootstrap-actions Path=s3://elasticmapreduce/bootstrap-actions/configure-hadoop,\ Name="Configuring infinite JVM reuse",Args=["-m","mapred.job.reuse.jvm.num.tasks=-1"] ``` # Hive application specifics for earlier AMI versions of Amazon EMR ## Log files Using Amazon EMR AMI versions 2.x and 3.x, Hive logs are saved to `/mnt/var/log/apps/`. In order to support concurrent versions of Hive, the version of Hive that you run determines the log file name, as shown in the following table. | Hive version | Log file name | | --- | --- | | 0.13.1 | hive.log Beginning with this version, Amazon EMR uses an unversioned file name, `hive.log`. Minor versions share the same log location as the major version. | | 0.11.0 | hive\$10110.log Minor versions of Hive 0.11.0, such as 0.11.0.1, share the same log file location as Hive 0.11.0. | | 0.8.1 | hive\$1081.log Minor versions of Hive 0.8.1, such as Hive 0.8.1.1, share the same log file location as Hive 0.8.1. | | 0.7.1 | hive\$107\$11.log Minor versions of Hive 0.7.1, such as Hive 0.7.1.3 and Hive 0.7.1.4, share the same log file location as Hive 0.7.1. | | 0.7 | hive\$107.log | | 0.5 | hive\$105.log | | 0.4 | hive.log | ## Split input functionality To implement split input functionality using Hive versions earlier than 0.13.1 (Amazon EMR AMI versions earlier 3.11.0), use the following: ``` hive> set hive.input.format=org.apache.hadoop.hive.ql.io.HiveCombineSplitsInputFormat; hive> set mapred.min.split.size=100000000; ``` This functionality was deprecated with Hive 0.13.1. To get the same split input format functionality in Amazon EMR AMI Version 3.11.0, use the following: ``` set hive.hadoop.supports.splittable.combineinputformat=true; ``` ## Thrift service ports Thrift is an RPC framework that defines a compact binary serialization format used to persist data structures for later analysis. Normally, Hive configures the server to operate on the following ports. | Hive version | Port number | | --- | --- | | Hive 0.13.1 | 10000 | | Hive 0.11.0 | 10004 | | Hive 0.8.1 | 10003 | | Hive 0.7.1 | 10002 | | Hive 0.7 | 10001 | | Hive 0.5 | 10000 | For more information about thrift services, see [http://wiki.apache.org/thrift/](http://wiki.apache.org/thrift/). ## Use Hive to recover partitions Amazon EMR includes a statement in the Hive query language that recovers the partitions of a table from table data located in Amazon S3. The following example shows this. ``` CREATE EXTERNAL TABLE (json string) raw_impression PARTITIONED BY (dt string) LOCATION 's3://elastic-mapreduce/samples/hive-ads/tables/impressions'; ALTER TABLE logs RECOVER PARTITIONS; ``` The partition directories and data must be at the location specified in the table definition and must be named according to the Hive convention: for example, `dt=2009-01-01`. **Note** After Hive 0.13.1 this capability is supported natively using `msck repair table` and therefore `recover partitions` is not supported. For more information, see [https://cwiki.apache.org/confluence/display/Hive/LanguageManual\$1DDL](https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DDL). ## Pass a Hive variable to a script To pass a variable into a Hive step using the AWS CLI, type the following command, replace *myKey* with the name of your EC2 key pair, and replace *amzn-s3-demo-bucket* with your bucket name. In this example, `SAMPLE` is a variable value preceded by the `-d` switch. This variable is defined in the Hive script as: `${SAMPLE}`. **Note** Linux line continuation characters (\$1) are included for readability. They can be removed or used in Linux commands. For Windows, remove them or replace with a caret (^). ``` aws emr create-cluster --name "Test cluster" --ami-version 3.9 \ --applications Name=Hue Name=Hive Name=Pig \ --use-default-roles --ec2-attributes KeyName=myKey \ --instance-type m3.xlarge --instance-count 3 \ --steps Type=Hive,Name="Hive Program",ActionOnFailure=CONTINUE,\ Args=[-f,s3://elasticmapreduce/samples/hive-ads/libs/response-time-stats.q,-d,\ INPUT=s3://elasticmapreduce/samples/hive-ads/tables,-d,OUTPUT=s3://amzn-s3-demo-bucket/hive-ads/output/,\ -d,SAMPLE=s3://elasticmapreduce/samples/hive-ads/] ``` ## Specify an external metastore location The following procedure shows you how to override the default configuration values for the Hive metastore location and start a cluster using the reconfigured metastore location. **To create a metastore located outside of the EMR cluster** 1. Create a MySQL or Aurora database using Amazon RDS. For information about how to create an Amazon RDS database, see [Getting started with Amazon RDS](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/CHAP_GettingStarted.html). 1. Modify your security groups to allow JDBC connections between your database and the **ElasticMapReduce-Master** security group. For information about how to modify your security groups for access, see [Amazon RDS security groups](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Overview.RDSSecurityGroups.html) in the *Amazon RDS User Guide*. 1. Set the JDBC configuration values in `hive-site.xml`: 1. Create a `hive-site.xml` configuration file containing the following: ``` javax.jdo.option.ConnectionURL jdbc:mariadb://hostname:3306/hive?createDatabaseIfNotExist=true JDBC connect string for a JDBC metastore javax.jdo.option.ConnectionUserName hive Username to use against metastore database javax.jdo.option.ConnectionPassword password Password to use against metastore database ``` *hostname* is the DNS address of the Amazon RDS instance running the database. *username* and *password* are the credentials for your database. For more information about connecting to MySQL and Aurora database instances, see [Connecting to a DB instance running the MySQL database engine](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_ConnectToInstance.html) and [Connecting to an Aurora DB cluster](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Aurora.Connecting.html) in the *Amazon RDS User Guide*. The JDBC drivers are installed by Amazon EMR. **Note** The value property should not contain any spaces or carriage returns. It should appear all on one line. 1. Save your `hive-site.xml` file to a location on Amazon S3, such as `s3://amzn-s3-demo-bucket/hive-site.xml`. 1. Create a cluster, specifying the Amazon S3 location of the customized `hive-site.xml` file. The following example command demonstrates an AWS CLI command that does this. **Note** Linux line continuation characters (\$1) are included for readability. They can be removed or used in Linux commands. For Windows, remove them or replace with a caret (^). ``` aws emr create-cluster --name "Test cluster" --ami-version 3.10 \ --applications Name=Hue Name=Hive Name=Pig \ --use-default-roles --ec2-attributes KeyName=myKey \ --instance-type m3.xlarge --instance-count 3 \ --bootstrap-actions Name="Install Hive Site Configuration",\ Path="s3://region.elasticmapreduce/libs/hive/hive-script",\ Args=["--base-path","s3://elasticmapreduce/libs/hive","--install-hive-site",\ "--hive-site=s3://amzn-s3-demo-bucket/hive-site.xml","--hive-versions","latest"] ``` ## Connect to Hive using JDBC To connect to Hive via JDBC requires you to download the JDBC driver and install a SQL client. The following example demonstrates using SQL Workbench/J to connect to Hive using JDBC. **To download JDBC drivers** 1. Download and extract the drivers appropriate to the versions of Hive that you want to access. The Hive version differs depending on the AMI that you choose when you create an Amazon EMR cluster. + Hive 0.13.1 JDBC drivers: [https://amazon-odbc-jdbc-drivers.s3.amazonaws.com/public/AmazonHiveJDBC\$11.0.4.1004.zip](https://amazon-odbc-jdbc-drivers.s3.amazonaws.com/public/AmazonHiveJDBC_1.0.4.1004.zip) + Hive 0.11.0 JDBC drivers: [https://mvnrepository.com/artifact/org.apache.hive/hive-jdbc/0.11.0](https://mvnrepository.com/artifact/org.apache.hive/hive-jdbc/0.11.0) + Hive 0.8.1 JDBC drivers: [https://mvnrepository.com/artifact/org.apache.hive/hive-jdbc/0.8.1](https://mvnrepository.com/artifact/org.apache.hive/hive-jdbc/0.8.1) 1. Install SQL Workbench/J. For more information, see [Installing and starting SQL Workbench/J](http://www.sql-workbench.net/manual/install.html) in the SQL Workbench/J Manual User's Manual. 1. Create an SSH tunnel to the cluster master node. The port for connection is different depending on the version of Hive. Example commands are provided in the tables below for Linux `ssh` users and PuTTY commands for Windows users **Linux SSH commands** [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-3x-hive.html) **Windows PuTTY tunnel settings** [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-3x-hive.html) 1. Add the JDBC driver to SQL Workbench. 1. In the **Select Connection Profile** dialog box, choose **Manage Drivers**. 1. Choose the **Create a new entry** (blank page) icon. 1. In the **Name** field, type **Hive JDBC**. 1. For **Library**, click the **Select the JAR file(s)** icon. 1. Select JAR files as shown in the following table. **** [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-3x-hive.html) 1. In the **Please select one driver** dialog box, select a driver according to the following table and click **OK**. [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-3x-hive.html) 1. When you return to the **Select Connection Profile** dialog box, verify that the **Driver** field is set to **Hive JDBC** and provide the JDBC connection string in the **URL** field according to the following table. [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-3x-hive.html) If your cluster uses AMI version 3.3.1 or later, in the **Select Connection Profile** dialog box, type **hadoop** in the **Username** field. # HBase application specifics for earlier AMI versions of Amazon EMR ## Supported HBase versions | HBase version | AMI version | AWS CLI configuration parameters | HBase version details | | --- | --- | --- | --- | | [0.94.18](https://svn.apache.org/repos/asf/hbase/branches/0.94/CHANGES.txt) | 3.1.0 and later | `--ami-version 3.1` `--ami-version 3.2` `--ami-version 3.3` `--applications Name=HBase` | [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-3x-hbase.html) | | [0.94.7](https://svn.apache.org/repos/asf/hbase/branches/0.94/CHANGES.txt) | 3.0-3.0.4 | `--ami-version 3.0` `--applications Name=HBase` | | [0.92](https://svn.apache.org/repos/asf/hbase/branches/0.92/CHANGES.txt) | 2.2 and later | `--ami-version 2.2 or later` `--applications Name=HBase` | ## HBase cluster prerequisites A cluster created using Amazon EMR AMI versions 2.x and 3.x should meet the following requirements for HBase. + The AWS CLI (optional)—To interact with HBase using the command line, download and install the latest version of the AWS CLI. For more information, see [Installing the AWS Command Line Interface](https://docs.aws.amazon.com/cli/latest/userguide/installing.html) in the *AWS Command Line Interface User Guide*. + At least two instances (optional)—The cluster's master node runs the HBase master server and Zookeeper, and task nodes run the HBase region servers. For best performance, HBase clusters should run on at least two EC2 instances, but you can run HBase on a single node for evaluation purposes. + Long-running cluster—HBase only runs on long-running clusters. By default, the CLI and Amazon EMR console create long-running clusters. + An Amazon EC2 key pair set (recommended)—To use the Secure Shell (SSH) network protocol to connect with the master node and run HBase shell commands, you must use an Amazon EC2 key pair when you create the cluster. + The correct AMI and Hadoop versions—HBase clusters are currently supported only on Hadoop 20.205 or later. + Ganglia (optional)—To monitor HBase performance metrics, install Ganglia when you create the cluster. + An Amazon S3 bucket for logs (optional)—The logs for HBase are available on the master node. If you'd like these logs copied to Amazon S3, specify an S3 bucket to receive log files when you create the cluster. ## Creating a cluster with HBase The following table lists options that are available when using the console to create a cluster with HBase using an Amazon EMR AMI release version. | Field | Action | | --- | --- | | Restore from backup | Specify whether to pre-load the HBase cluster with data stored in Amazon S3. | | Backup location | Specify the URI where the backup from which to restore resides in Amazon S3. | | Backup version | Optionally, specify the version name of the backup at Backup Location to use. If you leave this field blank, Amazon EMR uses the latest backup at Backup Location to populate the new HBase cluster. | | Schedule Regular Backups | Specify whether to schedule automatic incremental backups. The first backup is a full backup to create a baseline for future incremental backups. | | Consistent backup | Specify whether the backups should be consistent. A consistent backup is one that pauses write operations during the initial backup stage, synchronization across nodes. Any write operations thus paused are placed in a queue and resume when synchronization completes. | | Backup frequency | The number of days/hours/minutes between scheduled backups. | | Backup location | The Amazon S3 URI where backups are stored. The backup location for each HBase cluster should be different to ensure that differential backups stay correct. | | Backup start time | Specify when the first backup should occur. You can set this to now, which causes the first backup to start as soon as the cluster is running, or enter a date and time in [ISO format](http://www.w3.org/TR/NOTE-datetime). For example, 2012-06-15T20:00Z would set the start time to June 15, 2012 at 8PM UTC. | The following example AWS CLI command launches a cluster with HBase and other applications: **Note** Linux line continuation characters (\$1) are included for readability. They can be removed or used in Linux commands. For Windows, remove them or replace with a caret (^). ``` aws emr create-cluster --name "Test cluster" --ami-version 3.3 \ --applications Name=Hue Name=Hive Name=Pig Name=HBase \ --use-default-roles --ec2-attributes KeyName=myKey \ --instance-type c1.xlarge --instance-count 3 --termination-protected ``` After the connection between the Hive and HBase clusters has been made (as shown in the previous procedure), you can access the data stored on the HBase cluster by creating an external table in Hive. The following example, when run from the Hive prompt, creates an external table that references data stored in an HBase table called `inputTable`. You can then reference `inputTable` in Hive statements to query and modify data stored in the HBase cluster. **Note** The following example uses **protobuf-java-2.4.0a.jar** in AMI 2.3.3, but you should modify the example to match your version. To check which version of the Protocol Buffers JAR you have, run the command at the Hive command prompt: `! ls /home/hadoop/lib;`. ``` add jar lib/emr-metrics-1.0.jar ; add jar lib/protobuf-java-2.4.0a.jar ; set hbase.zookeeper.quorum=ec2-107-21-163-157.compute-1.amazonaws.com ; create external table inputTable (key string, value string) stored by 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' with serdeproperties ("hbase.columns.mapping" = ":key,f1:col1") tblproperties ("hbase.table.name" = "t1"); select count(*) from inputTable ; ``` ## Customizing HBase configuration Although the default settings should work for most applications, you have the flexibility to modify your HBase configuration settings. To do this, run one of two bootstrap action scripts: + **configure-hbase-daemons**—Configures properties of the master, regionserver, and zookeeper daemons. These properties include heap size and options to pass to the Java Virtual Machine (JVM) when the HBase daemon starts. You set these properties as arguments in the bootstrap action. This bootstrap action modifies the /home/hadoop/conf/hbase-user-env.sh configuration file on the HBase cluster. + **configure-hbase**—Configures HBase site-specific settings such as the port the HBase master should bind to and the maximum number of times the client CLI client should retry an action. You can set these one-by-one, as arguments in the bootstrap action, or you can specify the location of an XML configuration file in Amazon S3. This bootstrap action modifies the /home/hadoop/conf/hbase-site.xml configuration file on the HBase cluster. **Note** These scripts, like other bootstrap actions, can only be run when the cluster is created; you cannot use them to change the configuration of an HBase cluster that is currently running. When you run the **configure-hbase** or **configure-hbase-daemons** bootstrap actions, the values you specify override the default values. Any values that you don't explicitly set receive the default values. Configuring HBase with these bootstrap actions is analogous to using bootstrap actions in Amazon EMR to configure Hadoop settings and Hadoop daemon properties. The difference is that HBase does not have per-process memory options. Instead, memory options are set using the `--daemon-opts` argument, where *daemon* is replaced by the name of the daemon to configure. ### Configure HBase daemons Amazon EMR provides a bootstrap action, `s3://region.elasticmapreduce/bootstrap-actions/configure-hbase-daemons`, that you can use to change the configuration of HBase daemons, where *region* is the region into which you're launching your HBase cluster. To configure HBase daemons using the AWS CLI, add the `configure-hbase-daemons` bootstrap action when you launch the cluster to configure one or more HBase daemons. You can set the following properties. | Property | Description | | --- | --- | | hbase-master-opts | Options that control how the JVM runs the master daemon. If set, these override the default HBASE\$1MASTER\$1OPTS variables. | | regionserver-opts | Options that control how the JVM runs the region server daemon. If set, these override the default HBASE\$1REGIONSERVER\$1OPTS variables. | | zookeeper-opts | Options that control how the JVM runs the zookeeper daemon. If set, these override the default HBASE\$1ZOOKEEPER\$1OPTS variables. | For more information about these options, see [hbase-env.sh](https://hbase.apache.org/book.html#hbase.env.sh) in the HBase documentation. A bootstrap action to configure values for `zookeeper-opts` and `hbase-master-opts` is shown in the following example. **Note** Linux line continuation characters (\$1) are included for readability. They can be removed or used in Linux commands. For Windows, remove them or replace with a caret (^). ``` aws emr create-cluster --name "Test cluster" --ami-version 3.3 \ --applications Name=Hue Name=Hive Name=Pig Name=HBase \ --use-default-roles --ec2-attributes KeyName=myKey \ --instance-type c1.xlarge --instance-count 3 --termination-protected \ --bootstrap-actions Path=s3://elasticmapreduce/bootstrap-actions/configure-hbase-daemons,\ Args=["--hbase-zookeeper-opts=-Xmx1024m -XX:GCTimeRatio=19","--hbase-master-opts=-Xmx2048m","--hbase-regionserver-opts=-Xmx4096m"] ``` ### Configure HBase site settings Amazon EMR provides a bootstrap action, `s3://elasticmapreduce/bootstrap-actions/configure-hbase`, that you can use to change the configuration of HBase. You can set configuration values one-by-one, as arguments in the bootstrap action, or you can specify the location of an XML configuration file in Amazon S3. Setting configuration values one-by-one is useful if you only need to set a few configuration settings. Setting them using an XML file is useful if you have many changes to make, or if you want to save your configuration settings for reuse. **Note** You can prefix the Amazon S3 bucket name with a region prefix, such as `s3://region.elasticmapreduce/bootstrap-actions/configure-hbase`, where *region* is the region into which you're launching your HBase cluster. This bootstrap action modifies the `/home/hadoop/conf/hbase-site.xml` configuration file on the HBase cluster. The bootstrap action can only be run when the HBase cluster is launched. For more information about the HBase site settings that you can configure, see [Default configuration](http://hbase.apache.org/book.html#config.files) in the HBase documentation. Set the `configure-hbase` bootstrap action when you launch the HBase cluster and specify the values in `hbase-site.xml` to change. **To specify individual HBase site settings using the AWS CLI** + To change the `hbase.hregion.max.filesize` setting, type the following command and replace *myKey* with the name of your Amazon EC2 key pair. **Note** Linux line continuation characters (\$1) are included for readability. They can be removed or used in Linux commands. For Windows, remove them or replace with a caret (^). ``` aws emr create-cluster --name "Test cluster" --ami-version 3.3 \ --applications Name=Hue Name=Hive Name=Pig Name=HBase \ --use-default-roles --ec2-attributes KeyName=myKey \ --instance-type c1.xlarge --instance-count 3 --termination-protected \ --bootstrap-actions Path=s3://elasticmapreduce/bootstrap-actions/configure-hbase,Args=["-s","hbase.hregion.max.filesize=52428800"] ``` **To specify HBase site settings with an XML file using the AWS CLI** 1. Create a custom version of `hbase-site.xml`. Your custom file must be valid XML. To reduce the chance of introducing errors, start with the default copy of `hbase-site.xml`, located on the Amazon EMR HBase master node at `/home/hadoop/conf/hbase-site.xml`, and edit a copy of that file instead of creating a file from scratch. You can give your new file a new name, or leave it as `hbase-site.xml`. 1. Upload your custom `hbase-site.xml` file to an Amazon S3 bucket. It should have permissions set so the AWS account that launches the cluster can access the file. If the AWS account launching the cluster also owns the Amazon S3 bucket, it has access. 1. Set the **configure-hbase** bootstrap action when you launch the HBase cluster, and include the location of your custom `hbase-site.xml` file. The following example sets the HBase site configuration values to those specified in the file `s3://amzn-s3-demo-bucket/my-hbase-site.xml`. Type the following command, replace *myKey* with the name of your EC2 key pair, and replace *amzn-s3-demo-bucket* with the name of your Amazon S3 bucket. **Note** Linux line continuation characters (\$1) are included for readability. They can be removed or used in Linux commands. For Windows, remove them or replace with a caret (^). ``` aws emr create-cluster --name "Test cluster" --ami-version 3.3 \ --applications Name=Hue Name=Hive Name=Pig Name=HBase \ --use-default-roles --ec2-attributes KeyName=myKey \ --instance-type c1.xlarge --instance-count 3 --termination-protected \ --bootstrap-actions Path=s3://elasticmapreduce/bootstrap-actions/configure-hbase,Args=["--site-config-file","s3://amzn-s3-demo-bucket/config.xml"] ``` If you specify more than one option to customize HBase operation, you must prepend each key-value pair with a `-s` option switch, as shown in the following example: ``` --bootstrap-actions s3://elasticmapreduce/bootstrap-actions/configure-hbase,Args=["-s","zookeeper.session.timeout=60000"] ``` With the proxy set and the SSH connection open, you can view the HBase UI by opening a browser window with http://*master-public-dns-name*:60010/master-status, where *master-public-dns-name* is the public DNS address of the master node in the HBase cluster. You can view the current HBase logs by using SSH to connect to the master node, and navigating to the `mnt/var/log/hbase` directory. These logs are not available after the cluster is terminated unless you enable logging to Amazon S3 when the cluster is launched. ## Back up and restore HBase Amazon EMR provides the ability to back up your HBase data to Amazon S3, either manually or on an automated schedule. You can perform both full and incremental backups. After you have a backed-up version of HBase data, you can restore that version to an HBase cluster. You can restore to an HBase cluster that is currently running, or launch a new cluster pre-populated with backed-up data. During the backup process, HBase continues to execute write commands. Although this ensures that your cluster remains available throughout the backup, there is the risk of inconsistency between the data being backed up and any write operations being executed in parallel. To understand the inconsistencies that might arise, you have to consider that HBase distributes write operations across the nodes in its cluster. If a write operation happens after a particular node is polled, that data is not included in the backup archive. You may even find that earlier writes to the HBase cluster (sent to a node that has already been polled) might not be in the backup archive, whereas later writes (sent to a node before it was polled) are included. If a consistent backup is required, you must pause writes to HBase during the initial portion of the backup process, synchronization across nodes. You can do this by specifying the `--consistent` parameter when requesting a backup. With this parameter, writes during this period are queued and executed as soon as the synchronization completes. You can also schedule recurring backups, which resolves any inconsistencies over time, as data that is missed on one backup pass is backed up on the following pass. When you back up HBase data, you should specify a different backup directory for each cluster. An easy way to do this is to use the cluster identifier as part of the path specified for the backup directory. For example, `s3://amzn-s3-demo-bucket/backups/j-3AEXXXXXX16F2`. This ensures that any future incremental backups reference the correct HBase cluster. When you are ready to delete old backup files that are no longer needed, we recommend that you first do a full backup of your HBase data. This ensures that all data is preserved and provides a baseline for future incremental backups. After the full backup is done, you can navigate to the backup location and manually delete the old backup files. The HBase backup process uses S3DistCp for the copy operation, which has certain limitations regarding temporary file storage space. ### Back up and restore HBase using the console The console provides the ability to launch a new cluster and populate it with data from a previous HBase backup. It also gives you the ability to schedule periodic incremental backups of HBase data. Additional backup and restore functionality, such as the ability to restore data to an already running cluster, do manual backups, and schedule automated full backups, is available using the CLI. **To populate a new cluster with archived HBase data using the console** 1. Navigate to the new Amazon EMR console and select **Switch to the old console** from the side navigation. For more information on what to expect when you switch to the old console, see [Using the old console](https://docs.aws.amazon.com/emr/latest/ManagementGuide/whats-new-in-console.html#console-opt-in). 1. Choose **Create cluster**. 1. In the **Software Configuration** section, for **Additional Applications**, choose **HBase** and **Configure and add**. 1. On the **Add Application** dialog box, check **Restore From Backup**. 1. For **Backup Location**, specify the location of the backup yto load into the new HBase cluster. This should be an Amazon S3 URL of the form `s3://amzn-s3-demo-bucket/backups/`. 1. For **Backup Version**, you have the option to specify the name of a backup version to load by setting a value. If you do not set a value for **Backup Version**, Amazon EMR loads the latest backup in the specified location. 1. Choose **Add** and proceed to create the cluster with other options as desired. **To schedule automated backups of HBase data using the console** 1. In the **Software Configuration** section, for **Additional Applications**, choose **HBase** and **Configure and add**. 1. Choose **Schedule Regular Backups**. 1. Specify whether the backups should be consistent. A consistent backup is one that pauses write operations during the initial backup stage, synchronization across nodes. Any write operations thus paused are placed in a queue and resume when the synchronization completes. 1. Set how often backups should occur by entering a number for **Backup Frequency** and choosing **Days**, **Hours**, or **Minutes**. The first automated backup that runs is a full backup; after that, Amazon EMR saves incremental backups based on the schedule that you specify. 1. Specify the location in Amazon S3 where the backups should be stored. Each HBase cluster should be backed up to a separate location in Amazon S3 to ensure that incremental backups are calculated correctly. 1. Specify when the first backup should occur by setting a value for **Backup Start Time**. You can set this to `now`, which causes the first backup to start as soon as the cluster is running, or enter a date and time in [ISO format](http://www.w3.org/TR/NOTE-datetime). For example, 2013-09-26T20:00Z, sets the start time to September 26, 2013 at 8PM UTC. 1. Choose **Add**. 1. Proceed with creating the cluster with other options as desired. ## Monitor HBase with CloudWatch Amazon EMR reports three metrics to CloudWatch that you can use to monitor your HBase backups. These metrics are pushed to CloudWatch at five-minute intervals, and are provided without charge. | Metric | Description | | --- | --- | | HBaseBackupFailed | Whether the last backup failed. This is set to 0 by default and updated to 1 if the previous backup attempt failed. This metric is only reported for HBase clusters. Use case: Monitor HBase backups Units: *Count* | | HBaseMostRecentBackupDuration | The amount of time it took the previous backup to complete. This metric is set regardless of whether the last completed backup succeeded or failed. While the backup is ongoing, this metric returns the number of minutes after the backup started. This metric is only reported for HBase clusters. Use case: Monitor HBase Backups Units: *Minutes* | | HBaseTimeSinceLastSuccessfulBackup | The number of elapsed minutes after the last successful HBase backup started on your cluster. This metric is only reported for HBase clusters. Use case: Monitor HBase backups Units: *Minutes* | ## Configure Ganglia for HBase You configure Ganglia for HBase using the **configure-hbase-for-ganglia** bootstrap action. This bootstrap action configures HBase to publish metrics to Ganglia. You must configure HBase and Ganglia when you launch the cluster; Ganglia reporting cannot be added to a running cluster. Ganglia also stores log files on the server at `/mnt/var/log/ganglia/rrds`. If you configured your cluster to persist log files to an Amazon S3 bucket, the Ganglia log files are persisted there as well. To launch a cluster with Ganglia for HBase, use the **configure-hbase-for-ganglia** bootstrap action as shown in the following example. **Note** Linux line continuation characters (\$1) are included for readability. They can be removed or used in Linux commands. For Windows, remove them or replace with a caret (^). ``` aws emr create-cluster --name "Test cluster" --ami-version 3.3 \ --applications Name=Hue Name=Hive Name=Pig Name=HBase Name=Ganglia \ --use-default-roles --ec2-attributes KeyName=myKey \ --instance-type c1.xlarge --instance-count 3 --termination-protected \ --bootstrap-actions Path=s3://elasticmapreduce/bootstrap-actions/configure-hbase-for-ganglia ``` After the cluster is launched with Ganglia configured, you can access the Ganglia graphs and reports using the graphical interface running on the master node. # Pig application specifics for earlier AMI versions of Amazon EMR ## Supported Pig versions The Pig version you can add to your cluster depends on the version of the Amazon EMR AMI and the version of Hadoop you are using. The table below shows which AMI versions and versions of Hadoop are compatible with the different versions of Pig. We recommend using the latest available version of Pig to take advantage of performance enhancements and new functionality. When you use the API to install Pig, the default version is used unless you specify `--pig-versions` as an argument to the step that loads Pig onto the cluster during the call to [RunJobFlow](https://docs.aws.amazon.com/ElasticMapReduce/latest/API/API_RunJobFlow.html). | Pig version | AMI version | Configuration parameters | Pig version details | | --- | --- | --- | --- | | 0.12.0[Release notes](http://pig.apache.org/releases.html#14+October%2C+2013%3A+release+0.12.0+available)[Documentation](http://pig.apache.org/docs/r0.12.0/) | 3.1.0 and later | `--ami-version 3.1` `--ami-version 3.2` `--ami-version 3.3` | Adds support for the following: [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-3x-pig.html) | | 0.11.1.1[Release notes](http://pig.apache.org/releases.html#1+April%2C+2013%3A+release+0.11.1+available)[Documentation](http://pig.apache.org/docs/r0.11.1/) | 2.2 and later | `--pig-versions 0.11.1.1` `--ami-version 2.2` | Improves performance of LOAD command with PigStorage if input resides in Amazon S3. | | 0.11.1[Release notes](http://pig.apache.org/releases.html#1+April%2C+2013%3A+release+0.11.1+available)[Documentation](http://pig.apache.org/docs/r0.11.1/) | 2.2 and later | `--pig-versions 0.11.1` `--ami-version 2.2` | Adds support for JDK 7, Hadoop 2, Groovy user-defined functions, SchemaTuple optimization, new operators, and more. For more information, see [Pig 0.11.1 change log](http://svn.apache.org/repos/asf/pig/tags/release-0.11.1/CHANGES.txt). | | 0.9.2.2[Release notes](http://pig.apache.org/releases.html#22+January%2C+2012%3A+release+0.9.2+available)[Documentation](http://pig.apache.org/docs/r0.9.2/index.html) | 2.2 and later | `--pig-versions 0.9.2.2` `--ami-version 2.2` | Adds support for Hadoop 1.0.3. | | 0.9.2.1[Release notes](http://pig.apache.org/releases.html#22+January%2C+2012%3A+release+0.9.2+available)[Documentation](http://pig.apache.org/docs/r0.9.2/index.html) | 2.2 and later | `--pig-versions 0.9.2.1` `--ami-version 2.2` | Adds support for MapR. | | 0.9.2[Release notes](http://pig.apache.org/releases.html#22+January%2C+2012%3A+release+0.9.2+available)[Documentation](http://pig.apache.org/docs/r0.9.2/index.html) | 2.2 and later | `--pig-versions 0.9.2` `--ami-version 2.2` | Includes several performance improvements and bug fixes. For complete information about the changes for Pig 0.9.2, go to the [Pig 0.9.2 change log](http://svn.apache.org/repos/asf/pig/tags/release-0.9.2/CHANGES.txt). | | 0.9.1[Release notes](http://pig.apache.org/releases.html#5+October%2C+2011%3A+release+0.9.1+available)[Documentation](http://pig.apache.org/docs/r0.9.1/) | 2.0 | `--pig-versions 0.9.1` `--ami-version 2.0` | | 0.6[Release notes](http://pig.apache.org/releases.html#1+March%2C+2010%3A+release+0.6.0+available) | 1.0 | `--pig-versions 0.6` `--ami-version 1.0` | | 0.3[Release notes](http://pig.apache.org/releases.html#25+June%2C+2009%3A+release+0.3.0+available) | 1.0 | `--pig-versions 0.3` `--ami-version 1.0` | ## Pig version details Amazon EMR supports certain Pig releases that might have additional Amazon EMR patches applied. You can configure which version of Pig to run on Amazon EMR clusters. For more information about how to do this, see [Apache Pig](emr-pig.md). The following sections describe different Pig versions and the patches applied to the versions loaded on Amazon EMR. ### Pig patches This section describes the custom patches applied to Pig versions available with Amazon EMR. #### Pig 0.11.1.1 patches The Amazon EMR version of Pig 0.11.1.1 is a maintenance release that improves performance of LOAD command with PigStorage if the input resides in Amazon S3. #### Pig 0.11.1 patches The Amazon EMR version of Pig 0.11.1 contains all the updates provided by the Apache Software Foundation and the cumulative Amazon EMR patches from Pig version 0.9.2.2. However, there are no new Amazon EMR-specific patches in Pig 0.11.1. #### Pig 0.9.2 patches Apache Pig 0.9.2 is a maintenance release of Pig. The Amazon EMR team has applied the following patches to the Amazon EMR version of Pig 0.9.2. | Patch | Description | | --- | --- | | PIG-1429 | Add the Boolean data type to Pig as a first class data type. For more information, go to [https://issues.apache.org/jira/browse/PIG-1429](https://issues.apache.org/jira/browse/PIG-1429). **Status:** Committed **Fixed in Apache Pig Version:** 0.10 | | PIG-1824 | Support import modules in Jython UDF. For more information, go to [https://issues.apache.org/jira/browse/PIG-1824](https://issues.apache.org/jira/browse/PIG-1824). **Status:** Committed **Fixed in Apache Pig Version:** 0.10 | | PIG-2010 | Bundle registered JARs on the distributed cache. For more information, go to [https://issues.apache.org/jira/browse/PIG-2010](https://issues.apache.org/jira/browse/PIG-2010). **Status:** Committed **Fixed in Apache Pig Version:** 0.11 | | PIG-2456 | Add a \$1/.pigbootup file where the user can specify default Pig statements. For more information, go to [https://issues.apache.org/jira/browse/PIG-2456](https://issues.apache.org/jira/browse/PIG-2456). **Status:** Committed **Fixed in Apache Pig Version:** 0.11 | | PIG-2623 | Support using Amazon S3 paths to register UDFs. For more information, go to [https://issues.apache.org/jira/browse/PIG-2623](https://issues.apache.org/jira/browse/PIG-2623). **Status:** Committed **Fixed in Apache Pig Version:** 0.10, 0.11 | #### Pig 0.9.1 patches The Amazon EMR team has applied the following patches to the Amazon EMR version of Pig 0.9.1. | Patch | Description | | --- | --- | | Support JAR files and Pig scripts in dfs | Add support for running scripts and registering JAR files stored in HDFS, Amazon S3, or other distributed file systems. For more information, go to [https://issues.apache.org/jira/browse/PIG-1505](https://issues.apache.org/jira/browse/PIG-1505). **Status:** Committed **Fixed in Apache Pig Version:** 0.8.0 | | Support multiple file systems in Pig | Add support for Pig scripts to read data from one file system and write it to another. For more information, go to [https://issues.apache.org/jira/browse/PIG-1564](https://issues.apache.org/jira/browse/PIG-1564). **Status:** Not Committed **Fixed in Apache Pig Version:** n/a | | Add Piggybank datetime and string UDFs | Add datetime and string UDFs to support custom Pig scripts. For more information, go to [https://issues.apache.org/jira/browse/PIG-1565](https://issues.apache.org/jira/browse/PIG-1565). **Status:** Not Committed **Fixed in Apache Pig Version:** n/a | ## Interactive and batch Pig clusters Amazon EMR enables you to run Pig scripts in two modes: + Interactive + Batch When you launch a long-running cluster using the console or the AWS CLI, you can connect using **ssh** into the master node as the Hadoop user and use the Grunt shell to develop and run your Pig scripts interactively. Using Pig interactively enables you to revise the Pig script more easily than batch mode. After you successfully revise the Pig script in interactive mode, you can upload the script to Amazon S3 and use batch mode to run the script in production. You can also submit Pig commands interactively on a running cluster to analyze and transform data as needed. In batch mode, you upload your Pig script to Amazon S3, and then submit the work to the cluster as a step. Pig steps can be submitted to a long-running cluster or a transient cluster. # Spark application specifics with earlier AMI versions of Amazon EMR ## Use Spark interactively or in batch mode Amazon EMR enables you to run Spark applications in two modes: + Interactive + Batch When you launch a long-running cluster using the console or the AWS CLI, you can connect using SSH into the master node as the Hadoop user and use the Spark shell to develop and run your Spark applications interactively. Using Spark interactively enables you to prototype or test Spark applications more easily than in a batch environment. After you successfully revise the Spark application in interactive mode, you can put that application JAR or Python program in the file system local to the master node of the cluster on Amazon S3. You can then submit the application as a batch workflow. In batch mode, upload your Spark script to Amazon S3 or the local master node file system, and then submit the work to the cluster as a step. Spark steps can be submitted to a long-running cluster or a transient cluster. ## Creating a cluster with Spark installed **To launch a cluster with Spark installed using the console** 1. Navigate to the new Amazon EMR console and select **Switch to the old console** from the side navigation. For more information on what to expect when you switch to the old console, see [Using the old console](https://docs.aws.amazon.com/emr/latest/ManagementGuide/whats-new-in-console.html#console-opt-in). 1. Choose **Create cluster**. 1. For **Software Configuration**, choose the AMI release version that you require. 1. For **Applications to be installed**, choose **Spark** from the list, then choose **Configure and add**. 1. Add arguments to change the Spark configuration as desired. For more information, see [Configure Spark](#emr-3x-spark-configure). Choose **Add**. 1. Select other options as necessary and then choose **Create cluster**. The following example shows how to create a cluster with Spark using Java: ``` AmazonElasticMapReduceClient emr = new AmazonElasticMapReduceClient(credentials); SupportedProductConfig sparkConfig = new SupportedProductConfig() .withName("Spark"); RunJobFlowRequest request = new RunJobFlowRequest() .withName("Spark Cluster") .withAmiVersion("3.11.0") .withNewSupportedProducts(sparkConfig) .withInstances(new JobFlowInstancesConfig() .withEc2KeyName("myKeyName") .withInstanceCount(1) .withKeepJobFlowAliveWhenNoSteps(true) .withMasterInstanceType("m3.xlarge") .withSlaveInstanceType("m3.xlarge") ); RunJobFlowResult result = emr.runJobFlow(request); ``` ## Configure Spark You configure Spark when you create a cluster by running the bootstrap action located at [awslabs/emr-bootstrap-actions/spark repository on Github](https://github.com/awslabs/emr-bootstrap-actions/tree/master/spark). For arguments that the bootstrap action accepts, see the [README](https://github.com/aws-samples/emr-bootstrap-actions/blob/master/spark/examples/README.md)in that repository. The bootstrap action configures properties in the `$SPARK_CONF_DIR/spark-defaults.conf` file. For more information about settings, see the Spark Configuration topic in Spark documentation. You can replace "latest" in the following URL with the version number of Spark that you are installing, for example, `2.2.0` [http://spark.apache.org/docs/latest/configuration.html](http://spark.apache.org/docs/latest/configuration.html). You can also configure Spark dynamically at the time of each application submission. A setting to automatically maximize the resource allocation for an executor is available using the `spark` configuration file. For more information, see [Overriding Spark default configuration settings](#emr-3x-spark-dynamic-configuration). ### Changing Spark default settings The following example shows how to create a cluster with `spark.executor.memory` set to 2G using the AWS CLI. **Note** Linux line continuation characters (\$1) are included for readability. They can be removed or used in Linux commands. For Windows, remove them or replace with a caret (^). ``` aws emr create-cluster --name "Spark cluster" --ami-version 3.11.0 \ --applications Name=Spark, Args=[-d,spark.executor.memory=2G] --ec2-attributes KeyName=myKey \ --instance-type m3.xlarge --instance-count 3 --use-default-roles ``` ### Submit work to Spark To submit work to a cluster, use a step to run the `spark-submit` script on your EMR cluster. Add the step using the `addJobFlowSteps` method in [AmazonElasticMapReduceClient](https://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/services/elasticmapreduce/AmazonElasticMapReduceClient.html): ``` AWSCredentials credentials = new BasicAWSCredentials(accessKey, secretKey); AmazonElasticMapReduceClient emr = new AmazonElasticMapReduceClient(credentials); StepFactory stepFactory = new StepFactory(); AddJobFlowStepsRequest req = new AddJobFlowStepsRequest(); req.withJobFlowId("j-1K48XXXXXXHCB"); List stepConfigs = new ArrayList(); StepConfig sparkStep = new StepConfig() .withName("Spark Step") .withActionOnFailure("CONTINUE") .withHadoopJarStep(stepFactory.newScriptRunnerStep("/home/hadoop/spark/bin/spark-submit","--class","org.apache.spark.examples.SparkPi","/home/hadoop/spark/lib/spark-examples-1.3.1-hadoop2.4.0.jar","10")); stepConfigs.add(sparkStep); req.withSteps(stepConfigs); AddJobFlowStepsResult result = emr.addJobFlowSteps(req); ``` ### Overriding Spark default configuration settings You may want to override Spark default configuration values on a per-application basis. You can do this when you submit applications using a step, which essentially passes options to `spark-submit`. For example, you may wish to change the memory allocated to an executor process by changing `spark.executor.memory`. You can supply the `--executor-memory` switch with an argument like the following: ``` /home/hadoop/spark/bin/spark-submit --executor-memory 1g --class org.apache.spark.examples.SparkPi /home/hadoop/spark/lib/spark-examples*.jar 10 ``` Similarly, you can tune `--executor-cores` and `--driver-memory`. In a step, you would provide the following arguments to the step: ``` --executor-memory 1g --class org.apache.spark.examples.SparkPi /home/hadoop/spark/lib/spark-examples*.jar 10 ``` You can also tune settings that may not have a built-in switch using the `--conf` option. For more information about other settings that are tunable, see the [Dynamically loading Spark properties](https://spark.apache.org/docs/latest/configuration.html#dynamically-loading-spark-properties) topic in the Apache Spark documentation. # S3DistCp utility differences with earlier AMI versions of Amazon EMR ## S3DistCp versions supported in Amazon EMR The following S3DistCp versions are supported in Amazon EMR AMI releases. S3DistCp versions after 1.0.7 are found on directly on the clusters. Use the JAR in `/home/hadoop/lib` for the latest features. | Version | Description | Release date | | --- | --- | --- | | 1.0.8 | Adds the --appendToLastFile, --requirePreviousManifest, and --storageClass options. | 3 January 2014 | | 1.0.7 | Adds the --s3ServerSideEncryption option. | 2 May 2013 | | 1.0.6 | Adds the --s3Endpoint option. | 6 August 2012 | | 1.0.5 | Improves the ability to specify which version of S3DistCp to run. | 27 June 2012 | | 1.0.4 | Improves the --deleteOnSuccess option. | 19 June 2012 | | 1.0.3 | Adds support for the --numberFiles and --startingIndex options. | 12 June 2012 | | 1.0.2 | Improves file naming when using groups. | 6 June 2012 | | 1.0.1 | Initial release of S3DistCp. | 19 January 2012 | ## Add an S3DistCp copy step to a cluster To add an S3DistCp copy step to a running cluster, type the following command, replace *j-3GYXXXXXX9IOK* with your cluster ID, and replace *amzn-s3-demo-bucket* with your Amazon S3 bucket name. **Note** Linux line continuation characters (\$1) are included for readability. They can be removed or used in Linux commands. For Windows, remove them or replace with a caret (^). ``` aws emr add-steps --cluster-id j-3GYXXXXXX9IOK \ --steps Type=CUSTOM_JAR,Name="S3DistCp step",Jar=/home/hadoop/lib/emr-s3distcp-1.0.jar,\ Args=["--s3Endpoint,s3-eu-west-1.amazonaws.com",\ "--src,s3://amzn-s3-demo-bucket/logs/j-3GYXXXXXX9IOJ/node/",\ "--dest,hdfs:///output",\ "--srcPattern,.*[a-zA-Z,]+"] ``` **Example Load Amazon CloudFront logs into HDFS** This example loads Amazon CloudFront logs into HDFS by adding a step to a running cluster. In the process, it changes the compression format from Gzip (the CloudFront default) to LZO. This is useful because data compressed using LZO can be split into multiple maps as it is decompressed, so you don't have to wait until the compression is complete, as you do with Gzip. This provides better performance when you analyze the data using Amazon EMR. This example also improves performance by using the regular expression specified in the `--groupBy` option to combine all of the logs for a given hour into a single file. Amazon EMR clusters are more efficient when processing a few, large, LZO-compressed files than when processing many, small, Gzip-compressed files. To split LZO files, you must index them and use the hadoop-lzo third-party library. To load Amazon CloudFront logs into HDFS, type the following command, replace *j-3GYXXXXXX9IOK* with your cluster ID, and replace *amzn-s3-demo-bucket* with your Amazon S3 bucket name. Linux line continuation characters (\$1) are included for readability. They can be removed or used in Linux commands. For Windows, remove them or replace with a caret (^). ``` aws emr add-steps --cluster-id j-3GYXXXXXX9IOK \ --steps Type=CUSTOM_JAR,Name="S3DistCp step",Jar=/home/hadoop/lib/emr-s3distcp-1.0.jar,\ Args=["--src,s3://amzn-s3-demo-bucket/cf","--dest,hdfs:///local",\ "--groupBy,.*XABCD12345678.([0-9]+-[0-9]+-[0-9]+-[0-9]+).*",\ "--targetSize,128", "--outputCodec,lzo","--deleteOnSuccess"] ``` Consider the case in which the preceding example is run over the following CloudFront log files. ``` s3://amzn-s3-demo-bucket/cf/XABCD12345678.2012-02-23-01.HLUS3JKx.gz s3://amzn-s3-demo-bucket/cf/XABCD12345678.2012-02-23-01.I9CNAZrg.gz s3://amzn-s3-demo-bucket/cf/XABCD12345678.2012-02-23-02.YRRwERSA.gz s3://amzn-s3-demo-bucket/cf/XABCD12345678.2012-02-23-02.dshVLXFE.gz s3://amzn-s3-demo-bucket/cf/XABCD12345678.2012-02-23-02.LpLfuShd.gz ``` S3DistCp copies, concatenates, and compresses the files into the following two files, where the file name is determined by the match made by the regular expression. ``` hdfs:///local/2012-02-23-01.lzo hdfs:///local/2012-02-23-02.lzo ```