View a markdown version of this page

AWS runtime for Apache Spark (emr-spark-8.1.0) on EKS - Amazon EMR
ReleasesRelease notesChanges and features

AWS runtime for Apache Spark (emr-spark-8.1.0) on EKS

This page describes the new and updated functionality for Amazon EMR that is specific to the Amazon EMR on EKS deployment. For details about Amazon EMR running on Amazon EC2 and about the Amazon EMR Spark 8.1.0 release in general, see AWS runtime for Apache Spark (emr-spark-8.1.0) in the Amazon EMR Release Guide.

AWS runtime for Apache Spark (emr-spark-8.1.0) on EKS

The following emr-spark-8.1.0 image tags are available for AWS runtime for Apache Spark on EKS.

Spark releases

The following emr-spark-8.1.0 image tags are available when you run Spark applications on EKS.

  • emr-spark-8.1.0-latest

  • emr-spark-8.1.0-20260812

  • emr-spark-8.1.0-rapids-latest

  • emr-spark-8.1.0-rapids-20260812

  • notebook-spark/emr-spark-8.1.0-latest

  • notebook-spark/emr-spark-8.1.0-20260812

  • notebook-spark/emr-spark-8.1.0-rapids-latest

  • notebook-spark/emr-spark-8.1.0-rapids-20260812

  • notebook-python/emr-spark-8.1.0-latest

  • notebook-python/emr-spark-8.1.0-20260812

  • livy/emr-spark-8.1.0-latest

  • livy/emr-spark-8.1.0-20260812

Release notes

Release notes for AWS runtime for Apache Spark (emr-spark-8.1.0) on EKS:

  • Supported applications ‐ AWS SDK for Java 2.x 2.44.5, Apache Spark 4.1.1-amzn-0, Apache Hudi 1.1.1-amzn-0, Apache Iceberg 1.11.0-amzn-0, Delta Lake 4.2.0-amzn-0, Apache Spark RAPIDS 26.04.2-amzn-0

  • Supported components ‐ emr-ddb, emr-goodies, hadoop-client, hudi, hudi-spark, iceberg, spark-kubernetes.

  • Supported configuration classifications

    For use with StartJobRun and CreateManagedEndpoint APIs:

    Classifications Descriptions

    core-site

    Change values in the core-site.xml Hadoop file.

    spark-metrics

    Change values in the metrics.properties Spark file.

    spark-defaults

    Change values in the spark-defaults.conf Spark file.

    spark-env

    Change values in the Spark environment.

    spark-hive-site

    Change values in the hive-site.xml Spark file.

    spark-log4j2

    Change values in the log4j2.properties Spark file.

    emr-job-submitter

    Configuration for job submitter pod.

    For use specifically with CreateManagedEndpoint APIs:

    Classifications Descriptions

    jeg-config

    Change values in Jupyter Enterprise Gateway jupyter_enterprise_gateway_config.py file.

    jupyter-kernel-overrides

    Change value for the Kernel Image in Jupyter Kernel Spec file.

    Configuration classifications allow you to customize applications. For more information, see Configure applications.

Changes and features

The following features are included with the emr-spark-8.1.0 release of AWS runtime for Apache Spark on EKS:

  • Spark Connect on Amazon EMR on EKS — Amazon EMR on EKS clusters running emr-spark-8.1.0 now support Spark Connect endpoints with token-based authentication. For setup and configuration, see Run interactive sessions with Amazon EMR on EKS through Spark Connect in the Amazon EMR on EKS Development Guide.

  • Amazon EMR on EKS job runner pod graceful termination — The Amazon EMR on EKS jobsubmitter.gracefulTermination configuration is enabled by default for emr-spark-8.1.0 and subsequent releases. For setup and configuration, see Using job submitter classification in the Amazon EMR on EKS Development Guide.

Resolved issues

  • Livy endpoint stability after Kubernetes token rotation — Fixed an issue where Amazon EMR on EKS managed Livy endpoints could become unusable after the Kubernetes service-account token rotates, previously requiring a pod restart to recover. Livy now automatically refreshes its credentials when the projected service-account token is rotated.

  • Fixed Spark Operator applications on IPv6-only EKS clusters — Spark Operator applications on IPv6-only EKS clusters no longer fail to submit due to a malformed Kubernetes API server URL for emr-spark-8.1.0 and subsequent releases. For setup and configuration, see Running EMR on EKS on IPv6 clusters in the Amazon EMR on EKS Development Guide.