Beanstalk Cluster architecture

Beanstalk Cluster uses the same Elastic Beanstalk application, application version, environment, and configuration option concepts as Beanstalk Standard. Beanstalk Standard is the EC2-based environment type. Beanstalk Cluster uses a different compute layer and configuration surface. This topic describes the differences by aspect and identifies the applicable environment type.

Compute model

In Beanstalk Standard, Elastic Beanstalk launches Amazon Elastic Compute Cloud (Amazon EC2) instances in an Auto Scaling group dedicated to the environment. The application runs directly on those instances. In a Beanstalk Cluster environment, Elastic Beanstalk instead runs the application as a container image on an Amazon EKS cluster that can be shared by your Beanstalk Cluster environments. Elastic Beanstalk creates and operates the cluster. Elastic Beanstalk schedules the application onto the cluster. Elastic Beanstalk isolates each environment on the cluster and reconciles the number of application replicas to match the configured values. You do not create a cluster, choose which cluster an environment runs on, or select its Kubernetes version.

Several of your environments can run on the same Amazon EKS cluster. Elastic Beanstalk places an environment on the cluster that serves the configured VPC subnets. It creates a cluster the first time those subnets are used; see Environment grouping. Amazon EKS Auto Mode provides node capacity. It adds and removes nodes to fit the containers that are scheduled. Because the cluster can be shared and node capacity is managed by Amazon EKS, instance counts are not configured through the aws:autoscaling:asg namespace. Instead, the number of application replicas is set with the min-replica and max-replica options in the aws:elasticbeanstalk:eks:environment:autoscaling namespace. For the replica bounds and the triggers that change the replica count, see Scaling Beanstalk Cluster environments.

Elastic Beanstalk resolves the environment's configuration from the option settings that you supply, and applies the resolved configuration when it creates or updates the environment. When the same configuration option is supplied more than once, the last occurrence wins. To change the configuration, update the option settings.

Differences from Beanstalk Standard

The following table summarizes the customer-facing differences between Beanstalk Standard and a Beanstalk Cluster environment. Each row links to the topic that covers the Elastic Beanstalk concept in depth.

Customer-provided and service-managed resources

Elastic Beanstalk creates and operates the Amazon EKS cluster that runs a Beanstalk Cluster environment. The cluster and node AWS Identity and Access Management (IAM) roles required by Amazon EKS are customer-provided. We recommend that you use the role names and AWS managed policies described in Permissions for Beanstalk Cluster, because Elastic Beanstalk requires every environment on a cluster to supply the same roles. An application role can also be provided for the running application through Amazon EKS Pod Identity. The application role is selected in the Elastic Beanstalk console during environment creation. For the complete IAM responsibility model and application-role procedure, see Permissions for Beanstalk Cluster.

VPC subnets for the environment are optional. When subnets are omitted, Elastic Beanstalk uses the public subnets of the default VPC. The subnet set determines which cluster runs the environment. For cluster assignment and the infrastructure that Elastic Beanstalk operates, see Environment grouping.

Elastic Beanstalk operates the application on the service-created cluster. It deploys the container image and applies rolling updates through the aws:elasticbeanstalk:eks:environment:deployment namespace. It reconciles the running number of application replicas to the min-replica and max-replica bounds in the configuration. It reports environment-level health through the health statuses described in Monitoring Beanstalk Cluster environments. Elastic Beanstalk tracks each cluster it assigns environments to as service-managed.

Application requirements and limitations

A Beanstalk Cluster environment requires an application that can run as a container image and as one or more identical, interchangeable replicas. A load balancer is optional. When a load balancer is configured, it distributes requests across application replicas. A stateless web service or API whose replicas hold no local state that must survive a restart meets this requirement. The request port, CPU and memory requests, number of application replicas, and deployment behavior are configured through the aws:elasticbeanstalk:eks:* options. Elastic Beanstalk provides the cluster capacity through Amazon EKS Auto Mode.

Confirm the following application requirements and limitations before creating a Beanstalk Cluster environment:

Configuration example

The following AWS CLI request sets the request port, memory request, number of running replicas, and deployment behavior for a Beanstalk Cluster environment. The settings use the aws:elasticbeanstalk:eks:* namespaces.

$ aws elasticbeanstalk update-environment \
    --environment-name my-cluster-env \
    --option-settings \
        Namespace=aws:elasticbeanstalk:eks:environment,OptionName=service-port,Value=8080 \
        Namespace=aws:elasticbeanstalk:eks:environment,OptionName=memory,Value=1Gi \
        Namespace=aws:elasticbeanstalk:eks:environment,OptionName=load-balancer-type,Value=ALB \
        Namespace=aws:elasticbeanstalk:eks:environment:autoscaling,OptionName=min-replica,Value=3 \
        Namespace=aws:elasticbeanstalk:eks:environment:autoscaling,OptionName=max-replica,Value=3 \
        Namespace=aws:elasticbeanstalk:eks:environment:deployment,OptionName=strategy,Value=RollingUpdate \
        Namespace=aws:elasticbeanstalk:eks:environment:deployment:strategy:rolling,OptionName=max-surge,Value=25% \
        Namespace=aws:elasticbeanstalk:eks:environment:deployment:strategy:rolling,OptionName=max-unavailable,Value=0

The strategy option accepts RollingUpdate or Recreate, which the console shows as a rolling update and as all at once. With RollingUpdate, max-surge bounds how many extra replicas Elastic Beanstalk starts during a deployment. The max-unavailable option bounds how many existing replicas it takes down at a time. Each option accepts a count or a percentage. The memory value uses Kubernetes quantity notation, such as 1Gi or 512Mi. For the full set of options, see Configuration options for Beanstalk Cluster environments.

Environment grouping

Elastic Beanstalk groups your Beanstalk Cluster environments onto clusters by the VPC subnets they use:

The first environment you create with a given set of subnets causes Elastic Beanstalk to create a cluster for it, which takes about ten minutes. Elastic Beanstalk reports this in the environment's events:

INFO  Creating CloudFormation stack for cluster infrastructure. This is a one-time operation and generally takes about 10 minutes. stack='beanstalk-cluster-uuid'
INFO  Starting cluster assignment. environment='my-cluster-env'
INFO  Successfully completed cluster assignment. environment='my-cluster-env' clusterArn='arn:aws:eks:us-east-1:111122223333:cluster/beanstalk-cluster-uuid'

Elastic Beanstalk places later environments that use the same subnets on that existing cluster. Their events report the assignment without the stack-creation message. Elastic Beanstalk names both the cluster and the AWS CloudFormation stack that creates it beanstalk-cluster-uuid.

The order in which Elastic Beanstalk considers subnets does not matter; the same subnets in a different order are the same set. Only the subnet set selects the cluster. When a cluster is already registered for the subnet set, the cluster and node role settings that you provide must match the roles registered for that cluster. Elastic Beanstalk rejects conflicting role settings; different role ARNs do not select a different cluster. You supply the observability role as well, and Elastic Beanstalk validates it against the cluster in the same way. The optional application role belongs to the environment and can differ between environments. Configure it with the environment-creation procedure in Configure an application role.

Elastic Beanstalk does not limit how many environments share a cluster. Amazon EKS Auto Mode adds nodes to fit the containers scheduled on the cluster. Cluster sharing does not add an environment-specific scaling limit. Each environment remains subject to its configured replica or autoscaling limits, applicable service quotas, and available capacity. To run an environment on a separate cluster, create it with a different set of subnets.

Isolation between environments that share compute

Subnets are the primary boundary between environments. Because the subnet set selects the cluster, giving a group of environments its own subnets gives that group its own cluster, its own nodes, and its own network. Within a single cluster, Elastic Beanstalk isolates network traffic between Beanstalk Cluster environments by default, and options in the aws:elasticbeanstalk:eks:environment namespace let you permit specific environments to communicate or place an environment on dedicated nodes.

For the boundary each choice gives you, the options that widen it, and what a shared cluster does not separate, see Multi-tenancy for Beanstalk Cluster environments.

Managed infrastructure configuration

Elastic Beanstalk creates each cluster with a fixed configuration that you do not choose:

Because Elastic Beanstalk sets these itself, you configure your application through the aws:elasticbeanstalk:eks:* options rather than configuring the cluster. See Configuration options for Beanstalk Cluster environments.

Cluster configuration drift

Elastic Beanstalk operates a cluster it created only while that cluster matches the expected service-managed configuration. If the infrastructure no longer matches that configuration, Elastic Beanstalk detects configuration drift, pauses cluster maintenance, and reports an environment event:

ERROR  Cluster drift detected for environment 'my-cluster-env'. what changed. Service will skip cluster maintenance for this environment.

While a cluster is drifted:

Drift is recoverable. To recover, revert the change that caused it, so that the cluster matches the configuration that Elastic Beanstalk expects again. The drift event names what changed, which tells you what to revert. Elastic Beanstalk re-evaluates the cluster on the next environment operation, and resumes managing it when the configuration matches. Retry the operation that failed.

If you can't restore the expected configuration, contact AWS Support.

To avoid drift, make environment changes through Elastic Beanstalk operations and configuration options rather than by changing the cluster directly. See Configuration options for Beanstalk Cluster environments.

Cluster deletion

Elastic Beanstalk schedules cluster deletion three hours after you terminate its last environment. If you create another environment with the same subnets during this interval, Elastic Beanstalk cancels the pending cleanup and reuses the existing cluster.