Troubleshooting your Elastic Beanstalk environment

This chapter provides guidance for troubleshooting issues with your Elastic Beanstalk environment. It provides the following information.

Using AWS Systems Manager Elastic Beanstalk runbooks

You can use Systems Manager to troubleshoot your Elastic Beanstalk environments. To help you get started quickly, Systems Manager provides predefined Automation runbooks for Elastic Beanstalk. An Automation runbook is a type of Systems Manager document that defines actions to perform on your environment's instances and other AWS resources.

The document AWSSupport-TroubleshootElasticBeanstalk is an Automation runbook designed to help identify a number of common issues that can degrade your Elastic Beanstalk environment. To do so, it checks components of your environment, including the following: EC2 instances, the VPC, AWS CloudFormation stack, load balancers, Auto Scaling groups, and network configuration associated with security group rules, route tables, and ACLs.

It also provides an option to upload bundled log files from your environment to AWS Support.

For more information, see AWSSupport-TroubleshootElasticBeanstalk in the AWS Systems Manager Automation runbook reference.

After all of the steps successfully complete, the output gives troubleshooting steps and recommendations to restore your environment to a healthy state.

General guidance for troubleshooting your Elastic Beanstalk environment

Error messages can appear on the Events page in the console, in logs, or on the Health page. You can also take actions to recover from a degraded environment that was caused by a recent change. If the health of your environment changes to Red, try the following:

Environments that access secrets and parameters with environment variables

Event: Instance deployment failed to get one or more secrets

This message indicates that Elastic Beanstalk was not able to fetch one or more of the secrets specified during your application deployment.

For examples of these commands, see AWS CLI examples for Elastic Beanstalk. For more information about the API actions for these operations, see the AWS Elastic Beanstalk API Reference.

Event: Instance deployment detected one or more multiline environment values, which are not supported for this platform

Multiline variables are not supported for Amazon Linux 2 platforms, excluding Docker and ECS managed Docker platforms. For available options to proceed, see Multiline values.

Event: CreateEnvironment fails when a secret is specified

When CreateEnvironment fails and you have secrets as environment variables, you need to address the underlying issue and then use UpdateEnvironment to complete the environment setup. Do not use RestartAppServer, as it will not be sufficient to bring the environment up in this situation. For examples of these commands, see AWS CLI examples for Elastic Beanstalk. For more information about the API actions for these operations, see the AWS Elastic Beanstalk API Reference.

Environment creation and instance launches

Event: Failed to Launch Environment

This event occurs when Elastic Beanstalk attempts to launch an environment and encounters failures along the way. Previous events on the Events page will alert you to the root cause of this issue.

Event: Create environment operation is complete, but with command timeouts. Try increasing the timeout period.

Your application may take a long time to deploy if you use configuration files that run commands on the instance, download large files, or install packages. Increase the command timeout to give your application more time to start running during deployments.

Event: The following resource(s) failed to create: [AWSEBInstanceLaunchWaitCondition]

This message indicates that your environment's Amazon EC2 instances did not communicate to Elastic Beanstalk that they were launched successfully. This can occur if the instances do not have Internet connectivity. If you configured your environment to launch instances in a private VPC subnet, ensure that the subnet has a NAT to allow the instances to connect to Elastic Beanstalk.

Event: A Service Role is required in this region. Please add a Service Role option to the environment.

Elastic Beanstalk uses a service role to monitor the resources in your environment and support managed platform updates. See Managing Elastic Beanstalk service roles for more information.

Deployments

Issue: Application becomes unavailable during deployments

Because Elastic Beanstalk uses a drop-in upgrade process, there might be a few seconds of downtime. Use rolling deployments to minimize the effect of deployments on your production environments.

Event: Failed to create the AWS Elastic Beanstalk application version

Your application source bundle may be too large, or you may have reached the application version quota.

Event: Update environment operation is complete, but with command timeouts. Try increasing the timeout period.

Your application may take a long time to deploy if you use configuration files that run commands on the instance, download large files, or install packages. Increase the command timeout to give your application more time to start running during deployments.

Health

Event: CPU Utilization Exceeds 95.00%

Try running more instances, or choose a different instance type.

Event: Elastic Load Balancer awseb-myapp Has Zero Healthy Instances

If your application appears to be working, make sure that your application’s health check URL is configured correctly. If not, check the Health screen and environment logs for more information.

Event: Elastic Load Balancer awseb-myapp Cannot Be Found

Your environment's load balancer may have been removed out-of-band. Only make changes to your environment's resources with the configuration options and extensibility provided by Elastic Beanstalk. Rebuild your environment or launch a new one.

Event: EC2 Instance Launch Failure. Waiting for a New EC2 Instance to Launch...

Availability for your environment's instance type may be low, or you may have reached the instance quota for your account. Check the service health dashboard to ensure that the Elastic Compute Cloud (Amazon EC2) service is green, or request a quota increase.

Configuration

Event: The stack stack_id associated with environment environment-ID is in stack-status state

The underlying AWS CloudFormation stack of your environment may be in a *_FAILED status. This status must be remedied in order to continue Elastic Beanstalk operations on your environment. For more information, see Recovering your Elastic Beanstalk environment from an invalid state.

Event: You cannot configure an Elastic Beanstalk environment with values for both the Elastic Load Balancing Target option and Application Healthcheck URL option

The Target option in the aws:elb:healthcheck namespace is deprecated. Remove the Target option namespace) from your environment and try updating again.

Event: ELB cannot be attached to multiple subnets in the same AZ

This message can be seen if you try to move a load balancer between subnets in the same Availability Zone. Changing subnets on the load balancer requires moving it out of the original availability zone(s) and then back into the original with the desired subnets. During the process, all of your instances will be migrated between AZs, causing significant downtime. Instead, consider creating a new environment and perform a CNAME swap.

Troubleshooting Docker containers

Event: Failed to pull Docker image :latest: Invalid repository name (), only [a-z0-9-_.] are allowed. Tail the logs for more details.

Check the syntax of the dockerrun.aws.json file using a JSON validator. Also verify the dockerfile contents against the requirements described in Preparing your Docker image for deployment to Elastic Beanstalk

Event: No EXPOSE directive found in Dockerfile, abort deployment

The Dockerfile or the dockerrun.aws.json file does not declare the container port. Use the EXPOSE instruction (Dockerfile) or Ports block (dockerrun.aws.json file) to expose a port for incoming traffic.

Event: Failed to download authentication credentials repository from bucket name

The dockerrun.aws.json provides an invalid EC2 key pair and/or S3 bucket for the .dockercfg file. Or, the instance profile does not have GetObject authorization for the S3 bucket. Verify that the .dockercfg file contains a valid S3 bucket and EC2 key pair. Grant permissions for the action s3:GetObject to the IAM role in the instance profile. For details, go to Managing Elastic Beanstalk instance profiles

Event: Activity execution failed, because: WARNING: Invalid auth configuration file

Your authentication file (config.json) is not formatted correctly. See Using images from a private repository in Elastic Beanstalk

FAQ

Question: How can I change my application URL from myapp.us-west-2.elasticbeanstalk.com to www.myapp.com?

In a DNS server, register a CNAME record such as www.mydomain.com CNAME mydomain.elasticbeanstalk.com.

Question: How do I specify a specific Availability Zone for my Elastic Beanstalk application?

You can pick a specific Availability Zone by using the APIs, CLI, Eclipse plugin, or Visual Studio plugin. For instructions about using the Elastic Beanstalk console to specify an Availability Zone, see Auto Scaling your Elastic Beanstalk environment instances.

Question: How do I change my environment's instance type?

To change your environment's instance type go to the environment configuration page and choose Edit in the Instances configuration category. Then, select a new instance type and choose Apply to update your environment. After this, Elastic Beanstalk terminates all running instances and replaces them with new ones.

Question: How do I determine if anyone made configuration changes to an environment?

To see this information, in the navigation pane of the Elastic Beanstalk console choose Change history to display a list of configuration changes for all environments. This list includes the date and time of the change, the configuration parameter and value it was changed to, and the IAM user that made the change. For more information, see Change history.

Question: Can I prevent Amazon EBS volumes from being deleted when instances are terminated?

Instances in your environment use Amazon EBS for storage; however, the root volume is deleted when an instance is terminated by Auto Scaling. We don'trecommend that you store state or other data on your instances. If needed, you can prevent volumes from being deleted with the AWS CLI: $ aws ec2 modify-instance-attribute -b '/dev/sdc=<vol-id>:false as described in the AWS CLI Reference.

Question: How do I delete personal information from my Elastic Beanstalk application?

AWS resources that your Elastic Beanstalk application uses might store personal information. When you terminate an environment, Elastic Beanstalk terminates the resources that it created. Resources you added using configuration files are also terminated. However, if you created AWS resources outside of your Elastic Beanstalk environment and associated them with your application, you might need to manually check that personal information that your application might have stored isn't retained. Throughout this developer guide, whenever we discuss creating additional resources, we also mention when you should consider deleting them.

Common Errors

This topic lists common error messages encountered when using the EB CLI and possible solutions. If you encounter an error message not shown here, use the Feedback links to let us know about it.

ERROR: An error occurred while handling git command. Error code: 128 Error: fatal: Not a valid object name HEAD

Cause: This error message is shown when you have initialized a Git repository but have not yet committed. The EB CLI looks for the HEAD revision when your project folder contains a Git repository.

Solution: Add the files in your project folder to the staging area and commit:

~/my-app$ git add .
~/my-app$ git commit -m "First commit"

ERROR: This branch does not have a default environment. You must either specify an environment by typing "eb status my-env-name" or set a default environment by typing "eb use my-env-name".

Cause: When you create a new branch in git, it is not attached to an Elastic Beanstalk environment by default.

Solution: Run eb list to see a list of available environments. Then run eb use env-name to use one of the available environments.

ERROR: 2.0+ Platforms require a service role. You can provide one with --service-role option

Cause: If you specify an environment name with eb create (for example, eb create my-env), the EB CLI will not attempt to create a service role for you. If you don't have the default service role, the above error is shown.

Solution: Run eb create without an environment name and follow the prompts to create the default service role.

Deployment errors

Your Elastic Beanstalk deployment might response with a 404 (if your application failed to launch) or 500 (if your application fails during runtime) response. To troubleshoot many common issues, you can use the EB CLI to check the status of your deployment, view its logs, gain access to your EC2 instance with SSH, or to open the AWS Management Console page for your application environment.