# Configure IAM service roles for Amazon EMR permissions to AWS services and resources
Configure service roles for Amazon EMR
Amazon EMR and applications such as Hadoop and Spark need permissions to access other AWS resources and perform actions when they run. Each cluster in Amazon EMR must have a *service role* and a role for the Amazon EC2 *instance profile*. For more information, see [IAM roles](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles.html) and [Using instance profiles](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_use_switch-role-ec2_instance-profiles.html) in the *IAM User Guide*. The IAM policies attached to these roles provide permissions for the cluster to interoperate with other AWS services on behalf of a user.
An additional role, the Auto Scaling role, is required if your cluster uses automatic scaling in Amazon EMR. The AWS service role for EMR Notebooks is required if you use EMR Notebooks.
Amazon EMR provides default roles and default managed policies that determine permissions for each role. Managed policies are created and maintained by AWS, so they are updated automatically if service requirements change. See [AWS managed policies](https://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies_managed-vs-inline.html#aws-managed-policies.html) in the *IAM User Guide*.
If you are creating a cluster or notebook for the first time in an account, roles for Amazon EMR do not yet exist. After you create them, you can view the roles, the policies attached to them, and the permissions allowed or denied by the policies in the IAM console ([https://console.aws.amazon.com/iam/](https://console.aws.amazon.com/iam/)). You can specify default roles for Amazon EMR to create and use, you can create your own roles and specify them individually when you create a cluster to customize permissions, and you can specify default roles to be used when you create a cluster using the AWS CLI. For more information, see [Customize IAM roles with Amazon EMR](emr-iam-roles-custom.md).
## Modifying identity-based policies for permissions to pass service roles for Amazon EMR
The Amazon EMR full-permissions default managed policies incorporate `iam:PassRole` security configurations, including the following:
+ `iam:PassRole` permissions only for specific default Amazon EMR roles.
+ `iam:PassedToService` conditions that allow you to use the policy with only specified AWS services, such as `elasticmapreduce.amazonaws.com` and `ec2.amazonaws.com`.
You can view the JSON version of the [AmazonEMRFullAccessPolicy\$1v2](https://console.aws.amazon.com/iam/home#policies/arn:aws:iam::aws:policy/service-role/AmazonEMRFullAccessPolicy_v2) and [AmazonEMRServicePolicy\$1v2](https://console.aws.amazon.com/iam/home#policies/arn:aws:iam::aws:policy/service-role/AmazonEMRServicePolicy_v2) policies in the IAM console. We recommend that you create new clusters with the v2 managed policies.
## Service role summary
The following table lists the IAM service roles associated with Amazon EMR for quick reference.
| Function | Default role | Description | Default managed policy |
| --- | --- | --- | --- |
| [Service role for Amazon EMR (EMR role)](emr-iam-role.md) | `EMR_DefaultRole_V2` | Allows Amazon EMR to call other AWS services on your behalf when provisioning resources and performing service-level actions. This role is required for all clusters. | `AmazonEMRServicePolicy_v2` A service-linked role is required to request Spot Instances. If this role doesn't exist, the Amazon EMR service role must have permission to create it or a permission error occurs. If you plan to request Spot Instances, you must update this policy to include a statement that allows the creation of this service-linked role. For more information, see [Service role for Amazon EMR (EMR role)](emr-iam-role.md) and [Service-linked role for Spot Instance requests](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/spot-requests.html#service-linked-roles-spot-instance-requests) in the *Amazon EC2 User Guide*. |
| [Service role for cluster EC2 instances (EC2 instance profile)](emr-iam-role-for-ec2.md) | `EMR_EC2_DefaultRole` | Application processes that run on top of the Hadoop ecosystem on cluster instances use this role when they call other AWS services. For accessing data in Amazon S3 using EMRFS, you can specify different roles to be assumed based on the location of data in Amazon S3. For example, multiple teams can access a single Amazon S3 data "storage account." For more information, see [Configure IAM roles for EMRFS requests to Amazon S3](emr-emrfs-iam-roles.md). This role is required for all clusters. | `AmazonElasticMapReduceforEC2Role`. For more information, see [Service role for cluster EC2 instances (EC2 instance profile)](emr-iam-role-for-ec2.md). |
| [Service role for automatic scaling in Amazon EMR (Auto Scaling role)](emr-iam-role-automatic-scaling.md) | `EMR_AutoScaling_DefaultRole` | Allows additional actions for dynamically scaling environments. Required only for clusters that use automatic scaling in Amazon EMR. For more information, see [Using automatic scaling with a custom policy for instance groups in Amazon EMR](emr-automatic-scaling.md). | `AmazonElasticMapReduceforAutoScalingRole`. For more information, see [Service role for automatic scaling in Amazon EMR (Auto Scaling role)](emr-iam-role-automatic-scaling.md). |
| [Service role for EMR Notebooks](emr-managed-notebooks-service-role.md) | `EMR_Notebooks_DefaultRole` | Provides permissions that an EMR notebook needs to access other AWS resources and perform actions. Required only if EMR Notebooks is used. | `AmazonElasticMapReduceEditorsRole`. For more information, see [Service role for EMR Notebooks](emr-managed-notebooks-service-role.md). `S3FullAccessPolicy` is also attached by default. Following is the contents of this policy. JSON
****
```
{
"Version":"2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:*"
],
"Resource": [
"*"
],
"Sid": "AllowS3"
}
]
}
``` |
| [Service-Linked Role](using-service-linked-roles.md) | `AWSServiceRoleForEMRCleanup` | Amazon EMR automatically creates a service-linked role. If the service for Amazon EMR has lost the ability to clean up Amazon EC2 resources, Amazon EMR can use this role to clean up. If a cluster uses Spot Instances, the permissions policy attached to the [Service role for Amazon EMR (EMR role)](emr-iam-role.md) must allow the creation of a service-linked role. For more information, see [Using service-linked roles for Amazon EMR](using-service-linked-roles.md). | `AmazonEMRCleanupPolicy` |
**Topics**
+ [
## Modifying identity-based policies for permissions to pass service roles for Amazon EMR
](#emr-iam-roles-passrole)
+ [
## Service role summary
](#emr-iam-roles-summary)
+ [
# IAM service roles used by Amazon EMR
](emr-iam-service-roles.md)
+ [
# Customize IAM roles with Amazon EMR
](emr-iam-roles-custom.md)
+ [
# Configure IAM roles for EMRFS requests to Amazon S3
](emr-emrfs-iam-roles.md)
+ [
# Use resource-based policies for Amazon EMR access to AWS Glue Data Catalog
](emr-iam-roles-glue.md)
+ [
# Use IAM roles with applications that call AWS services directly
](emr-iam-roles-calling.md)
+ [
# Allow users and groups to create and modify roles
](emr-iam-roles-create-permissions.md)
# IAM service roles used by Amazon EMR
Service roles used by Amazon EMR
Amazon EMR uses IAM service roles to perform actions on your behalf when provisioning cluster resources, running applications, dynamically scaling resources, and creating and running EMR Notebooks. Amazon EMR uses the following roles when interacting with other AWS services. Each role has a unique function within Amazon EMR. The topics in this section describe the role function and provide the default roles and permissions policy for each role.
If you have application code on your cluster that calls AWS services directly, you may need to use the SDK to specify roles. For more information, see [Use IAM roles with applications that call AWS services directly](emr-iam-roles-calling.md).
**Topics**
+ [
# Service role for Amazon EMR (EMR role)
](emr-iam-role.md)
+ [
# Service role for cluster EC2 instances (EC2 instance profile)
](emr-iam-role-for-ec2.md)
+ [
# Service role for automatic scaling in Amazon EMR (Auto Scaling role)
](emr-iam-role-automatic-scaling.md)
+ [
# Service role for EMR Notebooks
](emr-managed-notebooks-service-role.md)
+ [
# Using service-linked roles for Amazon EMR
](using-service-linked-roles.md)
# Service role for Amazon EMR (EMR role)
Amazon EMR role
The Amazon EMR role defines the allowable actions for Amazon EMR when it provisions resources and performs service-level tasks that aren't performed in the context of an Amazon EC2 instance running within a cluster. For example, the service role is used to provision EC2 instances when a cluster launches.
+ The default role name is `EMR_DefaultRole_V2`.
+ The Amazon EMR scoped default managed policy attached to `EMR_DefaultRole_V2` is `AmazonEMRServicePolicy_v2`. This v2 policy replaces the deprecated default managed policy, `AmazonElasticMapReduceRole`.
`AmazonEMRServicePolicy_v2` depends on scoped down access to resources that Amazon EMR provisions or uses. When you use this policy, you need to pass the user tag `for-use-with-amazon-emr-managed-policies = true` when provisioning the cluster. Amazon EMR will automatically propagate those tags. Additionally, you may need to manually add a user tag to specific types of resources, such as EC2 security groups that were not created by Amazon EMR. See [Tagging resources to use managed policies](emr-managed-iam-policies.md#manually-tagged-resources).
**Important**
Amazon EMR uses this Amazon EMR service role and the `AWSServiceRoleForEMRCleanup` role to clean up cluster resources in your account that you no longer use, such as Amazon EC2 instances. You must include actions for the role policies to delete or terminate the resources. Otherwise, Amazon EMR can’t perform these cleanup actions, and you might incur costs for unused resources that remain on the cluster.
The following shows the contents of the current `AmazonEMRServicePolicy_v2` policy. You can also see the current content of the [https://console.aws.amazon.com/iam/home#policies/arn:aws:iam::aws:policy/service-role/AmazonEMRServicePolicy_v2](https://console.aws.amazon.com/iam/home#policies/arn:aws:iam::aws:policy/service-role/AmazonEMRServicePolicy_v2) managed policy on the IAM console.
------
#### [ JSON ]
****
```
{
"Version":"2012-10-17",
"Statement": [
{
"Sid": "CreateInTaggedNetwork",
"Effect": "Allow",
"Action": [
"ec2:CreateNetworkInterface",
"ec2:RunInstances",
"ec2:CreateFleet",
"ec2:CreateLaunchTemplate",
"ec2:CreateLaunchTemplateVersion"
],
"Resource": [
"arn:aws:ec2:*:*:subnet/*",
"arn:aws:ec2:*:*:security-group/*"
],
"Condition": {
"StringEquals": {
"aws:ResourceTag/for-use-with-amazon-emr-managed-policies": "true"
}
}
},
{
"Sid": "CreateWithEMRTaggedLaunchTemplate",
"Effect": "Allow",
"Action": [
"ec2:CreateFleet",
"ec2:RunInstances",
"ec2:CreateLaunchTemplateVersion"
],
"Resource": [
"arn:aws:ec2:*:*:launch-template/*"
],
"Condition": {
"StringEquals": {
"aws:ResourceTag/for-use-with-amazon-emr-managed-policies": "true"
}
}
},
{
"Sid": "CreateEMRTaggedLaunchTemplate",
"Effect": "Allow",
"Action": [
"ec2:CreateLaunchTemplate"
],
"Resource": [
"arn:aws:ec2:*:*:launch-template/*"
],
"Condition": {
"StringEquals": {
"aws:RequestTag/for-use-with-amazon-emr-managed-policies": "true"
}
}
},
{
"Sid": "CreateEMRTaggedInstancesAndVolumes",
"Effect": "Allow",
"Action": [
"ec2:RunInstances",
"ec2:CreateFleet"
],
"Resource": [
"arn:aws:ec2:*:*:instance/*",
"arn:aws:ec2:*:*:volume/*"
],
"Condition": {
"StringEquals": {
"aws:RequestTag/for-use-with-amazon-emr-managed-policies": "true"
}
}
},
{
"Sid": "ResourcesToLaunchEC2",
"Effect": "Allow",
"Action": [
"ec2:RunInstances",
"ec2:CreateFleet",
"ec2:CreateLaunchTemplate",
"ec2:CreateLaunchTemplateVersion"
],
"Resource": [
"arn:aws:ec2:*:*:network-interface/*",
"arn:aws:ec2:*::image/ami-*",
"arn:aws:ec2:*:*:key-pair/*",
"arn:aws:ec2:*:*:capacity-reservation/*",
"arn:aws:ec2:*:*:placement-group/pg-*",
"arn:aws:ec2:*:*:fleet/*",
"arn:aws:ec2:*:*:dedicated-host/*",
"arn:aws:resource-groups:*:*:group/*"
]
},
{
"Sid": "ManageEMRTaggedResources",
"Effect": "Allow",
"Action": [
"ec2:CreateLaunchTemplateVersion",
"ec2:DeleteLaunchTemplate",
"ec2:DeleteNetworkInterface",
"ec2:ModifyInstanceAttribute",
"ec2:TerminateInstances"
],
"Resource": [
"*"
],
"Condition": {
"StringEquals": {
"aws:ResourceTag/for-use-with-amazon-emr-managed-policies": "true"
}
}
},
{
"Sid": "ManageTagsOnEMRTaggedResources",
"Effect": "Allow",
"Action": [
"ec2:CreateTags",
"ec2:DeleteTags"
],
"Resource": [
"arn:aws:ec2:*:*:instance/*",
"arn:aws:ec2:*:*:volume/*",
"arn:aws:ec2:*:*:network-interface/*",
"arn:aws:ec2:*:*:launch-template/*"
],
"Condition": {
"StringEquals": {
"aws:ResourceTag/for-use-with-amazon-emr-managed-policies": "true"
}
}
},
{
"Sid": "CreateNetworkInterfaceNeededForPrivateSubnet",
"Effect": "Allow",
"Action": [
"ec2:CreateNetworkInterface"
],
"Resource": [
"arn:aws:ec2:*:*:network-interface/*"
],
"Condition": {
"StringEquals": {
"aws:RequestTag/for-use-with-amazon-emr-managed-policies": "true"
}
}
},
{
"Sid": "TagOnCreateTaggedEMRResources",
"Effect": "Allow",
"Action": [
"ec2:CreateTags"
],
"Resource": [
"arn:aws:ec2:*:*:network-interface/*",
"arn:aws:ec2:*:*:instance/*",
"arn:aws:ec2:*:*:volume/*",
"arn:aws:ec2:*:*:launch-template/*"
],
"Condition": {
"StringEquals": {
"ec2:CreateAction": [
"RunInstances",
"CreateFleet",
"CreateLaunchTemplate",
"CreateNetworkInterface"
]
}
}
},
{
"Sid": "TagPlacementGroups",
"Effect": "Allow",
"Action": [
"ec2:CreateTags",
"ec2:DeleteTags"
],
"Resource": [
"arn:aws:ec2:*:*:placement-group/pg-*"
]
},
{
"Sid": "ListActionsForEC2Resources",
"Effect": "Allow",
"Action": [
"ec2:DescribeAccountAttributes",
"ec2:DescribeCapacityReservations",
"ec2:DescribeDhcpOptions",
"ec2:DescribeImages",
"ec2:DescribeInstances",
"ec2:DescribeInstanceTypeOfferings",
"ec2:DescribeLaunchTemplates",
"ec2:DescribeNetworkAcls",
"ec2:DescribeNetworkInterfaces",
"ec2:DescribePlacementGroups",
"ec2:DescribeRouteTables",
"ec2:DescribeSecurityGroups",
"ec2:DescribeSubnets",
"ec2:DescribeVolumes",
"ec2:DescribeVolumeStatus",
"ec2:DescribeVpcAttribute",
"ec2:DescribeVpcEndpoints",
"ec2:DescribeVpcs"
],
"Resource": [
"*"
]
},
{
"Sid": "CreateDefaultSecurityGroupWithEMRTags",
"Effect": "Allow",
"Action": [
"ec2:CreateSecurityGroup"
],
"Resource": [
"arn:aws:ec2:*:*:security-group/*"
],
"Condition": {
"StringEquals": {
"aws:RequestTag/for-use-with-amazon-emr-managed-policies": "true"
}
}
},
{
"Sid": "CreateDefaultSecurityGroupInVPCWithEMRTags",
"Effect": "Allow",
"Action": [
"ec2:CreateSecurityGroup"
],
"Resource": [
"arn:aws:ec2:*:*:vpc/*"
],
"Condition": {
"StringEquals": {
"aws:ResourceTag/for-use-with-amazon-emr-managed-policies": "true"
}
}
},
{
"Sid": "TagOnCreateDefaultSecurityGroupWithEMRTags",
"Effect": "Allow",
"Action": [
"ec2:CreateTags"
],
"Resource": [
"arn:aws:ec2:*:*:security-group/*"
],
"Condition": {
"StringEquals": {
"aws:RequestTag/for-use-with-amazon-emr-managed-policies": "true",
"ec2:CreateAction": "CreateSecurityGroup"
}
}
},
{
"Sid": "ManageSecurityGroups",
"Effect": "Allow",
"Action": [
"ec2:AuthorizeSecurityGroupEgress",
"ec2:AuthorizeSecurityGroupIngress",
"ec2:RevokeSecurityGroupEgress",
"ec2:RevokeSecurityGroupIngress"
],
"Resource": [
"*"
],
"Condition": {
"StringEquals": {
"aws:ResourceTag/for-use-with-amazon-emr-managed-policies": "true"
}
}
},
{
"Sid": "CreateEMRPlacementGroups",
"Effect": "Allow",
"Action": [
"ec2:CreatePlacementGroup"
],
"Resource": [
"arn:aws:ec2:*:*:placement-group/pg-*"
]
},
{
"Sid": "DeletePlacementGroups",
"Effect": "Allow",
"Action": [
"ec2:DeletePlacementGroup"
],
"Resource": [
"*"
]
},
{
"Sid": "AutoScaling",
"Effect": "Allow",
"Action": [
"application-autoscaling:DeleteScalingPolicy",
"application-autoscaling:DeregisterScalableTarget",
"application-autoscaling:DescribeScalableTargets",
"application-autoscaling:DescribeScalingPolicies",
"application-autoscaling:PutScalingPolicy",
"application-autoscaling:RegisterScalableTarget"
],
"Resource": [
"*"
]
},
{
"Sid": "ResourceGroupsForCapacityReservations",
"Effect": "Allow",
"Action": [
"resource-groups:ListGroupResources"
],
"Resource": [
"*"
]
},
{
"Sid": "AutoScalingCloudWatch",
"Effect": "Allow",
"Action": [
"cloudwatch:PutMetricAlarm",
"cloudwatch:DeleteAlarms",
"cloudwatch:DescribeAlarms"
],
"Resource": [
"arn:aws:cloudwatch:*:*:alarm:*_EMR_Auto_Scaling"
]
},
{
"Sid": "PassRoleForAutoScaling",
"Effect": "Allow",
"Action": [
"iam:PassRole"
],
"Resource": [
"arn:aws:iam::*:role/EMR_AutoScaling_DefaultRole"
],
"Condition": {
"StringLike": {
"iam:PassedToService": "application-autoscaling.amazonaws.com*"
}
}
},
{
"Sid": "PassRoleForEC2",
"Effect": "Allow",
"Action": [
"iam:PassRole"
],
"Resource": [
"arn:aws:iam::*:role/EMR_EC2_DefaultRole"
],
"Condition": {
"StringLike": {
"iam:PassedToService": "ec2.amazonaws.com*"
}
}
},
{
"Sid": "CreateAndModifyEmrServiceVPCEndpoint",
"Effect": "Allow",
"Action": [
"ec2:ModifyVpcEndpoint",
"ec2:CreateVpcEndpoint"
],
"Resource": [
"arn:aws:ec2:*:*:vpc-endpoint/*",
"arn:aws:ec2:*:*:subnet/*",
"arn:aws:ec2:*:*:security-group/*",
"arn:aws:ec2:*:*:vpc/*"
],
"Condition": {
"StringEquals": {
"aws:ResourceTag/for-use-with-amazon-emr-managed-policies": "true"
}
}
},
{
"Sid": "CreateEmrServiceVPCEndpoint",
"Effect": "Allow",
"Action": [
"ec2:CreateVpcEndpoint"
],
"Resource": [
"arn:aws:ec2:*:*:vpc-endpoint/*"
],
"Condition": {
"StringEquals": {
"aws:RequestTag/for-use-with-amazon-emr-managed-policies": "true",
"aws:RequestTag/Name": "emr-service-vpce"
}
}
},
{
"Sid": "TagEmrServiceVPCEndpoint",
"Effect": "Allow",
"Action": [
"ec2:CreateTags"
],
"Resource": [
"arn:aws:ec2:*:*:vpc-endpoint/*"
],
"Condition": {
"StringEquals": {
"ec2:CreateAction": "CreateVpcEndpoint",
"aws:RequestTag/for-use-with-amazon-emr-managed-policies": "true",
"aws:RequestTag/Name": "emr-service-vpce"
}
}
}
]
}
```
------
Your service role should use the following trust policy.
**Important**
The following trust policy includes the [https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_condition-keys.html#condition-keys-sourcearn](https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_condition-keys.html#condition-keys-sourcearn) and [https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_condition-keys.html#condition-keys-sourceaccount](https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_condition-keys.html#condition-keys-sourceaccount) global condition keys, which limit the permissions that you give Amazon EMR to particular resources in your account. Using them can protect you against [the confused deputy problem](https://docs.aws.amazon.com/IAM/latest/UserGuide/confused-deputy.html).
------
#### [ JSON ]
****
```
{
"Version":"2012-10-17",
"Statement": [
{
"Sid": "AllowSTSAssumerole",
"Effect": "Allow",
"Action": [
"sts:AssumeRole"
],
"Resource": "arn:aws:iam::123456789012:role/EMRServiceRole",
"Condition": {
"StringEquals": {
"aws:SourceAccount": "123456789012"
},
"ArnLike": {
"aws:SourceArn": "arn:aws:elasticmapreduce:*:123456789012:*"
}
}
}
]
}
```
------
# Service role for cluster EC2 instances (EC2 instance profile)
EC2 instance profile
The service role for cluster EC2 instances (also called the EC2 instance profile for Amazon EMR) is a special type of service role that is assigned to every EC2 instance in an Amazon EMR cluster when the instance launches. Application processes that run on top of the Hadoop ecosystem assume this role for permissions to interact with other AWS services.
For more information about service roles for EC2 instances, see [Using an IAM role to grant permissions to applications running on Amazon EC2 instances](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_use_switch-role-ec2.html) in the *IAM User Guide*.
**Important**
The default service role for cluster EC2 instances and its associated AWS default managed policy, `AmazonElasticMapReduceforEC2Role` are on the path to deprecation, with no replacement AWS managed policies provided. You'll need to create and specify an instance profile to replace the deprecated role and default policy.
## Default role and managed policy
Default role and managed policy
+ The default role name is `EMR_EC2_DefaultRole`.
+ The `EMR_EC2_DefaultRole` default managed policy, `AmazonElasticMapReduceforEC2Role`, is nearing end of support. Instead of using a default managed policy for the EC2 instance profile, apply resource-based policies to S3 buckets and other resources that Amazon EMR needs, or use your own customer-managed policy with an IAM role as an instance profile. For more information, see [Creating a service role for cluster EC2 instances with least-privilege permissions](#emr-ec2-role-least-privilege).
The following shows the contents of version 3 of `AmazonElasticMapReduceforEC2Role`.
------
#### [ JSON ]
****
```
{
"Version":"2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Resource": [
"*"
],
"Action": [
"cloudwatch:*",
"dynamodb:*",
"ec2:Describe*",
"elasticmapreduce:Describe*",
"elasticmapreduce:ListBootstrapActions",
"elasticmapreduce:ListClusters",
"elasticmapreduce:ListInstanceGroups",
"elasticmapreduce:ListInstances",
"elasticmapreduce:ListSteps",
"kinesis:CreateStream",
"kinesis:DeleteStream",
"kinesis:DescribeStream",
"kinesis:GetRecords",
"kinesis:GetShardIterator",
"kinesis:MergeShards",
"kinesis:PutRecord",
"kinesis:SplitShard",
"rds:Describe*",
"s3:*",
"sdb:*",
"sns:*",
"sqs:*",
"glue:CreateDatabase",
"glue:UpdateDatabase",
"glue:DeleteDatabase",
"glue:GetDatabase",
"glue:GetDatabases",
"glue:CreateTable",
"glue:UpdateTable",
"glue:DeleteTable",
"glue:GetTable",
"glue:GetTables",
"glue:GetTableVersions",
"glue:CreatePartition",
"glue:BatchCreatePartition",
"glue:UpdatePartition",
"glue:DeletePartition",
"glue:BatchDeletePartition",
"glue:GetPartition",
"glue:GetPartitions",
"glue:BatchGetPartition",
"glue:CreateUserDefinedFunction",
"glue:UpdateUserDefinedFunction",
"glue:DeleteUserDefinedFunction",
"glue:GetUserDefinedFunction",
"glue:GetUserDefinedFunctions"
],
"Sid": "AllowCLOUDWATCH"
}
]
}
```
------
Your service role should use the following trust policy.
------
#### [ JSON ]
****
```
{
"Version":"2012-10-17",
"Statement": [
{
"Sid": "AllowSTSAssumerole",
"Effect": "Allow",
"Action": [
"sts:AssumeRole"
],
"Resource": "arn:aws:iam::123456789012:role/EMR_EC2_DefaultRole"
}
]
}
```
------
## Creating a service role for cluster EC2 instances with least-privilege permissions
Create a least privilege instance profile
As a best practice, we strongly recommend that you create a service role for cluster EC2 instances and permissions policy that has the minimum permissions to other AWS services required by your application.
The default managed policy, `AmazonElasticMapReduceforEC2Role`, provides permissions that make it easy to launch an initial cluster. However, `AmazonElasticMapReduceforEC2Role` is on the path to deprecation and Amazon EMR will not provide a replacement AWS managed default policy for the deprecated role. To launch an initial cluster, you need to provide a customer managed resource-based or ID-based policy.
The following policy statements provide examples of the permissions required for different features of Amazon EMR. We recommend that you use these permissions to create a permissions policy that restricts access to only those features and resources that your cluster requires. All example policy statements use the *us-west-2* Region and the fictional AWS account ID *123456789012*. Replace these as appropriate for your cluster.
For more information about creating and specifying custom roles, see [Customize IAM roles with Amazon EMR](emr-iam-roles-custom.md).
**Note**
If you create a custom EMR role for EC2, follow the basic work flow, which automatically creates an instance profile of the same name. Amazon EC2 allows you to create instance profiles and roles with different names, but Amazon EMR does not support this configuration, and it results in an "invalid instance profile" error when you create the cluster.
### Reading and writing data to Amazon S3 using EMRFS
When an application running on an Amazon EMR cluster references data using the `s3://mydata` format, Amazon EMR uses the EC2 instance profile to make the request. Clusters typically read and write data to Amazon S3 in this way, and Amazon EMR uses the permissions attached to the service role for cluster EC2 instances by default. For more information, see [Configure IAM roles for EMRFS requests to Amazon S3](emr-emrfs-iam-roles.md).
Because IAM roles for EMRFS will fall back to the permissions attached to the service role for cluster EC2 instances, as a best practice, we recommend that you use IAM roles for EMRFS, and limit the EMRFS and Amazon S3 permissions attached to the service role for cluster EC2 instances.
The sample statement below demonstrates the permissions that EMRFS requires to make requests to Amazon S3.
+ *my-data-bucket-in-s3-for-emrfs-reads-and-writes* specifies the bucket in Amazon S3 where the cluster reads and writes data and all sub-folders using */\$1*. Add only those buckets and folders that your application requires.
+ The policy statement that allows `dynamodb` actions is required only if EMRFS consistent view is enabled. *EmrFSMetadata* specifies the default folder for EMRFS consistent view.
------
#### [ JSON ]
****
```
{
"Version":"2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:AbortMultipartUpload",
"s3:CreateBucket",
"s3:DeleteObject",
"s3:GetBucketVersioning",
"s3:GetObject",
"s3:GetObjectTagging",
"s3:GetObjectVersion",
"s3:ListBucket",
"s3:ListBucketMultipartUploads",
"s3:ListBucketVersions",
"s3:ListMultipartUploadParts",
"s3:PutBucketVersioning",
"s3:PutObject",
"s3:PutObjectTagging"
],
"Resource": [
"arn:aws:s3:::my-data-bucket-in-s3-for-emrfs-reads-and-writes",
"arn:aws:s3:::my-data-bucket-in-s3-for-emrfs-reads-and-writes/*"
],
"Sid": "AllowS3Abortmultipartupload"
},
{
"Effect": "Allow",
"Action": [
"dynamodb:CreateTable",
"dynamodb:BatchGetItem",
"dynamodb:BatchWriteItem",
"dynamodb:PutItem",
"dynamodb:DescribeTable",
"dynamodb:DeleteItem",
"dynamodb:GetItem",
"dynamodb:Scan",
"dynamodb:Query",
"dynamodb:UpdateItem",
"dynamodb:DeleteTable",
"dynamodb:UpdateTable"
],
"Resource": [
"arn:aws:dynamodb:*:123456789012:table/EmrFSMetadata"
],
"Sid": "AllowDYNAMODBCreatetable"
},
{
"Effect": "Allow",
"Action": [
"cloudwatch:PutMetricData",
"dynamodb:ListTables",
"s3:ListBucket"
],
"Resource": [
"*"
],
"Sid": "AllowCLOUDWATCHPutmetricdata"
},
{
"Effect": "Allow",
"Action": [
"sqs:GetQueueUrl",
"sqs:ReceiveMessage",
"sqs:DeleteQueue",
"sqs:SendMessage",
"sqs:CreateQueue"
],
"Resource": [
"arn:aws:sqs:*:123456789012:EMRFS-Inconsistency-*"
],
"Sid": "AllowSQSGetqueueurl"
}
]
}
```
------
### Archiving log files to Amazon S3
The following policy statement allows the Amazon EMR cluster to archive log files to the Amazon S3 location specified. In the example below, when the cluster was created, *s3://MyLoggingBucket/MyEMRClusterLogs* was specified using the **Log folder S3 location** in the console, using the `--log-uri` option from the AWS CLI, or using the `LogUri` parameter in the `RunJobFlow` command. For more information, see [Archive log files to Amazon S3](emr-plan-debugging.md#emr-plan-debugging-logs-archive).
------
#### [ JSON ]
****
```
{
"Version":"2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:PutObject"
],
"Resource": [
"arn:aws:s3:::MyLoggingBucket/MyEMRClusterLogs/*"
],
"Sid": "AllowS3Putobject"
}
]
}
```
------
### Using the AWS Glue Data Catalog
The following policy statement allows actions that are required if you use the AWS Glue Data Catalog as the metastore for applications. For more information, see [Using the AWS Glue Data Catalog as the metastore for Spark SQL](https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-spark-glue.html), [Using the AWS Glue Data Catalog as the metastore for Hive](https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-hive-metastore-glue.html), and [Using Presto with the AWS Glue Data Catalog](https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-presto-glue.html) in the *Amazon EMR Release Guide*.
------
#### [ JSON ]
****
```
{
"Version":"2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"glue:CreateDatabase",
"glue:UpdateDatabase",
"glue:DeleteDatabase",
"glue:GetDatabase",
"glue:GetDatabases",
"glue:CreateTable",
"glue:UpdateTable",
"glue:DeleteTable",
"glue:GetTable",
"glue:GetTables",
"glue:GetTableVersions",
"glue:CreatePartition",
"glue:BatchCreatePartition",
"glue:UpdatePartition",
"glue:DeletePartition",
"glue:BatchDeletePartition",
"glue:GetPartition",
"glue:GetPartitions",
"glue:BatchGetPartition",
"glue:CreateUserDefinedFunction",
"glue:UpdateUserDefinedFunction",
"glue:DeleteUserDefinedFunction",
"glue:GetUserDefinedFunction",
"glue:GetUserDefinedFunctions"
],
"Resource": [
"*"
],
"Sid": "AllowGLUECreatedatabase"
}
]
}
```
------
# Service role for automatic scaling in Amazon EMR (Auto Scaling role)
Auto Scaling role
The Auto Scaling role for Amazon EMR performs a similar function as the service role, but allows additional actions for dynamically scaling environments.
+ The default role name is `EMR_AutoScaling_DefaultRole`.
+ The default managed policy attached to `EMR_AutoScaling_DefaultRole` is `AmazonElasticMapReduceforAutoScalingRole`.
The contents of version 1 of `AmazonElasticMapReduceforAutoScalingRole` are shown below.
------
#### [ JSON ]
****
```
{
"Version":"2012-10-17",
"Statement": [
{
"Action": [
"cloudwatch:DescribeAlarms",
"elasticmapreduce:ListInstanceGroups",
"elasticmapreduce:ModifyInstanceGroups"
],
"Effect": "Allow",
"Resource": [
"*"
],
"Sid": "AllowCLOUDWATCHDescribealarms"
}
]
}
```
------
Your service role should use the following trust policy.
**Important**
The following trust policy includes the [https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_condition-keys.html#condition-keys-sourcearn](https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_condition-keys.html#condition-keys-sourcearn) and [https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_condition-keys.html#condition-keys-sourceaccount](https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_condition-keys.html#condition-keys-sourceaccount) global condition keys, which limit the permissions that you give Amazon EMR to particular resources in your account. Using them can protect you against [the confused deputy problem](https://docs.aws.amazon.com/IAM/latest/UserGuide/confused-deputy.html).
------
#### [ JSON ]
****
```
{
"Version":"2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"sts:AssumeRole"
],
"Resource": "arn:aws:iam::123456789012:role/ApplicationAutoScalingEMRRole",
"Condition": {
"StringEquals": {
"aws:SourceAccount": "123456789012"
},
"ArnLike": {
"aws:SourceArn": "arn:aws:application-autoscaling:*:123456789012:scalable-target/*"
}
},
"Sid": "AllowSTSAssumerole"
}
]
}
```
------
# Service role for EMR Notebooks
EMR Notebooks role
Each EMR notebook needs permissions to access other AWS resources and perform actions. The IAM policies attached to this service role provide permissions for the notebook to interoperate with other AWS services. When you create a notebook using the AWS Management Console, you specify an *AWS service role*. You can use the default role, `EMR_Notebooks_DefaultRole`, or specify a role that you create. If a notebook has not been created before, you can choose to create the default role.
+ The default role name is `EMR_Notebooks_DefaultRole`.
+ The default managed policies attached to `EMR_Notebooks_DefaultRole` are `AmazonElasticMapReduceEditorsRole` and `S3FullAccessPolicy`.
Your service role should use the following trust policy.
**Important**
The following trust policy includes the [https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_condition-keys.html#condition-keys-sourcearn](https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_condition-keys.html#condition-keys-sourcearn) and [https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_condition-keys.html#condition-keys-sourceaccount](https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_condition-keys.html#condition-keys-sourceaccount) global condition keys, which limit the permissions that you give Amazon EMR to particular resources in your account. Using them can protect you against [the confused deputy problem](https://docs.aws.amazon.com/IAM/latest/UserGuide/confused-deputy.html).
------
#### [ JSON ]
****
```
{
"Version":"2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"sts:AssumeRole"
],
"Resource": "arn:aws:iam::123456789012:role/EMRServiceRole",
"Condition": {
"StringEquals": {
"aws:SourceAccount": "123456789012"
},
"ArnLike": {
"aws:SourceArn": "arn:aws:elasticmapreduce:*:123456789012:*"
}
},
"Sid": "AllowSTSAssumerole"
}
]
}
```
------
The contents of version 1 of `AmazonElasticMapReduceEditorsRole` are as follows.
------
#### [ JSON ]
****
```
{
"Version":"2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"ec2:AuthorizeSecurityGroupEgress",
"ec2:AuthorizeSecurityGroupIngress",
"ec2:CreateSecurityGroup",
"ec2:DescribeSecurityGroups",
"ec2:RevokeSecurityGroupEgress",
"ec2:CreateNetworkInterface",
"ec2:CreateNetworkInterfacePermission",
"ec2:DeleteNetworkInterface",
"ec2:DeleteNetworkInterfacePermission",
"ec2:DescribeNetworkInterfaces",
"ec2:ModifyNetworkInterfaceAttribute",
"ec2:DescribeTags",
"ec2:DescribeInstances",
"ec2:DescribeSubnets",
"ec2:DescribeVpcs",
"elasticmapreduce:ListInstances",
"elasticmapreduce:DescribeCluster",
"elasticmapreduce:ListSteps"
],
"Resource": [
"*"
],
"Sid": "AllowEC2Authorizesecuritygroupegress"
},
{
"Effect": "Allow",
"Action": [
"ec2:CreateTags"
],
"Resource": [
"arn:aws:ec2:*:*:network-interface/*"
],
"Condition": {
"ForAllValues:StringEquals": {
"aws:TagKeys": [
"aws:elasticmapreduce:editor-id",
"aws:elasticmapreduce:job-flow-id"
]
}
},
"Sid": "AllowEC2Createtags"
}
]
}
```
------
Following is the contents of the `S3FullAccessPolicy`. The `S3FullAccessPolicy` allows your service role for EMR Notebooks to perform all Amazon S3 actions on objects in your AWS account. When you create a custom service role for EMR Notebooks, you must give your service role Amazon S3 permissions.
------
#### [ JSON ]
****
```
{
"Version":"2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:*"
],
"Resource": [
"*"
],
"Sid": "AllowS3"
}
]
}
```
------
You can scope down read and write access for your service role to the Amazon S3 location where you want to save your notebook files. Use the following minimum set of Amazon S3 permissions.
```
"s3:PutObject",
"s3:GetObject",
"s3:GetEncryptionConfiguration",
"s3:ListBucket",
"s3:DeleteObject"
```
If your Amazon S3 bucket is encrypted, you must include the following permissions for AWS Key Management Service.
```
"kms:Decrypt",
"kms:GenerateDataKey",
"kms:ReEncryptFrom",
"kms:ReEncryptTo",
"kms:DescribeKey"
```
When you link Git repositories to your notebook and need to create a secret for the repository, you must add the `secretsmanager:GetSecretValue` permission in the IAM policy attached to the service role for Amazon EMR notebooks. An example policy is demonstrated below:
------
#### [ JSON ]
****
```
{
"Version":"2012-10-17",
"Statement": [
{
"Sid": "VisualEditor0",
"Effect": "Allow",
"Action": [
"secretsmanager:GetSecretValue"
],
"Resource": [
"*"
]
}
]
}
```
------
## EMR Notebooks service role permissions
This table lists the actions that EMR Notebooks takes using the service role, along with the permissions that are needed for each action.
****
| Action | Permissions |
| --- | --- |
| Establish a secure network channel between a notebook and an Amazon EMR cluster, and perform necessary cleanup actions. | "ec2:CreateNetworkInterface",
"ec2:CreateNetworkInterfacePermission",
"ec2:DeleteNetworkInterface",
"ec2:DeleteNetworkInterfacePermission",
"ec2:DescribeNetworkInterfaces",
"ec2:ModifyNetworkInterfaceAttribute",
"ec2:AuthorizeSecurityGroupEgress",
"ec2:AuthorizeSecurityGroupIngress",
"ec2:CreateSecurityGroup",
"ec2:DescribeSecurityGroups",
"ec2:RevokeSecurityGroupEgress",
"ec2:DescribeTags",
"ec2:DescribeInstances",
"ec2:DescribeSubnets",
"ec2:DescribeVpcs",
"elasticmapreduce:ListInstances",
"elasticmapreduce:DescribeCluster",
"elasticmapreduce:ListSteps"
|
| Use Git credentials stored in AWS Secrets Manager to link Git repositories to a notebook. | "secretsmanager:GetSecretValue"
|
| Apply AWS tags to the network interface and default security groups that EMR Notebooks creates while setting up the secure network channel. For more information, see [Tagging AWS resources](https://docs.aws.amazon.com/general/latest/gr/aws_tagging.html). | "ec2:CreateTags"
|
| Access or upload notebook files and metadata to Amazon S3. | "s3:PutObject",
"s3:GetObject",
"s3:GetEncryptionConfiguration",
"s3:ListBucket",
"s3:DeleteObject"
The following permissions are only required if you use an encrypted Amazon S3 bucket. "kms:Decrypt",
"kms:GenerateDataKey",
"kms:ReEncryptFrom",
"kms:ReEncryptTo",
"kms:DescribeKey"
|
## EMR Notebooks updates to AWS managed policies
View details about updates to AWS managed policies for EMR Notebooks since March 1, 2021.
| Change | Description | Date |
| --- | --- | --- |
| AmazonElasticMapReduceEditorsRole - Added permissions | EMR Notebooks added `ec2:describeVPCs` and `elastmicmapreduce:ListSteps` permissions to `AmazonElasticMapReduceEditorsRole`. | Feb 8, 2023 |
| EMR Notebooks started tracking changes | EMR Notebooks started tracking changes for its AWS managed policies. | Feb 8, 2023 |
# Using service-linked roles for Amazon EMR
Using service-linked roles
Amazon EMR uses AWS Identity and Access Management (IAM) [service-linked roles](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_terms-and-concepts.html#iam-term-service-linked-role). A service-linked role is a unique type of IAM role that is linked directly to Amazon EMR. Service-linked roles are predefined by Amazon EMR and include all the permissions that the service requires to call other AWS services on your behalf.
**Topics**
+ [
# Using service-linked roles for Amazon EMR for cleanup
](using-service-linked-roles-cleanup.md)
+ [
# Using service-linked roles with Amazon EMR for write-ahead logging
](using-service-linked-roles-wal.md)
For information about other services that support service-linked roles, see [AWS services that work with IAM](https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_aws-services-that-work-with-iam.html) and look for the services that have **Yes** in the **Service-linked roles** column. Choose a **Yes** with a link to view the service-linked role documentation for that service.
# Using service-linked roles for Amazon EMR for cleanup
Cleanup role
Amazon EMR uses AWS Identity and Access Management (IAM) [service-linked roles](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_terms-and-concepts.html#iam-term-service-linked-role). A service-linked role is a unique type of IAM role that is linked directly to Amazon EMR. Service-linked roles are predefined by Amazon EMR and include all the permissions that the service requires to call other AWS services on your behalf.
Service-linked roles work together with the Amazon EMR service role and Amazon EC2 instance profile for Amazon EMR. For more information about the service role and instance profile, see [Configure IAM service roles for Amazon EMR permissions to AWS services and resources](emr-iam-roles.md).
A service-linked role makes setting up Amazon EMR easier because you don’t have to manually add the necessary permissions. Amazon EMR defines the permissions of its service-linked roles, and unless defined otherwise, only Amazon EMR can assume its roles. The defined permissions include the trust policy and the permissions policy, and that permissions policy cannot be attached to any other IAM entity.
You can delete this service-linked role for Amazon EMR only after you delete any related resources and terminate all EMR clusters in the account. This protects your Amazon EMR resources so that you can't inadvertently remove permission to access the resources.
## Using service-linked roles for cleanup
Amazon EMR uses the service-based **AWSServiceRoleForEMRCleanup** role to grant Amazon EMR permission to terminate and delete Amazon EC2 resources on your behalf if the Amazon EMR service-linked role loses that capability. Amazon EMR creates the service-linked role automatically during cluster creation if it doesn't already exist.
The AWSServiceRoleForEMRCleanup service-linked role trusts the following services to assume the role:
+ `elasticmapreduce.amazonaws.com`
The AWSServiceRoleForEMRCleanup service-linked role permissions policy allows Amazon EMR to complete the following actions on the specified resources:
+ Action: `DescribeInstances` on `ec2`
+ Action: `DescribeLaunchTemplates` on `ec2`
+ Action: `DeleteLaunchTemplate` on `ec2`
+ Action: `DescribeSpotInstanceRequests` on `ec2`
+ Action: `ModifyInstanceAttribute` on `ec2`
+ Action: `TerminateInstances` on `ec2`
+ Action: `CancelSpotInstanceRequests` on `ec2`
+ Action: `DeleteNetworkInterface` on `ec2`
+ Action: `DescribeInstanceAttribute` on `ec2`
+ Action: `DescribeVolumeStatus` on `ec2`
+ Action: `DescribeVolumes` on `ec2`
+ Action: `DetachVolume` on `ec2`
+ Action: `DeleteVolume` on `ec2`
+ Action: `DescribePlacementGroups` on `ec2`
+ Action: `DeletePlacementGroup` on `ec2`
You must configure permissions to allow an IAM entity (such as a user, group, or role) to create, edit, or delete a service-linked role.
## Creating a service-linked role for Amazon EMR
You don't need to manually create the AWSServiceRoleForEMRCleanup role. When you launch a cluster, either for the first time or when the AWSServiceRoleForEMRCleanup service-linked role is not present, Amazon EMR creates the AWSServiceRoleForEMRCleanup service-linked role for you. You must have permissions to create a service-linked role. For an example statement that adds this capability to the permissions policy of an IAM entity (such as a user, group, or role):
Add the following statement to the permissions policy for the IAM entity that needs to create the service-linked role.
```
{
"Sid": "ElasticMapReduceServiceLinkedRole",
"Effect": "Allow",
"Action": "iam:CreateServiceLinkedRole",
"Resource": "arn:aws:iam::*:role/aws-service-role/elasticmapreduce.amazonaws.com*/AWSServiceRoleForEMRCleanup*",
"Condition": {
"StringEquals": {
"iam:AWSServiceName": [
"elasticmapreduce.amazonaws.com",
"elasticmapreduce.amazonaws.com.rproxy.govskope.ca.cn"
]
}
}
}
```
**Important**
If you used Amazon EMR before October 24, 2017, when service-linked roles weren't supported, then Amazon EMR created the AWSServiceRoleForEMRCleanup service-linked role in your account. For more information, see [A new role appeared in my IAM account](https://docs.aws.amazon.com/IAM/latest/UserGuide/troubleshoot_roles.html#troubleshoot_roles_new-role-appeared).
## Editing a service-linked role for Amazon EMR
Amazon EMR doesn't allow you to edit the AWSServiceRoleForEMRCleanup service-linked role. After you create a service-linked role, you can't change the name of the service-linked role because various entities might reference the service-linked role. However, you can edit the description of the service-linked role using IAM.
### Editing a service-linked role description (IAM console)
You can use the IAM console to edit the description of a service-linked role.
**To edit the description of a service-linked role (console)**
1. In the navigation pane of the IAM console, choose **Roles**.
1. Choose the name of the role to modify.
1. To the right of the **Role description**, choose **Edit**.
1. Enter a new description in the box and choose **Save changes**.
### Editing a service-linked role description (IAM CLI)
You can use IAM commands from the AWS Command Line Interface to edit the description of a service-linked role.
**To change the description of a service-linked role (CLI)**
1. (Optional) To view the current description for a role, use the following commands:
```
$ aws iam get-role --role-name role-name
```
Use the role name, not the ARN, to refer to roles with the CLI commands. For example, if a role has the following ARN: `arn:aws:iam::123456789012:role/myrole`, you refer to the role as **myrole**.
1. To update a service-linked role's description, use one of the following commands:
```
$ aws iam update-role-description --role-name role-name --description description
```
### Editing a service-linked role description (IAM API)
You can use the IAM API to edit the description of a service-linked role.
**To change the description of a service-linked role (API)**
1. (Optional) To view the current description for a role, use the following command:
IAM API: [GetRole](https://docs.aws.amazon.com/IAM/latest/APIReference/API_GetRole.html)
1. To update a role's description, use the following command:
IAM API: [UpdateRoleDescription](https://docs.aws.amazon.com/IAM/latest/APIReference/API_UpdateRoleDescription.html)
## Deleting a service-linked role for Amazon EMR
If you no longer need to use a feature or service that requires a service-linked role, we recommend that you delete that service-linked role. That way, you don't have an unused entity that is not being actively monitored or maintained. However, you must clean up your service-linked role before you can delete it.
### Cleaning up a service-linked role
Before you can use IAM to delete a service-linked role, you must first confirm that the service-linked role has no active sessions and remove any resources used by the service-linked role.
**To check whether the service-linked role has an active session in the IAM console**
1. Open the IAM console at [https://console.aws.amazon.com/iam/](https://console.aws.amazon.com/iam/).
1. In the navigation pane, choose **Roles**. Select the name (not the check box) of the AWSServiceRoleForEMRCleanup service-linked role.
1. On the **Summary** page for the selected service-linked role, choose **Access Advisor**.
1. On the **Access Advisor** tab, review the recent activity for the service-linked role.
**Note**
If you are unsure whether Amazon EMR is using the AWSServiceRoleForEMRCleanup service-linked role, you can try to delete the service-linked role. If the service is using the service-linked role, then the deletion fails and you can view the Regions where the service-linked role is being used. If the service-linked role is being used, then you must wait for the session to end before you can delete the service-linked role. You cannot revoke the session for a service-linked role.
**To remove Amazon EMR resources used by the AWSServiceRoleForEMRCleanup**
+ Terminate all clusters in your account. For more information, see [Terminate an Amazon EMR cluster in the starting, running, or waiting states](UsingEMR_TerminateJobFlow.md).
### Deleting a service-linked role (IAM console)
You can use the IAM console to delete a service-linked role.
**To delete a service-linked role (console)**
1. Open the IAM console at [https://console.aws.amazon.com/iam/](https://console.aws.amazon.com/iam/).
1. In the navigation pane, choose **Roles**. Select the check box next to AWSServiceRoleForEMRCleanup, not the name or row itself.
1. For **Role actions** at the top of the page, choose **Delete role**.
1. In the confirmation dialog box, review the service last accessed data, which shows when each of the selected roles last accessed an AWS service. This helps you to confirm whether the role is currently active. To proceed, choose **Yes, Delete**.
1. Watch the IAM console notifications to monitor the progress of the service-linked role deletion. Because the IAM service-linked role deletion is asynchronous, after you submit the service-linked role for deletion, the deletion task can succeed or fail. If the task fails, you can choose **View details** or **View Resources** from the notifications to learn why the deletion failed. If the deletion fails because there are resources in the service that are being used by the role, then the reason for the failure includes a list of resources.
### Deleting a service-linked role (IAM CLI)
You can use IAM commands from the AWS Command Line Interface to delete a service-linked role. Because a service-linked role cannot be deleted if it is being used or has associated resources, you must submit a deletion request. If these conditions are not met, that request can be denied.
**To delete a service-linked role (CLI)**
1. To check the status of the deletion task, you must capture the `deletion-task-id` from the response. Type the following command to submit a service-linked role deletion request:
```
$ aws iam [delete-service-linked-role](https://docs.aws.amazon.com/cli/latest/reference/iam/delete-service-linked-role.html) --role-name AWSServiceRoleForEMRCleanup
```
1. Type the following command to check the status of the deletion task:
```
$ aws iam [get-service-linked-role-deletion-status](https://docs.aws.amazon.com/cli/latest/reference/iam/get-service-linked-role-deletion-status.html) --deletion-task-id deletion-task-id
```
The status of the deletion task can be `NOT_STARTED`, `IN_PROGRESS`, `SUCCEEDED`, or `FAILED`. If the deletion fails, the call returns the reason that it failed so that you can troubleshoot.
### Deleting a service-linked role (IAM API)
You can use the IAM API to delete a service-linked role. Because a service-linked role cannot be deleted if it is being used or has associated resources, you must submit a deletion request. If these conditions are not met, that request can be denied.
**To delete a service-linked role (API)**
1. To submit a deletion request for a service-linked role, call [DeleteServiceLinkedRole](https://docs.aws.amazon.com/IAM/latest/APIReference/API_DeleteServiceLinkedRole.html). In the request, specify the AWSServiceRoleForEMRCleanup role name.
To check the status of the deletion task, you must capture the `DeletionTaskId` from the response.
1. To check the status of the deletion, call [GetServiceLinkedRoleDeletionStatus](https://docs.aws.amazon.com/IAM/latest/APIReference/API_GetServiceLinkedRoleDeletionStatus.html). In the request, specify the `DeletionTaskId`.
The status of the deletion task can be `NOT_STARTED`, `IN_PROGRESS`, `SUCCEEDED`, or `FAILED`. If the deletion fails, the call returns the reason that it failed so that you can troubleshoot.
## Supported Regions for AWSServiceRoleForEMRCleanup
Amazon EMR supports using the AWSServiceRoleForEMRCleanup service-linked role in the following Regions.
****
| Region name | Region identity | Support in Amazon EMR |
| --- | --- | --- |
| US East (N. Virginia) | us-east-1 | Yes |
| US East (Ohio) | us-east-2 | Yes |
| US West (N. California) | us-west-1 | Yes |
| US West (Oregon) | us-west-2 | Yes |
| Asia Pacific (Mumbai) | ap-south-1 | Yes |
| Asia Pacific (Osaka) | ap-northeast-3 | Yes |
| Asia Pacific (Seoul) | ap-northeast-2 | Yes |
| Asia Pacific (Singapore) | ap-southeast-1 | Yes |
| Asia Pacific (Sydney) | ap-southeast-2 | Yes |
| Asia Pacific (Tokyo) | ap-northeast-1 | Yes |
| Canada (Central) | ca-central-1 | Yes |
| Europe (Frankfurt) | eu-central-1 | Yes |
| Europe (Ireland) | eu-west-1 | Yes |
| Europe (London) | eu-west-2 | Yes |
| Europe (Paris) | eu-west-3 | Yes |
| South America (São Paulo) | sa-east-1 | Yes |
# Using service-linked roles with Amazon EMR for write-ahead logging
WAL role
Amazon EMR uses AWS Identity and Access Management (IAM) [service-linked roles](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_terms-and-concepts.html#iam-term-service-linked-role). A service-linked role is a unique type of IAM role that is linked directly to Amazon EMR. Service-linked roles are predefined by Amazon EMR and include all the permissions that the service requires to call other AWS services on your behalf.
Service-linked roles work together with the Amazon EMR service role and Amazon EC2 instance profile for Amazon EMR. For more information about the service role and instance profile, see [Configure IAM service roles for Amazon EMR permissions to AWS services and resources](emr-iam-roles.md).
A service-linked role makes setting up Amazon EMR easier because you don’t have to manually add the necessary permissions. Amazon EMR defines the permissions of its service-linked roles, and unless defined otherwise, only Amazon EMR can assume its roles. The defined permissions include the trust policy and the permissions policy, and that permissions policy cannot be attached to any other IAM entity.
You can delete this service-linked role for Amazon EMR only after you delete their related resources and terminate all EMR clusters in the account. This protects your Amazon EMR resources so that you can't inadvertently remove permission to access the resources.
## Service-linked role permissions for write-ahead logging (WAL)
Amazon EMR uses the service-linked role **AWSServiceRoleForEMRWAL** to retrieve a cluster status.
The AWSServiceRoleForEMRWAL service-linked role trusts the following services to assume the role:
+ `emrwal.amazonaws.com`
The [`EMRDescribeClusterPolicyForEMRWAL`](EMRDescribeClusterPolicyForEMRWAL.md) permissions policy for the service-linked role allows Amazon EMR to complete the following actions on the specified resources:
+ Action: `DescribeCluster` on `*`
You must configure permissions to allow an IAM entity (in this case, Amazon EMR WAL) to create, edit, or delete a service-linked role. Add the following statements as needed to the permissions policy for your instance profile:
## CreateServiceLinkedRole
**To allow an IAM entity to create the AWSServiceRoleForEMRWAL service-linked role**
Add the following statement to the permissions policy for the IAM entity that needs to create the service-linked role:
```
{
"Effect": "Allow",
"Action": [
"iam:CreateServiceLinkedRole",
"iam:PutRolePolicy"
],
"Resource": "arn:aws:iam::*:role/aws-service-role/emrwal.amazonaws.com*/AWSServiceRoleForEMRWAL*",
"Condition": {
"StringLike": {
"iam:AWSServiceName": [
"emrwal.amazonaws.com",
"elasticmapreduce.amazonaws.com.rproxy.govskope.ca.cn"
]
}
}
}
```
## UpdateRoleDescription
**To allow an IAM entity to edit the description of the AWSServiceRoleForEMRWAL service-linked role**
Add the following statement to the permissions policy for the IAM entity that needs to edit the description of a service-linked role:
```
{
"Effect": "Allow",
"Action": [
"iam:UpdateRoleDescription"
],
"Resource": "arn:aws:iam::*:role/aws-service-role/emrwal.amazonaws.com*/AWSServiceRoleForEMRWAL*",
"Condition": {
"StringLike": {
"iam:AWSServiceName": [
"emrwal.amazonaws.com",
"elasticmapreduce.amazonaws.com.rproxy.govskope.ca.cn"
]
}
}
}
```
## DeleteServiceLinkedRole
**To allow an IAM entity to delete the AWSServiceRoleForEMRWAL service-linked role**
Add the following statement to the permissions policy for the IAM entity that needs to delete a service-linked role:
```
{
"Effect": "Allow",
"Action": [
"iam:DeleteServiceLinkedRole",
"iam:GetServiceLinkedRoleDeletionStatus"
],
"Resource": "arn:aws:iam::*:role/aws-service-role/elasticmapreduce.amazonaws.com*/AWSServiceRoleForEMRCleanup*",
"Condition": {
"StringLike": {
"iam:AWSServiceName": [
"emrwal.amazonaws.com",
"elasticmapreduce.amazonaws.com.rproxy.govskope.ca.cn"
]
}
}
}
```
## Creating a service-linked role for Amazon EMR
You don't need to manually create the AWSServiceRoleForEMRWAL role. Amazon EMR creates this service-linked role automatically when you create a WAL workspace with the EMRWAL CLI or from AWS CloudFormation, or HBase will create the service-linked role when you configure a workspace for Amazon EMR WAL and the service-linked role doesn't yet exist. You must have permissions to create a service-linked role. For example statements that add this capability to the permissions policy of an IAM entity (such as a user, group, or role), see the prior section, [Service-linked role permissions for write-ahead logging (WAL)](#using-service-linked-roles-permissions-wal).
## Editing a service-linked role for Amazon EMR
Amazon EMR doesn't allow you to edit the AWSServiceRoleForEMRWAL service-linked role. After you create a service-linked role, you can't change the name of the service-linked role because various entities might reference the service-linked role. However, you can edit the description of the service-linked role using IAM.
### Editing a service-linked role description (IAM console)
You can use the IAM console to edit the description of a service-linked role.
**To edit the description of a service-linked role (console)**
1. In the navigation pane of the IAM console, choose **Roles**.
1. Choose the name of the role to modify.
1. To the right of the **Role description**, choose **Edit**.
1. Enter a new description in the box and choose **Save changes**.
### Editing a service-linked role description (IAM CLI)
You can use IAM commands from the AWS Command Line Interface to edit the description of a service-linked role.
**To change the description of a service-linked role (CLI)**
1. (Optional) To view the current description for a role, use the following commands:
```
$ aws iam get-role --role-name role-name
```
Use the role name, not the ARN, to refer to roles with the CLI commands. For example, if a role has the following ARN: `arn:aws:iam::123456789012:role/myrole`, you refer to the role as **myrole**.
1. To update a service-linked role's description, use one of the following commands:
```
$ aws iam update-role-description --role-name role-name --description description
```
### Editing a service-linked role description (IAM API)
You can use the IAM API to edit the description of a service-linked role.
**To change the description of a service-linked role (API)**
1. (Optional) To view the current description for a role, use the following command:
IAM API: [GetRole](https://docs.aws.amazon.com/IAM/latest/APIReference/API_GetRole.html)
1. To update a role's description, use the following command:
IAM API: [UpdateRoleDescription](https://docs.aws.amazon.com/IAM/latest/APIReference/API_UpdateRoleDescription.html)
## Deleting a service-linked role for Amazon EMR
If you no longer need to use a feature or service that requires a service-linked role, we recommend that you delete that service-linked role. That way, you don't have an unused entity that is not being actively monitored or maintained. However, you must clean up your service-linked role before you can delete it.
**Note**
The write-ahead logging operation isn't affected if you delete the AWSServiceRoleForEMRWAL role, but Amazon EMR won't auto-delete the logs that it created once your EMR cluster terminates. Therefore, you'll need to manually delete the Amazon EMR WAL logs if you delete the service-linked role.
### Cleaning up a service-linked role
Before you can use IAM to delete a service-linked role, you must first confirm that the role has no active sessions and remove any resources used by the role.
**To check whether the service-linked role has an active session in the IAM console**
1. Open the IAM console at [https://console.aws.amazon.com/iam/](https://console.aws.amazon.com/iam/).
1. In the navigation pane, choose **Roles**. Select the name (not the check box) of the AWSServiceRoleForEMRWAL role.
1. On the **Summary** page for the selected role, choose **Access Advisor**.
1. On the **Access Advisor** tab, review the recent activity for the service-linked role.
**Note**
If you are unsure whether Amazon EMR is using the AWSServiceRoleForEMRWAL role, you can try to delete the service-linked role. If the service is using the role, then the deletion fails and you can view the Regions where the service-linked role is being used. If the service-linked role is being used, then you must wait for the session to end before you can delete the service-linked role. You cannot revoke the session for a service-linked role.
**To remove Amazon EMR resources used by the AWSServiceRoleForEMRWAL**
+ Terminate all clusters in your account. For more information, see [Terminate an Amazon EMR cluster in the starting, running, or waiting states](UsingEMR_TerminateJobFlow.md).
### Deleting a service-linked role (IAM console)
You can use the IAM console to delete a service-linked role.
**To delete a service-linked role (console)**
1. Open the IAM console at [https://console.aws.amazon.com/iam/](https://console.aws.amazon.com/iam/).
1. In the navigation pane, choose **Roles**. Select the check box next to AWSServiceRoleForEMRWAL, not the name or row itself.
1. For **Role actions** at the top of the page, choose **Delete role**.
1. In the confirmation dialog box, review the service last accessed data, which shows when each of the selected roles last accessed an AWS service. This helps you to confirm whether the role is currently active. To proceed, choose **Yes, Delete**.
1. Watch the IAM console notifications to monitor the progress of the service-linked role deletion. Because the IAM service-linked role deletion is asynchronous, after you submit the role for deletion, the deletion task can succeed or fail. If the task fails, you can choose **View details** or **View Resources** from the notifications to learn why the deletion failed. If the deletion fails because there are resources in the service that are being used by the role, then the reason for the failure includes a list of resources.
### Deleting a service-linked role (IAM CLI)
You can use IAM commands from the AWS Command Line Interface to delete a service-linked role. Because a service-linked role cannot be deleted if it is being used or has associated resources, you must submit a deletion request. If these conditions are not met, that request can be denied.
**To delete a service-linked role (CLI)**
1. To check the status of the deletion task, you must capture the `deletion-task-id` from the response. Type the following command to submit a service-linked role deletion request:
```
$ aws iam [delete-service-linked-role](https://docs.aws.amazon.com/cli/latest/reference/iam/delete-service-linked-role.html) --role-name AWSServiceRoleForEMRWAL
```
1. Type the following command to check the status of the deletion task:
```
$ aws iam [get-service-linked-role-deletion-status](https://docs.aws.amazon.com/cli/latest/reference/iam/get-service-linked-role-deletion-status.html) --deletion-task-id deletion-task-id
```
The status of the deletion task can be `NOT_STARTED`, `IN_PROGRESS`, `SUCCEEDED`, or `FAILED`. If the deletion fails, the call returns the reason that it failed so that you can troubleshoot.
### Deleting a service-linked role (IAM API)
You can use the IAM API to delete a service-linked role. Because a service-linked role cannot be deleted if it is being used or has associated resources, you must submit a deletion request. If these conditions are not met, that request can be denied.
**To delete a service-linked role (API)**
1. To submit a deletion request for a service-linked role, call [DeleteServiceLinkedRole](https://docs.aws.amazon.com/IAM/latest/APIReference/API_DeleteServiceLinkedRole.html). In the request, specify the AWSServiceRoleForEMRWAL role name.
To check the status of the deletion task, you must capture the `DeletionTaskId` from the response.
1. To check the status of the deletion, call [GetServiceLinkedRoleDeletionStatus](https://docs.aws.amazon.com/IAM/latest/APIReference/API_GetServiceLinkedRoleDeletionStatus.html). In the request, specify the `DeletionTaskId`.
The status of the deletion task can be `NOT_STARTED`, `IN_PROGRESS`, `SUCCEEDED`, or `FAILED`. If the deletion fails, the call returns the reason that it failed so that you can troubleshoot.
## Supported Regions for AWSServiceRoleForEMRWAL
Amazon EMR supports using the AWSServiceRoleForEMRWAL service-linked role in the following Regions.
****
| Region name | Region identity | Support in Amazon EMR |
| --- | --- | --- |
| US East (N. Virginia) | us-east-1 | Yes |
| US East (Ohio) | us-east-2 | Yes |
| US West (N. California) | us-west-1 | Yes |
| US West (Oregon) | us-west-2 | Yes |
| Asia Pacific (Mumbai) | ap-south-1 | Yes |
| Asia Pacific (Singapore) | ap-southeast-1 | Yes |
| Asia Pacific (Sydney) | ap-southeast-2 | Yes |
| Asia Pacific (Tokyo) | ap-northeast-1 | Yes |
| Europe (Frankfurt) | eu-central-1 | Yes |
| Europe (Ireland) | eu-west-1 | Yes |
# Customize IAM roles with Amazon EMR
You may want to customize the IAM service roles and permissions to limit privileges according to your security requirements. To customize permissions, we recommend that you create new roles and policies. Begin with the permissions in the managed policies for the default roles (for example, `AmazonElasticMapReduceforEC2Role` and `AmazonElasticMapReduceRole`). Then, copy and paste the contents to new policy statements, modify the permissions as appropriate, and attach the modified permissions policies to the roles that you create. You must have the appropriate IAM permissions to work with roles and policies. For more information, see [Allow users and groups to create and modify roles](emr-iam-roles-create-permissions.md).
If you create a custom EMR role for EC2, follow the basic work flow, which automatically creates an instance profile of the same name. Amazon EC2 allows you to create instance profiles and roles with different names, but Amazon EMR does not support this configuration, and it results in an "invalid instance profile" error when you create the cluster.
**Important**
Inline policies are not automatically updated when service requirements change. If you create and attach inline policies, be aware that service updates might occur that suddenly cause permissions errors. For more information, see [Managed Policies and Inline Policies](https://docs.aws.amazon.com/IAM/latest/UserGuide/policies_managed-vs-inline.html) in the *IAM User Guide* and [Specify custom IAM roles when you create a cluster](#emr-iam-roles-launch-jobflow).
For more information about working with IAM roles, see the following topics in the *IAM User Guide*:
+ [Creating a role to delegate permissions to an AWS service](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_create_for-service.html)
+ [Modifying a role](https://docs.aws.amazon.com/IAM/latest/UserGuide/modifying-role.html)
+ [Deleting a role](https://docs.aws.amazon.com/IAM/latest/UserGuide/deleting-roles.html)
## Specify custom IAM roles when you create a cluster
You specify the service role for Amazon EMR and the role for the Amazon EC2 instance profile when you create a cluster. The user who is creating clusters needs permissions to retrieve and assign roles to Amazon EMR and EC2 instances. Otherwise, a **account is not authorized to call EC2** error occurs. For more information, see [Allow users and groups to create and modify roles](emr-iam-roles-create-permissions.md).
### Use the console to specify custom roles
When you create a cluster, you can specify a custom service role for Amazon EMR, a custom role for the EC2 instance profile, and a custom Auto Scaling role using **Advanced options**. When you use **Quick options**, the default service role and the default role for the EC2 instance profile are specified. For more information, see [IAM service roles used by Amazon EMR](emr-iam-service-roles.md).
------
#### [ Console ]
**To specify custom IAM roles with the console**
When you create a cluster with the console, you must specify a custom service role for Amazon EMR and a custom role for the EC2 instance profile. For more information, see [IAM service roles used by Amazon EMR](emr-iam-service-roles.md).
1. Sign in to the AWS Management Console, and open the Amazon EMR console at [https://console.aws.amazon.com/emr](https://console.aws.amazon.com/emr).
1. Under **EMR on EC2** in the left navigation pane, choose **Clusters**, and then choose **Create cluster**.
1. Under **Security configuration and permissions**, find the **IAM role for instance profile** and **Service role for Amazon EMR** fields. For each role type, select a role from the list. Only roles within your account that have the appropriate trust policy for that role type are listed.
1. Choose any other options that apply to your cluster.
1. To launch your cluster, choose **Create cluster**.
------
### Use the AWS CLI to specify custom roles
You can specify a service role for Amazon EMR and a service role for cluster EC2 instances explicitly using options with the `create-cluster` command from the AWS CLI. Use the `--service-role` option to specify the service role. Use the `InstanceProfile` argument of the `--ec2-attributes` option to specify the role for the EC2 instance profile.
The Auto Scaling role is specified using a separate option, `--auto-scaling-role`. For more information, see [Using automatic scaling with a custom policy for instance groups in Amazon EMR](emr-automatic-scaling.md).
**To specify custom IAM roles using the AWS CLI**
+ The following command specifies the custom service role, *MyCustomServiceRoleForEMR*, and a custom role for the EC2 instance profile, *MyCustomServiceRoleForClusterEC2Instances*, when launching a cluster. This example uses the default Amazon EMR role.
**Note**
Linux line continuation characters (\$1) are included for readability. They can be removed or used in Linux commands. For Windows, remove them or replace with a caret (^).
```
aws emr create-cluster --name "Test cluster" --release-label emr-7.12.0 \
--applications Name=Hive Name=Pig --service-role MyCustomServiceRoleForEMR \
--ec2-attributes InstanceProfile=MyCustomServiceRoleForClusterEC2Instances,\
KeyName=myKey --instance-type m5.xlarge --instance-count 3
```
You can use these options to specify default roles explicitly rather than using the `--use-default-roles` option. The `--use-default-roles` option specifies the service role and the role for the EC2 instance profile defined in the `config` file for the AWS CLI.
The following example demonstrates the contents of a `config` file for the AWS CLI the specifies custom roles for Amazon EMR. With this configuration file, when the `--use-default-roles` option is specified, the cluster is created using the *MyCustomServiceRoleForEMR* and *MyCustomServiceRoleForClusterEC2Instances*. By default, the `config` file specifies the default `service_role` as `AmazonElasticMapReduceRole` and the default `instance_profile` as `EMR_EC2_DefaultRole`.
```
[default]
output = json
region = us-west-1
aws_access_key_id = myAccessKeyID
aws_secret_access_key = mySecretAccessKey
emr =
service_role = MyCustomServiceRoleForEMR
instance_profile = MyCustomServiceRoleForClusterEC2Instances
```
# Configure IAM roles for EMRFS requests to Amazon S3
Configure IAM roles for EMRFS
**Note**
The EMRFS role mapping capability described on this page has been improved upon with the introduction of Amazon S3 Access Grants in Amazon EMR 6.15.0. For a scalable access control solution for your data in Amazon S3, we recommend that you use [S3 Access Grants with Amazon EMR](emr-access-grants.md).
When an application running on a cluster references data using the `s3://mydata` format, Amazon EMR uses EMRFS to make the request. To interact with Amazon S3, EMRFS assumes the permissions policies that are attached to your [Amazon EC2 instance profile](emr-iam-role-for-ec2.md). The same Amazon EC2 instance profile is used regardless of the user or group running the application or the location of the data in Amazon S3.
If you have a cluster with multiple users who need different levels of access to data in Amazon S3 through EMRFS, you can set up a security configuration with IAM roles for EMRFS. EMRFS can assume a different service role for cluster EC2 instances based on the user or group making the request, or based on the location of data in Amazon S3. Each IAM role for EMRFS can have different permissions for data access in Amazon S3. For more information about the service role for cluster EC2 instances, see [Service role for cluster EC2 instances (EC2 instance profile)](emr-iam-role-for-ec2.md).
Using custom IAM roles for EMRFS is supported in Amazon EMR versions 5.10.0 and later. If you use an earlier version or have requirements beyond what IAM roles for EMRFS provide, you can create a custom credentials provider instead. For more information, see [Authorizing access to EMRFS data in Amazon S3](https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-plan-credentialsprovider).
When you use a security configuration to specify IAM roles for EMRFS, you set up role mappings. Each role mapping specifies an IAM role that corresponds to identifiers. These identifiers determine the basis for access to Amazon S3 through EMRFS. The identifiers can be users, groups, or Amazon S3 prefixes that indicate a data location. When EMRFS makes a request to Amazon S3, if the request matches the basis for access, EMRFS has cluster EC2 instances assume the corresponding IAM role for the request. The IAM permissions attached to that role apply instead of the IAM permissions attached to the service role for cluster EC2 instances.
The users and groups in a role mapping are Hadoop users and groups that are defined on the cluster. Users and groups are passed to EMRFS in the context of the application using it (for example, YARN user impersonation). The Amazon S3 prefix can be a bucket specifier of any depth (for example, `s3://amzn-s3-demo-bucket` or `s3://amzn-s3-demo-bucket/myproject/mydata`). You can specify multiple identifiers within a single role mapping, but they all must be of the same type.
**Important**
IAM roles for EMRFS provide application-level isolation between users of the application. It does not provide host level isolation between users on the host. Any user with access to the cluster can bypass the isolation to assume any of the roles.
When a cluster application makes a request to Amazon S3 through EMRFS, EMRFS evaluates role mappings in the top-down order that they appear in the security configuration. If a request made through EMRFS doesn't match any identifier, EMRFS falls back to using the service role for cluster EC2 instances. For this reason, we recommend that the policies attached to this role limit permissions to Amazon S3. For more information, see [Service role for cluster EC2 instances (EC2 instance profile)](emr-iam-role-for-ec2.md).
## Configure roles
Before you set up a security configuration with IAM roles for EMRFS, plan and create the roles and permission policies to attach to the roles. For more information, see [How do roles for EC2 instances work?](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_use_switch-role-ec2.html) in the *IAM User Guide*. When creating permissions policies, we recommend that you start with the managed policy attached to the default Amazon EMR role for EC2, and then edit this policy according to your requirements. The default role name is `EMR_EC2_DefaultRole`, and the default managed policy to edit is `AmazonElasticMapReduceforEC2Role`. For more information, see [Service role for cluster EC2 instances (EC2 instance profile)](emr-iam-role-for-ec2.md).
### Updating trust policies to assume role permissions
Each role that EMRFS uses must have a trust policy that allows the cluster's Amazon EMR role for EC2 to assume it. Similarly, the cluster's Amazon EMR role for EC2 must have a trust policy that allows EMRFS roles to assume it.
The following example trust policy is attached to roles for EMRFS. The statement allows the default Amazon EMR role for EC2 to assume the role. For example, if you have two fictitious EMRFS roles, `EMRFSRole_First` and `EMRFSRole_Second`, this policy statement is added to the trust policies for each of them.
------
#### [ JSON ]
****
```
{
"Version":"2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"sts:AssumeRole"
],
"Resource": "arn:aws:iam::123456789012:role/EMR_EC2_DefaultRole",
"Sid": "AllowSTSAssumerole"
}
]
}
```
------
In addition, the following example trust policy statement is added to the `EMR_EC2_DefaultRole` to allow the two fictitious EMRFS roles to assume it.
------
#### [ JSON ]
****
```
{
"Version":"2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"sts:AssumeRole"
],
"Resource": [
"arn:aws:iam::123456789012:role/EMRFSRole_First",
"arn:aws:iam::123456789012:role/EMRFSRole_Second"
],
"Sid": "AllowSTSAssumerole"
}
]
}
```
------
**To update the trust policy of an IAM role**
Open the IAM console at [https://console.aws.amazon.com/iam/](https://console.aws.amazon.com/iam/).
1. Choose **Roles**, enter the name of the role in **Search**, and then select its **Role name**.
1. Choose **Trust relationships**, **Edit trust relationship**.
1. Add a trust statement according to the **Policy document** according to the guidelines above, and then choose **Update trust policy**.
### Specifying a role as a key user
If a role allows access to a location in Amazon S3 that is encrypted using an AWS KMS key, make sure that the role is specified as a key user. This gives the role permission to use the KMS key. For more information, see [Key policies in AWS KMS](https://docs.aws.amazon.com//kms/latest/developerguide/key-policies.html#key-policy-default-allow-users) in the *AWS Key Management Service Developer Guide*.
## Set up a security configuration with IAM roles for EMRFS
**Important**
If none of the IAM roles for EMRFS that you specify apply, EMRFS falls back to the Amazon EMR role for EC2. Consider customizing this role to restrict permissions to Amazon S3 as appropriate for your application and then specifying this custom role instead of `EMR_EC2_DefaultRole` when you create a cluster. For more information, see [Customize IAM roles with Amazon EMR](emr-iam-roles-custom.md) and [Specify custom IAM roles when you create a cluster](emr-iam-roles-custom.md#emr-iam-roles-launch-jobflow).
**To specify IAM roles for EMRFS requests to Amazon S3 using the console**
1. Create a security configuration that specifies role mappings:
1. In the Amazon EMR console, select **Security configurations**, **Create**.
1. Type a **Name** for the security configuration. You use this name to specify the security configuration when you create a cluster.
1. Choose **Use IAM roles for EMRFS requests to Amazon S3**.
1. Select an **IAM role** to apply, and under **Basis for access** select an identifier type (**Users**, **Groups**, or **S3 prefixes**) from the list and enter corresponding identifiers. If you use multiple identifiers, separate them with a comma and no space. For more information about each identifier type, see the [JSON configuration reference](#emrfs-seccfg-json) below.
1. Choose **Add role** to set up additional role mappings as described in the previous step.
1. Set up other security configuration options as appropriate and choose **Create**. For more information, see [Create a security configuration with the Amazon EMR console or with the AWS CLI](emr-create-security-configuration.md).
1. Specify the security configuration you created above when you create a cluster. For more information, see [Specify a security configuration for an Amazon EMR cluster](emr-specify-security-configuration.md).
**To specify IAM roles for EMRFS requests to Amazon S3 using the AWS CLI**
1. Use the `aws emr create-security-configuration` command, specifying a name for the security configuration, and the security configuration details in JSON format.
The example command shown below creates a security configuration with the name `EMRFS_Roles_Security_Configuration`. It is based on a JSON structure in the file `MyEmrfsSecConfig.json`, which is saved in the same directory where the command is executed.
```
aws emr create-security-configuration --name EMRFS_Roles_Security_Configuration --security-configuration file://MyEmrFsSecConfig.json.
```
Use the following guidelines for the structure of the `MyEmrFsSecConfig.json` file. You can specify this structure along with structures for other security configuration options. For more information, see [Create a security configuration with the Amazon EMR console or with the AWS CLI](emr-create-security-configuration.md).
The following is an example JSON snippet for specifying custom IAM roles for EMRFS within a security configuration. It demonstrates role mappings for the three different identifier types, followed by a parameter reference.
```
{
"AuthorizationConfiguration": {
"EmrFsConfiguration": {
"RoleMappings": [{
"Role": "arn:aws:iam::123456789101:role/allow_EMRFS_access_for_user1",
"IdentifierType": "User",
"Identifiers": [ "user1" ]
},{
"Role": "arn:aws:iam::123456789101:role/allow_EMRFS_access_to_demo_s3_buckets",
"IdentifierType": "Prefix",
"Identifiers": [ "s3://amzn-s3-demo-bucket1/","s3://amzn-s3-demo-bucket2/" ]
},{
"Role": "arn:aws:iam::123456789101:role/allow_EMRFS_access_for_AdminGroup",
"IdentifierType": "Group",
"Identifiers": [ "AdminGroup" ]
}]
}
}
}
```
[\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-emrfs-iam-roles.html)
1. Use the `aws emr create-cluster` command to create a cluster and specify the security configuration you created in the previous step.
The following example creates a cluster with default core Hadoop applications installed. The cluster uses the security configuration created above as `EMRFS_Roles_Security_Configuration` and also uses a custom Amazon EMR role for EC2, `EC2_Role_EMR_Restrict_S3`, which is specified using the `InstanceProfile` argument of the `--ec2-attributes` parameter.
**Note**
Linux line continuation characters (\$1) are included for readability. They can be removed or used in Linux commands. For Windows, remove them or replace with a caret (^).
```
aws emr create-cluster --name MyEmrFsS3RolesCluster \
--release-label emr-7.12.0 --ec2-attributes InstanceProfile=EC2_Role_EMR_Restrict_S3,KeyName=MyKey \
--instance-type m5.xlarge --instance-count 3 \
--security-configuration EMRFS_Roles_Security_Configuration
```
# Use resource-based policies for Amazon EMR access to AWS Glue Data Catalog
Resource-based policies for AWS Glue
If you use AWS Glue in conjunction with Hive, Spark, or Presto in Amazon EMR, AWS Glue supports resource-based policies to control access to Data Catalog resources. These resources include databases, tables, connections, and user-defined functions. For more information, see [AWS Glue Resource Policies](https://docs.aws.amazon.com/glue/latest/dg/glue-resource-policies.html) in the *AWS Glue Developer Guide*.
When using resource-based policies to limit access to AWS Glue from within Amazon EMR, the principal that you specify in the permissions policy must be the role ARN associated with the EC2 instance profile that is specified when a cluster is created. For example, for a resource-based policy attached to a catalog, you can specify the role ARN for the default service role for cluster EC2 instances, *EMR\$1EC2\$1DefaultRole* as the `Principal`, using the format shown in the following example:
```
arn:aws:iam::acct-id:role/EMR_EC2_DefaultRole
```
The *acct-id* can be different from the AWS Glue account ID. This enables access from EMR clusters in different accounts. You can specify multiple principals, each from a different account.
# Use IAM roles with applications that call AWS services directly
Applications running on the EC2 instances of a cluster can use the EC2 instance profile to obtain temporary security credentials when calling AWS services.
The versions of Hadoop available with Amazon EMR release 2.3.0 and later have already been updated to make use of IAM roles. If your application runs strictly on top of the Hadoop architecture, and does not directly call any service in AWS, it should work with IAM roles with no modification.
If your application calls services in AWS directly, you need to update it to take advantage of IAM roles. This means that instead of obtaining account credentials from `/etc/hadoop/conf/core-site.xml` on the EC2 instances in the cluster, your application uses an SDK to access the resources using IAM roles, or calls the EC2 instance metadata to obtain the temporary credentials.
**To access AWS resources with IAM roles using an SDK**
+ The following topics show how to use several of the AWS SDKs to access temporary credentials using IAM roles. Each topic starts with a version of an application that does not use IAM roles and then walks you through the process of converting that application to use IAM roles.
+ [Using IAM roles for Amazon EC2 instances with the SDK for Java](https://docs.aws.amazon.com/sdk-for-java/latest/developer-guide/java-dg-roles.html) in the *AWS SDK for Java Developer Guide*
+ [Using IAM roles for Amazon EC2 instances with the SDK for .NET](https://docs.aws.amazon.com/sdk-for-net/latest/developer-guide/net-dg-roles.html) in the *AWS SDK for .NET Developer Guide*
+ [Using IAM roles for Amazon EC2 instances with the SDK for PHP](https://docs.aws.amazon.com/sdk-for-php/latest/developer-guide/php-dg-roles.html) in the *AWS SDK for PHP Developer Guide*
+ [Using IAM roles for Amazon EC2 instances with the SDK for Ruby](https://docs.aws.amazon.com/sdk-for-ruby/latest/developer-guide/ruby-dg-roles.html) in the *AWS SDK for Ruby Developer Guide*
**To obtain temporary credentials from EC2 instance metadata**
+ Call the following URL from an EC2 instance that is running with the specified IAM role, which returns the associated temporary security credentials (AccessKeyId, SecretAccessKey, SessionToken, and Expiration). The following example uses the default instance profile for Amazon EMR, `EMR_EC2_DefaultRole`.
```
GET http://169.254.169.254/latest/meta-data/iam/security-credentials/EMR_EC2_DefaultRole
```
For more information about writing applications that use IAM roles, see [Granting applications that run on Amazon EC2 instances access to AWS resources](https://docs.aws.amazon.com/IAM/latest/UserGuide/role-usecase-ec2app.html).
For more information about temporary security credentials, see [Using temporary security credentials](https://docs.aws.amazon.com/STS/latest/UsingSTS/using-temp-creds.html) in the *Using Temporary Security Credentials* guide.
# Allow users and groups to create and modify roles
IAM principals (users and groups) who create, modify, and specify roles for a cluster, including default roles, must be allowed to perform the following actions. For details about each action, see [Actions](https://docs.aws.amazon.com/IAM/latest/APIReference/API_Operations.html) in the *IAM API Reference*.
+ `iam:CreateRole`
+ `iam:PutRolePolicy`
+ `iam:CreateInstanceProfile`
+ `iam:AddRoleToInstanceProfile`
+ `iam:ListRoles`
+ `iam:GetPolicy`
+ `iam:GetInstanceProfile`
+ `iam:GetPolicyVersion`
+ `iam:AttachRolePolicy`
+ `iam:PassRole`
The `iam:PassRole` permission allows cluster creation. The remaining permissions allow the creation of the default roles.
For information about assigning permissions to a user, see [Changing permissions for a user](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_users_change-permissions.html) in the *IAM User Guide*.