/start
```
## Set up IAM Identity Center authentication mode for Amazon EMR Studio
To prepare AWS IAM Identity Center for EMR Studio, you must configure your identity source and provision users and groups. Provisioning is the process of making user and group information available for use by IAM Identity Center and by applications that use IAM Identity Center. For more information, see [User and group provisioning](https://docs.aws.amazon.com/singlesignon/latest/userguide/users-groups-provisioning.html#user-group-provision).
EMR Studio supports using the following identity providers for IAM Identity Center:
+ **AWS Managed Microsoft AD and self-managed Active Directory** – For more information, see [Connect to your Microsoft AD directory](https://docs.aws.amazon.com/singlesignon/latest/userguide/manage-your-identity-source-ad.html).
+ **SAML-based providers** – For a full list, see [Supported identity providers](https://docs.aws.amazon.com/singlesignon/latest/userguide/supported-idps.html).
+ **The IAM Identity Center directory** – For more information, see [Manage identities in IAM Identity Center](https://docs.aws.amazon.com/singlesignon/latest/userguide/manage-your-identity-source-sso.html).
**To set up IAM Identity Center for EMR Studio**
1. To set up IAM Identity Center for EMR Studio, you need the following:
+ A management account in your AWS organization if you use multiple accounts in your organization.
**Note**
You should only use your management account to enable IAM Identity Center and *provision* users and groups. After you set up IAM Identity Center, use a member account to create an EMR Studio and *assign* users and groups. To learn more about AWS terminology, see [AWS Organizations terminology and concepts](https://docs.aws.amazon.com/organizations/latest/userguide/orgs_getting-started_concepts.html).
+ If you enabled IAM Identity Center before November 25, 2019, you might have to enable applications that use IAM Identity Center for the accounts in your AWS organization. For more information, see [Enable IAM Identity Center-integrated applications in AWS accounts](https://docs.aws.amazon.com/singlesignon/latest/userguide/app-enablement.html#enable-app-enablement).
+ Make sure that you have the prerequisites listed on the [IAM Identity Center prerequisites](https://docs.aws.amazon.com/singlesignon/latest/userguide/prereqs.html) page.
1. Follow the instructions in [Enable IAM Identity Center](https://docs.aws.amazon.com/singlesignon/latest/userguide/step1.html) to enable IAM Identity Center in the AWS Region where you want to create the EMR Studio.
1. Connect IAM Identity Center to your identity provider and provision the users and groups that you want to assign to the Studio.
****
[\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-studio-authentication.html)
You can now assign users and groups from your Identity Store to an EMR Studio. For instructions, see [Assign a user or group to an EMR Studio](emr-studio-manage-users.md#emr-studio-assign-users-groups).
# Create an EMR Studio service role
## About the EMR Studio service role
Each EMR Studio uses an IAM role with permissions that let the Studio interact with other AWS services. This service role must include permissions that allow EMR Studio to establish a secure network channel between Workspaces and clusters, to store notebook files in Amazon S3 Control, and to access the AWS Secrets Manager while linking a Workspace to a Git repository.
Use the Studio service role (instead of session policies) to define all Amazon S3 access permissions for storing notebook files, and to define AWS Secrets Manager access permissions.
## How to create a service role for EMR Studio on Amazon EC2 or Amazon EKS
1. Follow the instructions in [Creating a role to delegate permissions to an AWS service](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_create_for-service.html) to create the service role with the following trust policy.
**Important**
The following trust policy includes the [https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_condition-keys.html#condition-keys-sourcearn](https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_condition-keys.html#condition-keys-sourcearn) and [https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_condition-keys.html#condition-keys-sourceaccount](https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_condition-keys.html#condition-keys-sourceaccount) global condition keys to limit the permissions that you give EMR Studio to particular resources in your account. Doing so can protect you against [the confused deputy problem](https://docs.aws.amazon.com/IAM/latest/UserGuide/confused-deputy.html).
------
#### [ JSON ]
****
```
{
"Version":"2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"sts:AssumeRole"
],
"Resource": "arn:aws:iam::123456789012:role/EMRStudioServiceRole",
"Condition": {
"StringEquals": {
"aws:SourceAccount": "123456789012"
},
"ArnLike": {
"aws:SourceArn": "arn:aws:elasticmapreduce:*:123456789012:*"
}
},
"Sid": "AllowSTSAssumerole"
}
]
}
```
------
1. Remove the default role permissions. Then, include the permissions from the following sample IAM permissions policy. Alternatively, you can create a custom policy that uses the [EMR Studio service role permissions](#emr-studio-service-role-permissions-table).
**Important**
For Amazon EC2 tag-based access control with to work with EMR Studio, you must set access for the `ModifyNetworkInterfaceAttribute` API as shown the following policy.
For EMR Studio to work with the service role, you must not change the following statements: `AllowAddingEMRTagsDuringDefaultSecurityGroupCreation` and `AllowAddingTagsDuringEC2ENICreation`.
To use the example policy, you must tag the following resources with the key `"for-use-with-amazon-emr-managed-policies"` and value `"true"`.
Your Amazon Virtual Private Cloud (VPC) for EMR Studio.
Each subnet that you want to use with the Studio.
Any custom EMR Studio security groups. You must tag any security groups that you created during the EMR Studio preview period if you want to continue to use them.
Secrets maintained in AWS Secrets Manager that Studio users use to link Git repositories to a Workspace.
You can apply tags to resources using the **Tags** tab on the relevant resource screen in the AWS Management Console.
Where applicable, change the `*` in `"Resource":"*"` in the following policy to specify the Amazon Resource Name (ARN) of the resources that the statement covers for your use case.
------
#### [ JSON ]
****
```
{
"Version":"2012-10-17",
"Statement": [
{
"Sid": "AllowEMRReadOnlyActions",
"Effect": "Allow",
"Action": [
"elasticmapreduce:ListInstances",
"elasticmapreduce:DescribeCluster",
"elasticmapreduce:ListSteps"
],
"Resource": [
"*"
]
},
{
"Sid": "AllowEC2ENIActionsWithEMRTags",
"Effect": "Allow",
"Action": [
"ec2:CreateNetworkInterfacePermission",
"ec2:DeleteNetworkInterface"
],
"Resource": [
"arn:aws:ec2:*:*:network-interface/*"
],
"Condition": {
"StringEquals": {
"aws:ResourceTag/for-use-with-amazon-emr-managed-policies": "true"
}
}
},
{
"Sid": "AllowEC2ENIAttributeAction",
"Effect": "Allow",
"Action": [
"ec2:ModifyNetworkInterfaceAttribute"
],
"Resource": [
"arn:aws:ec2:*:*:instance/*",
"arn:aws:ec2:*:*:network-interface/*",
"arn:aws:ec2:*:*:security-group/*"
]
},
{
"Sid": "AllowEC2SecurityGroupActionsWithEMRTags",
"Effect": "Allow",
"Action": [
"ec2:AuthorizeSecurityGroupEgress",
"ec2:AuthorizeSecurityGroupIngress",
"ec2:RevokeSecurityGroupEgress",
"ec2:RevokeSecurityGroupIngress",
"ec2:DeleteNetworkInterfacePermission"
],
"Resource": [
"*"
],
"Condition": {
"StringEquals": {
"aws:ResourceTag/for-use-with-amazon-emr-managed-policies": "true"
}
}
},
{
"Sid": "AllowDefaultEC2SecurityGroupsCreationWithEMRTags",
"Effect": "Allow",
"Action": [
"ec2:CreateSecurityGroup"
],
"Resource": [
"arn:aws:ec2:*:*:security-group/*"
],
"Condition": {
"StringEquals": {
"aws:RequestTag/for-use-with-amazon-emr-managed-policies": "true"
}
}
},
{
"Sid": "AllowDefaultEC2SecurityGroupsCreationInVPCWithEMRTags",
"Effect": "Allow",
"Action": [
"ec2:CreateSecurityGroup"
],
"Resource": [
"arn:aws:ec2:*:*:vpc/*"
],
"Condition": {
"StringEquals": {
"aws:ResourceTag/for-use-with-amazon-emr-managed-policies": "true"
}
}
},
{
"Sid": "AllowAddingEMRTagsDuringDefaultSecurityGroupCreation",
"Effect": "Allow",
"Action": [
"ec2:CreateTags"
],
"Resource": [
"arn:aws:ec2:*:*:security-group/*"
],
"Condition": {
"StringEquals": {
"aws:RequestTag/for-use-with-amazon-emr-managed-policies": "true",
"ec2:CreateAction": "CreateSecurityGroup"
}
}
},
{
"Sid": "AllowEC2ENICreationWithEMRTags",
"Effect": "Allow",
"Action": [
"ec2:CreateNetworkInterface"
],
"Resource": [
"arn:aws:ec2:*:*:network-interface/*"
],
"Condition": {
"StringEquals": {
"aws:RequestTag/for-use-with-amazon-emr-managed-policies": "true"
}
}
},
{
"Sid": "AllowEC2ENICreationInSubnetAndSecurityGroupWithEMRTags",
"Effect": "Allow",
"Action": [
"ec2:CreateNetworkInterface"
],
"Resource": [
"arn:aws:ec2:*:*:subnet/*",
"arn:aws:ec2:*:*:security-group/*"
],
"Condition": {
"StringEquals": {
"aws:ResourceTag/for-use-with-amazon-emr-managed-policies": "true"
}
}
},
{
"Sid": "AllowAddingTagsDuringEC2ENICreation",
"Effect": "Allow",
"Action": [
"ec2:CreateTags"
],
"Resource": [
"arn:aws:ec2:*:*:network-interface/*"
],
"Condition": {
"StringEquals": {
"ec2:CreateAction": "CreateNetworkInterface"
}
}
},
{
"Sid": "AllowEC2ReadOnlyActions",
"Effect": "Allow",
"Action": [
"ec2:DescribeSecurityGroups",
"ec2:DescribeNetworkInterfaces",
"ec2:DescribeTags",
"ec2:DescribeInstances",
"ec2:DescribeSubnets",
"ec2:DescribeVpcs"
],
"Resource": [
"*"
]
},
{
"Sid": "AllowSecretsManagerReadOnlyActionsWithEMRTags",
"Effect": "Allow",
"Action": [
"secretsmanager:GetSecretValue"
],
"Resource": [
"arn:aws:secretsmanager:*:*:secret:*"
],
"Condition": {
"StringEquals": {
"aws:ResourceTag/for-use-with-amazon-emr-managed-policies": "true"
}
}
},
{
"Sid": "AllowWorkspaceCollaboration",
"Effect": "Allow",
"Action": [
"iam:GetUser",
"iam:GetRole",
"iam:ListUsers",
"iam:ListRoles",
"sso:GetManagedApplicationInstance",
"sso-directory:SearchUsers"
],
"Resource": [
"*"
]
}
]
}
```
------
1. Give your service role read and write access to your Amazon S3 location for EMR Studio. Use the following minimum set of permissions. For more information, see the [Amazon S3: Allows read and write access to objects in an S3 Bucket, programmatically and in the console](https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_examples_s3_rw-bucket-console.html) example.
```
"s3:PutObject",
"s3:GetObject",
"s3:GetEncryptionConfiguration",
"s3:ListBucket",
"s3:DeleteObject"
```
If you encrypt your Amazon S3 bucket, include the following permissions for AWS Key Management Service.
```
"kms:Decrypt",
"kms:GenerateDataKey",
"kms:ReEncryptFrom",
"kms:ReEncryptTo",
"kms:DescribeKey"
```
1. If you want to control access to Git secrets at user level, add tag-based permissions to `secretsmanager:GetSecretValue` in the EMR Studio **user role policy**, and remove permissions to `secretsmanager:GetSecretValue` policy from the EMR Studio **service role policy**. For more information on setting fine-grained user permissions, see [Create permissions policies for EMR Studio users](emr-studio-user-permissions.md#emr-studio-permissions-policies).
## Minimum service role for EMR Serverless
If you want to run interactive workloads with EMR Serverless through EMR Studio notebooks, use the same trust policy that you use to set up EMR Studio in the previous section, [How to create a service role for EMR Studio on Amazon EC2 or Amazon EKS](#emr-studio-service-role-instructions).
For your IAM policy, the minimum viable policy has permissions as follows. Update `bucket-name` with the name of the bucket that you plan to use when you configure your EMR Studio and Workspace. EMR Studio uses the bucket back up the Workspaces and notebook files in your Studio.
------
#### [ JSON ]
****
```
{
"Version":"2012-10-17",
"Statement": [
{
"Sid": "ObjectActions",
"Effect": "Allow",
"Action": [
"s3:PutObject",
"s3:GetObject",
"s3:DeleteObject"
],
"Resource": [
"arn:aws:s3:::bucket-name/*"
]
},
{
"Sid": "BucketActions",
"Effect": "Allow",
"Action": [
"s3:ListBucket",
"s3:GetEncryptionConfiguration"
],
"Resource": [
"arn:aws:s3:::bucket-name"
]
}
]
}
```
------
If you plan to use an encrypted Amazon S3 bucket, add the following permissions on your policy:
```
"kms:Decrypt",
"kms:GenerateDataKey",
"kms:ReEncryptFrom",
"kms:ReEncryptTo",
"kms:DescribeKey"
```
## EMR Studio service role permissions
The following table lists the operations that EMR Studio performs using the service role, along with the IAM actions required for each operation.
| Operation | Actions |
| --- | --- |
| Establish a secure network channel between a Workspace and an EMR cluster, and perform necessary cleanup actions. | "ec2:CreateNetworkInterface",
"ec2:CreateNetworkInterfacePermission",
"ec2:DeleteNetworkInterface",
"ec2:DeleteNetworkInterfacePermission",
"ec2:DescribeNetworkInterfaces",
"ec2:ModifyNetworkInterfaceAttribute",
"ec2:AuthorizeSecurityGroupEgress",
"ec2:AuthorizeSecurityGroupIngress",
"ec2:CreateSecurityGroup",
"ec2:DescribeSecurityGroups",
"ec2:RevokeSecurityGroupEgress",
"ec2:DescribeTags",
"ec2:DescribeInstances",
"ec2:DescribeSubnets",
"ec2:DescribeVpcs",
"elasticmapreduce:ListInstances",
"elasticmapreduce:DescribeCluster",
"elasticmapreduce:ListSteps"
|
| Use Git credentials stored in AWS Secrets Manager to link Git repositories to a Workspace. | "secretsmanager:GetSecretValue"
|
| Apply AWS tags to the network interface and default security groups that EMR Studio creates while setting up the secure network channel. For more information, see [Tagging AWS resources](https://docs.aws.amazon.com/general/latest/gr/aws_tagging.html). | "ec2:CreateTags"
|
| Access or upload notebook files and metadata to Amazon S3. | "s3:PutObject",
"s3:GetObject",
"s3:GetEncryptionConfiguration",
"s3:ListBucket",
"s3:DeleteObject"
If you use an encrypted Amazon S3 bucket, include the following permissions. "kms:Decrypt",
"kms:GenerateDataKey",
"kms:ReEncryptFrom",
"kms:ReEncryptTo",
"kms:DescribeKey"
|
| Enable and configure Workspace collaboration. | "iam:GetUser",
"iam:GetRole",
"iam:ListUsers",
"iam:ListRoles",
"sso:GetManagedApplicationInstance",
"sso-directory:SearchUsers",
"sso:DescribeApplication",
"sso:DescribeInstance"
|
| [ Encrypt EMR Studio workspace notebooks and files using customer managed keys (CMK) with AWS Key Management Service](https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-studio-workspace-storage-encryption) | "kms:Decrypt",
"kms:GenerateDataKey",
"kms:ReEncryptFrom",
"kms:ReEncryptTo",
"kms:DescribeKey"
|
# Configure EMR Studio user permissions for Amazon EC2 or Amazon EKS
You must configure user permissions policies for Amazon EMR Studio so that you can set fine-grained user and group permissions. For information about how user permissions work in EMR Studio, see [Access control](how-emr-studio-works.md#emr-studio-access-control) in [How Amazon EMR Studio works](how-emr-studio-works.md).
**Note**
The permissions covered in this section don't enforce data access control. To manage access to input datasets, you should configure permissions for the clusters that your Studio uses. For more information, see [Security in Amazon EMR](emr-security.md).
## Create an EMR Studio user role for IAM Identity Center authentication mode
You must create an EMR Studio user role when you use IAM Identity Center authentication mode.
**To create a user role for EMR Studio**
1. Follow the instructions in [Creating a role to delegate permissions to an AWS service](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_create_for-service.html) in the *AWS Identity and Access Management User Guide* to create a user role.
When you create the role, use the following trust relationship policy.
------
#### [ JSON ]
****
```
{
"Version":"2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"sts:AssumeRole",
"sts:SetContext"
],
"Resource": "arn:aws:iam::123456789012:role/EMRStudioServiceRole",
"Sid": "AllowSTSAssumerole"
}
]
}
```
------
1. Remove the default role permissions and policies.
1. Before you assign users and groups to a Studio, attach your EMR Studio session policies to the user role. For instructions on how to create session policies, see [Create permissions policies for EMR Studio users](#emr-studio-permissions-policies).
## Create permissions policies for EMR Studio users
Refer to the following sections to create permissions policies for EMR Studio.
**Topics**
+ [Create the permissions policies](#emr-studio-permissions-policies-create)
+ [Set ownership for Workspace collaboration](#emr-studio-workspace-collaboration-permissions)
+ [Create user-level Git secrets policy](#emr-studio-permissions-policies-git)
+ [Attach the permissions policy to your IAM identity](#emr-studio-permissions-policies-attach)
**Note**
To set Amazon S3 access permissions for storing notebook files, and to set AWS Secrets Manager access permissions to read secrets when you link Workspaces to Git repositories, use the EMR Studio service role.
### Create the permissions policies
Create one or more IAM permissions policies that specify what actions a user can take in your Studio. For example, you can create three separate policies for [basic](), [intermediate](), and [advanced]() Studio user types with the example policies on this page.
For a breakdown of each Studio operation that a user might perform, and the minimum IAM actions that are required to perform each operation, see [AWS Identity and Access Management permissions for EMR Studio users](#emr-studio-iam-permissions-table). For steps to create the policies, see [Creating IAM policies](https://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies_create-console.html) in the *IAM User Guide*.
Your permissions policy must include the following statements.
```
{
"Sid": "AllowAddingTagsOnSecretsWithEMRStudioPrefix",
"Effect": "Allow",
"Action": "secretsmanager:TagResource",
"Resource": "arn:aws:secretsmanager:*:*:secret:emr-studio-*"
},
{
"Sid": "AllowPassingServiceRoleForWorkspaceCreation",
"Action": "iam:PassRole",
"Resource": [
"arn:aws:iam::*:role/your-emr-studio-service-role"
],
"Effect": "Allow"
}
```
### Set ownership for Workspace collaboration
Workspace collaboration lets multiple users work simultaneously in the same Workspace and can be configured with the **Collaboration** panel in the Workspace UI. In order to see and use the **Collaboration** panel, a user must have the following permissions. Any user with these permissions can see and use the **Collaboration** panel.
```
"elasticmapreduce:UpdateEditor",
"elasticmapreduce:PutWorkspaceAccess",
"elasticmapreduce:DeleteWorkspaceAccess",
"elasticmapreduce:ListWorkspaceAccessIdentities"
```
To restrict access to the **Collaboration** panel, you can use tag-based access control. When a user creates a Workspace, EMR Studio applies a default tag with a key of `creatorUserId` whose value is the ID of the user creating the Workspace.
**Note**
EMR Studio adds the `creatorUserId` tag to Workspaces created after November 16, 2021. To restrict who can configure collaboration for workspaces that you created before this date, we recommend that you manually add the `creatorUserId` tag to your Workspace, and then use tag-based access control in your user permissions policies.
The following example statement allows a user to configure collaboration for any Workspace with the tag key `creatorUserId` whose value matches the user's ID (indicated by the policy variable `aws:userId`). In other words, the statement lets a user configure collaboration for the Workspaces that they create. To learn more about policy variables, see [IAM policy elements: Variables and tags](https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_variables.html) in the *IAM User Guide*.
```
{
"Sid": "UserRolePermissionsForCollaboration",
"Action": [
"elasticmapreduce:UpdateEditor",
"elasticmapreduce:PutWorkspaceAccess",
"elasticmapreduce:DeleteWorkspaceAccess",
"elasticmapreduce:ListWorkspaceAccessIdentities"
],
"Resource": "*",
"Effect": "Allow",
"Condition": {
"StringEquals": {
"elasticmapreduce:ResourceTag/creatorUserId": "${aws:userid}"
}
}
}
```
### Create user-level Git secrets policy
**Topics**
+ [To use user-level permissions](#emr-studio-permissions-policies-user)
+ [To transition from service-level permissions to user-level permissions](#emr-studio-permissions-policies-transition)
+ [To use service-level permissions](#emr-studio-permissions-policies-service)
#### To use user-level permissions
EMR Studio automatically adds the `for-use-with-amazon-emr-managed-user-policies` tag when it creates Git secrets. If you want to control access to Git secrets at the user level, add tag-based permissions to the EMR Studio **user role policy** with `secretsmanager:GetSecretValue` as shown in the [To transition from service-level permissions to user-level permissions](#emr-studio-permissions-policies-transition) section below.
If you have existing permissions for `secretsmanager:GetSecretValue` in the EMR Studio **service role policy**, you should remove those permissions.
#### To transition from service-level permissions to user-level permissions
**Note**
The `for-use-with-amazon-emr-managed-user-policies` tag ensures that the permissions from **Step 1** below grant the creator of the workspace access to the Git secret. However, if you linked Git repositories before September 1, 2023, then the corresponding Git secrets will be denied access because they don't have the `for-use-with-amazon-emr-managed-user-policies` tag applied. To apply user-level permissions, you must recreate the old secrets from JupyterLab and link the appropriate Git repositories again.
For more information about policy variables, see [IAM policy elements: Variables and tags](https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_variables.html) in the *IAM User Guide*.
1. Add the following permissions to the the [EMR Studio **user role policy**](emr-studio-service-role.md). It uses the `for-use-with-amazon-emr-managed-user-policies` key with value `"${aws:userid}"`.
```
{
"Sid": "AllowSecretsManagerReadOnlyActionsWithEMRTags",
"Effect": "Allow",
"Action": "secretsmanager:GetSecretValue",
"Resource": "arn:aws:secretsmanager:*:*:secret:*",
"Condition": {
"StringEquals": {
"secretsmanager:ResourceTag/for-use-with-amazon-emr-managed-user-policies": "${aws:userid}"
}
}
}
```
1. If present, remove the following permission from the [EMR Studio **service role policy**](emr-studio-service-role.md). Because the service role policy applies to all secrets defined by each user, you only need to do this one time.
```
{
"Sid": "AllowSecretsManagerReadOnlyActionsWithEMRTags",
"Effect": "Allow",
"Action": [
"secretsmanager:GetSecretValue"
],
"Resource": "arn:aws:secretsmanager:*:*:secret:*",
"Condition": {
"StringEquals": {
"aws:ResourceTag/for-use-with-amazon-emr-managed-policies": "true"
}
}
}
```
#### To use service-level permissions
As of September 1, 2023, EMR Studio automatically adds the `for-use-with-amazon-emr-managed-user-policies` tag for user-level access control. Because this is an added capability, you can continue to use service-level access that's available through the `GetSecretValue` permission in the [EMR Studio service role](emr-studio-service-role.md).
For secrets created before September 1, 2023, EMR Studio didn't add the `for-use-with-amazon-emr-managed-user-policies` tag. To keep using service-level permissions, simply retain your existing [EMR Studio service role](emr-studio-service-role.md) and user role permissions. However, to restrict who can access an individual secret, we recommend that you follow the steps in [To use user-level permissions](#emr-studio-permissions-policies-user) to manually add the `for-use-with-amazon-emr-managed-user-policies` tag to your secrets, and then use tag-based access control in your user permissions policies.
For more information about policy variables, see [IAM policy elements: Variables and tags](https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_variables.html) in the *IAM User Guide*.
### Attach the permissions policy to your IAM identity
The following table summarizes which IAM identity you attach a permissions policy to, depending on your EMR Studio authentication mode. For instructions on how to attach a policy, see [Adding and removing IAM identity permissions](https://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies_manage-attach-detach.html).
****
| If you use... | Attach the policy to... |
| --- | --- |
| IAM authentication | Your IAM identities (users, groups of users, or roles). For example, you can attach a permissions policy to a user in your AWS account. |
| IAM federation with an external identity provider (IdP) | The IAM role or roles that you create for your external IdP. For example, an IAM for SAML 2.0 federation. EMR Studio uses the permissions that you attach to your IAM role(s) for users with federated access to a Studio. |
| IAM Identity Center | Your Amazon EMR Studio user role. |
## Example user policies
The following basic user policy allows most EMR Studio actions, but does not let a user create new Amazon EMR clusters.
### Basic policy
**Important**
The example policy does not include the `CreateStudioPresignedUrl` permission, which you must allow for a user when you use IAM authentication mode. For more information, see [Assign a user or group to an EMR Studio](emr-studio-manage-users.md#emr-studio-assign-users-groups).
The example policy includes `Condition` elements to enforce tag-based access control (TBAC) so that you can use the policy with the example service role for EMR Studio. For more information, see [Create an EMR Studio service role](emr-studio-service-role.md).
------
#### [ JSON ]
****
```
{
"Version":"2012-10-17",
"Statement": [
{
"Sid": "AllowDefaultEC2SecurityGroupsCreationInVPCWithEMRTags",
"Effect": "Allow",
"Action": [
"ec2:CreateSecurityGroup"
],
"Resource": [
"arn:aws:ec2:*:*:vpc/*"
],
"Condition": {
"StringEquals": {
"aws:ResourceTag/for-use-with-amazon-emr-managed-policies": "true"
}
}
},
{
"Sid": "AllowAddingEMRTagsDuringDefaultSecurityGroupCreation",
"Effect": "Allow",
"Action": [
"ec2:CreateTags"
],
"Resource": [
"arn:aws:ec2:*:*:security-group/*"
],
"Condition": {
"StringEquals": {
"aws:RequestTag/for-use-with-amazon-emr-managed-policies": "true",
"ec2:CreateAction": "CreateSecurityGroup"
}
}
},
{
"Sid": "AllowSecretManagerListSecrets",
"Action": [
"secretsmanager:ListSecrets"
],
"Resource": [
"*"
],
"Effect": "Allow"
},
{
"Sid": "AllowSecretCreationWithEMRTagsAndEMRStudioPrefix",
"Effect": "Allow",
"Action": [
"secretsmanager:CreateSecret"
],
"Resource": [
"arn:aws:secretsmanager:*:*:secret:emr-studio-*"
],
"Condition": {
"StringEquals": {
"aws:RequestTag/for-use-with-amazon-emr-managed-policies": "true"
}
}
},
{
"Sid": "AllowAddingTagsOnSecretsWithEMRStudioPrefix",
"Effect": "Allow",
"Action": [
"secretsmanager:TagResource"
],
"Resource": [
"arn:aws:secretsmanager:*:*:secret:emr-studio-*"
]
},
{
"Sid": "AllowPassingServiceRoleForWorkspaceCreation",
"Action": [
"iam:PassRole"
],
"Resource": [
"arn:aws:iam::*:role/your-emr-studio-service-role>"
],
"Effect": "Allow"
},
{
"Sid": "AllowS3ListAndLocationPermissions",
"Action": [
"s3:ListAllMyBuckets",
"s3:ListBucket",
"s3:GetBucketLocation"
],
"Resource": [
"arn:aws:s3:::*"
],
"Effect": "Allow"
},
{
"Sid": "AllowS3ReadOnlyAccessToLogs",
"Action": [
"s3:GetObject"
],
"Resource": [
"arn:aws:s3:::aws-logs-aws-111122223333>-region>/elasticmapreduce/*"
],
"Effect": "Allow"
},
{
"Sid": "AllowConfigurationForWorkspaceCollaboration",
"Action": [
"elasticmapreduce:UpdateEditor",
"elasticmapreduce:PutWorkspaceAccess",
"elasticmapreduce:DeleteWorkspaceAccess",
"elasticmapreduce:ListWorkspaceAccessIdentities"
],
"Resource": [
"*"
],
"Effect": "Allow",
"Condition": {
"StringEquals": {
"elasticmapreduce:ResourceTag/creatorUserId": "${aws:userId}"
}
}
},
{
"Sid": "DescribeNetwork",
"Effect": "Allow",
"Action": [
"ec2:DescribeVpcs",
"ec2:DescribeSubnets",
"ec2:DescribeSecurityGroups"
],
"Resource": [
"*"
]
},
{
"Sid": "ListIAMRoles",
"Effect": "Allow",
"Action": [
"iam:ListRoles"
],
"Resource": [
"*"
]
}
]
}
```
------
The following intermediate user policy allows most EMR Studio actions, and lets a user create new Amazon EMR clusters using a cluster template.
### Intermediate policy
**Important**
The example policy does not include the `CreateStudioPresignedUrl` permission, which you must allow for a user when you use IAM authentication mode. For more information, see [Assign a user or group to an EMR Studio](emr-studio-manage-users.md#emr-studio-assign-users-groups).
The example policy includes `Condition` elements to enforce tag-based access control (TBAC) so that you can use the policy with the example service role for EMR Studio. For more information, see [Create an EMR Studio service role](emr-studio-service-role.md).
------
#### [ JSON ]
****
```
{
"Version":"2012-10-17",
"Statement": [
{
"Sid": "AllowEMRBasicActions",
"Action": [
"elasticmapreduce:CreateEditor",
"elasticmapreduce:DescribeEditor",
"elasticmapreduce:ListEditors",
"elasticmapreduce:StartEditor",
"elasticmapreduce:StopEditor",
"elasticmapreduce:DeleteEditor",
"elasticmapreduce:OpenEditorInConsole",
"elasticmapreduce:AttachEditor",
"elasticmapreduce:DetachEditor",
"elasticmapreduce:CreateRepository",
"elasticmapreduce:DescribeRepository",
"elasticmapreduce:DeleteRepository",
"elasticmapreduce:ListRepositories",
"elasticmapreduce:LinkRepository",
"elasticmapreduce:UnlinkRepository",
"elasticmapreduce:DescribeCluster",
"elasticmapreduce:ListInstanceGroups",
"elasticmapreduce:ListBootstrapActions",
"elasticmapreduce:ListClusters",
"elasticmapreduce:ListSteps",
"elasticmapreduce:CreatePersistentAppUI",
"elasticmapreduce:DescribePersistentAppUI",
"elasticmapreduce:GetPersistentAppUIPresignedURL",
"elasticmapreduce:GetOnClusterAppUIPresignedURL"
],
"Resource": [
"*"
],
"Effect": "Allow"
},
{
"Sid": "AllowEMRContainersBasicActions",
"Action": [
"emr-containers:DescribeVirtualCluster",
"emr-containers:ListVirtualClusters",
"emr-containers:DescribeManagedEndpoint",
"emr-containers:ListManagedEndpoints",
"emr-containers:DescribeJobRun",
"emr-containers:ListJobRuns"
],
"Resource": [
"*"
],
"Effect": "Allow"
},
{
"Sid": "AllowRetrievingManagedEndpointCredentials",
"Effect": "Allow",
"Action": [
"emr-containers:GetManagedEndpointSessionCredentials"
],
"Resource": [
"arn:aws:emr-containers:us-west-1:123456789012:/virtualclusters/virtual-cluster-id/endpoints/managed-endpoint-id"
],
"Condition": {
"StringEquals": {
"emr-containers:ExecutionRoleArn": [
"arn:aws:iam::123456789012:role/emr-on-eks-execution-role"
]
}
}
},
{
"Sid": "AllowSecretManagerListSecrets",
"Action": [
"secretsmanager:ListSecrets"
],
"Resource": [
"*"
],
"Effect": "Allow"
},
{
"Sid": "AllowSecretCreationWithEMRTagsAndEMRStudioPrefix",
"Effect": "Allow",
"Action": [
"secretsmanager:CreateSecret"
],
"Resource": [
"arn:aws:secretsmanager:*:*:secret:emr-studio-*"
],
"Condition": {
"StringEquals": {
"aws:RequestTag/for-use-with-amazon-emr-managed-policies": "true"
}
}
},
{
"Sid": "AllowAddingTagsOnSecretsWithEMRStudioPrefix",
"Effect": "Allow",
"Action": [
"secretsmanager:TagResource"
],
"Resource": [
"arn:aws:secretsmanager:*:*:secret:emr-studio-*"
]
},
{
"Sid": "AllowClusterTemplateRelatedIntermediateActions",
"Action": [
"servicecatalog:DescribeProduct",
"servicecatalog:DescribeProductView",
"servicecatalog:DescribeProvisioningParameters",
"servicecatalog:ProvisionProduct",
"servicecatalog:SearchProducts",
"servicecatalog:UpdateProvisionedProduct",
"servicecatalog:ListProvisioningArtifacts",
"servicecatalog:ListLaunchPaths",
"servicecatalog:DescribeRecord",
"cloudformation:DescribeStackResources"
],
"Resource": [
"*"
],
"Effect": "Allow"
},
{
"Sid": "AllowPassingServiceRoleForWorkspaceCreation",
"Action": [
"iam:PassRole"
],
"Resource": [
"arn:aws:iam::*:role/your-emr-studio-service-role"
],
"Effect": "Allow"
},
{
"Sid": "AllowS3ListAndLocationPermissions",
"Action": [
"s3:ListAllMyBuckets",
"s3:ListBucket",
"s3:GetBucketLocation"
],
"Resource": [
"arn:aws:s3:::*"
],
"Effect": "Allow"
},
{
"Sid": "AllowS3ReadOnlyAccessToLogs",
"Action": [
"s3:GetObject"
],
"Resource": [
"arn:aws:s3:::aws-logs-123456789012-us-east-1/elasticmapreduce/*"
],
"Effect": "Allow"
},
{
"Sid": "AllowConfigurationForWorkspaceCollaboration",
"Action": [
"elasticmapreduce:UpdateEditor",
"elasticmapreduce:PutWorkspaceAccess",
"elasticmapreduce:DeleteWorkspaceAccess",
"elasticmapreduce:ListWorkspaceAccessIdentities"
],
"Resource": [
"*"
],
"Effect": "Allow",
"Condition": {
"StringEquals": {
"elasticmapreduce:ResourceTag/creatorUserId": "${aws:userId}"
}
}
},
{
"Sid": "DescribeNetwork",
"Effect": "Allow",
"Action": [
"ec2:DescribeVpcs",
"ec2:DescribeSubnets",
"ec2:DescribeSecurityGroups"
],
"Resource": [
"*"
]
},
{
"Sid": "ListIAMRoles",
"Effect": "Allow",
"Action": [
"iam:ListRoles"
],
"Resource": [
"*"
]
},
{
"Sid": "AllowServerlessActions",
"Action": [
"emr-serverless:CreateApplication",
"emr-serverless:UpdateApplication",
"emr-serverless:DeleteApplication",
"emr-serverless:ListApplications",
"emr-serverless:GetApplication",
"emr-serverless:StartApplication",
"emr-serverless:StopApplication",
"emr-serverless:StartJobRun",
"emr-serverless:CancelJobRun",
"emr-serverless:ListJobRuns",
"emr-serverless:GetJobRun",
"emr-serverless:GetDashboardForJobRun",
"emr-serverless:AccessInteractiveEndpoints"
],
"Resource": [
"*"
],
"Effect": "Allow"
},
{
"Sid": "AllowPassingRuntimeRoleForRunningServerlessJob",
"Action": [
"iam:PassRole"
],
"Resource": [
"arn:aws:iam::*:role/serverless-runtime-role"
],
"Effect": "Allow"
}
]
}
```
------
The following advanced user policy allows all EMR Studio actions, and lets a user create new Amazon EMR clusters using a cluster template or by providing a cluster configuration.
### Advanced policy
**Important**
The example policy does not include the `CreateStudioPresignedUrl` permission, which you must allow for a user when you use IAM authentication mode. For more information, see [Assign a user or group to an EMR Studio](emr-studio-manage-users.md#emr-studio-assign-users-groups).
The example policy includes `Condition` elements to enforce tag-based access control (TBAC) so that you can use the policy with the example service role for EMR Studio. For more information, see [Create an EMR Studio service role](emr-studio-service-role.md).
------
#### [ JSON ]
****
```
{
"Version":"2012-10-17",
"Statement": [
{
"Sid": "AllowEMRBasicActions",
"Action": [
"elasticmapreduce:CreateEditor",
"elasticmapreduce:DescribeEditor",
"elasticmapreduce:ListEditors",
"elasticmapreduce:StartEditor",
"elasticmapreduce:StopEditor",
"elasticmapreduce:DeleteEditor",
"elasticmapreduce:OpenEditorInConsole",
"elasticmapreduce:AttachEditor",
"elasticmapreduce:DetachEditor",
"elasticmapreduce:CreateRepository",
"elasticmapreduce:DescribeRepository",
"elasticmapreduce:DeleteRepository",
"elasticmapreduce:ListRepositories",
"elasticmapreduce:LinkRepository",
"elasticmapreduce:UnlinkRepository",
"elasticmapreduce:DescribeCluster",
"elasticmapreduce:ListInstanceGroups",
"elasticmapreduce:ListBootstrapActions",
"elasticmapreduce:ListClusters",
"elasticmapreduce:ListSteps",
"elasticmapreduce:CreatePersistentAppUI",
"elasticmapreduce:DescribePersistentAppUI",
"elasticmapreduce:GetPersistentAppUIPresignedURL",
"elasticmapreduce:GetOnClusterAppUIPresignedURL"
],
"Resource": [
"*"
],
"Effect": "Allow"
},
{
"Sid": "AllowEMRContainersBasicActions",
"Action": [
"emr-containers:DescribeVirtualCluster",
"emr-containers:ListVirtualClusters",
"emr-containers:DescribeManagedEndpoint",
"emr-containers:ListManagedEndpoints",
"emr-containers:DescribeJobRun",
"emr-containers:ListJobRuns"
],
"Resource": [
"*"
],
"Effect": "Allow"
},
{
"Sid": "AllowRetrievingManagedEndpointCredentials",
"Effect": "Allow",
"Action": [
"emr-containers:GetManagedEndpointSessionCredentials"
],
"Resource": [
"arn:aws:emr-containers:*:123456789012:/virtualclusters/virtual-cluster-id/endpoints/managed-endpoint-id"
],
"Condition": {
"StringEquals": {
"emr-containers:ExecutionRoleArn": [
"arn:aws:iam::123456789012:role/emr-on-eks-execution-role"
]
}
}
},
{
"Sid": "AllowSecretManagerListSecrets",
"Action": [
"secretsmanager:ListSecrets"
],
"Resource": [
"*"
],
"Effect": "Allow"
},
{
"Sid": "AllowSecretCreationWithEMRTagsAndEMRStudioPrefix",
"Effect": "Allow",
"Action": [
"secretsmanager:CreateSecret"
],
"Resource": [
"arn:aws:secretsmanager:*:*:secret:emr-studio-*"
],
"Condition": {
"StringEquals": {
"aws:RequestTag/for-use-with-amazon-emr-managed-policies": "true"
}
}
},
{
"Sid": "AllowAddingTagsOnSecretsWithEMRStudioPrefix",
"Effect": "Allow",
"Action": [
"secretsmanager:TagResource"
],
"Resource": [
"arn:aws:secretsmanager:*:*:secret:emr-studio-*"
]
},
{
"Sid": "AllowClusterTemplateRelatedIntermediateActions",
"Action": [
"servicecatalog:DescribeProduct",
"servicecatalog:DescribeProductView",
"servicecatalog:DescribeProvisioningParameters",
"servicecatalog:ProvisionProduct",
"servicecatalog:SearchProducts",
"servicecatalog:UpdateProvisionedProduct",
"servicecatalog:ListProvisioningArtifacts",
"servicecatalog:ListLaunchPaths",
"servicecatalog:DescribeRecord",
"cloudformation:DescribeStackResources"
],
"Resource": [
"*"
],
"Effect": "Allow"
},
{
"Sid": "AllowEMRCreateClusterAdvancedActions",
"Action": [
"elasticmapreduce:RunJobFlow"
],
"Resource": [
"*"
],
"Effect": "Allow"
},
{
"Sid": "AllowPassingServiceRoleForWorkspaceCreation",
"Action": [
"iam:PassRole"
],
"Resource": [
"arn:aws:iam::*:role/your-emr-studio-service-role",
"arn:aws:iam::*:role/EMR_DefaultRole_V2",
"arn:aws:iam::*:role/EMR_EC2_DefaultRole"
],
"Effect": "Allow"
},
{
"Sid": "AllowS3ListAndLocationPermissions",
"Action": [
"s3:ListAllMyBuckets",
"s3:ListBucket",
"s3:GetBucketLocation"
],
"Resource": [
"arn:aws:s3:::*"
],
"Effect": "Allow"
},
{
"Sid": "AllowS3ReadOnlyAccessToLogs",
"Action": [
"s3:GetObject"
],
"Resource": [
"arn:aws:s3:::aws-logs-123456789012-us-east-1/elasticmapreduce/*"
],
"Effect": "Allow"
},
{
"Sid": "AllowConfigurationForWorkspaceCollaboration",
"Action": [
"elasticmapreduce:UpdateEditor",
"elasticmapreduce:PutWorkspaceAccess",
"elasticmapreduce:DeleteWorkspaceAccess",
"elasticmapreduce:ListWorkspaceAccessIdentities"
],
"Resource": [
"*"
],
"Effect": "Allow",
"Condition": {
"StringEquals": {
"elasticmapreduce:ResourceTag/creatorUserId": "${aws:userId}"
}
}
},
{
"Sid": "SageMakerDataWranglerForEMRStudio",
"Effect": "Allow",
"Action": [
"sagemaker:CreatePresignedDomainUrl",
"sagemaker:DescribeDomain",
"sagemaker:ListDomains",
"sagemaker:ListUserProfiles"
],
"Resource": [
"*"
]
},
{
"Sid": "DescribeNetwork",
"Effect": "Allow",
"Action": [
"ec2:DescribeVpcs",
"ec2:DescribeSubnets",
"ec2:DescribeSecurityGroups"
],
"Resource": [
"*"
]
},
{
"Sid": "ListIAMRoles",
"Effect": "Allow",
"Action": [
"iam:ListRoles"
],
"Resource": [
"*"
]
},
{
"Sid": "AllowServerlessActions",
"Action": [
"emr-serverless:CreateApplication",
"emr-serverless:UpdateApplication",
"emr-serverless:DeleteApplication",
"emr-serverless:ListApplications",
"emr-serverless:GetApplication",
"emr-serverless:StartApplication",
"emr-serverless:StopApplication",
"emr-serverless:StartJobRun",
"emr-serverless:CancelJobRun",
"emr-serverless:ListJobRuns",
"emr-serverless:GetJobRun",
"emr-serverless:GetDashboardForJobRun",
"emr-serverless:AccessInteractiveEndpoints"
],
"Resource": [
"*"
],
"Effect": "Allow"
},
{
"Sid": "AllowPassingRuntimeRoleForRunningServerlessJob",
"Action": [
"iam:PassRole"
],
"Resource": [
"arn:aws:iam::*:role/serverless-runtime-role"
],
"Effect": "Allow"
},
{
"Sid": "AllowCodeWhisperer",
"Effect": "Allow",
"Action": [
"codewhisperer:GenerateRecommendations"
],
"Resource": [
"*"
]
},
{
"Sid": "AllowAthenaSQL",
"Action": [
"athena:StartQueryExecution",
"athena:StopQueryExecution",
"athena:GetQueryExecution",
"athena:GetQueryRuntimeStatistics",
"athena:GetQueryResults",
"athena:ListQueryExecutions",
"athena:BatchGetQueryExecution",
"athena:GetNamedQuery",
"athena:ListNamedQueries",
"athena:BatchGetNamedQuery",
"athena:UpdateNamedQuery",
"athena:DeleteNamedQuery",
"athena:ListDataCatalogs",
"athena:GetDataCatalog",
"athena:ListDatabases",
"athena:GetDatabase",
"athena:ListTableMetadata",
"athena:GetTableMetadata",
"athena:ListWorkGroups",
"athena:GetWorkGroup",
"athena:CreateNamedQuery",
"athena:GetPreparedStatement",
"glue:CreateDatabase",
"glue:DeleteDatabase",
"glue:GetDatabase",
"glue:GetDatabases",
"glue:UpdateDatabase",
"glue:CreateTable",
"glue:DeleteTable",
"glue:BatchDeleteTable",
"glue:UpdateTable",
"glue:GetTable",
"glue:GetTables",
"glue:BatchCreatePartition",
"glue:CreatePartition",
"glue:DeletePartition",
"glue:BatchDeletePartition",
"glue:UpdatePartition",
"glue:GetPartition",
"glue:GetPartitions",
"glue:BatchGetPartition",
"kms:ListAliases",
"kms:ListKeys",
"kms:DescribeKey",
"lakeformation:GetDataAccess",
"s3:GetObject",
"s3:ListBucket",
"s3:ListBucketMultipartUploads",
"s3:ListMultipartUploadParts",
"s3:AbortMultipartUpload",
"s3:PutObject",
"s3:PutBucketPublicAccessBlock",
"s3:ListAllMyBuckets"
],
"Resource": [
"*"
],
"Effect": "Allow"
}
]
}
```
------
The following user policy contains the minimum user permissions that are required to use an EMR Serverless interactive application with EMR Studio Workspaces.
### EMR Serverless interactive policy
In this example policy that has user permissions for EMR Serverless interactive applications with EMR Studio, replace the placeholders for *serverless-runtime-role* and *emr-studio-service-role* with your correct [EMR Studio service role](emr-studio-service-role.md) and [EMR Serverless runtime role](https://docs.aws.amazon.com/emr/latest/EMR-Serverless-UserGuide/security-iam-runtime-role.html).
------
#### [ JSON ]
****
```
{
"Version":"2012-10-17",
"Statement": [
{
"Sid": "AllowServerlessActions",
"Action": [
"emr-serverless:CreateApplication",
"emr-serverless:UpdateApplication",
"emr-serverless:DeleteApplication",
"emr-serverless:ListApplications",
"emr-serverless:GetApplication",
"emr-serverless:StartApplication",
"emr-serverless:StopApplication",
"emr-serverless:StartJobRun",
"emr-serverless:CancelJobRun",
"emr-serverless:ListJobRuns",
"emr-serverless:GetJobRun",
"emr-serverless:GetDashboardForJobRun",
"emr-serverless:AccessInteractiveEndpoints"
],
"Resource": [
"*"
],
"Effect": "Allow"
},
{
"Sid": "AllowEMRBasicActions",
"Action": [
"elasticmapreduce:CreateEditor",
"elasticmapreduce:DescribeEditor",
"elasticmapreduce:ListEditors",
"elasticmapreduce:UpdateStudio",
"elasticmapreduce:StartEditor",
"elasticmapreduce:StopEditor",
"elasticmapreduce:DeleteEditor",
"elasticmapreduce:OpenEditorInConsole",
"elasticmapreduce:AttachEditor",
"elasticmapreduce:DetachEditor",
"elasticmapreduce:CreateStudio",
"elasticmapreduce:DescribeStudio",
"elasticmapreduce:DeleteStudio",
"elasticmapreduce:ListStudios",
"elasticmapreduce:CreateStudioPresignedUrl"
],
"Resource": [
"*"
],
"Effect": "Allow"
},
{
"Sid": "AllowPassingRuntimeRoleForRunningEMRServerlessJob",
"Action": [
"iam:PassRole"
],
"Resource": [
"arn:aws:iam::*:role/serverless-runtime-role"
],
"Effect": "Allow"
},
{
"Sid": "AllowPassingServiceRoleForWorkspaceCreation",
"Action": [
"iam:PassRole"
],
"Resource": [
"arn:aws:iam::*:role/emr-studio-service-role"
],
"Effect": "Allow"
},
{
"Sid": "AllowS3ListAndGetPermissions",
"Action": [
"s3:ListAllMyBuckets",
"s3:ListBucket",
"s3:GetBucketLocation",
"s3:GetObject"
],
"Resource": [
"arn:aws:s3:::*"
],
"Effect": "Allow"
},
{
"Sid": "DescribeNetwork",
"Effect": "Allow",
"Action": [
"ec2:DescribeVpcs",
"ec2:DescribeSubnets",
"ec2:DescribeSecurityGroups"
],
"Resource": [
"*"
]
},
{
"Sid": "ListIAMRoles",
"Effect": "Allow",
"Action": [
"iam:ListRoles"
],
"Resource": [
"*"
]
}
]
}
```
------
## AWS Identity and Access Management permissions for EMR Studio users
The following table includes each Amazon EMR Studio operation that a user might perform, and lists the minimum IAM actions needed to perform that operation. You allow these actions in your IAM permissions policies (when you use IAM authentication) or in your user role session policies (when you use IAM Identity Center authentication) for EMR Studio.
The table also displays the operations allowed in each of example permissions policy for EMR Studio. For more information about the example permissions policies, see [Create permissions policies for EMR Studio users](#emr-studio-permissions-policies).
| Action | Basic | Intermediate | Advanced | Associated actions |
| --- | --- | --- | --- | --- |
| Create and delete Workspaces | Yes | Yes | Yes | "elasticmapreduce:CreateEditor",
"elasticmapreduce:DescribeEditor",
"elasticmapreduce:ListEditors",
"elasticmapreduce:DeleteEditor"
|
| View the Collaboration panel, enable Workspace collaboration, and add collaborators. For more information, see [Set ownership for Workspace collaboration](#emr-studio-workspace-collaboration-permissions). | Yes | Yes | Yes | "elasticmapreduce:UpdateEditor",
"elasticmapreduce:PutWorkspaceAccess",
"elasticmapreduce:DeleteWorkspaceAccess",
"elasticmapreduce:ListWorkspaceAccessIdentities"
|
| See a list of Amazon S3 Control storage buckets in the same account as the Studio when creating a new EMR cluster, and access container logs when using a web UI to debug applications | Yes | Yes | Yes | "s3:ListAllMyBuckets",
"s3:ListBucket",
"s3:GetBucketLocation",
"s3:GetObject"
|
| Access Workspaces | Yes | Yes | Yes | "elasticmapreduce:DescribeEditor",
"elasticmapreduce:ListEditors",
"elasticmapreduce:StartEditor",
"elasticmapreduce:StopEditor",
"elasticmapreduce:OpenEditorInConsole"
|
| Attach or detach existing Amazon EMR clusters associated with the Workspace | Yes | Yes | Yes | "elasticmapreduce:AttachEditor",
"elasticmapreduce:DetachEditor",
"elasticmapreduce:ListClusters",
"elasticmapreduce:DescribeCluster",
"elasticmapreduce:ListInstanceGroups",
"elasticmapreduce:ListBootstrapActions"
|
| Attach or detach Amazon EMR on EKS clusters | Yes | Yes | Yes | "elasticmapreduce:AttachEditor",
"elasticmapreduce:DetachEditor",
"emr-containers:ListVirtualClusters",
"emr-containers:DescribeVirtualCluster",
"emr-containers:ListManagedEndpoints",
"emr-containers:DescribeManagedEndpoint",
"emr-containers:GetManagedEndpointSessionCredentials"
|
| Attach or detach EMR Serverless applications that are associated with the Workspace | No | Yes | Yes | "elasticmapreduce:AttachEditor",
"elasticmapreduce:DetachEditor",
"emr-serverless:GetApplication",
"emr-serverless:StartApplication",
"emr-serverless:ListApplications",
"emr-serverless:GetDashboardForJobRun",
"emr-serverless:AccessInteractiveEndpoints",
"iam:PassRole"
The `PassRole` permission is required to pass the EMR Serverless job runtime role. For more information, see [Job runtime roles](https://docs.aws.amazon.com/emr/latest/EMR-Serverless-UserGuide/security-iam-runtime-role.html) in the *Amazon EMR Serverless User Guide*. |
| Debug Amazon EMR on EC2 jobs with persistent application user interfaces | Yes | Yes | Yes | "elasticmapreduce:CreatePersistentAppUI",
"elasticmapreduce:DescribePersistentAppUI",
"elasticmapreduce:GetPersistentAppUIPresignedURL",
"elasticmapreduce:ListClusters",
"elasticmapreduce:ListSteps",
"elasticmapreduce:DescribeCluster",
"s3:ListBucket",
"s3:GetObject"
|
| Debug Amazon EMR on EC2 jobs with on-cluster application user interfaces | Yes | Yes | Yes | "elasticmapreduce:GetOnClusterAppUIPresignedURL"
|
| Debug Amazon EMR on EKS job runs using the Spark History Server | Yes | Yes | Yes | "elasticmapreduce:CreatePersistentAppUI",
"elasticmapreduce:DescribePersistentAppUI",
"elasticmapreduce:GetPersistentAppUIPresignedURL",
"emr-containers:ListVirtualClusters",
"emr-containers:DescribeVirtualCluster",
"emr-containers:ListJobRuns",
"emr-containers:DescribeJobRun",
"s3:ListBucket",
"s3:GetObject"
|
| Create and delete Git repositories | Yes | Yes | Yes | "elasticmapreduce:CreateRepository",
"elasticmapreduce:DeleteRepository",
"elasticmapreduce:ListRepositories",
"elasticmapreduce:DescribeRepository",
"secretsmanager:CreateSecret",
"secretsmanager:ListSecrets",
"secretsmanager:TagResource"
|
| Link and unlink Git repositories | Yes | Yes | Yes | "elasticmapreduce:LinkRepository",
"elasticmapreduce:UnlinkRepository",
"elasticmapreduce:ListRepositories",
"elasticmapreduce:DescribeRepository"
|
| Create new clusters from predefined cluster templates | No | Yes | Yes | "servicecatalog:SearchProducts",
"servicecatalog:DescribeProduct",
"servicecatalog:DescribeProductView",
"servicecatalog:DescribeProvisioningParameters",
"servicecatalog:ProvisionProduct",
"servicecatalog:UpdateProvisionedProduct",
"servicecatalog:ListProvisioningArtifacts",
"servicecatalog:DescribeRecord",
"servicecatalog:ListLaunchPaths",
"cloudformation:DescribeStackResources",
"elasticmapreduce:ListClusters",
"elasticmapreduce:DescribeCluster"
|
| Provide a cluster configuration to create new clusters. | No | No | Yes | "elasticmapreduce:RunJobFlow",
"iam:PassRole",
"elasticmapreduce:ListClusters",
"elasticmapreduce:DescribeCluster"
|
| [Assign a user to a Studio when you use IAM authentication mode.](emr-studio-manage-users.md#emr-studio-assign-users-groups) | No | No | No | "elasticmapreduce:CreateStudioPresignedUrl"
|
| Describe network objects. | Yes | Yes | Yes | JSON
****
```
{
"Version":"2012-10-17",
"Statement": [
{
"Sid": "DescribeNetwork",
"Effect": "Allow",
"Action": [
"ec2:DescribeVpcs",
"ec2:DescribeSubnets",
"ec2:DescribeSecurityGroups"
],
"Resource": [
"*"
]
}
]
}
``` |
| List IAM roles. | Yes | Yes | Yes | JSON
****
```
{
"Version":"2012-10-17",
"Statement": [
{
"Sid": "ListIAMRoles",
"Effect": "Allow",
"Action": [
"iam:ListRoles"
],
"Resource": [
"*"
]
}
]
}
``` |
| [Connect to EMR Studio from Amazon SageMaker AI Studio and use the Data Wrangler visual interface.](https://aws.amazon.com/blogs/machine-learning/prepare-data-from-amazon-emr-for-machine-learning-using-amazon-sagemaker-data-wrangler/) | No | No | Yes | "sagemaker:CreatePresignedDomainUrl",
"sagemaker:DescribeDomain",
"sagemaker:ListDomains",
"sagemaker:ListUserProfiles"
|
| [Use Amazon CodeWhisperer in your EMR Studio.](emr-studio-codewhisperer.md) | No | No | Yes | "codewhisperer:GenerateRecommendations"
|
| [Access Amazon Athena SQL editor from your EMR Studio.](emr-studio-athena.md) This list might not include all of the permissions that you need to use all Athena features. For the most up-to-date list, see the [Athena full access policy](https://docs.aws.amazon.com/athena/latest/ug/managed-policies.html#amazonathenafullaccess-managed-policy). | No | No | Yes | "athena:StartQueryExecution",
"athena:StopQueryExecution",
"athena:GetQueryExecution",
"athena:GetQueryRuntimeStatistics",
"athena:GetQueryResults",
"athena:ListQueryExecutions",
"athena:BatchGetQueryExecution",
"athena:GetNamedQuery",
"athena:ListNamedQueries",
"athena:BatchGetNamedQuery",
"athena:UpdateNamedQuery",
"athena:DeleteNamedQuery",
"athena:ListDataCatalogs",
"athena:GetDataCatalog",
"athena:ListDatabases",
"athena:GetDatabase",
"athena:ListTableMetadata",
"athena:GetTableMetadata",
"athena:ListWorkGroups",
"athena:GetWorkGroup",
"athena:CreateNamedQuery",
"athena:GetPreparedStatement",
"glue:CreateDatabase",
"glue:DeleteDatabase",
"glue:GetDatabase",
"glue:GetDatabases",
"glue:UpdateDatabase",
"glue:CreateTable",
"glue:DeleteTable",
"glue:BatchDeleteTable",
"glue:UpdateTable",
"glue:GetTable",
"glue:GetTables",
"glue:BatchCreatePartition",
"glue:CreatePartition",
"glue:DeletePartition",
"glue:BatchDeletePartition",
"glue:UpdatePartition",
"glue:GetPartition",
"glue:GetPartitions",
"glue:BatchGetPartition",
"kms:ListAliases",
"kms:ListKeys",
"kms:DescribeKey",
"lakeformation:GetDataAccess",
"s3:GetBucketLocation",
"s3:GetBucketLocation",
"s3:GetObject",
"s3:ListBucket",
"s3:ListBucketMultipartUploads",
"s3:ListMultipartUploadParts",
"s3:AbortMultipartUpload",
"s3:PutObject",
"s3:PutBucketPublicAccessBlock",
"s3:ListAllMyBuckets"
|
# Create an EMR Studio
You can create an EMR Studio for your team with the Amazon EMR console or the AWS CLI. Creating a Studio instance is part of setting up Amazon EMR Studio.
**Prerequisites**
Before you create a Studio, make sure you've completed the previous tasks in [Set up an EMR Studio](emr-studio-set-up.md).
To create a Studio using the AWS CLI, you should have the latest version installed. For more information, see [Installing or updating the latest version of the AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html).
**Important**
Deactivate proxy management tools such as FoxyProxy or SwitchyOmega in the browser before you create a Studio. Active proxies can result in a **Network Failure ** error message when you choose **Create Studio**.
Amazon EMR provides you with a simple console experience to create a Studio, so you can quickly get started with the default settings. to run interactive workloads or batch jobs with the default settings. Creating a EMR Studio also creates an EMR Serverless application ready for your interactive jobs.
If you want full control over your Studio's settings, you can choose **Custom**, which lets you configure all of the additional settings.
------
#### [ Interactive workloads ]
**To create a EMR Studio for interactive workloads**
1. Open the Amazon EMR console at [https://console.aws.amazon.com/emr](https://console.aws.amazon.com/emr).
1. Under **EMR Studio** on the left navigation, choose **Getting started**. You can also create a new Studio from the **Studios** page.
1. Amazon EMR provides default settings for you if you're creating a EMR Studio for interactive workloads, but you can edit these settings. Configurable settings include the EMR Studio's name, the S3 location for your Workspace, the service role to use, the Workspace(s) you want to use, EMR Serverless application name, and the associated runtime role.
1. Choose **Create Studio and launch Workspace** to finish and navigate to the **Studios** page. Your new Studio appears in the list with details such as **Studio name**, **Creation date**, and **Studio access URL**. Your Workspace opens in a new tab in your browser.
------
#### [ Batch jobs ]
**To create a EMR Studio for interactive workloads**
1. Open the Amazon EMR console at [https://console.aws.amazon.com/emr](https://console.aws.amazon.com/emr).
1. Under **EMR Studio** on the left navigation, choose **Getting started**. You can also create a new Studio from the **Studios** page.
1. Amazon EMR provides default settings for you if you're creating a EMR Studio for batch jobs, but you can edit these settings. Configurable settings include the EMR Studio's name, EMR Serverless application name, and the associated runtime role.
1. Choose **Create Studio and launch Workspace** to finish and navigate to the **Studios** page. Your new Studio appears in the list with details such as **Studio name**, **Creation date**, and **Studio access URL**. Your EMR Studio opens in a new tab in your browser.
------
#### [ Custom settings ]
**To create a EMR Studio with custom settings**
1. Open the Amazon EMR console at [https://console.aws.amazon.com/emr](https://console.aws.amazon.com/emr).
1. Under **EMR Studio** on the left navigation, choose **Getting started**. You can also create a new Studio from the **Studios** page.
1. Choose **Create a Studio** to open the **Create a Studio** page.
1. Enter a **Studio name**.
1. Choose to create a new S3 bucket or use an existing location.
1. Choose the Workspace to add to the Studio. You can add up to 3 Workspaces.
1. Under **Authentication**, choose an authentication mode for the Studio and provide information according to the following table. To learn more about authentication for EMR Studio, see [Choose an authentication mode for Amazon EMR Studio](emr-studio-authentication.md).
****
[\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-studio-create-studio.html)
1. For VPC, choose an Amazon Virtual Private Cloud (**VPC**) for the Studio from the dropdown list.
1. Under **Subnets**, select a maximum of five subnets in your VPC to associate with the Studio. You have the option to add more subnets after you create the Studio.
1. For **Security groups**, choose either the default security groups or custom security groups. For more information, see [Define security groups to control EMR Studio network traffic](emr-studio-security-groups.md).
****
[\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-studio-create-studio.html)
1. Add tags to your Studio and other resources. For more information about tags, see [Tag clusters](https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-plan-tags.html).
1. Choose **Create Studio and launch Workspace** to finish and navigate to the **Studios** page. Your new Studio appears in the list with details such as **Studio name**, **Creation date**, and **Studio access URL**.
After you create a Studio, follow the instructions in [Assign a user or group to an EMR Studio](emr-studio-manage-users.md#emr-studio-assign-users-groups).
------
#### [ CLI ]
**Note**
Linux line continuation characters (\$1) are included for readability. They can be removed or used in Linux commands. For Windows, remove them or replace with a caret (^).
**Example – Create an EMR Studio that uses IAM for authentication**
The following example AWS CLI command creates an EMR Studio with IAM authentication mode. When you use IAM authentication or federation for the Studio, you don't specify a `--user-role`.
To let federated users log in using the Studio URL and credentials for your identity provider (IdP), specify your `--idp-auth-url` and `--idp-relay-state-parameter-name`. For a list of IdP authentication URLs and RelayState names, see [Identity provider RelayState parameters and authentication URLs](#emr-studio-idp-reference-table).
```
aws emr create-studio \
--name \
--auth-mode IAM \
--vpc-id \
--subnet-ids ... \
--service-role \
--user-role studio-user-role-name \
--workspace-security-group-id \
--engine-security-group-id \
--default-s3-location \
--idp-auth-url \
--idp-relay-state-parameter-name
```
**Example – Create an EMR Studio that uses Identity Center for authentication**
The following AWS CLI example command creates an EMR Studio that uses IAM Identity Center authentication mode. When you use IAM Identity Center authentication, you must specify a `--user-role`.
For more information about IAM Identity Center authentication mode, see [Set up IAM Identity Center authentication mode for Amazon EMR Studio](emr-studio-authentication.md#emr-studio-enable-sso).
```
aws emr create-studio \
--name \
--auth-mode SSO \
--vpc-id \
--subnet-ids ... \
--service-role \
--user-role \
--workspace-security-group-id \
--engine-security-group-id \
--default-s3-location
--trusted-identity-propagation-enabled \
--idc-user-assignment OPTIONAL \
--idc-instance-arn
```
**Example – CLI output for `aws emr create-studio`**
The following is an example of the output that appears after you create a Studio.
```
{
StudioId: "es-123XXXXXXXXX",
Url: "https://es-123XXXXXXXXX.emrstudio-prod.us-east-1.amazonaws.com"
}
```
For more information about the `create-studio` command, see [https://docs.aws.amazon.com/cli/latest/reference/emr/create-studio.html](https://docs.aws.amazon.com/cli/latest/reference/emr/create-studio.html).
------
## Identity provider RelayState parameters and authentication URLs
When you use IAM federation, and you want users to log in using your Studio URL and credentials for your identity provider (IdP), you can specify your **Identity provider (IdP) login URL** and **RelayState** parameter name when you [Create an EMR Studio](#emr-studio-create-studio).
The following table shows the standard authentication URL and RelayState parameter name for some popular identity providers.
| Identity provider | Parameter | Authentication URL |
| --- | --- | --- |
| Auth0 | RelayState | https://.auth0.com/samlp/ |
| Google accounts | RelayState | https://accounts.google.com/o/saml2/initsso?idpid=&spid=&forceauthn=false |
| Microsoft Azure | RelayState | https://myapps.microsoft.com/signin//?tenantId= |
| Okta | RelayState | https://.okta.com/app///sso/saml |
| PingFederate | TargetResource | https:///idp//startSSO.ping?PartnerSpId= |
| PingOne | TargetResource | https://sso.connect.pingidentity.com/sso/sp/initsso?saasid=&idpid= |
# Assign and manage EMR Studio users
After you create an EMR Studio, you can assign users and groups to it. The method you use to assign, update, and remove users depends on the Studio authentication mode.
+ When you use IAM authentication mode, you configure EMR Studio user assignment and permissions in IAM or with IAM and your identity provider.
+ With IAM Identity Center authentication mode, you use the Amazon EMR management console or the AWS CLI to manage users.
To learn more about authentication for Amazon EMR Studio, see [Choose an authentication mode for Amazon EMR Studio](emr-studio-authentication.md).
## Assign a user or group to an EMR Studio
------
#### [ IAM ]
When you use [Set up IAM authentication mode for Amazon EMR Studio](emr-studio-authentication.md#emr-studio-iam-authentication), you must allow the `CreateStudioPresignedUrl` action in a user's IAM permissions policy and restrict the user to a particular Studio. You can include `CreateStudioPresignedUrl` in your [User permissions for IAM authentication mode](how-emr-studio-works.md#emr-studio-iam-authorization) or use a separate policy.
To restrict a user to a Studio (or set of Studios), you can use attribute-based access control (ABAC) or specify the Amazon Resource Name (ARN) of a Studio in the `Resource` element of the permissions policy.
**Example Assign a user to a Studio using a Studio ARN**
The following example policy gives a user access to a particular EMR Studio by allowing the `CreateStudioPresignedUrl` action and specifying the Studio's Amazon Resource Name (ARN) in the `Resource` element.
****
```
{
"Version":"2012-10-17",
"Statement": [
{
"Sid": "AllowCreateStudioPresignedUrl",
"Effect": "Allow",
"Action": [
"elasticmapreduce:CreateStudioPresignedUrl"
],
"Resource": [
"arn:aws:elasticmapreduce:us-east-1:123456789012:studio/studio-id"
]
}
]
}
```
**Example Assign a user to a Studio with ABAC for IAM authentication**
There are multiple ways to configure attribute-based access control (ABAC) for a Studio. For example, you might attach one or more tags to an EMR Studio, and then create an IAM policy that restricts the `CreateStudioPresignedUrl` action to a particular Studio or set of Studios with those tags.
You can add tags during or after Studio creation. To add tags to an existing Studio, you can use the [AWS CLI`emr add-tags`](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/emr/add-tags.html) command. The following example adds a tag with the key-value pair `Team = Data Analytics` to an EMR Studio.
```
aws emr add-tags --resource-id --tags Team="Data Analytics"
```
The following example permissions policy allows the `CreateStudioPresignedUrl` action for EMR Studios with the tag key-value pair `Team = DataAnalytics`. For more information about using tags to control access, see [Controlling access to and for a users and roles using tags](https://docs.aws.amazon.com/IAM/latest/UserGuide/access_iam-tags.html) or [Controlling access to AWS resources using tags](https://docs.aws.amazon.com/IAM/latest/UserGuide/access_tags.html).
****
```
{
"Version":"2012-10-17",
"Statement": [
{
"Sid": "AllowCreateStudioPresignedUrl",
"Effect": "Allow",
"Action": [
"elasticmapreduce:CreateStudioPresignedUrl"
],
"Resource": [
"arn:aws:elasticmapreduce:*:123456789012:studio/*"
],
"Condition": {
"StringEquals": {
"elasticmapreduce:ResourceTag/Team": "Data Analytics"
}
}
}
]
}
```
**Example Assign a user to a Studio using the aws:SourceIdentity global condition key**
When you use IAM federation, you can use the global condition key `aws:SourceIdentity` in a permissions policy to give users Studio access when they assume your IAM role for federation.
You must first configure your identity provider (IdP) to return an identifying string, such as an email address or username, when a user authenticates and assumes your IAM role for federation. IAM sets the global condition key `aws:SourceIdentity` to the identifying string returned by your IdP.
For more information, see the [How to relate IAM role activity to corporate identity](https://aws.amazon.com/blogs/security/how-to-relate-iam-role-activity-to-corporate-identity/) blog post in the AWS Security Blog and the [aws:SourceIdentity](https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_condition-keys.html#condition-keys-sourceidentity) entry in the global condition keys reference.
The following example policy allows the `CreateStudioPresignedUrl` action and gives users with an `aws:SourceIdentity` that matches the ** access to the EMR Studio specified by **.
****
```
{
"Version":"2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"elasticmapreduce:CreateStudioPresignedUrl"
],
"Resource": [
"arn:aws:elasticmapreduce:us-east-1:123456789012:studio/studio-name"
],
"Condition": {
"StringLike": {
"aws:SourceIdentity": "example-source-identity"
}
},
"Sid": "AllowELASTICMAPREDUCECreatestudiopresignedurl"
}
]
}
```
------
#### [ IAM Identity Center ]
When you assign a user or group to an EMR Studio, you specify a session policy that defines fine-grained permissions, such as the ability to create a new EMR cluster, for that user or group. Amazon EMR stores these session policy mappings. You can update a user or group's session policy after assignment.
**Note**
The final permissions for a user or group is an intersection of the permissions defined in your EMR Studio user role and the permissions defined in the session policy for that user or group. If a user belongs to more than one group assigned to the Studio, EMR Studio uses a union of permissions for that user.
**To assign users or groups to an EMR Studio using the Amazon EMR console**
1. Navigate to the new Amazon EMR console and select **Switch to the old console** from the side navigation. For more information on what to expect when you switch to the old console, see [Using the old console](https://docs.aws.amazon.com/emr/latest/ManagementGuide/whats-new-in-console.html#console-opt-in).
1. Choose **EMR Studio** from the left navigation.
1. Choose your Studio name from the **Studios** list, or select the Studio and choose **View details**, to open the Studio detail page.
1. Choose **Add Users** to see the **Users** and **Groups** search table.
1. Select the **Users** tab or the **Groups** tab, and enter a search term in the search bar to find a user or group.
1. Select one or more users or groups from the search results list. You can switch back and forth between the **Users** tab and the **Groups** tab.
1. After you select users and groups to add to the Studio, choose **Add**. You should see the users and groups appear in the **Studio users** list. It might take a few seconds for the list to refresh.
1. Follow the instructions in [Update permissions for a user or group assigned to a Studio](#emr-studio-update-user) to refine the Studio permissions for a user or group.
**To assign a user or group to an EMR Studio using the AWS CLI**
Insert your own values for the following `create-studio-session-mapping` arguments. For more information about the `create-studio-session-mapping` command, see the [https://docs.aws.amazon.com/cli/latest/reference/emr/create-studio-session-mapping.html](https://docs.aws.amazon.com/cli/latest/reference/emr/create-studio-session-mapping.html).
+ **`--studio-id`** – The ID of the Studio you want to assign the user or group to. For instructions on how to retrieve a Studio ID, see [View Studio details](emr-studio-manage-studio.md#emr-studio-get-studio-id).
+ `--identity-name` – The name of the user or group from the Identity Store. For more information, see [UserName](https://docs.aws.amazon.com/singlesignon/latest/IdentityStoreAPIReference/API_User.html#singlesignon-Type-User-UserName) for users and [DisplayName](https://docs.aws.amazon.com/singlesignon/latest/IdentityStoreAPIReference/API_Group.html#singlesignon-Type-Group-DisplayName) for groups in the *Identity Store API Reference*.
+ **`--identity-type`** – Use either `USER` or `GROUP` to specify the identity type.
+ **`--session-policy-arn`** – The Amazon Resource Name (ARN) for the session policy you want to associate with the user or group. For example, `arn:aws:iam:::policy/EMRStudio_Advanced_User_Policy`. For more information, see [Create permissions policies for EMR Studio users](emr-studio-user-permissions.md#emr-studio-permissions-policies).
```
aws emr create-studio-session-mapping \
--studio-id \
--identity-name \
--identity-type \
--session-policy-arn
```
**Note**
Linux line continuation characters (\$1) are included for readability. They can be removed or used in Linux commands. For Windows, remove them or replace with a caret (^).
Use the `get-studio-session-mapping` command to verify the new assignment. Replace ** with the IAM Identity Center name of the user or group that you updated.
```
aws emr get-studio-session-mapping \
--studio-id \
--identity-type \
--identity-name \
```
------
## Update permissions for a user or group assigned to a Studio
------
#### [ IAM ]
To update user or group permissions when you use IAM authentication mode, use IAM to change the IAM permissions policies attached to your IAM identities (users, groups, or roles).
For more information, see [User permissions for IAM authentication mode](how-emr-studio-works.md#emr-studio-iam-authorization).
------
#### [ IAM Identity Center ]
****To update EMR Studio permissions for a user or group using the console****
1. Navigate to the new Amazon EMR console and select **Switch to the old console** from the side navigation. For more information on what to expect when you switch to the old console, see [Using the old console](https://docs.aws.amazon.com/emr/latest/ManagementGuide/whats-new-in-console.html#console-opt-in).
1. Choose **EMR Studio** from the left navigation.
1. Choose your Studio name from the **Studios** list, or select the Studio and choose **View details**, to open the Studio detail page.
1. In the** Studio users** list on the Studio detail page, search for the user or group you want to update. You can search by name or identity type.
1. Select the user or group that you want to update and choose **Assign policy** to open the **Session policy** dialog box.
1. Select a policy to apply to the user or group that you chose in step 5, and choose **Apply policy**. The **Studio users** list should display the policy name in the **Session policy** column for the user or group that you updated.
**To update EMR Studio permissions for a user or group using the AWS CLI**
Insert your own values for the following `update-studio-session-mappings` arguments. For more information about the `update-studio-session-mappings` command, see the [https://docs.aws.amazon.com/cli/latest/reference/emr/update-studio-session-mapping.html](https://docs.aws.amazon.com/cli/latest/reference/emr/update-studio-session-mapping.html).
```
aws emr update-studio-session-mapping \
--studio-id \
--identity-name \
--session-policy-arn \
--identity-type \
```
Use the `get-studio-session-mapping` command to verify the new session policy assignment. Replace ** with the IAM Identity Center name of the user or group that you updated.
```
aws emr get-studio-session-mapping \
--studio-id \
--identity-type \
--identity-name \
```
------
## Remove a user or group from a Studio
------
#### [ IAM ]
To remove a user or group from an EMR Studio when you use IAM authentication mode, you must revoke the user's access to the Studio by reconfiguring the user's IAM permissions policy.
In the following example policy, assume that you have an EMR Studio with the tag key-value pair `Team = Quality Assurance`. According to the policy, the user can access Studios tagged with the `Team` key whose value is equal to either `Data Analytics` or `Quality Assurance`. To remove the user from the Studio tagged with `Team = Quality Assurance`, remove `Quality Assurance` from the list of tag values.
------
#### [ JSON ]
****
```
{
"Version":"2012-10-17",
"Statement": [
{
"Sid": "AllowCreateStudioPresignedUrl",
"Effect": "Allow",
"Action": [
"elasticmapreduce:CreateStudioPresignedUrl"
],
"Resource": [
"arn:aws:elasticmapreduce:us-east-1:123456789012:studio/*"
],
"Condition": {
"StringEquals": {
"elasticmapreduce:ResourceTag/Team": [
"Data Analytics",
"Quality Assurance"
]
}
}
}
]
}
```
------
------
#### [ IAM Identity Center ]
****To remove a user or group from an EMR Studio using the console****
1. Navigate to the new Amazon EMR console and select **Switch to the old console** from the side navigation. For more information on what to expect when you switch to the old console, see [Using the old console](https://docs.aws.amazon.com/emr/latest/ManagementGuide/whats-new-in-console.html#console-opt-in).
1. Choose **EMR Studio** from the left navigation.
1. Choose your Studio name from the **Studios** list, or select the Studio and choose **View details**, to open the Studio detail page.
1. In the** Studio users** list on the Studio detail page, find the user or group you want to remove from the Studio. You can search by name or identity type.
1. Select the user or group that you want to delete, choose **Delete** and confirm. The user or group that you deleted disappears from the **Studio users** list.
**To remove a user or group from an EMR Studio using the AWS CLI**
Insert your own values for the following `delete-studio-session-mapping` arguments. For more information about the `delete-studio-session-mapping` command, see the [https://docs.aws.amazon.com/cli/latest/reference/emr/delete-studio-session-mapping.html](https://docs.aws.amazon.com/cli/latest/reference/emr/delete-studio-session-mapping.html).
```
aws emr delete-studio-session-mapping \
--studio-id \
--identity-type \
--identity-name \
```
------
# Monitor, update and delete Amazon EMR Studio resources
This section includes instructions to help you monitor, update, or delete an EMR Studio resource. For information about assigning users or updating user permissions, see [Assign and manage EMR Studio users](emr-studio-manage-users.md).
## View Studio details
------
#### [ Console ]
****To view details about an EMR Studio with the new console****
1. Open the Amazon EMR console at [https://console.aws.amazon.com/emr](https://console.aws.amazon.com/emr).
1. Under **EMR Studio** on the left navigation, choose **Studios**.
1. Select the Studio from the **Studios** list to open the Studio detail page. The Studio detail page includes **Studio setting** information, such as the Studio **Description**, **VPC**, and **Subnets**.
------
#### [ CLI ]
**To retrieve details for an EMR Studio by Studio ID using the AWS CLI**
Use the following `describe-studio` AWS CLI command to fetch detailed information about a particular EMR Studio. For more information, see the [https://docs.aws.amazon.com/cli/latest/reference/emr/describe-studio.html](https://docs.aws.amazon.com/cli/latest/reference/emr/describe-studio.html).
```
aws emr describe-studio \
--studio-id \
```
**To retrieve a list of EMR Studios using the AWS CLI**
Use the following `list-studios` AWS CLI command. For more information, see the [https://docs.aws.amazon.com/cli/latest/reference/emr/list-studios.html](https://docs.aws.amazon.com/cli/latest/reference/emr/list-studios.html).
```
aws emr list-studios
```
The following is an example return value for the `list-studios` command in JSON format.
```
{
"Studios": [
{
"AuthMode": "IAM",
"VpcId": "vpc-b21XXXXX",
"Name": "example-studio-name",
"Url": "https://es-7HWP74SNGDXXXXXXXXXXXXXXX.emrstudio-prod.us-east-1.amazonaws.com",
"CreationTime": 1605672582.781,
"StudioId": "es-7HWP74SNGDXXXXXXXXXXXXXXX",
"Description": "example studio description"
}
]
}
```
------
## Monitor Amazon EMR Studio actions
### View EMR Studio and API activity
EMR Studio is integrated with AWS CloudTrail, a service that provides a record of actions taken by a user, by an IAM role, or by another AWS service in EMR Studio. CloudTrail captures API calls for EMR Studio as events. You can view events using the CloudTrail console at [https://console.aws.amazon.com/cloudtrail/](https://console.aws.amazon.com/cloudtrail/).
EMR Studio events provide information such as which Studio or IAM user makes a request, and what kind of request it is.
**Note**
On-cluster actions such as running notebook jobs do not emit AWS CloudTrail.
You can also create a trail for continuous delivery of EMR Studio CloudTrail events to an Amazon S3 bucket. For more information, see the *[AWS CloudTrail User Guide](https://docs.aws.amazon.com/awscloudtrail/latest/userguide/cloudtrail-user-guide.html)*.
**Example CloudTrail Event: a user Calls the DescribeStudio API**
The following is an example AWS CloudTrail event that is created when a user, `admin`, calls the [DescribeStudio](https://docs.aws.amazon.com/emr/latest/APIReference/API_DescribeStudio.html) API. CloudTrail records the user name as `admin`.
**Note**
To protect Studio details, the EMR Studio API event for DescribeStudio excludes a value for `responseElements`.
```
{
"eventVersion":"1.08",
"userIdentity":{
"type":"IAMUser",
"principalId":"AIDXXXXXXXXXXXXXXXXXX",
"arn":"arn:aws:iam::653XXXXXXXXX:user/admin",
"accountId":"653XXXXXXXXX",
"accessKeyId":"AKIAIOSFODNN7EXAMPLE",
"userName":"admin"
},
"eventTime":"2021-01-07T19:13:58Z",
"eventSource":"elasticmapreduce.amazonaws.com",
"eventName":"DescribeStudio",
"awsRegion":"us-east-1",
"sourceIPAddress":"72.XX.XXX.XX",
"userAgent":"aws-cli/1.18.188 Python/3.8.5 Darwin/18.7.0 botocore/1.19.28",
"requestParameters":{
"studioId":"es-9O5XXXXXXXXXXXXXXXXXXXXXX"
},
"responseElements":null,
"requestID":"0fxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
"eventID":"b0xxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
"readOnly":true,
"eventType":"AwsApiCall",
"managementEvent":true,
"eventCategory":"Management",
"recipientAccountId":"653XXXXXXXXX"
}
```
### View Spark user and job activity
To view Spark job activity by Amazon EMR Studio users, you can configure user impersonation on a cluster. With user impersonation, each Spark job that is submitted from a Workspace is associated with the Studio user who ran the code.
When user impersonation is enabled, Amazon EMR creates an HDFS user directory on the cluster's primary node for each user that runs code in the Workspace. For example, if user `studio-user-1@example.com` runs code, you can connect to the primary node and see that `hadoop fs -ls /user` has a directory for `studio-user-1@example.com`.
To set up Spark user impersonation, set the following properties in the following configuration classifications:
+ `core-site`
+ `livy-conf`
```
[
{
"Classification": "core-site",
"Properties": {
"hadoop.proxyuser.livy.groups": "*",
"hadoop.proxyuser.livy.hosts": "*"
}
},
{
"Classification": "livy-conf",
"Properties": {
"livy.impersonation.enabled": "true"
}
}
]
```
To view history server pages, see [Debug applications and jobs with EMR Studio](emr-studio-debug.md). You can also connect to the primary node of the cluster using SSH to view application web interfaces. For more information, see [View web interfaces hosted on Amazon EMR clusters](emr-web-interfaces.md).
## Update an Amazon EMR Studio
After you create an EMR Studio, you can update the following attributes using the AWS CLI:
+ Name
+ Description
+ Default S3 location
+ Subnets
**To update an EMR Studio using the AWS CLI**
Use the `update-studio` AWS CLI command to update an EMR Studio. For more information, see the [https://docs.aws.amazon.com/cli/latest/reference/emr/update-studio.html](https://docs.aws.amazon.com/cli/latest/reference/emr/update-studio.html).
**Note**
You can associated a Studio with a maximum of 5 subnets. These subnets must belong to the same VPC as the Studio. The list of subnet IDs that you submit to the `update-studio` command can include new subnet IDs, but must also include all of the subnet IDs that you already associated with the Studio. You can't remove subnets from a Studio.
```
aws emr update-studio \
--studio-id \
--name \
--subnet-ids \
```
To verify the changes, use the `describe-studio` AWS CLI command and specify your Studio ID. For more information, see the [https://docs.aws.amazon.com/cli/latest/reference/emr/describe-studio.html](https://docs.aws.amazon.com/cli/latest/reference/emr/describe-studio.html).
```
aws emr describe-studio \
--studio-id \
```
## Delete an Amazon EMR Studio and Workspaces
When you delete a Studio, EMR Studio deletes all of the IAM Identity Center user and group assignments that are associated with the Studio.
**Note**
When you delete a Studio, Amazon EMR does *not* delete the Workspaces associated with that Studio. You must delete the Workspaces in your Studio separately.
**Delete Workspaces**
------
#### [ Console ]
Since each EMR Studio Workspace is an EMR notebook instance, you can use the Amazon EMR management console to delete Workspaces. You can delete Workspaces using the Amazon EMR console before or after you delete your Studio
**To delete a Workspace using the Amazon EMR console**
1. Navigate to the new Amazon EMR console and select **Switch to the old console** from the side navigation. For more information on what to expect when you switch to the old console, see [Using the old console](https://docs.aws.amazon.com/emr/latest/ManagementGuide/whats-new-in-console.html#console-opt-in).
1. Choose **Notebooks**.
1. Select the Workspace(s) that you want to delete.
1. Choose **Delete**, then choose **Delete** again to confirm.
1. Follow the instructions for [Deleting objects](https://docs.aws.amazon.com/AmazonS3/latest/user-guide/delete-objects.html) in the *Amazon Simple Storage Service* *Console User Guide* to remove the notebook files associated with the deleted Workspace from Amazon S3.
------
#### [ EMR Studio UI ]
------
#### [ From the Workspace UI ]
**Delete a Workspace and its associated backup files from EMR Studio**
1. Log in to your EMR Studio with your Studio access URL and choose **Workspaces** from the left navigation.
1. Find your Workspace in the list, then select the check box next to its name. You can select multiple Workspaces to delete at the same time.
1. Choose **Delete** in the upper right of the **Workspaces** list and confirm that you want to delete the selected Workspaces. Choose **Delete** to confirm.
1. If you want to remove the notebook files that were associated with the deleted Workspace from Amazon S3, follow the instructions for [Deleting objects](https://docs.aws.amazon.com/AmazonS3/latest/user-guide/delete-objects.html) in the *Amazon Simple Storage Service* *Console User Guide*. If you did not create the Studio, consult your Studio administrator to determine the Amazon S3 backup location for the deleted Workspace.
------
#### [ From the Workspaces list ]
**Delete a Workspace and its associated backup files from the Workspaces list**
1. Navigate to the **Workspace**s list in the console.
1. Select the Workspace that you want to delete from the list and then choose **Actions**.
1. Choose **Delete**.
1. If you want to remove the notebook files that were associated with the deleted Workspace from Amazon S3, follow the instructions for [Deleting objects](https://docs.aws.amazon.com/AmazonS3/latest/user-guide/delete-objects.html) in the *Amazon Simple Storage Service* *Console User Guide*. If you did not create the Studio, consult your Studio administrator to determine the Amazon S3 backup location for the deleted Workspace.
------
------
**Delete an EMR Studio**
------
#### [ Console ]
****To delete an EMR Studio with the new console****
1. Open the Amazon EMR console at [https://console.aws.amazon.com/emr](https://console.aws.amazon.com/emr).
1. Under **EMR Studio** on the left navigation, choose **Studios**.
1. Select the Studio from the **Studios** list with the toggle to the left of the Studio name . Choose **Delete**.
------
#### [ Old console ]
****To delete an EMR Studio with the old console****
1. Open the Amazon EMR console at [https://console.aws.amazon.com/elasticmapreduce/home](https://console.aws.amazon.com/elasticmapreduce/home).
1. Choose **EMR Studio** from the left navigation.
1. Select the Studio from the **Studios** list and choose **Delete**.
------
#### [ CLI ]
**To delete an EMR Studio with the AWS CLI**
Use the `delete-studio` AWS CLI command to delete an EMR Studio. For more information, see the [https://docs.aws.amazon.com/cli/latest/reference/emr/delete-studio.html](https://docs.aws.amazon.com/cli/latest/reference/emr/delete-studio.html).
```
aws emr delete-studio --studio-id
```
------
# Encrypting EMR Studio workspace notebooks and files
In EMR Studio, you can create and configure different workspaces to organize and run notebooks. These workspaces store notebooks and related files in your specified Amazon S3 bucket. By default, these files are encrypted with Amazon S3-managed keys (SSE-S3) with server-side encryption as the base level of encryption. You can also choose to use customer managed KMS keys (SSE-KMS) to encrypt your files. You can do so by using the Amazon EMR management console or through the AWS CLI and AWS SDK when creating an EMR Studio.
EMR Studio workspace storage encryption is available in all the [Regions](https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-studio-considerations.html#emr-studio-considerations-general) where EMR Studio is available.
## Prerequisites
Before you can encrypt EMR Studio workspace notebook and files, you must use AWS Key Management Service to [ create a symmetric customer manager key (CMK)](https://docs.aws.amazon.com/kms/latest/developerguide/create-keys.html#create-symmetric-cmk) in the same AWS account and Region as your EMR Studio.
The resource policy of your AWS KMS must have the necessary access permissions for your EMR Studio's service role. The following is a sample IAM policy granting minimum access permissions for EMR Studio Workspace storage encryption:
```
{
"Sid": "AllowEMRStudioServiceRoleAccess",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam:::role/"
},
"Action": [
"kms:Decrypt",
"kms:GenerateDataKey",
"kms:ReEncryptFrom",
"kms:ReEncryptTo",
"kms:DescribeKey"
],
"Resource": "*",
"Condition": {
"StringEquals": {
"kms:CallerAccount": "",
"kms:EncryptionContext:aws:s3:arn": "arn:aws:s3:::",
"kms:ViaService": "s3..amazonaws.com"
}
}
}
```
Your EMR Studio service role must also have the access permissions to use your AWS KMS key. The following is a sample IAM policy granting the minimum access permissions for EMR Studio Workspace storage encryption:
------
#### [ JSON ]
****
```
{
"Version":"2012-10-17",
"Statement": [
{
"Sid": "AllowEMRStudioWorkspaceStorageEncryptionAccess",
"Effect": "Allow",
"Action": [
"kms:Decrypt",
"kms:GenerateDataKey",
"kms:ReEncryptFrom",
"kms:ReEncryptTo",
"kms:DescribeKey"
],
"Resource": [
"arn:aws:kms:*:123456789012:key/12345678-1234-1234-1234-123456789012"
]
}
]
}
```
------
## Create a new EMR Studio
Follow these steps to create a new EMR Studio that uses workspace storage encryption.
1. Open the Amazon EMR console at [https://console.aws.amazon.com/elasticmapreduce/](https://console.aws.amazon.com/elasticmapreduce/).
1. Choose **Studios**, then choose **Create Studio**.
1. For **S3 location for storage**, enter or choose an Amazon S3 path. This is the Amazon S3 location where Amazon EMR stores workspace notebooks and files.
1. For **Service role**, enter or choose an IAM role. This is the IAM role that Amazon EMR assumes.
1. Choose **Encrypt Workspace files with your own AWS KMS key**.
1. Enter or choose an AWS KMS key to use to encrypt workspace notebooks and files in Amazon S3.
1. Choose **Create Studio** or **Create Studio and Launch Workspaces**.
1. Choose **Encrypt Workspace files with your own AWS KMS key**.
1. Enter or choose an AWS KMS to use to encrypt workspace notebooks and files in Amazon S3.
1. Choose **Save Changes**.
The following steps demonstrate how to update an EMR Studio and set up workspace storage encryption.
1. Open the Amazon EMR console at [https://console.aws.amazon.com/elasticmapreduce/](https://console.aws.amazon.com/elasticmapreduce/).
1. Choose **an existing EMR Studio from the list**, then choose **Edit**.
1. Choose **Encrypt Workspace files with your own AWS KMS key**.
1. Enter or choose an AWS KMS to use to encrypt workspace notebooks and files in Amazon S3.
1. Choose **Save Changes**.
# Define security groups to control EMR Studio network traffic
## About the EMR Studio security groups
Amazon EMR Studio uses two security groups to control network traffic between Workspaces in the Studio and an attached Amazon EMR cluster running on Amazon EC2:
+ An **engine security group** that uses port 18888 to communicate with an attached Amazon EMR cluster running on Amazon EC2.
+ A **Workspace security group** associated with the Workspaces in a Studio. This security group includes an outbound HTTPS rule to allow the Workspace to route traffic to the internet and must allow outbound traffic to the internet on port 443 to enable linking Git repositories to a Workspace.
EMR Studio uses these security groups in addition to any security groups associated with an EMR cluster attached to a Workspace.
You must create these security groups when you use the AWS CLI to create a Studio.
**Note**
You can customize the security groups for EMR Studio with rules tailored to your environment, but you must include the rules noted on this page. Your Workspace security group can't allow any inbound traffic, and the engine security group must allow inbound traffic from the Workspace security group.
**Use the Default EMR Studio Security Groups**
When you use the Amazon EMR console, you can choose the following default security groups. The default security groups are created by EMR Studio on your behalf, and include the minimum required inbound and outbound rules for Workspaces in an EMR Studio.
+ `DefaultEngineSecurityGroup`
+ `DefaultWorkspaceSecurityGroupGit` or `DefaultWorkspaceSecurityGroupWithoutGit`
## Prerequisites
To create the security groups for EMR Studio, you need an Amazon Virtual Private Cloud (VPC) for the Studio. You choose this VPC when you create the security groups. This should be the same VPC that you specify when you create the Studio. If you plan to use Amazon Amazon EMR on EKS with EMR Studio, choose the VPC for your Amazon EKS cluster worker nodes.
## Instructions
Follow the instructions in [Creating a security group](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/working-with-security-groups.html#creating-security-group) in the *Amazon EC2 User Guide for Linux Instances* to create an engine security group and a Workspace security group in your VPC. The security groups must include the rules summarized in the following tables.
When you create security groups for EMR Studio, note the IDs for both. You specify each security group by ID when you create a Studio.
**Engine security group**
EMR Studio uses port 18888 to communicate with an attached cluster.
**Inbound rules**
[\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-studio-security-groups.html)
**Workspace security group**
This security group is associated with the Workspaces in an EMR Studio.
**Outbound rules**
[\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-studio-security-groups.html)
# Create AWS CloudFormation templates for Amazon EMR Studio
## About EMR Studio cluster templates
You can create AWS CloudFormation templates to help EMR Studio users launch new Amazon EMR clusters in a Workspace. CloudFormation templates are formatted text files in JSON or YAML. In a template, you describe a stack of AWS resources and tell CloudFormation how to provision those resources for you. For EMR Studio, you can create one or more templates that describe an Amazon EMR cluster.
You organize your templates in AWS Service Catalog. AWS Service Catalog lets you create and manage commonly deployed IT services called *products* on AWS. You collect your templates as products in a *portfolio* that you share with your EMR Studio users. After you create cluster templates, Studio users can launch a new cluster for a Workspace with one of your templates. Users must have permission to create new clusters from templates. You can set user permissions in your [EMR Studio permissions policies](emr-studio-user-permissions.md).
To learn more about CloudFormation templates, see [Templates](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/cfn-whatis-concepts.html#w2ab1b5c15b7) in the *AWS CloudFormation User Guide*. For more information about AWS Service Catalog, see [What is AWS Service Catalog](https://docs.aws.amazon.com/servicecatalog/latest/adminguide/introduction.html).
The following video demonstrates how to set up cluster templates in AWS Service Catalog for EMR Studio. You can also learn more in the [Build a self-service environment for each line of business using Amazon EMR and Service Catalog](https://aws.amazon.com/blogs/big-data/build-a-self-service-environment-for-each-line-of-business-using-amazon-emr-and-aws-service-catalog/) blog post.
[](http://www.youtube.com/watch?v=https://www.youtube.com/embed/9w_TXTdFLpo)
### Optional template parameters
You can include additional options in the [https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/parameters-section-structure.html](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/parameters-section-structure.html) section of your template. *Parameters* let Studio users input or select custom values for a cluster. For example, you could add a parameter that lets users select a particular Amazon EMR release. For more information, see [Parameters](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/parameters-section-structure.html) in the *CloudFormation User Guide*.
The following example `Parameters` section defines additional input parameters such as `ClusterName`, `EmrRelease` version, and `ClusterInstanceType`.
```
Parameters:
ClusterName:
Type: "String"
Default: "Cluster_Name_Placeholder"
EmrRelease:
Type: "String"
Default: "emr-6.2.0"
AllowedValues:
- "emr-6.2.0"
- "emr-5.32.0"
ClusterInstanceType:
Type: "String"
Default: "m5.xlarge"
AllowedValues:
- "m5.xlarge"
- "m5.2xlarge"
```
When you add parameters, Studio users see additional form options after selecting a cluster template. The following image shows additional form options for **EmrRelease** version, **ClusterName**, and **InstanceType**.
![\[Screenshot of the additional inputs in the Studio user interface when a user selects a cluster template with parameters.\]](http://docs.aws.amazon.com/emr/latest/ManagementGuide/images/cluster-template-parameters-studio-ui.jpg)
## Prerequisites
Before you create a cluster template, make sure you have IAM permissions to access the Service Catalog administrator console view. You also need the required IAM permissions to perform Service Catalog administrative tasks. For more information, see [Grant permissions to Service Catalog administrators](https://docs.aws.amazon.com/servicecatalog/latest/adminguide/getstarted-iamadmin.html).
## Create EMR cluster templates
**To create EMR cluster templates using Service Catalog**
1. Create one or more CloudFormation templates. Where you store your templates is up to you. Since templates are formatted text files, you can upload them to Amazon S3 or keep them in your local file system. To learn more about CloudFormation templates, see [Templates](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/cfn-whatis-concepts.html#w2ab1b5c15b7) in the *AWS CloudFormation User Guide*.
Use the following rules to name your templates, or check your names against the pattern `[a-zA-Z0-9][a-zA-Z0-9._-]*`.
+ Template names must start with a letter or a number.
+ Template names can only consist of letters, numbers, periods (.), underscores (\$1), and hyphens (-).
Each cluster template that you create must include the following options:
**Input parameters**
+ ClusterName – A name for the cluster to help users identify it after it has been provisioned.
**Output**
+ `ClusterId` – The ID of the newly-provisioned EMR cluster.
Following is an example CloudFormation template in YAML format for a cluster with two nodes. The example template includes the required template options and defines additional input parameters for `EmrRelease` and `ClusterInstanceType`.
```
awsTemplateFormatVersion: 2010-09-09
Parameters:
ClusterName:
Type: "String"
Default: "Example_Two_Node_Cluster"
EmrRelease:
Type: "String"
Default: "emr-6.2.0"
AllowedValues:
- "emr-6.2.0"
- "emr-5.32.0"
ClusterInstanceType:
Type: "String"
Default: "m5.xlarge"
AllowedValues:
- "m5.xlarge"
- "m5.2xlarge"
Resources:
EmrCluster:
Type: AWS::EMR::Cluster
Properties:
Applications:
- Name: Spark
- Name: Livy
- Name: JupyterEnterpriseGateway
- Name: Hive
EbsRootVolumeSize: '10'
Name: !Ref ClusterName
JobFlowRole: EMR_EC2_DefaultRole
ServiceRole: EMR_DefaultRole_V2
ReleaseLabel: !Ref EmrRelease
VisibleToAllUsers: true
LogUri:
Fn::Sub: 's3://aws-logs-${AWS::AccountId}-${AWS::Region}/elasticmapreduce/'
Instances:
TerminationProtected: false
Ec2SubnetId: 'subnet-ab12345c'
MasterInstanceGroup:
InstanceCount: 1
InstanceType: !Ref ClusterInstanceType
CoreInstanceGroup:
InstanceCount: 1
InstanceType: !Ref ClusterInstanceType
Market: ON_DEMAND
Name: Core
Outputs:
ClusterId:
Value:
Ref: EmrCluster
Description: The ID of the EMR cluster
```
1. Create a portfolio for your cluster templates in the same AWS account as your Studio.
1. Open the AWS Service Catalog console at [https://console.aws.amazon.com/servicecatalog/](https://console.aws.amazon.com/servicecatalog/).
1. Choose **Portfolios** in the left navigation menu.
1. Enter the requested information on the **Create portfolio** page.
1. Choose **Create**. AWS Service Catalog creates the portfolio and displays the portfolio details.
1. Use the following steps to add your cluster templates as AWS Service Catalog products.
1. Navigate to the **Products** page under **Administration** in the AWS Service Catalog management console.
1. Choose **Upload new product**.
1. Enter a **Product name** and **Owner**.
1. Specify your template file under **Version details**.
1. Choose **Review** to review your product settings, then choose **Create product**.
1. Complete the following steps to add your products to your portfolio.
1. Navigate to the **Products** page in the AWS Service Catalog management console.
1. Choose your product, choose **Actions**, then choose **Add product to portfolio**.
1. Choose your portfolio, then choose **Add product to portfolio**.
1. Create a launch constraint for your products. A launch constraint is an IAM role that specifies user permissions for launching a product. You can tailor your launch constraints, but must allow permissions to use CloudFormation, Amazon EMR, and AWS Service Catalog. For more information and instructions, see [Service Catalog launch constraints](https://docs.aws.amazon.com/servicecatalog/latest/adminguide/constraints-launch.html).
1. Apply your launch constraint to each product in your portfolio. You must apply the launch constraint to each product individually.
1. Select your portfolio from the **Portfolios** page in the AWS Service Catalog management console.
1. Choose the **Constraints** tab and choose **Create constraint**.
1. Choose your product and choose **Launch** under **Constraint type**. Choose **Continue**.
1. Select your launch constraint role in the **Launch constraint** section, then choose **Create**.
1. Grant access to your portfolio.
1. Select your portfolio from the **Portfolios** page in the AWS Service Catalog management console.
1. Expand the **Groups, roles, and users** tab and choose **Add groups, roles, users**.
1. Search for your EMR Studio IAM role in the **Roles** tab, select your role, and choose **Add access**.
****
[\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-studio-cluster-templates.html)
# Establish access and permissions for Git-based repositories
EMR Studio supports the following Git-based services:
+ [AWS CodeCommit](https://aws.amazon.com/codecommit)
+ [GitHub](https://github.com)
+ [Bitbucket](https://bitbucket.org/)
+ [GitLab](https://about.gitlab.com/)
To let EMR Studio users associate a Git repository with a Workspace, set up the following access and permissions requirements. You can also configure Git-based repositories that you host in a private network by following the instructions in [Configure a privately hosted Git repository for EMR Studio](#emr-studio-private-git-repo).
**Cluster internet access**
Both Amazon EMR clusters running on Amazon EC2 and Amazon EMR on EKS clusters attached to Studio Workspaces must be in a private subnet that uses a network address translation (NAT) gateway, or they must be able to access the internet through a virtual private gateway. For more information, see [Amazon VPC options when you launch a cluster](emr-clusters-in-a-vpc.md).
The security groups that you use with EMR Studio must also include an outbound rule that allows Workspaces to route traffic to the internet from an attached EMR cluster. For more information, see [Define security groups to control EMR Studio network traffic](emr-studio-security-groups.md).
If the network interface is in a public subnet, it won't be able to communicate with the internet through an internet gateway (IGW).
**Permissions for AWS Secrets Manager**
To let EMR Studio users access Git repositories with secrets stored in AWS Secrets Manager, add a permissions policy to the [service role for EMR Studio](emr-studio-service-role.md) that allows the `secretsmanager:GetSecretValue` operation.
For information about how to link Git-based repositories to Workspaces, see [Link Git-based repositories to an EMR Studio Workspace](emr-studio-git-repo.md).
## Configure a privately hosted Git repository for EMR Studio
Use the following instructions to configure privately hosted repositories for Amazon EMR Studio. Provide a configuration file with information about your DNS and Git servers. EMR Studio uses this information to configure Workspaces that can route traffic to your self-managed repositories.
**Note**
If you configure `DnsServerIpV4`, EMR Studio uses your DNS server to resolve both your `GitServerDnsName` and AWS Endpoints, but it's strongly recommended to avoid resolving AWS Endpoints with your `DnsServerIpV4` as this can disrupt essential service functionalities.
**Prerequisites**
Before you configure a privately hosted Git repository for EMR Studio, you need an Amazon S3 storage location where EMR Studio can back up the Workspaces and notebook files in the Studio. Use the same S3 bucket that you specify when you create a Studio.
**To configure one or more privately hosted Git repositories for EMR Studio**
1. Create a configuration file using the following template. Include the following values for each Git server that you want to specify in your configuration:
+ **`DnsServerIpV4`** - The IPv4 address of your DNS server. If you provide values for both `DnsServerIpV4` and `GitServerIpV4List`, the value for `DnsServerIpV4` takes precedence and EMR Studio uses `DnsServerIpV4` to resolve your `GitServerDnsName`.
**Note**
To use privately hosted Git repositories, your DNS server must allow inbound access from EMR Studio. We urge you to secure your DNS server against other, unauthorized access.
+ **`GitServerDnsName`** - The DNS name of your Git server. For example `"git.example.com"`.
+ **`GitServerIpV4List`** - A list of IPv4 addresses that belong to your Git servers.
```
[
{
"Type": "PrivatelyHostedGitConfig",
"Value": [
{
"DnsServerIpV4": "<10.24.34.xxx>",
"GitServerDnsName": "",
"GitServerIpV4List": [
"",
""
]
},
{
"DnsServerIpV4": "<10.24.34.xxx>",
"GitServerDnsName": "",
"GitServerIpV4List": [
"",
""
]
}
]
}
]
```
1. Save your configuration file as `configuration.json`.
1. Upload the configuration file into your default Amazon S3 storage location in a folder called `life-cycle-configuration`. For example, if your default S3 location is `s3://amzn-s3-demo-bucket/workspace`, your configuration file would be in `s3://amzn-s3-demo-bucket/workspace/life-cycle-configuration/configuration.json`.
**Important**
We urge you to restrict access to your `life-cycle-configuration` folder to Studio administrators and to your EMR Studio service role, and that you secure `configuration.json` against unauthorized access. For instructions, see [Controlling access to a bucket with user policies](https://docs.aws.amazon.com/AmazonS3/latest/userguide/walkthrough1.html) or [Security Best Practices for Amazon S3](https://docs.aws.amazon.com/AmazonS3/latest/userguide/security-best-practices.html).
For upload instructions, see [Creating a folder](https://docs.aws.amazon.com/AmazonS3/latest/userguide/using-folders.html#create-folder) and [Uploading objects](https://docs.aws.amazon.com/AmazonS3/latest/userguide/upload-objects.html) in the *Amazon Simple Storage Service User Guide*. To apply your configuration to an existing Workspace, close and restart the Workspace after you upload your configuration file to Amazon S3.
# Optimize Spark jobs in EMR Studio
When running a Spark job using EMR Studio, there are a few steps you can take to help ensure that you're optimizing your Amazon EMR cluster resources.
## Prolong your Livy session
If you use Apache Livy along with Spark on your Amazon EMR cluster, we recommend that you increase your Livy session timeout by doing one of the following:
+ When you create an Amazon EMR cluster, set this configuration classification in the **Enter Configuration** field.
```
[
{
"Classification": "livy-conf",
"Properties": {
"livy.server.session.timeout": "8h"
}
}
]
```
+ For an already-running EMR cluster, connect to your cluster using `ssh` and set the `livy-conf` configuration classification in `/etc/livy/conf/livy.conf`.
```
[
{
"Classification": "livy-conf",
"Properties": {
"livy.server.session.timeout": "8h"
}
}
]
```
You may need to restart Livy after changing the configuration.
+ If you don't want your Livy session to time out at all, set the property `livy.server.session.timeout-check` to `false` in `/etc/livy/conf/livy.conf`.
## Run Spark in cluster mode
In cluster mode, the Spark driver runs on a core node instead of on the primary node, improving resource utilization on the primary node.
To run your Spark application in cluster mode instead of the default client mode, choose **Cluster** mode when you set **Deploy mode** while configuring your Spark step in your new Amazon EMR cluster. For more information, see [Cluster mode overview](https://spark.apache.org/docs/latest/cluster-overview.html) in the Apache Spark documentation.
## Increase Spark driver memory
To increase the Spark driver memory, configure your Spark session using the `%%configure` magic command in your EMR notebook, as in the following example.
```
%%configure -f
{"driverMemory": "6000M"}
```