# Managing Lake Formation permissions Managing Lake Formation permissions Lake Formation provides central access controls for data in your data lake. You can define security policy-based rules for your users and applications by role in Lake Formation, and integration with AWS Identity and Access Management authenticates those users and roles. Once the rules are defined, Lake Formation enforces your access controls at table, column, and ro-level granularity for users of Amazon Redshift Spectrum and Amazon Athena. **Topics** + [ # Granting data location permissions ](granting-location-permissions.md) + [ # Granting permissions on Data Catalog resources ](granting-catalog-permissions.md) + [ # Permissions example scenario ](security-permissions-example-scenario.md) + [ # Data filtering and cell-level security in Lake Formation ](data-filtering.md) + [ # Viewing database and table permissions in Lake Formation ](viewing-permissions.md) + [ # Revoking permission using the Lake Formation console ](revoking-permssions-console-all.md) + [ # Cross-account data sharing in Lake Formation ](cross-account-permissions.md) + [ # Accessing and viewing shared Data Catalog tables and databases ](viewing-shared-resources.md) + [ # Creating resource links ](creating-resource-links.md) + [ # Accessing tables across Regions ](data-access-across-region.md) # Granting data location permissions Data location permissions in AWS Lake Formation enable principals to create and alter Data Catalog resources that point to designated registered Amazon S3 locations. Data location permissions work in addition to Lake Formation data permissions to secure information in your data lake. Lake Formation does not use the AWS Resource Access Manager (AWS RAM) service for data location permission grants, so you don't need to accept resource share invitations for data location permissions. You can grant data location permissions by using the Lake Formation console, API, or AWS Command Line Interface (AWS CLI). **Note** For a grant to succeed, you must first register the data location with Lake Formation. **See Also:** [Underlying data access control](access-control-underlying-data.md#data-location-permissions) **Topics** + [ # Granting data location permissions (same account) ](granting-location-permissions-local.md) + [ # Granting data location permissions (external account) ](granting-location-permissions-external.md) + [ # Granting permissions on a data location shared with your account ](regranting-locations.md) # Granting data location permissions (same account) Follow these steps to grant data location permissions to principals in your AWS account. You can grant permissions by using the Lake Formation console, the API, or the AWS Command Line Interface (AWS CLI). ------ #### [ AWS Management Console ] **To grant data location permissions (same account)** 1. Open the AWS Lake Formation console at [https://console.aws.amazon.com/lakeformation/](https://console.aws.amazon.com/lakeformation/). Sign in as a data lake administrator or as a principal who has grant permissions on the desired data location. 1. In the navigation pane, under **Permissions**, choose **Data locations**. 1. Choose **Grant**. 1. In the **Grant permissions** dialog box, ensure that the **My account** tile is selected. Then provide the following information: + For **IAM users and roles**, choose one or more principals. + For **SAML and Amazon Quick users and groups**, enter one or more Amazon Resource Names (ARNs) for users or groups federated through SAML or ARNs for Amazon Quick users or groups. Enter one ARN at a time, and press **Enter** after each ARN. For information about how to construct the ARNs, see [Lake Formation grant and revoke AWS CLI commands](lf-permissions-reference.md#perm-command-format). + For **Storage locations**, choose **Browse**, and choose an Amazon Simple Storage Service (Amazon S3) storage location. The location must be registered with Lake Formation. Choose **Browse** again to add another location. You can also type the location, but ensure that you precede the location with `s3://`. + For **Registered account location**, enter the AWS account ID where the location is registered. This defaults to your account ID. In a cross-account scenario, data lake administrators in a recipient account can specify the owner account here when granting the data location permission to other principals in the recipient account. + (Optional) To enable the selected principals to grant data location permissions on the selected location, select **Grantable**. ![\[In the Grant permissions dialog box, the user datalake_user and storage location s3://retail/transactions/q119 are selected.\]](http://docs.aws.amazon.com/lake-formation/latest/dg/images/grant-location-dialog-local.png) 1. Choose **Grant**. ------ #### [ AWS CLI ] **To grant data location permissions (same account)** + Run a `grant-permissions` command, and grant `DATA_LOCATION_ACCESS` to the principal, specifying the Amazon S3 path as the resource. **Example** The following example grants data location permissions on `s3://retail` to user `datalake_user1`. ``` aws lakeformation grant-permissions --principal DataLakePrincipalIdentifier=arn:aws:iam:::user/datalake_user1 --permissions "DATA_LOCATION_ACCESS" --resource '{ "DataLocation": {"ResourceArn":"arn:aws:s3:::retail"}}' ``` **Example** The following example grants data location permissions on `s3://retail` to `ALLIAMPrincipals` group. ``` aws lakeformation grant-permissions --principal DataLakePrincipalIdentifier=111122223333:IAMPrincipals --permissions "DATA_LOCATION_ACCESS" --resource '{ "DataLocation": {"ResourceArn":"arn:aws:s3:::retail", "CatalogId": "111122223333"}}' ``` ------ **See Also:** [Lake Formation permissions reference](lf-permissions-reference.md) # Granting data location permissions (external account) Follow these steps to grant data location permissions to an external AWS account or organization. You can grant permissions by using the Lake Formation console, the API, or the AWS Command Line Interface (AWS CLI). **Before you begin** Ensure that all cross-account access prerequisites are satisfied. For more information, see [Prerequisites](cross-account-prereqs.md). ------ #### [ AWS Management Console ] **To grant data location permissions (external account, console)** 1. Open the AWS Lake Formation console at [https://console.aws.amazon.com/lakeformation/](https://console.aws.amazon.com/lakeformation/). Sign in as a data lake administrator. 1. In the navigation pane, under **Permissions**, choose **Data locations**, and then choose **Grant**. 1. In the **Grant permissions** dialog box, choose the **External account** tile. 1. Provide the following information: + For **AWS account ID or AWS organization ID**, enter valid AWS account numbers, organization IDs, or organizational unit IDs. Press **Enter** after each ID. An organization ID consists of "o-" followed by 10 to 32 lower-case letters or digits. An organizational unit ID consists of "ou-" followed by 4 to 32 lowercase letters or digits (the ID of the root that contains the OU). This string is followed by a second "-" (hyphen) and 8 to 32 additional lowercase letters or digits. + Under **Storage locations**, choose **Browse**, and choose an Amazon Simple Storage Service (Amazon S3) storage location. The location must be registered with Lake Formation. ![\[The Grant permission dialog has the External account radio button selected, an AWS account specified, and a storage location specified.\]](http://docs.aws.amazon.com/lake-formation/latest/dg/images/grant-location-dialog-external.png) 1. Select **Grantable**. 1. Choose **Grant**. ------ #### [ AWS CLI ] **To grant data location permissions (external account, AWS CLI)** + To grant permissions to an external AWS account, enter a command similar to the following. ``` aws lakeformation grant-permissions --principal DataLakePrincipalIdentifier=111122223333 --permissions "DATA_LOCATION_ACCESS" --permissions-with-grant-option "DATA_LOCATION_ACCESS" --resource '{ "DataLocation": {"CatalogId":"123456789012","ResourceArn":"arn:aws:s3::retail/transactions/2020q1"}}' ``` This command grants `DATA_LOCATION_ACCESS` with the grant option to account 1111-2222-3333 on the Amazon S3 location `s3://retail/transactions/2020q1`, which is owned by account 1234-5678-9012. To grant permissions to an organization, enter a command similar to the following. ``` aws lakeformation grant-permissions --principal DataLakePrincipalIdentifier=arn:aws:organizations::111122223333:organization/o-abcdefghijkl --permissions "DATA_LOCATION_ACCESS" --permissions-with-grant-option "DATA_LOCATION_ACCESS" --resource '{"DataLocation": {"CatalogId":"123456789012","ResourceArn":"arn:aws:s3::retail/transactions/2020q1"}}' ``` This command grants `DATA_LOCATION_ACCESS` with grant option to the organization `o-abcdefghijkl` on the Amazon S3 location `s3://retail/transactions/2020q1`, which is owned by account 1234-5678-9012. To grant permissions to a principal in an external AWS account, enter a command similar to the following. ``` aws lakeformation grant-permissions --principal DataLakePrincipalIdentifier=arn:aws:iam::111122223333:user/datalake_user1 --permissions "DATA_LOCATION_ACCESS" --resource '{ "DataLocation": {"ResourceArn":"arn:aws:s3::retail/transactions/2020q1", "CatalogId": "123456789012"}}' ``` This command grants `DATA_LOCATION_ACCESS` to a principal in account 1111-2222-3333 on the Amazon S3 location `s3://retail/transactions/2020q1`, which is owned by account 1234-5678-9012. **Example** The following example grants data location permissions on `s3://retail` to `ALLIAMPrincipals` group in an external account. ``` aws lakeformation grant-permissions --principal DataLakePrincipalIdentifier=111122223333:IAMPrincipals --permissions "DATA_LOCATION_ACCESS" --resource '{ "DataLocation": {"ResourceArn":"arn:aws:s3:::retail", "CatalogId": "123456789012"}}' ``` ------ **See Also:** [Lake Formation permissions reference](lf-permissions-reference.md) # Granting permissions on a data location shared with your account After a Data Catalog resource is shared with your AWS account, as a data lake administrator, you can grant permissions on the resource to other principals in your account. If the `ALTER` permission is granted on a shared table, and the table points to a registered Amazon S3 location, you must also grant data location permissions on the location. Likewise, if the `CREATE_TABLE` or `ALTER` permission is granted on a shared database and the database has a location property that points to a registered location, you must also grant data location permissions on the location. To grant data location permissions on a shared location to a principal in your account, your account must have been granted the `DATA_LOCATION_ACCESS` permission on the location with the grant option. When you then grant `DATA_LOCATION_ACCESS` to another principal in your account, you must include the Data Catalog ID (AWS account ID) of the owner account. The owner account is the account that registered the location. You can use the AWS Lake Formation console, the API, or the AWS Command Line Interface (AWS CLI to grant data location permissions. **To grant permissions on a data location shared with your account (console)** + Follow the steps in [Granting data location permissions (same account)](granting-location-permissions-local.md). For **Storage locations**, you must type the locations. For **Registered account location**, enter the AWS account ID of the owner account. **To grant permissions on a data location shared with your account (AWS CLI)** + Enter one of the following commands to grant permissions to either a user or a role. ``` aws lakeformation grant-permissions --principal DataLakePrincipalIdentifier=arn:aws:iam:::user/ --permissions "DATA_LOCATION_ACCESS" --resource '{ "DataLocation": {"CatalogId":"","ResourceArn":"arn:aws:s3:::"}}' aws lakeformation grant-permissions --principal DataLakePrincipalIdentifier=arn:aws:iam:::role/ --permissions "DATA_LOCATION_ACCESS" --resource '{ "DataLocation": {"CatalogId":"","ResourceArn":"arn:aws:s3:::"}}' ``` # Granting permissions on Data Catalog resources Granting data permissions You can grant **Data permissions** to principals in AWS Lake Formation so that the principals can create and manage Data Catalog resources, and can access underlying data. You can grant **Data lake permissions** on catalogs, databases, tables, and views. When you grant permissions on tables, you can limit access to specific table columns or rows for even more fine-grained access control. You can grant permissions on individual catalogs, databases, tables and views, or with a single grant operation, you can grant permissions on all databases, tables and views in a catalog or a database. If you grant permissions on all tables in a database to IAM principals, you are implicitly granting the `DESCRIBE` permission on the database. The database then appears on the **Databases** page on the console, and is returned by the `GetDatabases` API operation. The same principle applies at the catalog level - when you receive permissions for databases within a catalog, you also get `DESCRIBE` permissions for that catalog. **Important** The implicit `DESCRIBE` permission applies only when granting permissions to IAM principals within the same AWS account. For cross-account resources, you must explicitly grant `DESCRIBE` permissions. The automatic `DESCRIBE` permission grant doesn't apply when using attribute-based access control (ABAC). When granting permissions on all tables in a database using attributes, Lake Formation doesn't implicitly grant `DESCRIBE` permission to the database. You can grant permissions by using either the named resource method or the Lake Formation tag-based access control (LF-TBAC) method. You can grant permissions to principals in the same AWS account or to external accounts or organizations. When you grant to external accounts or organizations, you are sharing Data Catalog objects that you own with those accounts or organizations. Principals in those accounts or organizations can then access Data Catalog objects that you own and the underlying data. **Note** Currently, the LF-TBAC method supports granting cross-account permissions to IAM principals, AWS accounts, organizations, and organizational units (OUs). When you grant permissions to external accounts or organizations, you must include the grant option. Only the data lake administrator in the external account can access the shared objects until the administrator grants permissions on the shared objects to other principals in the external account. You can grant Data Catalog permissions by using the AWS Lake Formation console, the API, or the AWS Command Line Interface (AWS CLI). **Note** When you delete a Data Catalog object, all permissions that are associated with the object become invalid. Recreating the same resource with the same name, will not recover Lake Formation permissions. Users will have to setup new permissions again. **See also:** [Sharing Data Catalog tables and databases across AWS Accounts](sharing-catalog-resources.md) [Metadata access control](access-control-metadata.md) [Lake Formation permissions reference](lf-permissions-reference.md) # IAM permissions required to grant or revoke Lake Formation permissions IAM permissions required to grant Lake Formation permissions All principals, including the data lake administrator, need the following AWS Identity and Access Management (IAM) permissions to grant or revoke AWS Lake Formation Data Catalog permissions or data location permissions with the Lake Formation API or the AWS CLI: + `lakeformation:GrantPermissions` + `lakeformation:BatchGrantPermissions` + `lakeformation:RevokePermissions` + `lakeformation:BatchRevokePermissions` + `glue:GetTable`, `glue:GetDatabase`, or `glue:GetCatalog` for a table, database, or catalog that you're granting permissions using the named resource method. **Note** Data lake administrators have implicit Lake Formation permissions to grant and revoke Lake Formation permissions. But they still need the IAM permissions on the Lake Formation grant and revoke API operations. IAM roles with `AWSLakeFormationDataAdmin` AWS managed policy cannot add new data lake administrators because this policy contains an explicit deny for the Lake Formation API operation, `PutDataLakeSetting`. The following IAM policy is recommended for principals who are not data lake administrators and who want to grant or revoke permissions using the Lake Formation console. ------ #### [ JSON ] **** ``` { "Version":"2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "lakeformation:ListPermissions", "lakeformation:GrantPermissions", "lakeformation:BatchGrantPermissions", "lakeformation:RevokePermissions", "lakeformation:BatchRevokePermissions", "glue:GetCatalogs", "glue:GetDatabases", "glue:SearchTables", "glue:GetTables", "glue:GetCatalog", "glue:GetDatabase", "glue:GetTable", "iam:ListUsers", "iam:ListRoles", "sso-directory:DescribeUser", "sso-directory:DescribeGroup", "sso:DescribeInstance" ], "Resource": "*" } ] } ``` ------ All of the `glue:` and `iam:` permissions in this policy are available in the AWS managed policy `AWSGlueConsoleFullAccess`. To grant permissions by using Lake Formation tag-based access control (LF-TBAC), principals need additional IAM permissions. For more information, see [Lake Formation tag-based access control best practices and considerations](lf-tag-considerations.md) and [Lake Formation personas and IAM permissions reference](permissions-reference.md). **Cross-account permissions** Users who want to grant cross-account Lake Formation permissions by using the named resource method must also have the permissions in the `AWSLakeFormationCrossAccountManager` AWS managed policy. Data lake administrators need those same permissions for granting cross-account permissions, plus the AWS Resource Access Manager (AWS RAM) permission to enable granting permissions to organizations. For more information, see [Data lake administrator permissions](permissions-reference.md#persona-dl-admin). **The administrative user** A principal with administrative permissions—for example, with the `AdministratorAccess` AWS managed policy—has permissions to grant Lake Formation permissions and create data lake administrators. To deny a user or role access to Lake Formation administrator operations, attach or add into its policy a `Deny` statement for administrator API operations. ------ #### [ JSON ] **** ``` { "Version":"2012-10-17", "Statement": [ { "Action": [ "lakeformation:GetDataLakeSettings", "lakeformation:PutDataLakeSettings" ], "Effect": "Deny", "Resource": [ "*" ] } ] } ``` ------ **Important** To prevent users from adding themselves as an administrator with an extract, transform, and load (ETL) script, make sure that all non-administrator users and roles are denied access to these API operations. The `AWSLakeFormationDataAdmin` AWS managed policy contains an explict deny for the Lake Formation API operation, `PutDataLakeSetting` that prevents users from adding new data lake administrators. # Granting data permissions using the named resource method Using the named resources method The named Data Catalog resource method is a way of granting permissions to AWS Glue Data Catalog objects, such as catalogs, databases, tables, columns, and views, using a centralized approach. It allows you to define resource-based policies that control access to specific resources within your data lake. When you use the named resource method to grant permissions, you can specify the resource type and the permissions that you want to grant or revoke for that resource. You can also revoke the permission later if needed, thereby removing the permissions from the associated resources. You can grant permissions by using the AWS Lake Formation console, APIs, or the AWS Command Line Interface (AWS CLI). **Topics** + [ # Granting catalog permissions using the named resource method ](granting-multi-catalog-permissions.md) + [ # Granting database permissions using the named resource method ](granting-database-permissions.md) + [ # Granting table permissions using the named resource method ](granting-table-permissions.md) + [ # Granting permissions on views using the named resource method ](granting-view-permissions.md) # Granting catalog permissions using the named resource method Granting catalog permissions using the named resource method The following steps explain how to grant catalog permissions by using the named resource method. ------ #### [ Console ] Use the **Grant permissions** page on the Lake Formation console. The page is divided into the following sections: + **Principal type** – You can grant permissions to specific principals or use attribute tags. + **Principals** – The IAM users, roles, IAM Identity Center users and groups, SAML users and groups, AWS accounts, organizations, or organizational units to grant permissions. **Principal by attributes** – Add tag key-value pairs from IAMroles or IAM session tags. Principals with matching attributes receive access to the specified resource. + **LF-Tags or catalog resources** – The catalogs, databases, tables, views, or resource links to grant permissions on. + **Permissions** – The Lake Formation permissions to grant. **Note** To grant permissions on a database resource link, see [Granting resource link permissions](granting-link-permissions.md). 1. Open the **Grant permissions** page. Open the AWS Lake Formation console at [https://console.aws.amazon.com/lakeformation/](https://console.aws.amazon.com/lakeformation/), and sign in as a data lake administrator, the catalog creator, or an IAM user who has **Grantable permissions** on the catalog. Do one of the following: + In the navigation pane, under **Permissions**, choose **Data permissions**. Then choose **Grant**. + In the navigation pane, choose **Catalogs** under **Data Catalog**. Then, on the **Catalogs** page, choose a catalog, and from the **Actions** menu, under **Permissions**, choose **Grant**. **Note** You can grant permissions on a catalog through its resource link. To do so, on the **Catalogs** page, choose a catalog link container, and on the **Actions** menu, choose **Grant on target**. For more information, see [How resource links work in Lake Formation](resource-links-about.md). 1. Next, in the **Principal type** section, choose principals or specify attributes attached to the principals. ![\[The principal type section contains two tiles arranged horizontally, where each tile contains an option button and descriptive text. The options are Principals and Principals by attributes.Below the title are the principals.\]](http://docs.aws.amazon.com/lake-formation/latest/dg/images/grant-catalog-principal-type.png) ****Specify principals**** **IAM users and roles** Choose one or more users or roles from the **IAM users and roles** list. **IAM Identity Center** Choose one or more users or groups from the **Users and groups** list. Select **Add** to add more users or groups. **SAML users and groups** For **SAML and Quick users and groups**, enter one or more Amazon Resource Names (ARNs) for users or groups federated through SAML, or ARNs for Amazon Quick users or groups. Press Enter after each ARN. For information about how to construct the ARNs, see [Lake Formation grant and revoke AWS CLI commands](lf-permissions-reference.md#perm-command-format). Lake Formation integration with Quick is supported only for Quick Enterprise Edition. **External accounts** For **AWS account, AWS organization**, or **IAM Principal** enter one or more valid AWS account IDs, organization IDs, organizational unit IDs, or ARN for the IAM user or role. Press **Enter** after each ID. An organization ID consists of "o-" followed by 10–32 lower-case letters or digits. An organizational unit ID starts with "ou-" followed by 4–32 lowercase letters or digits (the ID of the root that contains the OU). This string is followed by a second "-" dash and 8 to 32 additional lowercase letters or digits. ****Principals by attributes**** **Attributes** Add the IAM tag key-value pairs from the IAM role. **Permission scope** Specify if you're granting permissions to principals with matching attributes in the same account or in another account. 1. In the **LF-Tags or catalog resources** section, choose **Named data catalog resources**. ![\[The LF-Tags or catalog resources section contains two tiles arranged horizontally, where each tile contains an option button and descriptive text. The options are Resources matched by LF-Tags, and Named data catalog resources. Below the tiles are two dropdown lists: Database and Table. The Database dropdown list has a tile beneath it containing the selected database name.\]](http://docs.aws.amazon.com/lake-formation/latest/dg/images/grant-target-resources-catalog.png) 1. Choose one or more catalogs from the **Catalogs** list. You can also choose one or more **Databases**, **Tables**, and/or **Data filters**. 1. In the **Catalog permissions** section, select permissions and grantable permissions. Under **Catalog permissions**, select one or more permissions to grant. ![\[The Permissions section the catalog permissions tile. Below the tiles is a group of check boxes for catalog permissions to grant. Check boxes include Super user, Create catalog, Create database, Alter, Drop, Describe, and Super. Below that group is another group of the same check boxes for grantable permissions.\]](http://docs.aws.amazon.com/lake-formation/latest/dg/images/grant-target-catalog-permissions-section.png) Choose **Super user** to grant unrestricted administrative privileges to perform any operation on all resources within the catalog (databases, tables, and views). **Note** After granting `Create database` or `Alter` on a catalog that has a location property that points to a registered location, be sure to also grant data location permissions on the location to the principals. For more information, see [Granting data location permissions](granting-location-permissions.md). 1. (Optional) Under **Grantable permissions**, select the permissions that the grant recipient can grant to other principals in their AWS account. This option is not supported when you are granting permissions to an IAM principal from an external account. 1. Choose **Grant**. The **Data permissions** page shows the permission details. If you used **Principals by attribute** option to grant permissions, you can view the permission grant to `ALLPrincipals` in the list. ------ #### [ AWS CLI ] For granting catalog permissions using AWS CLI, see [Creating Amazon Redshift federated catalogs](create-ns-catalog.md). ------ # Granting database permissions using the named resource method Granting database permissions using the named resource method The following steps explain how to grant database permissions by using the named resource method. ------ #### [ Console ] Use the **Grant permissions** page on the Lake Formation console. The page is divided into the following sections: + **Principal type** – The **Principals** section include the IAM users, roles, IAM Identity Center users and groups, SAML users and groups, AWS accounts, organizations, or organizational units to grant permissions. In the **Principals by attributes** section, you can specify the key and values for the attributes attached to the IAM roles. + **LF-Tags or catalog resources** – The databases, tables, views, or resource links to grant permissions on. + **Permissions** – The Lake Formation permissions to grant. **Note** To grant permissions on a database resource link, see [Granting resource link permissions](granting-link-permissions.md). 1. Open the **Grant permissions** page. Open the AWS Lake Formation console at [https://console.aws.amazon.com/lakeformation/](https://console.aws.amazon.com/lakeformation/), and sign in as a data lake administrator, the database creator, or an IAM user who has **Grantable permissions** on the database. Do one of the following: + In the navigation pane, under **Permissions**, choose **Data permissions**. Then choose **Grant**. + In the navigation pane, choose **Databases** under **Data Catalog**. Then, on the **Databases** page, choose a database, and from the **Actions** menu, under **Permissions**, choose **Grant**. **Note** You can grant permissions on a database through its resource link. To do so, on the **Databases** page, choose a resource link, and on the **Actions** menu, choose **Grant on target**. For more information, see [How resource links work in Lake Formation](resource-links-about.md). 1. In the **Principal type** section, specify principals or grant permissions to principals using attributes. ![\[The Principals section contains four tiles. Each tile contains a option button and text.\]](http://docs.aws.amazon.com/lake-formation/latest/dg/images/identity-center-grant-perm.png) **IAM users and roles** Choose one or more users or roles from the **IAM users and roles** list. **IAM Identity Center** Choose one or more users or groups from the **Users and groups** list. Select **Add** to add more users or groups. **SAML users and groups** For **SAML and Quick users and groups**, enter one or more Amazon Resource Names (ARNs) for users or groups federated through SAML, or ARNs for Amazon Quick users or groups. Press Enter after each ARN. For information about how to construct the ARNs, see [Lake Formation grant and revoke AWS CLI commands](lf-permissions-reference.md#perm-command-format). Lake Formation integration with Quick is supported only for Quick Enterprise Edition. **External accounts** For **AWS account, AWS organization**, or **IAM Principal** enter one or more valid AWS account IDs, organization IDs, organizational unit IDs, or ARN for the IAM user or role. Press **Enter** after each ID. An organization ID consists of "o-" followed by 10–32 lower-case letters or digits. An organizational unit ID starts with "ou-" followed by 4–32 lowercase letters or digits (the ID of the root that contains the OU). This string is followed by a second "-" dash and 8 to 32 additional lowercase letters or digits. Principals by attributes Specify the attribute key and value(s). If you choose more than one value, you are creating an attribute expression with an OR operator. This means that if any of the attribute tag values assigned to an IAM role or user match, the role/user gains access permissions on the resource. Choose the permission scope by specifying if you're granting permissions to principals with matching attributes in the same account or in another account. 1. In the **LF-Tags or catalog resources** section, choose **Named data catalog resources**. ![\[The LF-Tags or catalog resources section contains two tiles arranged horizontally, where each tile contains an option button and descriptive text. The options are Resources matched by LF-Tags, and Named data catalog resources. Below the tiles are two dropdown lists: Database and Table. The Database dropdown list has a tile beneath it containing the selected database name.\]](http://docs.aws.amazon.com/lake-formation/latest/dg/images/grant-target-resources-section-2.png) 1. Choose one or more databases from the **Database** list. You can also choose one or more **Tables** and/or **Data filters**. 1. In the **Permissions** section, select permissions and grantable permissions. Under **Database permissions**, select one or more permissions to grant. ![\[The Permissions section contains two tiles, arranged horizontally. Each tile contains a option button and text. The Database permissions tile is selected. The other tile, Column-based permissions, is disabled, because it relates to table permissions. Below the tiles is a group of check boxes for database permissions to grant. Check boxes include Create Table, Alter, Drop, Describe, and Super. Below that group is another group of the same check boxes for grantable permissions.\]](http://docs.aws.amazon.com/lake-formation/latest/dg/images/grant-target-db-permissions-section.png) **Note** After granting `Create Table` or `Alter` on a database that has a location property that points to a registered location, be sure to also grant data location permissions on the location to the principals. For more information, see [Granting data location permissions](granting-location-permissions.md). 1. (Optional) Under **Grantable permissions**, select the permissions that the grant recipient can grant to other principals in their AWS account. This option is not supported when you are granting permissions to an IAM principal from an external account. 1. Choose **Grant**. ------ #### [ AWS CLI ] You can grant database permissions by using the named resource method and the AWS Command Line Interface (AWS CLI). **To grant database permissions using the AWS CLI** + Run a `grant-permissions` command, and specify a database or the Data Catalog as the resource, depending on the permission being granted. In the following examples, replace ** with a valid AWS account ID. **Example – Grant to create a database** This example grants `CREATE_DATABASE` to user `datalake_user1`. Because the resource on which this permission is granted is the Data Catalog, the command specifies an empty `CatalogResource` structure as the `resource` parameter. ``` 1. aws lakeformation grant-permissions --principal DataLakePrincipalIdentifier=arn:aws:iam:::user/datalake_user1 --permissions "CREATE_DATABASE" --resource '{ "Catalog": {}}' ``` **Example – Grant to create tables in a designated database** The next example grants `CREATE_TABLE` on the database `retail` to user `datalake_user1`. ``` 1. aws lakeformation grant-permissions --principal DataLakePrincipalIdentifier=arn:aws:iam:::user/datalake_user1 --permissions "CREATE_TABLE" --resource '{ "Database": {"Name":"retail"}}' ``` **Example – Grant to an external AWS account with the Grant option** The next example grants `CREATE_TABLE` with the grant option on the database `retail` to external account 1111-2222-3333. ``` 1. aws lakeformation grant-permissions --principal DataLakePrincipalIdentifier=111122223333 --permissions "CREATE_TABLE" --permissions-with-grant-option "CREATE_TABLE" --resource '{ "Database": {"Name":"retail"}}' ``` **Example – Grant to an organization** The next example grants `ALTER` with the grant option on the database `issues` to the organization `o-abcdefghijkl`. ``` 1. aws lakeformation grant-permissions --principal DataLakePrincipalIdentifier=arn:aws:organizations::111122223333:organization/o-abcdefghijkl --permissions "ALTER" --permissions-with-grant-option "ALTER" --resource '{ "Database": {"Name":"issues"}}' ``` **Example - Grant to `ALLIAMPrincipals` in the same account** The next example grants `CREATE_TABLE` permission on the database `retail` to all principals in the same account. This option enables every principal in the account to create a table in the database and create a table resource link allowing integrated query engines to access shared databases and tables. This option is especially useful when a principal receives a cross-account grant, and does not have the permission to create resource links. In this scenario, the data lake administrator can create a placeholder database and grant `CREATE_TABLE` permission to the `ALLIAMPrincipal` group, enabling every IAM principal in the account to create resource links in the placeholder database. ``` 1. aws lakeformation grant-permissions --principal DataLakePrincipalIdentifier=111122223333:IAMPrincipals --permissions "CREATE_TABLE" --resource '{ "Database": {"Name":"temp","CatalogId":"111122223333"}}' ``` **Example - Grant to `ALLIAMPrincipals` in an external account** The next example grants `CREATE_TABLE` on the database `retail` to all principals in an external account. This option enables every principal in the account to create a table in the database. ``` 1. aws lakeformation grant-permissions --principal DataLakePrincipalIdentifier=111122223333:IAMPrincipals --permissions "CREATE_TABLE" --resource '{ "Database": {"Name":"retail","CatalogId":"123456789012"}}' ``` **Note** After granting `CREATE_TABLE` or `ALTER` on a database that has a location property that points to a registered location, be sure to also grant data location permissions on the location to the principals. For more information, see [Granting data location permissions](granting-location-permissions.md). ------ **See also** [Lake Formation permissions reference](lf-permissions-reference.md) [Granting permissions on a database or table shared with your account](regranting-shared-resources.md) [Accessing and viewing shared Data Catalog tables and databases](viewing-shared-resources.md) # Granting table permissions using the named resource method Granting table permissions using the named resource method You can use the Lake Formation console or AWS CLI to grant Lake Formation permissions on Data Catalog tables. You can grant permissions on individual tables, or with a single grant operation, you can grant permissions on all tables in a database. If you grant permissions on all tables in a database, you are implicitly granting the `DESCRIBE` permission on the database. The database then appears on the **Databases** page on the console, and is returned by the `GetDatabases` API operation. This automatic `DESCRIBE` permission grant doesn't apply when using attribute-based access control (ABAC). When granting permissions on all tables in a database using attributes, Lake Formation doesn't implicitly grant `DESCRIBE` permission to the database. When you choose `SELECT` as the permission to grant, you have the option to apply a column filter, row filter, or cell filter. ------ #### [ Console ] The following steps explain how to grant table permissions by using the named resource method and the **Grant data lake permissions** page on the Lake Formation console. The page is divided into these sections: + **Principals types** – The users, roles, AWS accounts, organizations, or organizational units to grant permissions to. You can also grant permissions to principals with matching attributes. + **LF-Tags or catalog resources** – The databases, tables, or resource links to grant permissions on. + **Permissions** – The Lake Formation permissions to grant. **Note** To grant permissions on a table resource link, see [Granting resource link permissions](granting-link-permissions.md). 1. Open the Grant permissions page. Open the AWS Lake Formation console at [https://console.aws.amazon.com/lakeformation/](https://console.aws.amazon.com/lakeformation/), and sign in as a data lake administrator, the table creator, or a user who has been granted permissions on the table with the grant option. Do one of the following: + In the navigation pane, choose **Data permissions** under **Permissions**. Then choose **Grant**. + In the navigation pane, choose **Tables**. Then, on the **Tables** page, choose a table, and on the **Actions** menu, under **Permissions**, choose **Grant**. **Note** You can grant permissions on a table through its resource link. To do so, on the **Tables** page, choose a resource link, and on the **Actions** menu, choose **Grant on target**. For more information, see [How resource links work in Lake Formation](resource-links-about.md). 1. Next, in the **Principal types** section, specify principals or principals with matching attrubutes to grant permissions. **IAM users and roles** Choose one or more users or roles from the **IAM users and roles** list. **IAM Identity Center** Choose one or more users or groups from the **Users and groups** list. **SAML users and groups** For **SAML and Quick users and groups**, enter one or more Amazon Resource Names (ARNs) for users or groups federated through SAML, or ARNs for Quick users or groups. Press Enter after each ARN. For information about how to construct the ARNs, see [Lake Formation grant and revoke AWS CLI commands](lf-permissions-reference.md#perm-command-format). Lake Formation integration with Quick is supported for Quick Enterprise Edition only. **External accounts** For **AWS account , AWS organization**, or **IAM Principal** enter one or more valid AWS account IDs, organization IDs, organizational unit IDs, or the ARN for the IAM user or role. Press **Enter** after each ID. An organization ID consists of "o-" followed by 10–32 lower-case letters or digits. An organizational unit ID starts with "ou-" followed by 4–32 lowercase letters or digits (the ID of the root that contains the OU). This string is followed by a second "-" character and 8 to 32 additional lowercase letters or digits. Principals by attributes Specify the attribute key and value(s). If you choose more than one value, you are creating an attribute expression with an OR operator. This means that if any of the attribute tag values assigned to an IAM role or user match, the role/user gains access permissions on the resource Choose the permission scope by specifying if you're granting permissions to principals with matching attributes in the same account or in another account. 1. In the **LF-Tags or catalog resources** section, choose a database. Then choose one or more tables, or **All tables**. ![\[The LF-Tags or catalog resources section contains two tiles arranged horizontally, where each tile contains an option button and descriptive text. The options are Resources matched by LF-Tags, and Named data catalog resources. Named data catalog resources is selected. Below the tiles are two dropdown lists: Database and Table. The Database dropdown list has a tile beneath it containing the selected database name. The Table dropdown list has a tile beneath it containing the selected table name.\]](http://docs.aws.amazon.com/lake-formation/latest/dg/images/grant-target-resources-tables-section-2.png) 1. **Specify the permissions with no data filtering.** In the **Permissions** section, select the table permissions to grant, and optionally select grantable permissions. ![\[The Table and column permissions section has two subsections: Table permissions and Grantable permissions. Each subsection has a check box for each possible Lake Formation permission: Alter, Insert, Drop, Delete, Select, Describe, and Super. The Super permission is set off to the right of the other permissions, and has a description: "This permission allows the principal to grant any of the permissions to the left, and supersedes those grantable permissions."\]](http://docs.aws.amazon.com/lake-formation/latest/dg/images/grant-table-permissions-section-no-filter.png) If you grant **Select**, the **Data permissions** section appears beneath the **Table and column permissions** section, with the **All data access** option selected by default. Accept the default. ![\[The section contains three tiles, arranged horizontally, each with an option button and a description. The option buttons are: All data access (selected), Simple column-based access, and Advanced cell-level filters.\]](http://docs.aws.amazon.com/lake-formation/latest/dg/images/grant-select-all-data-access.png) 1. Choose **Grant**. 1. **Specify the **Select** permission with data filtering** Select the **Select** permission. Don't select any other permissions. The **Data permissions** section appears beneath the **Table and column permissions** section. 1. Do one of the following: + Apply simple column filtering only. 1. Choose **Simple column-based access**. ![\[The top section is the Table and column permissions section. It is described in the preceding screenshot. It contains check boxes for table permissions and grantable permissions. The bottom section, Data permissions, has three tiles arranged horizontally, where each tile has an option button and description. The options are All data access, Simple column-based access, and Advanced cell-level filters. The Simple column-based access option is selected. Beneath the tiles is an option button group with the label Choose permission filter. The options are Include columns and Exclude columns. Beneath the option group is a Select columns dropdown list, and beneath that is a Grantable permissions subsection, which contains a single check box labeled Select.\]](http://docs.aws.amazon.com/lake-formation/latest/dg/images/grant-table-permissions-section-column-filter.png) 1. Choose whether to include or exclude columns, and then choose the columns to include or exclude. Only include lists are supported when granting permissions to an external AWS account or organization. 1. (Optional) Under **Grantable permissions**, turn on the grant option for the Select permission. If you include the grant option, the grant recipient can grant permissions only on the columns that you grant to them. **Note** You can also apply column filtering only by creating a data filter that specifies a column filter and specifies all rows as the row filter. However, this requires more steps. + Apply column, row, or cell filtering. 1. Choose **Advanced cell-level filters**. ![\[This section, titled Data permissions, is beneath the Table permissions section. It has three tiles arranged horizontally, where each tile has an option button and description. The options are All data access, Simple column-based access, and Advanced cell-level filters. The Advanced cell-level filters option is selected. Beneath the tiles is the label View existing permissions with an exposure triangle to the left. The existing permissions are not exposed. Below that is a section entitled Data filters to grant. To the right of the title are three buttons: Refresh, Manage filters, and Create new filter. Below the title and buttons is a text field with the placeholder text "Find filter". Below that is a table of existing filters. Each row has a check box at the left. The column headings are Filter name, Table, Database, and Table catalog ID. There are two rows. The filter name in the first row is restrict-pharma. The name in the second row is no-pharma.\]](http://docs.aws.amazon.com/lake-formation/latest/dg/images/grant-table-permissions-section-cell-filter.png) 1. (Optional) Expand **View existing permissions**. 1. (Optional) Choose **Create new filter**. 1. (Optional) To view details for the listed filters, or to create new or delete existing filters, choose **Manage filters**. The **Data filters** page opens in a new browser window. When you are finished on the **Data filters** page, return to the **Grant permissions** page, and if necessary, refresh the page to view any new data filters that you created. 1. Select one or more data filters to apply to the grant. **Note** If there are no data filters in the list, it means that no data filters were created for the selected table. 1. Choose **Grant**. ------ #### [ AWS CLI ] You can grant table permissions by using the named resource method and the AWS Command Line Interface (AWS CLI). **To grant table permissions using the AWS CLI** + Run a `grant-permissions` command, and specify a table as the resource. **Example – Grant on a single table - no filtering** The following example grants `SELECT` and `ALTER` to user `datalake_user1` in AWS account 1111-2222-3333 on the table `inventory` in the database `retail`. ``` 1. aws lakeformation grant-permissions --principal DataLakePrincipalIdentifier=arn:aws:iam::111122223333:user/datalake_user1 --permissions "SELECT" "ALTER" --resource '{ "Table": {"DatabaseName":"retail", "Name":"inventory"}}' ``` If you grant the `ALTER` permission on a table that has its underlying data in a registered location, be sure to also grant data location permissions on the location to the principals. For more information, see [Granting data location permissions](granting-location-permissions.md). **Example – Grant on All Tables with the Grant option - no filtering** The next example grants `SELECT` with the grant option on all tables in database `retail`. ``` aws lakeformation grant-permissions --principal DataLakePrincipalIdentifier=arn:aws:iam::111122223333:user/datalake_user1 --permissions "SELECT" --permissions-with-grant-option "SELECT" --resource '{ "Table": { "DatabaseName": "retail", "TableWildcard": {} } }' ``` **Example – Grant with simple column filtering** This next example grants `SELECT` on a subset of columns in the table `persons`. It uses simple column filtering. ``` aws lakeformation grant-permissions --principal DataLakePrincipalIdentifier=arn:aws:iam::111122223333:user/datalake_user1 --permissions "SELECT" --resource '{ "TableWithColumns": {"DatabaseName":"hr", "Name":"persons", "ColumnNames":["family_name", "given_name", "gender"]}}' ``` **Example – Grant with a data filter** This example grants `SELECT` on the `orders` table and applies the `restrict-pharma` data filter. ``` aws lakeformation grant-permissions --cli-input-json file://grant-params.json ``` The following are the contents of file `grant-params.json`. ``` { "Principal": {"DataLakePrincipalIdentifier": "arn:aws:iam::111122223333:user/datalake_user1"}, "Resource": { "DataCellsFilter": { "TableCatalogId": "111122223333", "DatabaseName": "sales", "TableName": "orders", "Name": "restrict-pharma" } }, "Permissions": ["SELECT"], "PermissionsWithGrantOption": ["SELECT"] } ``` ------ **See also** [Overview of Lake Formation permissions](lf-permissions-overview.md) [Data filtering and cell-level security in Lake Formation](data-filtering.md) [Lake Formation personas and IAM permissions reference](permissions-reference.md) [Granting resource link permissions](granting-link-permissions.md) [Accessing and viewing shared Data Catalog tables and databases](viewing-shared-resources.md) # Granting permissions on views using the named resource method The following steps explain how to grant permissions on views by using the named resource method and the **Grant permissions** page. The page is divided into the following sections: + **Principal types** – The IAM users, roles, IAM Identity Center users and groups, AWS accounts, organizations, or organizational units to grant permissions. You can also grant permissions to principals with matching attributes. + **LF-Tags or catalog resources** – The databases, tables, views, or resource links to grant permissions on. + **Permissions** – The data lake permissions to grant. ## Open the **Grant permissions** page 1. Open the AWS Lake Formation console at [https://console.aws.amazon.com/lakeformation/](https://console.aws.amazon.com/lakeformation/), and sign in as a data lake administrator, the database creator, or an IAM user who has **Grantable permissions** on the database. 1. Do one of the following: + In the navigation pane, under **Permissions**, choose **Data permissions**. Then choose **Grant**. + In the navigation pane, choose **Views** under **Data Catalog**. Then, on the **Views** page, choose a view, and from the **Actions** menu, under **Permissions**, choose **Grant**. **Note** You can grant permissions on a view through its resource link. To do so, on the **Views** page, choose a resource link, and on the **Actions** menu, choose **Grant on target**. For more information, see [How resource links work in Lake Formation](resource-links-about.md). ## Specify the principal types In the **Principal types** section, either choose Principals or Principals by attributes. If you choose Principals, the following options are available: **IAM users and roles** Choose one or more users or roles from the **IAM users and roles** list. **IAM Identity Center ** Choose one or more users or groups from the **Users and groups** list. **SAML users and groups** For **SAML and Quick users and groups**, enter one or more Amazon Resource Names (ARNs) for users or groups federated through SAML, or ARNs for Amazon Quick users or groups. Press Enter after each ARN. For information about how to construct the ARNs, see [Lake Formation grant and revoke AWS CLI commands](lf-permissions-reference.md#perm-command-format). Lake Formation integration with Quick is supported only for Quick Enterprise Edition. **External accounts** For **AWS account, AWS organization**, or **IAM Principal** enter one or more valid AWS account IDs, organization IDs, organizational unit IDs, or ARN for the IAM user or role. Press **Enter** after each ID. An organization ID consists of "o-" followed by 10–32 lower-case letters or digits. An organizational unit ID starts with "ou-" followed by 4–32 lowercase letters or digits (the ID of the root that contains the OU). This string is followed by a second "-" dash and 8 to 32 additional lowercase letters or digits. **See Also** + [Accessing and viewing shared Data Catalog tables and databases](viewing-shared-resources.md) **Principals by attributes** Specify the attribute key and value(s). If you choose more than one value, you are creating an attribute expression with an OR operator. This means that if any of the attribute tag values assigned to an IAM role or user match, the role/user gains access permissions on the resource Choose the permission scope by specifying if you're granting permissions to principals with matching attributes in the same account or in another account. ## Specify the views In the **LF-Tags or catalog resources** section, choose one or more views to grant permissions on. 1. Choose **Named data catalog resources**. 1. Choose one or more views from the **Views** list. You can also choose one or more catalogs, databases, tables, and/or data filters. Granting data lake permissions to `All tables` within a database will result in the grantee having permissions on all tables and views within the database. ## Specify the permissions In the **Permissions** section, select permissions and grantable permissions. ![\[The Permissions section has a group of check boxes for view permissions to grant. Check boxes include Select, Describe, Drop, and Super. Below that group is another group of the same check boxes for grantable permissions.\]](http://docs.aws.amazon.com/lake-formation/latest/dg/images/view-permissions.png) 1. Under **View permissions**, select one or more permissions to grant. 1. (Optional) Under **Grantable permissions**, select the permissions that the grant recipient can grant to other principals in their AWS account. This option is not supported when you are granting permissions to an IAM principal from an external account. 1. Choose **Grant**. **See Also** [Lake Formation permissions reference](lf-permissions-reference.md) [Granting permissions on a database or table shared with your account](regranting-shared-resources.md) # Lake Formation tag-based access control Tag-based access control Lake Formation tag-based access control (LF-TBAC) is an authorization strategy that defines permissions based on attributes. In Lake Formation, these attributes are called *LF-Tags*. You can attach LF-Tags to Data Catalog resources, and grant permissions to Lake Formation principals on those resources using these LF-Tags. Lake Formation allows operations on those resources when the principal has granted access to a tag value that matches the resource tag value. LF-TBAC is helpful in environments that are growing rapidly and helps with situations where policy management becomes cumbersome. LF-TBAC is the recommended method to use to grant Lake Formation permissions when there is a large number of Data Catalog objects including federated catalogs, databases, tables, and views. Lake Formation supports tag-based access control for federated catalogs of Amazon S3 tables, Amazon Redshift data warehouses, and federated data sources such as Amazon DynamoDB, SQL Server, and Snowflake. **Note** IAM tags are not the same as LF-Tags. These tags are not interchangeable. LF-Tags are used to grant Lake Formation permissions and IAM tags are used to define IAM policies. ## How Lake Formation tag-based access control works Each LF-Tag is a key-value pair, such as `department=sales` or `classification=restricted`. A key can have multiple defined values, such as `department=sales,marketing,engineering,finance`. To use the LF-TBAC method, data lake administrators and data engineers perform the following tasks. | Task | Task details | | --- | --- | | 1. Define the properties and relationships of LF-Tags. | - | | 2. Create the LF-Tag creators in Lake Formation. | [Adding LF-Tag creators](TBAC-adding-tag-creator.md) | | 3. Create the LF-Tag in Lake Formation. | [Creating LF-Tags](TBAC-creating-tags.md) | | 4. Assign LF-Tags to Data Catalog resources. | [Assigning LF-Tags to Data Catalog resources](TBAC-assigning-tags.md) | | 5. Grant permissions to other principals to assign LF-Tags to resources, optionally with the grant option. | [Managing LF-Tag value permissions](TBAC-granting-tags.md) | | 6. Grant LF-Tag expressions to principals, optionally with the grant option. | [Granting data lake permissions using the LF-TBAC method](granting-catalog-perms-TBAC.md) | | 7. (Recommended) After verifying that principals have access to the correct resources through the LF-TBAC method, revoke permissions that were granted by using the named resource method. | - | Consider the case where you must grant permissions to three principals on three databases and seven tables. ![\[Three figures of users are at the left, arranged vertically. At the right are three databases labeled A, B, and C, arranged vertically. Database A has two tables labeled A.1 and A.2, database B has tables labels B.1 and B.2, and Database C has three tables labeled C.1, C.2, and C.3. Seventeen arrows connect the users to the databases and tables, indicating grants on the databases and tables to the users.\]](http://docs.aws.amazon.com/lake-formation/latest/dg/images/TBAC_example_discreet.png) To achieve the permissions indicated in the preceding diagram by using the named resource method, you would have to make 17 grants, as follows (in pseudo-code). ``` GRANT CREATE_TABLE ON Database A TO PRINCIPAL 1 GRANT SELECT, INSERT ON Table A.1 TO PRINCIPAL 1 GRANT SELECT, INSERT ON Table A.2 TO PRINCIPAL 1 GRANT SELECT, INSERT ON Table B.2 TO PRINCIPAL 1 ... GRANT SELECT, INSERT ON Table A.2 TO PRINCIPAL 2 GRANT CREATE_TABLE ON Database B TO PRINCIPAL 2 ... GRANT SELECT, INSERT ON Table C.3 TO PRINCIPAL 3 ``` Now consider how you would grant permissions by using LF-TBAC. The following diagram indicates that you've assigned LF-Tags to databases and tables, and has granted permissions on LF-Tags to principals. In this example, the LF-Tags represent areas of the data lake that contain analytics for different modules of an enterprise resource planning (ERP) application suite. You can use them to control access to the analytics data for the various modules. All LF-Tags have the key `module` and possible values `Sales`, `Orders`, and `Customers`. An example LF-Tag looks like this: ``` module=Sales ``` The diagram shows only the LF-Tag values. ![\[Like the previous diagram, three figures of users are at the left, arranged vertically, and at the right are three databases labeled A, B, and C, arranged vertically. Database A has two tables labeled A.1 and A.2, database B has tables labels B.1 and B.2, and Database C has three tables labeled C.1, C.2, and C.3. There are no arrows between the users and the databases and tables. Instead, labeled "flags" next to the users indicate that user1 has been granted the LF-Tags Sales and Customers, user 2 has been granted the LF-Tag Orders, and user 3 has been granted the LF-Tag Customers. Flags next to the databases and tables indicate the following assignments ofLF-Tags to databases and tables: Database A: Sales. Table A1: A dimmed flag indicates that Sales was inherited from Database A. Table A2: Orders, but a dimmed flag indicates that Sales was inherited from Database A. Database B: Orders. Table B.1 and B.2 inherit Orders, and Table B.2 has Customers. Database C has Customers, and Tables C.1, C.2, and C.3 inherit Customers. The C tables don't have any other assignments.\]](http://docs.aws.amazon.com/lake-formation/latest/dg/images/TBAC_example_tags.png) **Tag assignments to Data Catalog resources and inheritance** Tables inherit LF-Tags from databases and columns inherit LF-Tags from tables. Inherited values can be overridden. In the preceding diagram, dimmed LF-Tags are inherited. Because of inheritance, the data lake administrator needs to make only the five following LF-Tag assignments to resources (in pseudo-code). ``` ASSIGN TAGS module=Sales TO database A ASSIGN TAGS module=Orders TO table A.2 ASSIGN TAGS module=Orders TO database B ASSIGN TAGS module=Customers TO table B.2 ASSIGN TAGS module=Customers TO database C ``` **Tag grants to principals** After assigning LF-Tags to the databases and tables, the data lake administrator must make only four grants of LF-Tags to principals, as follows (in pseudo-code). ``` GRANT TAGS module=Sales TO Principal 1 GRANT TAGS module=Customers TO Principal 1 GRANT TAGS module=Orders TO Principal 2 GRANT TAGS module=Customers TO Principal 3 ``` Now, a principal with the `module=Sales`LF-Tag can access Data Catalog resources with the `module=Sales` LF-Tag (for example, database A), a principal with the `module=Customers` LF-Tag can access resources with the `module=Customers` LF-Tag, and so on. The preceding grant commands are incomplete. This is because although they indicate through LF-Tags the Data Catalog resources that the principals have permissions on, they don't indicate exactly which Lake Formation permissions (such as `SELECT`, `ALTER`) the principals have on those resources. Therefore, the following pseudo-code commands are a more accurate representation of how Lake Formation permissions are granted on Data Catalog resources through LF-Tags. ``` GRANT (CREATE_TABLE ON DATABASES) ON TAGS module=Sales TO Principal 1 GRANT (SELECT, INSERT ON TABLES) ON TAGS module=Sales TO Principal 1 GRANT (CREATE_TABLE ON DATABASES) ON TAGS module=Customers TO Principal 1 GRANT (SELECT, INSERT ON TABLES) ON TAGS module=Customers TO Principal 1 GRANT (CREATE_TABLE ON DATABASES) ON TAGS module=Orders TO Principal 2 GRANT (SELECT, INSERT ON TABLES) ON TAGS module=Orders TO Principal 2 GRANT (CREATE_TABLE ON DATABASES) ON TAGS module=Customers TO Principal 3 GRANT (SELECT, INSERT ON TABLES) ON TAGS module=Customers TO Principal 3 ``` **Putting it together - Resulting permissions on resources** Given the LF-Tags assigned to the databases and tables in the preceding diagram, and the LF-Tags granted to the principals in the diagram, the following table lists the Lake Formation permissions that the principals have on the databases and tables. | Principal | Permissions Granted Through LF-Tags | | --- | --- | | Principal 1 | [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/lake-formation/latest/dg/tag-based-access-control.html) | | Principal 2 | [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/lake-formation/latest/dg/tag-based-access-control.html) | | Principal 3 | [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/lake-formation/latest/dg/tag-based-access-control.html) | **Bottom line** In this simple example, using five assignment operations and eight grant operations, the data lake administrator was able to specify 17 permissions. When there are tens of databases and hundreds of tables, the advantage of the LF-TBAC method over the named resource method becomes clear. In the hypothetical case of the need to grant every principal access to every resource, and where `n(P)` is the number of principals and `n(R)` is the number of resources: + With the named resource method, the number of grants required is `n(P)` ✕ `n(R)`. + With the LF-TBAC method, using a single LF-Tag, the total of the number of grants to principals and assignments to resources is `n(P)` \$1 `n(R)`. **See also** [Managing LF-Tags for metadata access control](managing-tags.md) [Granting data lake permissions using the LF-TBAC method](granting-catalog-perms-TBAC.md) **Topics** + [ ## How Lake Formation tag-based access control works ](#how-TBAC-works) + [ # Managing LF-Tags for metadata access control ](managing-tags.md) + [ # Managing LF-Tag expressions for metadata access control ](managing-tag-expressions.md) + [ # Managing LF-Tag value permissions ](TBAC-granting-tags.md) # Managing LF-Tags for metadata access control To use the Lake Formation tag-based access control (LF-TBAC) method to secure Data Catalog objects such as catalogs, databases, tables, views, and columns, you create LF-Tags, assign them to resources, and grant LF-Tag permissions to principals. Before you can assign LF-Tags to Data Catalog objects or grant permissions to principals, you need to define LF-Tags. Only a data lake administrator or a principal with LF-Tag creator permissions can create LF-Tags. **LF-Tag creators** LF-Tag creator is a non-admin principal who has permissions to create and manage LF-Tags. Data lake administrators can add LF-Tag creators using the Lake Formation console or CLI. LF-Tag creators have implicit Lake Formation permissions to update, and delete LF-Tags, to assign LF-Tags to resources, and to grant LF-Tag permissions and LF-Tag value permissions to other principals. With LF-Tag creator roles, data lake administrators can delegate tag management tasks such as creating and updating tag keys and values to non-admin principals. Data lake administrators can also grant LF-Tag creators grantable `Create LF-Tag` permissions. Then, the LF-Tag creator can grant the permission to create LF-Tags to other principals. You can grant two types of permissions on LF-Tags: + LF-Tag permissions - `Create LF-Tag`, `Alter`, and `Drop`. These permissions are required to create, update, and delete LF-Tags. Data lake administrators and LF-Tag creators implicitly have these permissions on the LF-Tags they create and can grant these permissions explicitly to principals to manage tags in the data lake. + LF-Tag key-value pair permissions - `Assign`, `Describe`, and `Grant with LF-Tag expressions`. These permissions are required to assign LF-Tags to Data Catalog objects, and to grant permissions on the resources to principals using Lake Formation tag-based access control. LF-Tag creators implicitly receive these permissions when creating LF-Tags. After receiving the `Create LF-Tag` permission and successfully creating LF-Tags, the LF-Tag creator can assign LF-Tags to resources and grant LF-Tag permissions (`Create LF-Tag`, `Alter`, `Drop`, and ) to other non-administrative principals to manage tags in the data lake. You can manage LF-Tags by using the Lake Formation console, the API, or the AWS Command Line Interface (AWS CLI). **Note** Data lake administrators have implicit Lake Formation permissions to create, update, and delete LF-Tags, to assign LF-Tags to resources, and to grant LF-Tag permissions to principals. For best practices and considerations, see [Lake Formation tag-based access control best practices and considerations](lf-tag-considerations.md) **Topics** + [ # Adding LF-Tag creators ](TBAC-adding-tag-creator.md) + [ # Creating LF-Tags ](TBAC-creating-tags.md) + [ # Updating LF-Tags ](TBAC-updating-tags.md) + [ # Deleting LF-Tags ](TBAC-deleting-tags.md) + [ # Listing LF-Tags ](TBAC-listing-tags.md) + [ # Assigning LF-Tags to Data Catalog resources ](TBAC-assigning-tags.md) + [ # Viewing LF-Tags assigned to a resource ](TBAC-view-resource-tags.md) + [ # Viewing the resources that a LF-Tag is assigned to ](TBAC-view-tag-resources.md) + [ ## Life cycle of a LF-Tag ](#lf-tag-life-cycle) + [ ## Comparison of Lake Formation tag-based access control to IAM attribute-based access control ](#TBAC-comparison-ABAC) **See also** [Managing LF-Tag value permissions](TBAC-granting-tags.md) [Granting data lake permissions using the LF-TBAC method](granting-catalog-perms-TBAC.md) [Lake Formation tag-based access control](tag-based-access-control.md) # Adding LF-Tag creators By default, data lake administrators can create, update, and delete LF-Tags, assign tags to Data Catalog objects, and grant tag permissions to principals. If you wish to delegate the tag creation and management operations to non-admin principals, the data lake administrator can create LF-Tag creator roles and grant Lake Formation `Create LF-Tag` permission to the roles. With grantable `Create LF-Tag` permission, LF-Tag creators can delegate tag creation and maintenance tasks to other non-administrative principals. For data lake administrators to assign LF-Tags to Data Catalog resources, they are required to grant themselves associate permissions on LF-Tags that were not created by them. **Note** Cross-account permission grants can include only `Describe` and `Associate` permissions. You can't grant `Create LF-Tag`, `Drop`, `Alter`, and `Grant with LFTag expressions` permissions to principals in a different account. **Topics** + [ ## IAM permissions required to create LF-Tags ](#tag-creator-permissions) + [ ## Add LF-Tag creators ](#add-lf-tag-creator) **See also** [Managing LF-Tag value permissions](TBAC-granting-tags.md) [Granting data lake permissions using the LF-TBAC method](granting-catalog-perms-TBAC.md) [Lake Formation tag-based access control](tag-based-access-control.md) ## IAM permissions required to create LF-Tags You must configure permissions to allow a Lake Formation principal to create LF-Tags. Add the following statement to the permissions policy for the principal that needs to be a LF-Tag creator. **Note** Although data lake administrators have implicit Lake Formation permissions to create, update, and delete LF-Tags, to assign LF-Tags to resources, and to grant LF-Tags to principals, data lake administrators also need the following IAM permissions. For more information, see [Lake Formation personas and IAM permissions reference](permissions-reference.md). ``` { "Sid": "Transformational", "Effect": "Allow", "Action": [ "lakeformation:AddLFTagsToResource", "lakeformation:RemoveLFTagsFromResource", "lakeformation:GetResourceLFTags", "lakeformation:ListLFTags", "lakeformation:CreateLFTag", "lakeformation:GetLFTag", "lakeformation:UpdateLFTag", "lakeformation:DeleteLFTag", "lakeformation:SearchTablesByLFTags", "lakeformation:SearchDatabasesByLFTags" ] } ``` Principals who assign LF-Tags to resources and grant LF-Tags to principals must have the same permissions, except for the `CreateLFTag`, `UpdateLFTag`, and `DeleteLFTag` permissions. ## Add LF-Tag creators A LF-Tag creator can create a LF-Tag, update tag key and values, delete tags, associate tags to Data Catalog resources, and grant permissions on Data Catalog resources to principals using LF-TBAC method. The LF-Tag creator can also grant these permissions to principals. You can create LF-Tag creator roles by using the AWS Lake Formation console, the API, or the AWS Command Line Interface (AWS CLI). ------ #### [ console ] **To add a LF-Tag creator** 1. Open the Lake Formation console at [https://console.aws.amazon.com/lakeformation/](https://console.aws.amazon.com/lakeformation/). Sign in as a datalake administrator. 1. In the navigation pane, under **Permissions**, choose **LF-Tags and permissions**. On the **LF-Tags and permissions** page, choose **LF-Tag creators** section and choose **Add LF-Tag creators**. ![\[LF-Tag creator details form with IAM user selection and permission options.\]](http://docs.aws.amazon.com/lake-formation/latest/dg/images/add-lf-tag-creator.png) 1. On the **Add LF-Tag creators** page, choose an IAM role or user who has the required permissions to create LF-Tags. 1. Enable `Create LF-Tag` permission check box. 1. (Optional) To enable the selected principals to grant `Create LF-Tag` permission to principals, choose Grantable `Create LF-Tag` permission. 1. Choose **Add**. ------ #### [ AWS CLI ] ``` aws lakeformation grant-permissions --cli-input-json file://grantCreate { "Principal": { "DataLakePrincipalIdentifier": "arn:aws:iam::123456789012:user/tag-manager" }, "Resource": { "Catalog": {} }, "Permissions": [ "CreateLFTag" ], "PermissionsWithGrantOption": [ "CreateLFTag" ] } ``` ------ The following are the permissions available for a LF-Tag creator role: | Permission | Description | | --- | --- | | Drop | A principal with this permission on a LF-Tag can delete a LF-Tag from the data lake. The principal gets implicit Describe permission on all tag values of a LF-Tag resource. | | Alter | A principal with this permission on a LF-Tag can add or remove tag value from a LF-Tag. The principal gets implicit Alter permission on all tag values of a LF-Tag. | | Describe | A principal with this permission on a LF-Tag can view the LF-Tag and its values when they assign LF-Tags to resources or grant permissions on LF-Tags. You can grant Describe on all key values or on specific values. | | Associate | A principal with this permission on a LF-Tag can assign the LF-Tag to a Data Catalog resource. Granting Associate implicitly grants Describe. | | Grant with LF-Tag expression | A principal with this permission on a LF-Tag can grant permissions on a Data Catalog resources using the LF-Tag key and values. Granting Grant with LF-Tag expression implicitly grants Describe. | These permissions are grantable. A principal who has been granted these permissions with the grant option can grant them to other principals. # Creating LF-Tags All LF-Tags must be defined in Lake Formation before they can be used. A LF-Tag consists of a key and one or more possible values for the key. After the data lake administrator has setup the required IAM permissions and Lake Formation permissions for the LF-Tag creator role, the principal can create a LF-Tag. The LF-Tag creator gets implicit permission to update or remove any tag value from the LF-Tag and delete the LF-Tag. You can create LF-Tags by using the AWS Lake Formation console, the API, or the AWS Command Line Interface (AWS CLI). ------ #### [ Console ] **To create a LF-Tag** 1. Open the Lake Formation console at [https://console.aws.amazon.com/lakeformation/](https://console.aws.amazon.com/lakeformation/). Sign in as a principal with LF-Tag creator permissions or as data lake administrator. 1. In the navigation pane, under **Permissions**, **LF-Tags and permissions**, choose **LF-Tags**. The **LF-Tags** page appears. ![\[The page has a 4-column table with column headings Key, Values, Owner account ID, and LF-Tag permissions. The table has 2 rows. Above the table are 4 buttons arranged horizontally: Delete (dimmed), Edit (dimmed), Grant permissions (dimmed) and Add tag. The page also has a search field with the placeholder text "Find tag". To the right of the search field is a page selector, showing the value "1" between left and right buttons, and a Settings icon.\]](http://docs.aws.amazon.com/lake-formation/latest/dg/images/policy-tags-page-2.png) 1. Choose **Add LF-Tag**. 1. In the **Add LF-Tag** dialog box, enter a key and one or more values. Each key must have at least one value. To enter multiple values, either enter a comma-delimited list and then press **Enter**, or enter one value at a time and choose **Add** after each one. The maximum number of values permitted is 1000. 1. Choose **Add tag**. ------ #### [ AWS CLI ] **To create a LF-Tag** + Enter a `create-lf-tag` command. The following example creates a LF-Tag with key `module` and values `Customers` and `Orders`. ``` aws lakeformation create-lf-tag --tag-key module --tag-values Customers Orders ``` ------ As tag creator , the principal gets `Alter` permission on this LF-Tag and can update or remove any tag value from this LF-Tag. The LF-Tag creator principal can also grant `Alter` permission to another principal to update and remove tag values on this LF-Tag. # Updating LF-Tags You update a LF-Tag that you have the `Alter` permission on by adding or deleting permitted key values. You can't change the LF-Tag key. To change the key, delete the LF-Tag and add one with the required key. In addition to `Alter` permission, you also need the `lakeformation:UpdateLFTag` IAM permission to update values. When you delete a LF-Tag value, no check is performed for the presence of that LF-Tag value on any Data Catalog resource. If the deleted LF-Tag value is associated with a resource, it is no longer visible for the resource, and any principals that were granted permissions on that key-value pair no longer have the permissions. Before deleting a LF-Tag value, you can optionally use the [`remove-lf-tags-from-resource` command](TBAC-assigning-tags.md#remove-tag-command) command to remove the LF-Tag from Data Catalog resources that have the value that you want to delete, and then retag the resource with the values that you want to keep. Only data lake administrators, the LF-Tag creator, and principals that have `Alter` permissions on the LF-Tag can update a LF-Tag. You can update a LF-Tag by using the AWS Lake Formation console, the API, or the AWS Command Line Interface (AWS CLI). ------ #### [ Console ] **To update a LF-Tag (console)** 1. Open the Lake Formation console at [https://console.aws.amazon.com/lakeformation/](https://console.aws.amazon.com/lakeformation/). Sign in as a data lake administrator, LF-Tag creator or a principal with `Alter` permission on the LF-Tag. 1. In the navigation pane, under **Permissions**, **LF-Tags and permissions**, choose **LF-Tags**. 1. On the **LF-Tags** page, select a LF-Tag, and then choose **Edit**. 1. In the **Edit LF-Tag** dialog box, add or remove LF-Tag values. To add multiple values, in the **Values** field, either enter a comma-delimited list and press **Enter**, or enter one value at a time or choose **Add** after each one. 1. Choose **Save**. ------ #### [ AWS CLI ] **To update a LF-Tag (AWS CLI)** + Enter an `update-lf-tag` command. Provide one or both of the following arguments: + `--tag-values-to-add` + `--tag-values-to-delete` **Example** The following example replaces the value `vp` with the value `vice-president` for the LF-Tag key `level`. ``` aws lakeformation update-lf-tag --tag-key level --tag-values-to-add vice-president --tag-values-to-delete vp ``` ------ # Deleting LF-Tags You can delete LF-Tags that are no longer in use. No check is performed for the presence of the LF-Tag on a Data Catalog resource. If the deleted LF-Tag is associated with a resource, it is no longer visible for the resource, and any principals that were granted permissions on that LF-Tag no longer have the permissions. Before deleting a LF-Tag, you can optionally use the [`remove-lf-tags-from-resource`](TBAC-assigning-tags.md#remove-tag-command) command to remove the LF-Tag from all resources. Only data lake administrators, the LF-Tag creator, or a principal that has `Drop` permission on the LF-Tag can delete a LF-Tag. In addition to the `Drop` permission, the principal also need `lakeformation:DeleteLFTag` IAM permission to delete a LF-Tag. You can delete a LF-Tag by using the AWS Lake Formation console, the API, or the AWS Command Line Interface (AWS CLI). ------ #### [ Console ] **To delete a LF-Tag (console)** 1. Open the Lake Formation console at [https://console.aws.amazon.com/lakeformation/](https://console.aws.amazon.com/lakeformation/). Sign in as a data lake administrator. 1. In the navigation pane, under **Permissions**, **LF-Tags and permissions**, choose **LF-Tags**. 1. On the **LF-Tags** page, select a LF-Tag, and then choose **Delete**. 1. In the **Delete tag environment?** dialog box, to confirm the deletion, enter the LF-Tag key value in the designated field and then choose **Delete**. ------ #### [ AWS CLI ] **To delete a LF-Tag (AWS CLI)** + Enter a `delete-lf-tag` command. Provide the key of the LF-Tag to delete. **Example** The following example deletes the LF-Tag with the key `region`. ``` aws lakeformation delete-lf-tag --tag-key region ``` ------ # Listing LF-Tags You can list the LF-Tags that you have the `Describe` or `Associate` permissions on. The values listed with each LF-Tag key are the values that you have permissions on. LF-Tag creator has implicit permissions to see the LF-Tags they have created. Data lake administrators can see all LF-Tags that are defined in the local AWS account and all LF-Tags for which the `Describe` and `Associate` permissions have been granted to the local account from external accounts. The data lake administrator can see all values for all LF-Tags. You can list LF-Tags by using the AWS Lake Formation console, the API, or the AWS Command Line Interface (AWS CLI). ------ #### [ Console ] **To list LF-Tags (console)** 1. Open the Lake Formation console at [https://console.aws.amazon.com/lakeformation/](https://console.aws.amazon.com/lakeformation/). Sign in as the LF-Tag creator, as a data lake administrator, or as a principal that has been granted permissions on LF-Tags and that has the `lakeformation:ListLFTags` IAM permission. 1. In the navigation pane, under **Permissions**, **LF-Tags and permissions**, choose **LF-Tags**. The **LF-Tags** page appears. ![\[The page has a 3-column table with column headings Key, Values, and Owner account ID. The table has 2 rows. Above the table are 4 buttons arranged horizontally: Reload page, Delete (dimmed), Edit (dimmed), and Add tag. The page also has a search field with the placeholder text "Find tag". To the right of the search field is a page selector, showing the value "1" between left and right buttons, and a Settings icon.\]](http://docs.aws.amazon.com/lake-formation/latest/dg/images/policy-tags-page-2.png) Check the **Owner account ID** column to determine the LF-Tags that were shared with your account from an external account. ------ #### [ AWS CLI ] **To list LF-Tags (AWS CLI)** + Run the following command as a data lake administrator or as a principal that has been granted permissions on LF-Tags and that has the `lakeformation:ListLFTags` IAM permission. ``` aws lakeformation list-lf-tags ``` The output is similar to the following. ``` { "LFTags": [ { "CatalogId": "111122223333", "TagKey": "level", "TagValues": [ "director", "vp", "c-level" ] }, { "CatalogId": "111122223333", "TagKey": "module", "TagValues": [ "Orders", "Sales", "Customers" ] } ] } ``` To also see LF-Tags that were granted from external accounts, include the command option `--resource-share-type ALL`. ``` aws lakeformation list-lf-tags --resource-share-type ALL ``` The output is similar to the following. Note the `NextToken` key, which indicates that there is more to list. ``` { "LFTags": [ { "CatalogId": "111122223333", "TagKey": "level", "TagValues": [ "director", "vp", "c-level" ] }, { "CatalogId": "111122223333", "TagKey": "module", "TagValues": [ "Orders", "Sales", "Customers" ] } ], "NextToken": "eyJleHBpcmF0aW...ZXh0Ijp0cnVlfQ==" } ``` Repeat the command, and add the `--next-token` argument to view any remaining local LF-Tags and LF-Tags that were granted from external accounts. LF-Tags from external accounts are always on a separate page. ``` aws lakeformation list-lf-tags --resource-share-type ALL --next-token eyJleHBpcmF0aW...ZXh0Ijp0cnVlfQ== ``` ``` { "LFTags": [ { "CatalogId": "123456789012", "TagKey": "region", "TagValues": [ "central", "south" ] } ] } ``` ------ #### [ API ] You can use the SDKs available for Lake Formation to lists the tags that the requester has permission to view. ``` import boto3 client = boto3.client('lakeformation') ... response = client.list_lf_tags( CatalogId='string', ResourceShareType='ALL', MaxResults=50' ) ``` This command returns a `dict` object with the following structure: ``` { 'LFTags': [ { 'CatalogId': 'string', 'TagKey': 'string', 'TagValues': [ 'string', ] }, ], 'NextToken': 'string' } ``` ------ For more information about the required permissions, see [Lake Formation personas and IAM permissions reference](permissions-reference.md). # Assigning LF-Tags to Data Catalog resources You can assign LF-Tags to Data Catalog resources (databases, tables, and columns) to control access to those resources. Only principals that are granted matching LF-Tags (and principals that are granted access with the named resource method) can access the resources. If a table inherits a LF-Tag from a database or a column inherits a LF-Tag from a table, you can override the inherited value by assigning a new value to the LF-Tag key. The maximum number of LF-Tags that you can assign to a resource is 50. **Topics** + [ ## Requirements for managing tags assigned to resources ](#manage-tags-reqs) + [ ## Assign LF-Tags to a table column ](#assign-tag-column) + [ ## Assign LF-Tags to a Data Catalog resource ](#assign-tag-catalog-resource) + [ ## Updating LF-Tags for a resource ](#update-tags) + [ ## Removing LF-Tag from a resource ](#remove-tag) ## Requirements for managing tags assigned to resources To assign a LF-Tag to a Data Catalog resource, you must: + Have the Lake Formation `ASSOCIATE` permission on the LF-Tag. + Have the IAM `lakeformation:AddLFTagsToResource` permission. + Have glue:GetDatabase permission on a Glue database. + Be the resource owner (creator), have the `Super` Lake Formation permission on the resource with the `GRANT` option, or have the following permissions with the `GRANT` option: + For databases in the same AWS account: `DESCRIBE`, `CREATE_TABLE`, `ALTER`, and `DROP` + For databases in an external account: `DESCRIBE`, `CREATE_TABLE` and `ALTER` + For tables (and columns): `DESCRIBE`, `ALTER`, `DROP`, `INSERT`, `SELECT`, and `DELETE` In addition, the LF-Tag and the resource that it is being assigned to must be in the same AWS account. To remove a LF-Tag from a Data Catalog resource, you must meet these requirements, and also have the `lakeformation:RemoveLFTagsFromResource` IAM permission. ## Assign LF-Tags to a table column **To assign LF-Tags to a table column (console)** 1. Open the Lake Formation console at [https://console.aws.amazon.com/lakeformation/](https://console.aws.amazon.com/lakeformation/). Sign in as a user who meets the requirements listed above. 1. In the navigation pane, choose **Tables**. 1. Choose a table name (not the option button next to the table name). 1. On the table details page, in the **Schema** section, choose **Edit schema**. 1. On the **Edit schema** page, select one or more columns, and then choose **Edit LF-Tags**. **Note** If you intend to add or delete columns and save a new version, do that first. Then edit the LF-Tags. The **Edit LF-Tags** dialog box appears, and displays any LF-Tags that are inherited from the table. ![\[The image is a screenshot of the Edit LF-Tags dialog window. The top part of the windows shows two inherited keys. The first inherited key has the key "level" and the value "director (inherited)". The second inherited key has the key "module" and the value "Orders (inherited)". Below those fields is an "Assign new LF-Tag" button. Below and to the right are the Cancel and Save buttons.\]](http://docs.aws.amazon.com/lake-formation/latest/dg/images/edit-policy-tags-for-columns-2a.png) 1. (Optional) For the **Values** list next to an **Inherited keys** field, choose a value to override the inherited value. 1. (Optional) Choose **Assign new LF-Tag**. Then for **Assigned keys**, choose a key, and for **Values**, choose a value for the key. ![\[The image is a screenshot of the Edit LF-Tags dialog window. The top part of the windows shows two inherited keys. The first inherited key has the key "level" and the value "director (inherited)". The second inherited key has the key "module" and the value "Orders (inherited)". Below this section, aligned horizontally, are these fields and controls : "Assigned keys" field, "Values" field, and a Remove button. The Assigned keys field contains the text "environment". The Values field is a drop-down list, with the values "Production" (highlighted) and "Customers". An "Assign new LF-Tag" button appears below the Assigned keys field. In the bottom right of the window are the Cancel and Save buttons.\]](http://docs.aws.amazon.com/lake-formation/latest/dg/images/edit-policy-tags-for-columns-2b.png) 1. (Optional) Choose **Assign new LF-Tag** again to add another LF-Tag. 1. Choose **Save**. ## Assign LF-Tags to a Data Catalog resource ------ #### [ Console ] **To assign LF-Tags to a Data Catalog database or table** 1. Open the Lake Formation console at [https://console.aws.amazon.com/lakeformation/](https://console.aws.amazon.com/lakeformation/). Sign in as a user who meets the requirements listed earlier. 1. In the navigation pane, under **Data catalog**, do one of the following: + To assign LF-Tags to databases, choose **Databases**. + To assign LF-Tags to tables, choose **Tables**. 1. Choose a database or table, and on the **Actions** menu, choose **Edit LF-Tags**. The **Edit LF-Tags: *resource-name*** dialog box appears. If a table inherits LF-Tags from its containing database, the window displays the inherited LF-Tags. Otherwise, it displays the text "There are no inherited LF-Tags associated with the resource." ![\[The image is a screenshot of the "Edit LF-Tags: inventory" dialog window. At the top are the fields "Inherited keys" (dimmed) and "Values". The Inherited keys field has the value "level" and the Values field has the value "director (inherited)". Below this section, aligned horizontally, are these fields and controls : "Assigned keys" field, "Values" field, and a Remove button. The Assigned keys field contains the text "module". The Values field is a drop-down list, with the values "Orders", "Sales", and "Customers" (highlighted). An "Assign new LF-Tag" button is below the Assigned keys field. In the bottom right of the window are Cancel and Save buttons.\]](http://docs.aws.amazon.com/lake-formation/latest/dg/images/edit-policy-tags-for-tables-2.png) 1. (Optional) If a table has inherited LF-Tags, for the **Values** list next to an **Inherited keys** field, you can choose a value to override the inherited value. 1. To assign new LF-Tags, perform these steps: 1. Choose **Assign new LF-Tag**. 1. In the **Assigned keys** field, choose a LF-Tag key, and in the **Values** field, choose a value. 1. (Optional) Choose **Assign new LF-Tag** again to assign an additional LF-Tag. 1. Choose **Save**. ------ #### [ AWS CLI ] **To assign LF-Tags to a Data Catalog resource** + Run the `add-lf-tags-to-resource` command. The following example assigns the LF-Tag `module=orders` to the table `orders` in the database `erp`. It uses the shortcut syntax for the `--lf-tags` argument. The `CatalogID` property for `--lf-tags` is optional. If not provided, the catalog ID of the resource (in this case, the table) is assumed. ``` aws lakeformation add-lf-tags-to-resource --resource '{ "Table": {"DatabaseName":"erp", "Name":"orders"}}' --lf-tags CatalogId=111122223333,TagKey=module,TagValues=orders ``` The following is the output if the command succeeds. ``` { "Failures": [] } ``` This next example assigns two LF-Tags to the `sales` table, and uses the JSON syntax for the `--lf-tags` argument. ``` aws lakeformation add-lf-tags-to-resource --resource '{ "Table": {"DatabaseName":"erp", "Name":"sales"}}' --lf-tags '[{"TagKey": "module","TagValues": ["sales"]},{"TagKey": "environment","TagValues": ["development"]}]' ``` This next example assigns the LF-Tag `level=director` to the `total` column of the table `sales`. ``` aws lakeformation add-lf-tags-to-resource --resource '{ "TableWithColumns": {"DatabaseName":"erp", "Name":"sales", "ColumnNames":["total"]}}' --lf-tags TagKey=level,TagValues=director ``` ------ ## Updating LF-Tags for a resource **To update a LF-Tag for a Data Catalog resource (AWS CLI)** + Use the `add-lf-tags-to-resource` command, as described in the previous procedure. Adding a LF-Tag with the same key as an existing LF-Tag, but with a different value updates the existing value. ## Removing LF-Tag from a resource **To remove a LF-Tag for a Data Catalog resource (AWS CLI)** + Run the `remove-lf-tags-from-resource` command. If a table has a LF-Tag value that overrides the value that is inherited from the parent database, removing that LF-Tag from the table restores the inherited value. This behavior also applies to a column that overrides key values inherited from the table. The following example removes the LF-Tag `level=director` from the `total` column of the `sales` table. The `CatalogID` property for `--lf-tags` is optional. If not provided, the catalog ID of the resource (in this case, the table) is assumed. ``` aws lakeformation remove-lf-tags-from-resource --resource ' { "TableWithColumns": { "DatabaseName": "erp", "Name": "sales", "ColumnNames":[ "total"]}}' --lf-tags CatalogId=111122223333,TagKey=level,TagValues=director ``` # Viewing LF-Tags assigned to a resource You can view the LF-Tags that are assigned to a Data Catalog resource. You must have the `DESCRIBE` or `ASSOCIATE` permission on a LF-Tag to view it. ------ #### [ Console ] **To view the LF-Tags that are assigned to a resource (console)** 1. Open the Lake Formation console at [https://console.aws.amazon.com/lakeformation/](https://console.aws.amazon.com/lakeformation/). Sign in as the data lake administrator, the resource owner, or a user who has been granted Lake Formation permissions on the resource. 1. In the navigation pane, under the heading **Data catalog**, do one of the following: + To view LF-Tags assigned to a database, choose **Databases**. + To view LF-Tags assigned to a table, choose **Tables**. 1. On the **Tables** or **Databases** page, choose the name of the database or table. Then on the details page, scroll down to the **LF-Tags** section. The following screenshot shows the LF-Tags assigned to a `customers` table, which is contained in the `retail` database. The `module` LF-Tag is inherited from the database. The `credit_limit` column has the `level=vp` LF-Tag assigned. ![\[The image is a screenshot of the LF-Tags section of the customers table detail page. The LF-Tags section contains a table with the following columns: Resource, Key, Value, and Inherited from. The table has 3 rows. Above the table is a text entry field with the "Find tags" placeholder text, and an Edit tags button. The paragraph that precedes the image describes the table values shown in the screenshot.\]](http://docs.aws.amazon.com/lake-formation/latest/dg/images/tags-for-resource-2.png) ------ #### [ AWS CLI ] **To view the LF-Tags that are assigned to a resource (AWS CLI)** + Enter a command similar to the following. ``` aws lakeformation get-resource-lf-tags --show-assigned-lf-tags --resource '{ "Table": {"CatalogId":"111122223333", "DatabaseName":"erp", "Name":"sales"}}' ``` The command returns the following output. ``` { "TableTags": [ { "CatalogId": "111122223333", "TagKey": "module", "TagValues": [ "sales" ] }, { "CatalogId": "111122223333", "TagKey": "environment", "TagValues": [ "development" ] } ], "ColumnTags": [ { "Name": "total", "Tags": [ { "CatalogId": "111122223333", "TagKey": "level", "TagValues": [ "director" ] } ] } ] } ``` This output shows only LF-Tags that are explicitly assigned, not inherited. If you want to see all LF-Tags on all columns, including inherited LF-Tags, omit the `--show-assigned-lf-tags` option. ------ # Viewing the resources that a LF-Tag is assigned to You can view all the Data Catalog resources that a particular LF-Tag key is assigned to. To do so, you need the following Lake Formation permissions: + `Describe` or `Associate` on the LF-Tag. + `Describe` or any other Lake Formation permission on the resource. In addition, you need the following AWS Identity and Access Management (IAM) permissions: + `lakeformation:SearchDatabasesByLFTags` + `lakeformation:SearchTablesByLFTags` ------ #### [ Console ] **To view the resources that a LF-Tag is assigned to (console)** 1. Open the Lake Formation console at [https://console.aws.amazon.com/lakeformation/](https://console.aws.amazon.com/lakeformation/). Sign in as a data lake administrator or as a user who meets the requirements listed earlier. 1. In the navigation pane, under **Permissions** and **LF-Tags and permissions**, choose **LF-Tags**. 1. Choose a LF-Tag key (not the option button next to the key name). The LF-Tag details page displays a list of resources that the LF-Tag has been assigned to. ![\[The image is a screenshot of the LF-Tag detail page for the key "module". The LF-Tag detail page has two sections. The top section displays the LF-Tag key and values. The bottom section displays the resources associated with that LF-Tag in a table with the following columns: Key, Values, Resource type, and Resource. The table has 12 rows, but only 7 are shown in the screenshot. The table rows show that the LF-Tag is assigned to a database, two of the tables in the database, and by inheritance, the columns of those tables.\]](http://docs.aws.amazon.com/lake-formation/latest/dg/images/resources-on-tags-2.png) ------ #### [ AWS CLI ] **To view the resources that a LF-Tag is assigned to** + Run a `search-tables-by-lf-tags` or `search-databases-by-lf-tags` command. **Example** The following example lists tables and columns that have the `level=vp` LF-Tag assigned. For each table and column listed, all assigned LF-Tags for the table or column are output, not just the search expression. ``` aws lakeformation search-tables-by-lf-tags --expression TagKey=level,TagValues=vp ``` ------ For more information about the required permissions, see [Lake Formation personas and IAM permissions reference](permissions-reference.md). ## Life cycle of a LF-Tag 1. The LF-Tag creator Michael creates a LF-Tag `module=Customers`. 1. Michael grants `Associate` on the LF-Tag to the data engineer Eduardo. Granting `Associate` implicitly grants `Describe`. 1. Michael grants `Super` on the table `Custs` to Eduardo with the grant option, so that Eduardo can assign LF-Tags to the table. For more information, see [Assigning LF-Tags to Data Catalog resources](TBAC-assigning-tags.md). 1. Eduardo assigns the LF-Tag `module=customers` to the table `Custs`. 1. Michael makes the following grant to data engineer Sandra (in pseudo-code). ``` GRANT (SELECT, INSERT ON TABLES) ON TAGS module=customers TO Sandra WITH GRANT OPTION ``` 1. Sandra makes the following grant to data analyst Maria. ``` GRANT (SELECT ON TABLES) ON TAGS module=customers TO Maria ``` Maria can now run queries on the `Custs` table. **See also** [Metadata access control](access-control-metadata.md) ## Comparison of Lake Formation tag-based access control to IAM attribute-based access control Attribute-based access control (ABAC) is an authorization strategy that defines permissions based on attributes. In AWS, these attributes are called *tags*. You can attach tags to IAM resources, including IAM entities (users or roles) and to AWS resources. You can create a single ABAC policy or small set of policies for your IAM principals. These ABAC policies can be designed to allow operations when the principal's tag matches the resource tag. ABAC is helpful in environments that are growing rapidly and helps with situations where policy management becomes cumbersome. Cloud security and governance teams use IAM to define access policies and security permissions for all resources including Amazon S3 buckets, Amazon EC2 instances and any resources you can reference with an ARN. The IAM policies define broad (coarse-grained) permissions to your data lake resources, for example, to allow or deny access at Amazon S3 bucket or prefix level or database level. For more information about IAM ABAC, see [What is ABAC for AWS?](https://docs.aws.amazon.com/IAM/latest/UserGuide/introduction_attribute-based-access-control.html) in the *IAM User Guide*. For example, you can create three roles with the `project-access` tag key. Set the tag value of the first role to `Dev`, the second to `Marketing`, and the third to `Support`. Assign tags with the appropriate value to resources. You can then use a single policy that allows access when the role and the resource are tagged with the same value for `project-access`. Data governance teams use Lake Formation to define fine-grained permissions to specific data lake resources. LF-Tags are assigned to Data Catalog resources (databases, tables, and columns) and are granted to principals. A principal with LF-Tags that match the LF-Tags of a resource can access that resource. Lake Formation permissions are secondary to IAM permissions. For example, if IAM permissions don't allow a user access to a data lake, Lake Formation doesn't grant access to any resource within that data lake to that user, even if the principal and resource have matching LF-Tags. Lake Formation tag-based access control (LF-TBAC) works with IAM ABAC to provide additional levels of permissions for your Lake Formation data and resources. + **Lake Formation TBAC permissions scale with innovation.** It's no longer necessary for an administrator to update existing policies to allow access to new resources. For example, assume that you use an IAM ABAC strategy with the `project-access` tag to provide access to specific databases within Lake Formation. Using LF-TBAC, the LF-Tag `Project=SuperApp` is assigned to specific tables or columns, and the same LF-Tag is granted to a developer for that project. Through IAM, the developer can access the database, and LF-TBAC permissions grant the developer further access to specific tables or columns within tables. If a new table is added to the project, the Lake Formation administrator only needs to assign the tag to the new table for the developer to be given access to the table. + **Lake Formation TBAC requires fewer IAM policies.** Because you use IAM policies to grant high level access to Lake Formation resources and Lake Formation TBAC for managing more precise data access, you create fewer IAM policies. + **Using Lake Formation TBAC, teams can change and grow quickly.** This is because permissions for new resources are automatically granted based on attributes. For example, if a new developer joins the project, it's easy to grant this developer access by associating the IAM role to the user and then assigning the required LF-Tags to the user. You don't have to change the IAM policy to support a new project or to create new LF-Tags. + **Finer-grained permissions are possible using Lake Formation TBAC.** IAM policies grant access to the top-level resources, such as Data Catalog databases or tables. Using **Lake Formation TBAC**, you can grant access to specific tables or columns that contain specific data values. **Note** IAM tags are not the same as LF-Tags. These tags are not interchangeable. LF-Tags are used to grant Lake Formation permissions and IAM tags are used to define IAM policies. # Managing LF-Tag expressions for metadata access control LF-Tag expressions are logical expressions composed of one or more LF-Tags (key-value pairs) used to grant permissions on AWS Glue Data Catalog resources. LF-Tag expressions allow you to define rules that govern access to your data resources based on their metadata tags. You can save these expressions and reuse them across multiple permission grants, ensuring consistency and making it straight-forward to manage changes to the tag ontology over time. Within a given LF-Tag expression, the tag keys are combined using the AND operation, while the values are combined using the OR operation. For example, the tag expression `content_type:Sales AND location:US` represents resources related to sales data in the US. You can create up to 1000 LF-Tag expressions in an AWS account. These expressions provide a flexible and scalable way to manage permissions based on metadata tags, ensuring that only authorized users or applications can access specific data resources based on the defined tag rules. LF-Tag expressions offer the following benefits: + **Reusability **– By defining and saving LF-Tag expressions, you no longer need to manually replicate the same expressions when assigning permissions to other resources or principals. + **Consistency **– Reusing LF-Tag expressions across multiple permission grants ensures consistency in how permissions are granted and managed. + **Tag ontology management** – LF-Tag expressions help manage changes to the tag ontology over time, as you can update the saved expressions instead of modifying individual permission grants. For more information about tag-based access control, please refer to the [Lake Formation tag-based access control](tag-based-access-control.md). **LF-Tag expression creators** LF-Tag expression creator is a principal who has permissions to create and manage LF-Tag expressions. Data lake administrators can add LF-Tag expression creators using the Lake Formation console, CLI, API, or SDK. LF-Tag expression creators have implicit Lake Formation permissions to create, update, and delete LF-Tag expressions, and to grant LF-Tag expression permissions to other principals. LF-Tag expression creators that are not data lake administrators receive implicit `Alter`, `Drop`, `Describe`, and `Grant with LF-Tag expression` permissions only for expressions they created. Data lake administrators can also grant LF-Tag expression creators grantable `Create LF-Tag expression` permissions. Then, the LF-Tag expression creator can grant the permission to create LF-Tag expressions to other principals. **Topics** + [ ## IAM permissions required to create LF-Tag expressions ](#tag-expression-creator-permissions) + [ ## Add LF-Tag expression creators ](#add-lf-tag-expression-creator) + [ # Creating LF-Tag expressions ](TBAC-creating-tag-expressions.md) + [ # Updating LF-Tag expressions ](TBAC-updating-expressions.md) + [ # Deleting LF-Tag expressions ](TBAC-deleting-expressions.md) + [ # Listing LF-Tag expressions ](TBAC-listing-expressions.md) **See also** [Managing LF-Tag value permissions](TBAC-granting-tags.md) [Granting data lake permissions using the LF-TBAC method](granting-catalog-perms-TBAC.md) [Lake Formation tag-based access control](tag-based-access-control.md) ## IAM permissions required to create LF-Tag expressions You must configure permissions to allow a Lake Formation principal to create LF-Tag expressions. Add the following statement to the permissions policy for the principal that needs to be an LF-Tag expression creator. **Note** Although data lake administrators have implicit Lake Formation permissions to create, update, and delete LF-Tags and LF-Tag expressions, to assign LF-Tags to resources, and to grant LF-Tags and LF-Tag expression permission to principals, data lake administrators also need the following IAM permissions. For more information, see [Lake Formation personas and IAM permissions reference](permissions-reference.md). ``` { "Sid": "Transformational", "Effect": "Allow", "Action": [ "lakeformation:AddLFTagsToResource", "lakeformation:RemoveLFTagsFromResource", "lakeformation:GetResourceLFTags", "lakeformation:ListLFTags", "lakeformation:CreateLFTag", "lakeformation:GetLFTag", "lakeformation:UpdateLFTag", "lakeformation:DeleteLFTag", "lakeformation:SearchTablesByLFTags", "lakeformation:SearchDatabasesByLFTags", "lakeformation:CreateLFTagExpression", "lakeformation:DeleteLFTagExpression", "lakeformation:UpdateLFTagExpression", "lakeformation:GetLFTagExpression", "lakeformation:ListLFTagExpressions", "lakeformation:GrantPermissions", "lakeformation:RevokePermissions", "lakeformation:BatchGrantPermissions", "lakeformation:BatchRevokePermissions" ] } ``` ## Add LF-Tag expression creators LF-Tag expression creators can create and save reusable LF-Tag expressions, update tag key and values, delete expressions, and grant permissions on Data Catalog resources to principals using LF-TBAC method. The LF-Tag expression creator can also grant these permissions to principals. You can create LF-Tag expression creator roles by using the AWS Lake Formation console, the API, or the AWS Command Line Interface (AWS CLI). ------ #### [ console ] **To add an LF-Tag expression creator** 1. Open the Lake Formation console at [https://console.aws.amazon.com/lakeformation/](https://console.aws.amazon.com/lakeformation/). Sign in as a data lake administrator. 1. In the navigation pane, under **Permissions**, choose **LF-Tags and permissions**. 1. Choose the **LF-Tag expressions** tab. 1. In the **LF-Tag expression creators** section, choose **Add LF-Tag expression creators**. ![\[Form to add LF-Tag expression creators with IAM user selection and permissions.\]](http://docs.aws.amazon.com/lake-formation/latest/dg/images/add-lf-tag-expression-creator.png) 1. On the **Add LF-Tag expression creators** page, choose an IAM role or user who has the required permissions to create LF-Tag expressions. 1. Select `Create LF-Tag expression` permission check box. 1. (Optional) To enable the selected principals to grant `Create LF-Tag expression` permission to principals, choose Grantable `Create LF-Tag expression` permission. 1. Choose **Add**. ------ #### [ AWS CLI ] ``` aws lakeformation grant-permissions --cli-input-json file://grantCreate { "Principal": { "DataLakePrincipalIdentifier": "arn:aws:iam::123456789012:user/tag-manager" }, "Resource": { "Catalog": {} }, "Permissions": [ "CreateLFTagExpression" ], "PermissionsWithGrantOption": [ "CreateLFTagExpression" ] } ``` ------ The LF-Tag expression creator role gets the ability to create, update, or delete LF-Tag expressions. | Permission | Description | | --- | --- | | Create | A principal with this permission can add LF-Tag expressions in the data lake. | | Drop | A principal with this permission on an LF-Tag expression can delete an LF-Tag expression from the data lake. | | Alter | A principal with this permission on an LF-Tag expression can update the expression body of an LF-Tag expression. | | Describe | A principal with this permission on an LF-Tag expression can view the contents of an LF-Tag expression. | | Grant with LF-Tag expression | This permission allows the recipient to use the tag expression as the resource when granting data or metadata access permissions. Granting Grant with LF-Tag expression implicitly grants Describe. | | Super | For LF-Tag expressions, the Super permission grants the ability to Describe, Alter, Drop, and grant permissions on the tag expression to other principals. | These permissions are grantable. A principal who has been granted these permissions with the grant option can grant them to other principals. # Creating LF-Tag expressions You need to define all LF-Tags in Lake Formation, and assign them to Data Catalog resources before they can be used to create expressions. An LF-Tag expression consists of one more keys and one or more possible values for each key. After the data lake administrator has set up the required IAM permissions and Lake Formation permissions for the LF-Tag expression creator role, the principal can create reusable LF-Tag expressions. The LF-Tag expression creator gets implicit permissions to update the expression body, and delete the LF-Tag expression. You can create LF-Tag expressions by using the AWS Lake Formation console, the API, or the AWS Command Line Interface (AWS CLI). ------ #### [ Console ] **To create an LF-Tag expression** 1. Open the Lake Formation console at [https://console.aws.amazon.com/lakeformation/](https://console.aws.amazon.com/lakeformation/). Sign in as a principal with LF-Tag expression creator permissions or as data lake administrator. 1. In the navigation pane, under **Permissions****, choose LF-Tags and permissions**. 1. Choose **LF-Tag expressions**. The **Add LF-Tag expressions** page appears. ![\[The page has fields to add a name, description, and a drop down to select expression body. Users can also have the option to grant permissions.\]](http://docs.aws.amazon.com/lake-formation/latest/dg/images/add-tag-expression.png) 1. Enter the following information: + Name – Enter a unique name for the expression. You can't update the expression name. + Description – Provide an optional description for the expression with the details of the expression. + Expression – Create the expression by specifying tag keys and their associated values. You can add up to 50 keys per expression. You must have `Grant with LF-Tags` Lake Formation permission on all tags in expression body. Each key must have at least one value. To enter multiple values, either enter a comma-delimited list and then press **Enter**, or enter one value at a time and choose **Add** after each one. The maximum number of values permitted per key is 1000. Lake Formation uses the AND/OR logic to combine multiple keys and values in an expression. Within a single (key : list of values) pair, the values are combined using the logical OR operator. For example, if the pair is (Department : [Sales, Marketing]), it means the tag matches if the resource has the Department tag with value Sales OR Marketing. When you specify multiple keys, the keys are joined by an AND logical operator. So if the full expression is (Department : [Sales, Marketing]) AND (Location : [US, Canada]), it matches resources that have the Department tag with value Sales OR Marketing, AND also have the Location tag with value US OR Canada. The following is another example with multiple keys and values: LF-Tag expression: (ContentType : [Video, Audio]) AND (Region : [Europe, Asia]) AND (Department : [Engineering, ProductManagement]). This expression would match resources that have: - The ContentType tag with value Video OR Audio AND - The Region tag with value Europe OR Asia AND - The Department tag with value Engineering OR ProductManagement. You can also save a tag expression when granting data lake permissions using LF-Tags. Choose the key and value pairs and choose the **Save as new expression** option. Enter a name that describes the expression. ![\[The page has fields to select expression body and a filed to enter a name.\]](http://docs.aws.amazon.com/lake-formation/latest/dg/images/save-expression-grant.png) 1. (Optional) Next, choose the users/roles, and the permissions on the expression that you want to grant to them in the account. You can also choose grantable permissions that allows the users to grant these permissions to other users in the account. You can't grant cross account permissions on the tag expressions. ![\[The page shows the fields to select permission to grant to other principals.\]](http://docs.aws.amazon.com/lake-formation/latest/dg/images/grant-expression-permissions.png) 1. Choose **Add **. ------ #### [ AWS CLI ] **To create an LF-Tag expression** + Enter a `create-lf-tag-expression` command. The following example creates an LF-Tag expression with the tag `Department` with values `Sales` and `Marketing`, AND the tag `Location` with the value `US`. ``` aws lakeformation create-lf-tag-expression \ -- name "my-tag-expression" \ -- catalog-id "123456789012" \ -- expression '{"Expression":[{"TagKey":"Department","TagValues":["Sales","Marketing"]},{"TagKey":"Location","TagValues":["US"]}]}' ``` This CLI command creates a new LF-Tag expression in the AWS Glue Data Catalog. The expression can be used grant permissions to Data Catalog resources such as databases, tables, views or columns based on their associated tags. In this example, the expression will match resources that have the `Department` key with values `Sales` or `Marketing`, and the `Location` key with the value `US`. ------ As a tag expression creator , the principal gets `Alter` permission on this LF-Tag expression and can update or remove the expression. The LF-Tag expression creator principal can also grant `Alter` permission to another principal to update and remove this expression. # Updating LF-Tag expressions Only data lake administrators, the LF-Tag expression creator, and principals that have `Alter` or `Super` permission on the LF-Tag expression can update an LF-Tag expression. In addition to `Alter` permission, you also need the `lakeformation:UpdateLFTagExpression` IAM permission and `Grant with LF-Tag` permission on all underlying keys-values on the new expression body to update expressions. You update an LF-Tag expression by updating the description, expression body and permissions granted on the expression. You can't change the name of the LF-Tag expression. To change the name, delete the LF-Tag expression and add one with the required parameters. You can update an LF-Tag expression by using the AWS Lake Formation console, the API, or the AWS Command Line Interface (AWS CLI). ------ #### [ Console ] **To update an LF-Tag expression** 1. Open the Lake Formation console at [https://console.aws.amazon.com/lakeformation/](https://console.aws.amazon.com/lakeformation/). Sign in as a data lake administrator, the LF-Tag creator or a principal with `Alter` permission on the LF-Tag. 1. In the navigation pane, under permissions, choose **LF-Tags and permissions**. 1. Choose **LF-Tag expressions** tab. 1. On the **LF-Tag expressions** section, select an LF-Tag expression, and then choose **Edit**. 1. In the **Edit LF-Tag expression** dialog box, update the description and update the expression body by adding or removing keys and values. To add multiple values, in the **Values** field, choose the values from the drop down. 1. Choose **Save**. ------ #### [ AWS CLI ] The update-lf-tag-expression command in Lake Formation allows you to update an existing LF-Tag expression. ``` aws lakeformation update-lf-tag-expression \ -- name expression_name\ -- description new_description \ -- catalog-id catalog_id \ -- expression '{"Expression": [{"TagKey": "tag_key", "TagValues": ["tag_value1", "tag_value2", ...]}]}' ``` Here's what the parameters in the provided command mean: + name – The name of the existing named tag expression that you want to update. + description – A new description for the expression. catalog-id – The ID of the Data Catalog where the named tag expression resides. + expression – The new tag expression string that you want to update the expression with. ------ # Deleting LF-Tag expressions You can delete LF-Tag expressions that are no longer in use. If you have granted permissions to principals on Data Catalog resources using the LF-Tag expression, they will no longer have the permissions. Only data lake administrators, the LF-Tag expression creator, or a principal with `Drop` permission on the LF-Tag expression can delete an LF-Tag expression. In addition to the `Drop` permission, the principal also needs `lakeformation:DeleteLFTagExpression` IAM permission to delete an LF-Tag expression. You can delete an LF-Tag expression by using the AWS Lake Formation console, the API, or the AWS Command Line Interface (AWS CLI). ------ #### [ Console ] **To delete an LF-Tag expression (console)** 1. Open the Lake Formation console at [https://console.aws.amazon.com/lakeformation/](https://console.aws.amazon.com/lakeformation/). Sign in as a data lake administrator, the LF-Tag expression creator, or a principal that has permissions to delete the expression. 1. In the navigation pane, under **Permissions**, choose **LF-Tags and permissions**. 1. Choose the **LF-Tag expression** tab. 1. On the **LF-Tag expressions** section, select an LF-Tag expression, and then choose **Delete**. 1. In the **Delete LF-Tag expression?** dialog box, to confirm the deletion, enter the LF-Tag expression name in the designated field and then choose **Delete**. ------ #### [ AWS CLI ] **To delete an LF-Tag (AWS CLI)** + Enter a `delete-lf-tag-expression` command. Provide the expression name and catalog ID to delete. **Example** The following example deletes the LF-Tag expression with the name `my-tag-expression` from the Data Catalog with ID `123456789012`. The `catalog-id` parameter is optional if you're using the same account as your AWS CLI configuration. After deleting an LF-Tag expression, Lake Formation cleans up the associated permission records for that expression. This includes both individual permission records and aggregate permission records that contain the deleted expression. ``` aws lakeformation delete-lf-tag-expression \ --name "my-tag-expression" \ --catalog-id "123456789012" ``` ------ # Listing LF-Tag expressions You can list the LF-Tag expressions that you have the Describe permissions on. Data lake administrators, LF-Tag expression creators, and Read-only administrators implicitly can see all tag expressions in their account. You can list LF-Tag expressions by using the AWS Lake Formation console, the API, or the AWS Command Line Interface (AWS CLI). ------ #### [ Console ] **To list LF-Tag expressions (console)** 1. Open the Lake Formation console at [https://console.aws.amazon.com/lakeformation/](https://console.aws.amazon.com/lakeformation/). Sign in as the LF-Tag expression creator, as a data lake administrator, or as a principal that has been granted permissions on LF-Tag expressions and that has the `lakeformation:ListLFTagExpressions` IAM permission. 1. In the navigation pane, under ** Permissions**, **LF-Tags and permissions**. 1. Choose **LF-Tag expressions** tab to see the expressions. This section shows the information about the existing LF-Tag expressions, including the expression name, the expression itself with links to the included tags, and options to create, edit, or delete expressions. ------ #### [ AWS CLI ] **To list LF-Tags (AWS CLI)** + To list LF-Tag expressions using the AWS CLI, you can use the list-lf-tag-expressions command. The request syntax is: ``` aws lakeformation list-lf-tag-expressions \ -- catalog-id "123456789012" \ -- max-items "100" \ -- next-token "next-token" ``` Where: + `catalog-id` is the AWS account ID of the Data Catalog you want to list tag expressions for . + `max-items` specifies the maximum number of tag expressions to return. If this parameter is not used, the default value is 100. + `next-token` is a continuation token if the results were truncated in a previous request. The response will include a list of LF-Tag expressions and a next token if applicable. ------ # Managing LF-Tag value permissions You can grant the `Drop`, `Alter` permissions on LF-Tags to principals to manage LF-Tag value expressions. You can also grant `Describe`, `Associate`, and `Grant with LF-Tag expressions` permissions on LF-Tags to principals to view the LF-Tags and assign them to Data Catalog resources (databases, tables, and columns). When LF-Tags are assigned to Data Catalog resources, you can use the Lake Formation tag-based access control (LF-TBAC) method to secure those resources. For more information, see [Lake Formation tag-based access control](tag-based-access-control.md). You can grant these permissions with the grant option so that other principals can grant them. The `Grant with LF-Tag expressions`, `Describe`, and `Associate` permissions are explained in [Add LF-Tag creators](TBAC-adding-tag-creator.md#add-lf-tag-creator). You can grant the `Describe` and `Associate` permissions on a LF-Tag to an external AWS account. A data lake administrator in that account can then grant those permissions to other principals in the account. Principals to whom the data lake administrator in the external account grants the `Associate` permission can then assign LF-Tags to Data Catalog resources that you shared with their account. When granting to an external account, you must include the grant option. You can grant permissions on LF-Tags by using the Lake Formation console, the API, or the AWS Command Line Interface (AWS CLI). **Topics** + [ # Listing LF-Tag permissions using the console ](TBAC-listing-tag-perms-console.md) + [ # Granting LF-Tag permissions using the console ](TBAC-granting-tags-console.md) + [ # Managing LF-Tag permissions using the AWS CLI ](TBAC-granting-revoking-tags-cli.md) For more information see [Managing LF-Tags for metadata access control](managing-tags.md) and [Lake Formation tag-based access control](tag-based-access-control.md). # Listing LF-Tag permissions using the console You can use the Lake Formation console to view the permissions granted on LF-Tags. You must be a LF-Tag creator, a data lake administrator, or have the `Describe` or `Associate` permission on a LF-Tag to see it. **To list LF-Tag permissions (console)** 1. Open the Lake Formation console at [https://console.aws.amazon.com/lakeformation/](https://console.aws.amazon.com/lakeformation/). Sign in as the LF-Tag creator, a data lake administrator, or as a user to whom the `Drop`, `Alter`, `Associate`, or `Describe` permissions on LF-Tags have been granted. 1. In the navigation pane, under **Permissions**, choose **LF-Tags and permissions**, and choose **LF-Tag permissions** section. The **LF-Tag permissions** section shows a table that contains principal, tag keys, values, and permissions. ![\[The page includes a table of permissions with the following columns: Principal, Principal type, Keys, Values, Permissions, and Grantable. There are five rows. To the left of each row is a radio button. Above the table are a search field and these buttons: Refresh, View, Revoke, and Grant. Because no row is initially selected, the View and Revoke buttons are disabled. The values in the first row are: Principal=arn:aws:iam::111122223333:user/datalake_admin, Principal type=IAM user, Keys=environment, Values=All values, Permissions=DESCRIBE, Grantable=DESCRIBE.\]](http://docs.aws.amazon.com/lake-formation/latest/dg/images/list-tag-permissions-page.png) # Granting LF-Tag permissions using the console The following steps explain how to grant permissions on LF-Tags by using the **Grant LF-Tag permissions** page on the Lake Formation console. The page is divided into these sections: + **Permission types** – The type of permission to grant. + **Principals** – The IAM users or roles, or SAML users or roles to grant permissions to. + **LF-Tag key-value pair permissions** permissions – The LF-Tag key-value pairs to grant permissions on. + **LF-Tag permissions** – The LF-Tags to grant permissions on. + **LF-Tag expression permissions** permissions – The LF-Tags to grant permissions on. + **Permissions** – The permissions to grant. ## Open the **Grant LF-Tag permissions** page 1. Open the Lake Formation console at [https://console.aws.amazon.com/lakeformation/](https://console.aws.amazon.com/lakeformation/). Sign in as the LF-Tag creator, a data lake administrator, or as a user LF-Tag permissions or LF-Tag key-value pair permissions on LF-Tags have been granted with the `Grant` option. 1. In the navigation pane, choose **LF-Tags and permissions**, choose **LF-Tag permissions** section. 1. Choose **Grant permissions**. ## Specify the permissions type In the **Permissions type** section, choose a permissions type. LF-Tag permissions Choose the **LF-Tag permissions** to allow principals to update LF-Tag values or delete LF-Tags. LF-Tag key-value pair permissions Choose the **LF-Tag key-value pair permissions** to allow principals to assign LF-Tags to Data Catalog resources, view LF-Tags and values, and grant LF-Tags based permissions on Data Catalog resources to principals. The options available in the following sections depend on the **Permissions type**. LF-Tag expression permissions Choose the **LF-Tag expression permissions** to allow principals to update expressions or delete expressions. ## Specify the principals **Note** You can't grant LF-Tag permissions (`Alter` and `Drop`) to external accounts or principals in another account. In the **Principals** section, choose a principal type and specify principals to grant permissions to. ![\[The principals section contains three tiles that are named in the following text. Each tile contains an option button and text. The IAM users and roles tile is selected, and an IAM users and roles dropdown list is below the tiles.\]](http://docs.aws.amazon.com/lake-formation/latest/dg/images/grant-tags-principals-section.png) **IAM users and roles** Choose one or more users or roles from the **IAM users and roles** list. **SAML users and groups** For **SAML and Quick users and groups**, enter one or more Amazon Resource Names (ARNs) for users or groups federated through SAML, or ARNs for Quick users or groups. Press **Enter** after each ARN. For information about how to construct the ARNs, see [Lake Formation grant and revoke AWS CLI commands](lf-permissions-reference.md#perm-command-format). Lake Formation integration with Quick is supported for Quick Enterprise Edition only. **External accounts** For **AWS account**, enter one or more valid AWS account IDs. Press **Enter** after each ID. An organization ID consists of "o-" followed by 10 to 32 lower-case letters or digits. An organizational unit ID starts with "ou-" followed by 4 to 32 lowercase letters or digits (the ID of the root that contains the OU). This string is followed by a second "-" dash and 8 to 32 additional lowercase letters or digits. For IAM principal, enter the ARN for the IAM user or role. ## Specify the LF-Tags To grant permissions on LF-Tags, in the **LF-Tag permissions** section, specify the LF-Tags to grant permissions on. ![\[The LF-Tags section shows two rows of fields, where each row, going from left to right, has a Key field, a Value field, and a Remove button. The Value field is a drop-down list. Beneath the two rows of fields is an Add LF-Tag button. The first row shows "module" in the Key field, and beneath the Values field are two small tiles that contain Orders and Sales, respectively, indicating that the use has chosen Orders and Sales as the values for the key module. Each tile has an X that you can click (like a close box) to delete the tile. The second row if fields is empty.\]](http://docs.aws.amazon.com/lake-formation/latest/dg/images/grant-tags-tags-section-2.png) + Choose one or more LF-Tag using the drop-down. ## Specify the LF-Tag key-value pairs 1. To grant permissions on LF-Tag key-value pairs, (you need to first choose choose **LF-Tag key-value pair permissions** as the **Permission type**) choose **Add LF-Tag key-value pair** to reveal the first row of fields for specifying LF-Tag key and values. ![\[Interface for adding LF-Tag key-value pairs and setting associated permissions.\]](http://docs.aws.amazon.com/lake-formation/latest/dg/images/tag-key-value-pair.png) 1. Position the cursor in the **Key** field, optionally start typing to narrow down the selection list, and select a LF-Tag key. 1. In the **Values** list, select one or more values, and then press **Tab** or click or tap outside the field to save the selected values. **Note** If one of the rows in the **Values** list has focus, pressing **Enter** selects or clears the check box. The selected values appear as tiles below the **Values** list. Choose the ✖ to remove a value. Choose **Remove** to remove the entire LF-Tag. 1. To add another LF-Tag, choose **Add LF-Tag** again, and repeat the previous two steps. ## Specify the LF-Tag expressions 1. To grant permissions on LF-Tag expressions, (you need to first choose choose **LF-Tag expression permissions** as the **Permission type**). ![\[Permission type selection interface with LF-Tag expression permissions highlighted.\]](http://docs.aws.amazon.com/lake-formation/latest/dg/images/tag-expression.png) 1. Choose a LF-Tag expression. 1. The selected expressions appear as tiles below the **LF-Tag expressions** list. Choose the ✖ to remove an expression. 1. To add another LF-Tag expression, choose another expression. ## Specify the permissions This section shows either the **LF-Tag permissions** or the **LF-Tag value permissions** based on the **Permission type** you chose in the previous step. Depending on the **Permission type** you chose to grant, select the **LF-Tag permissions** or **LF-Tag key-value pair permissions**, and grantable permissions. 1. Under **LF-Tag permissions**, select the permissions to grant. Granting **Drop** and **Alter** implicitly grants **Describe**. You need to grant **Alter** and **Drop** permissions on all tag values. 1. Under **LT-Tag key-value value permissions**, select the permissions to grant. Granting **Associate** implicitly grants **Describe**. Choose **Grant with LF-Tag expression** to allow the grant recipient to grant or revoke access permissions on Data Catalog resources using LF-TBAC method. 1. Under **LF-Tag expression permissions**, select the permissions to grant. Granting **Drop** and **Alter** implicitly grants **Describe**. Granting **Super** permission, grants all available permissions. 1. (Optional) Under **Grantable permissions**, select the permissions that the grant recipient can grant to other principals in their AWS account. 1. Choose **Grant**. # Managing LF-Tag permissions using the AWS CLI You can grant, revoke, and list permissions on LF-Tags by using the AWS Command Line Interface (AWS CLI). **To list LF-Tag permissions (AWS CLI)** + Enter a `list-permissions` command. You must be the LF-Tag creator, a data lake administrator, or have the `Drop`, `Alter`, `Describe`, `Associate`, `Grant with LF-Tag permissions` permission on a LF-Tag to see it. The following command requests all LF-Tags that you have permissions on. ``` aws lakeformation list-permissions --resource-type LF_TAG ``` The following is sample output for a data lake administrator, who sees all LF-Tags granted to all principals. Non-administrative users see only LF-Tags granted to them. LF-Tag permissions granted from an external account appear on a separate results page. To see them, repeat the command and supply the `--next-token` argument with the token returned from the previous command run. ``` { "PrincipalResourcePermissions": [ { "Principal": { "DataLakePrincipalIdentifier": "arn:aws:iam::111122223333:user/datalake_admin" }, "Resource": { "LFTag": { "CatalogId": "111122223333", "TagKey": "environment", "TagValues": [ "*" ] } }, "Permissions": [ "ASSOCIATE" ], "PermissionsWithGrantOption": [ "ASSOCIATE" ] }, { "Principal": { "DataLakePrincipalIdentifier": "arn:aws:iam::111122223333:user/datalake_user1" }, "Resource": { "LFTag": { "CatalogId": "111122223333", "TagKey": "module", "TagValues": [ "Orders", "Sales" ] } }, "Permissions": [ "DESCRIBE" ], "PermissionsWithGrantOption": [] }, ... ], "NextToken": "eyJzaG91bGRRdWVy...Wlzc2lvbnMiOnRydWV9" } ``` You can list all grants for a specific LF-Tag key. The following command returns all permissions granted on the LF-Tag `module`. ``` aws lakeformation list-permissions --resource-type LF_TAG --resource '{ "LFTag": {"CatalogId":"111122223333","TagKey":"module","TagValues":["*"]}}' ``` You can also list LF-Tag values granted to a specific principal for a specific LF-Tag. When supplying the `--principal` argument, you must supply the `--resource` argument. Therefore, the command can only effectively request the values granted to a specific principal for a specific LF-Tag key. The following command shows how to do this for the principal `datalake_user1` and the LF-Tag key `module`. ``` aws lakeformation list-permissions --principal DataLakePrincipalIdentifier=arn:aws:iam::111122223333:user/datalake_user1 --resource-type LF_TAG --resource '{ "LFTag": {"CatalogId":"111122223333","TagKey":"module","TagValues":["*"]}}' ``` The following is sample output. ``` { "PrincipalResourcePermissions": [ { "Principal": { "DataLakePrincipalIdentifier": "arn:aws:iam::111122223333:user/datalake_user1" }, "Resource": { "LFTag": { "CatalogId": "111122223333", "TagKey": "module", "TagValues": [ "Orders", "Sales" ] } }, "Permissions": [ "ASSOCIATE" ], "PermissionsWithGrantOption": [] } ] } ``` **To grant permissions on LF-Tags (AWS CLI)** 1. Enter a command similar to the following. This example grants to user `datalake_user1` the `Associate` permission on the LF-Tag with the key `module`. It grants permissions to view and assign all values for that key, as indicated by the asterisk (\$1). ``` aws lakeformation grant-permissions --principal DataLakePrincipalIdentifier=arn:aws:iam::111122223333:user/datalake_user1 --permissions "ASSOCIATE" --resource '{ "LFTag": {"CatalogId":"111122223333","TagKey":"module","TagValues":["*"]}}' ``` Granting the `Associate` permission implicitly grants the `Describe` permission. The next example grants `Associate` to the external AWS account 1234-5678-9012 on the LF-Tag with the key `module`, with the grant option. It grants permissions to view and assign only the values `sales` and `orders`. ``` aws lakeformation grant-permissions --principal DataLakePrincipalIdentifier=123456789012 --permissions "ASSOCIATE" --permissions-with-grant-option "ASSOCIATE" --resource '{ "LFTag": {"CatalogId":"111122223333","TagKey":"module","TagValues":["sales", "orders"]}}' ``` 1. Granting the `GrantWithLFTagExpression` permission implicitly grants the `Describe` permission. The next example grants `GrantWithLFTagExpression` to a user on the LF-Tag with the key `module`, with the grant option. It grants permissions to view and grant permissions on Data Catalog resources using only the values `sales` and `orders`. ``` aws lakeformation grant-permissions --principal DataLakePrincipalIdentifier=111122223333 --permissions "GrantWithLFTagExpression" --permissions-with-grant-option "GrantWithLFTagExpression" --resource '{ "LFTag": {"CatalogId":"111122223333","TagKey":"module","TagValues":["sales", "orders"]}}' ``` 1. The next example grants `Drop` permissions to a user on the LF-Tag with the key `module`, with the grant option. It grants permissions to delete the LF-Tag. To delete a LF-Tag, you need permissions on all values for that key. ``` aws lakeformation grant-permissions --principal DataLakePrincipalIdentifier=111122223333 --permissions "DROP" --permissions-with-grant-option "DROP" --resource '{ "LFTag": {"CatalogId":"111122223333","TagKey":"module","TagValues":["*"]}}' ``` 1. The next example grants `Alter` permissions to the user on the LF-Tag with the key `module`, with the grant option. It grants permissions to delete the LF-Tag. To update a LF-Tag, you need permissions on all values for that key. ``` aws lakeformation grant-permissions --principal DataLakePrincipalIdentifier=111122223333 --permissions "ALTER" --permissions-with-grant-option "ALTER" --resource '{ "LFTag": {"CatalogId":"111122223333","TagKey":"module","TagValues":["*"]}}' ``` **To revoke permissions on LF-Tags (AWS CLI)** + Enter a command similar to the following. This example revokes the `Associate` permission on the LF-Tag with the key `module` from user `datalake_user1`. ``` aws lakeformation revoke-permissions --principal DataLakePrincipalIdentifier=arn:aws:iam::111122223333:user/datalake_user1 --permissions "ASSOCIATE" --resource '{ "LFTag": {"CatalogId":"111122223333","TagKey":"module","TagValues":["*"]}}' ``` # Granting data lake permissions using the LF-TBAC method You can grant the `DESCRIBE` and `ASSOCIATE` Lake Formation permissions on LF-Tags to principals so that they can view the LF-Tags and assign them to Data Catalog resources (databases, tables, views, and columns). When LF-Tags are assigned to Data Catalog resources, you can use the Lake Formation tag-based access control (LF-TBAC) method to secure those resources. For more information, see [Lake Formation tag-based access control](tag-based-access-control.md). At first, only the data lake administrator can grant these permissions. If the data lake administrator grants these permissions with the grant option, other principals can grant them. The `DESCRIBE` and `ASSOCIATE` permissions are explained in [Lake Formation tag-based access control best practices and considerations](lf-tag-considerations.md). You can grant the `DESCRIBE` and `ASSOCIATE` permissions on a LF-Tag to an external AWS account. A data lake administrator in that account can then grant those permissions to other principals in the account. Principals to whom the data lake administrator in the external account grants the `ASSOCIATE` permission can then assign LF-Tags to Data Catalog resources that you shared with their account. When granting to an external account, you must include the grant option. You can grant permissions on LF-Tags by using the AWS Lake Formation console, the API, or the AWS Command Line Interface (AWS CLI). **Note** The following steps are not needed for S3 Tables catalogs. You can use LF-Tags to grant permissions on existing S3 Tables catalogs without deleting and recreating them. **Enabling LF-Tags support for existing federated catalogs that uses Lake Formation permissions** Follow these steps, if you have existing federated catalogs that are using Lake Formation permissions, such as Amazon Redshift or Amazon DynamoDB catalogs that were created before LF-Tags support was available for federated catalogs. 1. Delete the existing catalog – Call the `deleteCatalog` API operation to remove the existing federated catalog that uses Lake Formation permissions. 1. Create a new federated catalog – Create a new catalog and point the new catalog to your existing namespace/datashare. Use a new name for the catalog – This process updates your pre-existing federated catalogs to support LF-Tag functionality. If you want to use the same catalog name, contact AWS support team for assistance. **Topics** + [ ## Granting Data Catalog permissions ](#granting-cat-perms-TBAC-console) **See also** [Managing LF-Tag value permissions](TBAC-granting-tags.md) [Managing LF-Tags for metadata access control](managing-tags.md) [Lake Formation tag-based access control](tag-based-access-control.md) ## Granting Data Catalog permissions Granting Data Catalog permissions using the LF-TBAC Method Use the Lake Formation console or AWS CLI to grant Lake Formation permissions on Data Catalog databases, tables, views, and columns using the Lake Formation tag-based access control (LF-TBAC) method. ------ #### [ Console ] The following steps explain how to grant permissions by using the Lake Formation tag-based access control (LF-TBAC) method and the **Grant data lake permissions** page on the Lake Formation console. The page is divided into the following sections: + **Principals** – The users, roles, and AWS accounts to grant permissions to. + **LF-Tags or catalog resources** – The databases, tables, or resource links to grant permissions on. + **Permissions** – The Lake Formation permissions to grant. 1. **Open the Grant data lake permissions page.** Open the AWS Lake Formation console at [https://console.aws.amazon.com/lakeformation/](https://console.aws.amazon.com/lakeformation/), and sign in as a data lake administrator or as a user who has been granted Lake Formation permissions on Data Catalog resources through LF-TBAC with the grant option. In the navigation pane, under **Permissions**, choose **Data lake permissions**. Then choose **Grant**. 1. **Specify the principals.** In the **Principals** section, choose a principal type and then specify principals to grant permissions to. ![\[The Principals section contains four tiles that are named in the following text. Each tile contains a option button and text. The IAM Identity Center tile is selected, and users and groups dropdown list is below the tiles.\]](http://docs.aws.amazon.com/lake-formation/latest/dg/images/identity-center-grant-perm.png) **IAM users and roles** Choose one or more users or roles from the **IAM users and roles** list. **IAM Identity Center ** Choose one or more users or from the **Users and groups** list. **SAML users and groups** For **SAML and Quick users and groups**, enter one or more Amazon Resource Names (ARNs) for users or groups federated through SAML, or ARNs for Quick users or groups. Press Enter after each ARN. For information about how to construct the ARNs, see [Lake Formation grant and revoke AWS CLI commands](lf-permissions-reference.md#perm-command-format). Lake Formation integration with Quick is supported for Quick Enterprise Edition only. **External accounts** For **AWS accounts, AWS organization**, or **IAM principal** enter one or more valid AWS account IDs, organization IDs, organizational unit IDs, or ARN for the IAM user or role. Press **Enter** after each ID. An organization ID consists of "o-" followed by 10 to 32 lower-case letters or digits. An organizational unit ID starts with "ou-" followed by 4 to 32 lowercase letters or digits (the ID of the root that contains the OU). This string is followed by a second "-" dash and 8 to 32 additional lowercase letters or digits. 1. **Specify the LF-Tags.** Ensure that the **Resources matched by LF-Tags** option is chosen. Choose **LF-Tag key-value pairs** or **Saved LF-Tag expressions**. 1. If you choose the **LF-Tag key-value pairs** option, choose the keys and values. If you choose more than one value, you are creating a LF-Tag expression with an `OR` operator. This means that if any of the LF-Tag values match a LF-Tag assigned to a Data Catalog resource, you are granted permissions on the resource. ![\[The LF-Tag or catalog resources section contains two tiles arranged horizontally, where each tile contains an option button and descriptive text. The options are Resources matched by LF-Tags (recommended), and Named data catalog resources. Resources matched by LF-Tags is selected. Below the tiles are a Key field and a Values field arranged horizontally. The Key field contains "module" and the Values field is a dropdown list that contains three entries: Orders, Sales, and Customers. Each entry has a check box associated. The check box for Customers is selected. To the right of these two fields is a Remove button. At the bottom is an Add LF-Tag button, indicating that you can add another row containing the Key and Values fields and a Remove button.\]](http://docs.aws.amazon.com/lake-formation/latest/dg/images/grant-data-permissions-tags-2.png) 1. (Optional) Choose **Add LF-Tag key-value pair** again to specify another LF-Tag. If you specify more than one LF-Tag, you are creating a LF-Tag expression with an `AND` operator. The principal is granted permissions on a Data Catalog resource only if the resource was assigned a matching LF-Tag for each LF-Tag in the LF-Tag expression. 1. Choose **Save as a new expression** option to reuse the expression. You need `Create LF-Tag expression` to save expressions. For more information about LF-Tag expressions, see [Managing LF-Tag expressions for metadata access control](managing-tag-expressions.md). 1. **Specify the permissions.** Specify the permissions that you want to grant the principal on matching Data Catalog resources. Matching resources are those resources that were assigned LF-Tags that match one of the LF-Tag expressions granted to the principal. You can specify the permissions to grant on matching databases, matching tables, and matching views. ![\[Two sections of the page are shown. The Database permissions section contains check boxes for database permissions and grantable permissions. Beneath the Database section, the Table permissions section shows the check boxes for table permissions and grantable permissions.\]](http://docs.aws.amazon.com/lake-formation/latest/dg/images/grant-TBAC-DB-table-permissions.png) Under **Database permissions**, select the database permissions to grant to the principal on matching databases. Under **Table permissions**, select the table or view permissions to grant to the principal on matching tables and views. You can also choose `Select`, `Describe`, and `Drop` permissions from the **Table permissions** to apply on views. 1. Choose **Grant**. ------ #### [ AWS CLI ] You can use the AWS Command Line Interface (AWS CLI) and the Lake Formation tag-based access control (LF-TBAC) method to grant Lake Formation permissions on Data Catalog databases, tables, and columns. **Granting data lake permissions using the AWS CLI and the LF-TBAC method** + Use the `grant-permissions` command. **Example** The following example grants the LF-Tag expression "`module=*`" (all values of the LF-Tag key `module`) to user `datalake_user1`. That user will have the `CREATE_TABLE` permission on all matching databases—databases that have been assigned the LF-Tag with the key `module`, with any value. ``` aws lakeformation grant-permissions --principal DataLakePrincipalIdentifier=arn:aws:iam::111122223333:user/datalake_user1 --permissions "CREATE_TABLE" --resource '{ "LFTagPolicy": {"CatalogId":"111122223333","ResourceType":"DATABASE","Expression":[{"TagKey":"module","TagValues":["*"]}]}}' ``` **Example** The next example grants the LF-Tag expression "`(level=director) AND (region=west OR region=south)`" to user `datalake_user1`. That user will have the `SELECT`, `ALTER`, and `DROP` permissions with the grant option on matching tables—tables that have been assigned both `level=director` and (`region=west` or `region=south`). ``` aws lakeformation grant-permissions --principal DataLakePrincipalIdentifier=arn:aws:iam::111122223333:user/datalake_user1 --permissions "SELECT" "ALTER" "DROP" --permissions-with-grant-option "SELECT" "ALTER" "DROP" --resource '{ "LFTagPolicy": {"CatalogId":"111122223333","ResourceType":"TABLE","Expression": [{"TagKey": "level","TagValues": ["director"]},{"TagKey": "region","TagValues": ["west", "south"]}]}}' ``` **Example** This next example grants the LF-Tag expression "`module=orders`" to the AWS account 1234-5678-9012. The data lake administrator in that account can then grant the "`module=orders`" expression to principals in their account. Those principals will then have the `CREATE_TABLE` permission on matching databases owned by account 1111-2222-3333 and shared with account 1234-5678-9012 by using either the named resource method or the LF-TBAC method. ``` aws lakeformation grant-permissions --principal DataLakePrincipalIdentifier=123456789012 --permissions "CREATE_TABLE" --permissions-with-grant-option "CREATE_TABLE" --resource '{ "LFTagPolicy": {"CatalogId":"111122223333","ResourceType":"DATABASE","Expression":[{"TagKey":"module","TagValues":["orders"]}]}}' ``` ------ # Attribute-based access control Attribute-based access control In AWS Lake Formation, you can grant access on AWS Glue Data Catalog objects such as catalogs, databases, tables, and data filters using attributes that are IAM tags and session tags associated with IAM entities such as roles and users. For more information about using session tags, see [assume-role](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/sts/assume-role.html) in the AWS CLI user guide. Attribute-based access control (ABAC) is an authorization strategy that defines permissions based on attributes. AWS calls these attributes *tags*. You can use ABAC to grant access to principals within the same account or in another account on the Data Catalog resources. Any IAM principal with matching IAM tag or session tag keys and values gains access to the resource. You must have grantable permissions on the resources to make these grants. ABAC allows you to grant access to multiple users at the same time. When new users join the organization, their access to data can be automatically determined based on their attributes, such as their job function or department, without requiring administrators to manually assign specific roles or permissions. By using attributes instead of roles, ABAC provides a more streamlined and maintainable way to manage data access across diverse systems and environments, ultimately enhancing data governance and compliance. For more information about defining attributes, see [Define permissions based on attributes with ABAC authorization](https://docs.aws.amazon.com/IAM/latest/UserGuide/introduction_attribute-based-access-control.html). For information on limitations, considerations, and supported AWS Regions, see [Attribute-based access control considerations, limitations, and supported regions](abac-considerations.md). **Topics** + [ # Prerequisites for granting permissions using attributes ](abac-prerequisites.md) + [ # Granting permissions using attribute-based access control ](abac-granting-permissions.md) # Prerequisites for granting permissions using attributes Prerequisites To grant permissions using attribute-based access control (ABAC), you must complete the following prerequisites: + Update the **Data Catalog** **settings **to enable Lake Formation permissions for Data Catalog objects. For more information, see the [Change the default permission model or use hybrid access mode](https://docs.aws.amazon.com/lake-formation/latest/dg/initial-lf-config.html#setup-change-cat-settings) section. + Set the cross-account version settings to two or higher. + [Attach attributes](https://docs.aws.amazon.com/IAM/latest/UserGuide/tutorial_attribute-based-access-control.html) to the IAM entities that require access. + Only a data lake administrator or an IAM user with the required permissions can grant access on Data Catalog objects. For more information on required permissions, see [IAM permissions](https://docs.aws.amazon.com/lake-formation/latest/dg/required-permissions-for-grant.html). # Granting permissions using attribute-based access control Granting permissions This topic describes the steps you need to follow to grant attribute-based access permissions on Data Catalog resources. You can use the Lake Formation console or the AWS Command Line Interface (AWS CLI). ## Granting permissions using ABAC (AWS Management Console) 1. Open the Lake Formation console at [https://console.aws.amazon.com/lakeformation/](https://console.aws.amazon.com/lakeformation/), and sign in as a data lake administrator, the resource creator, or an IAM user who has **Grantable permissions** on the resource. 1. Do one of the following: + In the navigation pane, under **Permissions**, choose **Data lake permissions**. Then choose **Grant**. + In the navigation pane, choose **Catalogs** under **Data Catalog**. Then, choose a catalog object (catalogs, databases, tables, and data filters), and from the **Actions** menu under **Permissions**, and choose **Grant**. 1. On the **Grant permissions** page, choose **Principals by attribute**. 1. Specify the attribute key and value(s). If you choose more than one value, you are creating an attribute expression with an `OR` operator. This means that if any of the attribute tag values assigned to an IAM role or user match, the role/user gains access permissions on the resource. If you specify more than one attribute tag, you are creating an attribute expression with an `AND` operator. The principal is granted permissions on a Data Catalog resource only if the IAM role/user was assigned a matching tag for each attribute tag in the attribute expression. Review the resulting Cedar policy expression shown in the console. ![\[In the Grant permissions dialog box, an attribute expression is created.\]](http://docs.aws.amazon.com/lake-formation/latest/dg/images/abac-grant-permissions.png) 1. Choose the permission scope. If the grantees belong to an external account, choose **External account** and enter the AWS account ID. 1. Next, choose the Data Catalog account or in external accounts. You must have corresponding grantable permissions on the resources to successfully complete the permission grants. 1. Specify which actions you want to allow for principals (users or roles) that have matching attributes perform. Access is granted to IAM entities that have been assigned tags and values that match at least one of your specified attribute expressions. Review the Cedar policy expression in the console. For more information about Cedar, see [What is Cedar? \$1 Cedar Policy Language Reference GuideLink](https://docs.cedarpolicy.com/). 1. Next choose the Data Catalog resources to grant access. you can define these permissions for various Data Catalog resources, including catalogs, databases, tables, and data filters. 1. Choose **Grant**. This approach allows you to control access based on attributes, ensuring that only users or roles with the appropriate tags can perform specific actions on the designated resources. ## Granting permissions using ABAC (AWS CLI) The following example shows an attribute expression that must be met for receiving all available permissions on the resource. You can alternatively specify individual permissions such as `Select`, `Describe`, or `Drop`. The expression uses Cedar policy expression. For more information about Cedar, see [What is Cedar? \$1 Cedar Policy Language Reference GuideLink](https://docs.cedarpolicy.com/). This condition checks if the IAM principal has a `department` tag, and the `department` tag value equals `sales`. ``` aws lakeformation grant-permissions --principal '{"DataLakePrincipalIdentifier": "111122223333:IAMPrincipals"}' \ --resource '{"Database": {"CatalogId": 111122223333, "Name": "abac-db"}}' \ --permissions ALL \ --condition '{"Expression": "context.iam.principalTags.hasTag(\"department\") \ && context.iam.principalTags.getTag(\"department\") == \"sales\""}' ``` # Permissions example scenario The following scenario helps demonstrate how you can set up permissions to secure access to data in AWS Lake Formation. Shirley is a data administrator. She wants to set up a data lake for her company, AnyCompany. Currently, all data is stored in Amazon S3. John is a marketing manager and needs write access to customer purchasing information (contained in `s3://customerPurchases`). A marketing analyst, Diego, joins John this summer. John needs the ability to grant Diego access to perform queries on the data without involving Shirley. Mateo, from finance, needs access to query accounting data (for example, `s3://transactions`). He wants to query the transactions data in tables in a database (`Finance_DB`) that the finance team uses. His manager, Arnav, can give him access to the `Finance_DB`. Although he shouldn’t be able to modify accounting data, he needs the ability to convert data into a format (schema) suitable for forecasting. This data will be stored in a separate bucket (`s3://financeForecasts`) that he can modify. To summarize: + Shirley is the data lake administrator. + John requires `CREATE_DATABASE` and `CREATE_TABLE` permission to create new databases and tables in the Data Catalog. + John also requires `SELECT`, `INSERT`, and `DELETE` permissions on tables he creates. + Diego requires `SELECT` permission on the table to run queries. The employees of AnyCompany perform the following actions to set up permissions. The API operations shown in this scenario show a simplified syntax for clarity. 1. Shirley registers the Amazon S3 path containing customer purchasing information with Lake Formation. ``` RegisterResource(ResourcePath("s3://customerPurchases"), false, Role_ARN ) ``` 1. Shirley grants John access to the Amazon S3 path containing customer purchasing information. ``` GrantPermissions(John, S3Location("s3://customerPurchases"), [DATA_LOCATION_ACCESS]) ) ``` 1. Shirley grants John permission to create databases. ``` GrantPermissions(John, catalog, [CREATE_DATABASE]) ``` 1. John creates the database `John_DB`. John automatically has `CREATE_TABLE` permission on that database because he created it. ``` CreateDatabase(John_DB) ``` 1. John creates the table `John_Table` pointing to `s3://customerPurchases`. Because he created the table, he has all permissions on it, and can grant permissions on it. ``` CreateTable(John_DB, John_Table) ``` 1. John allows his analyst, Diego, access to the table `John_Table`. ``` GrantPermissions(Diego, John_Table, [SELECT]) ``` 1. John allows his analyst, Diego, access to the `s3://customerPurchases/London/`. Because Shirley already registered `s3://customerPurchases`, its subfolders are registered with Lake Formation. ``` GrantDataLakePrivileges( 123456789012/datalake, Diego, [DATA_LOCATION_ACCESS], [], S3Location("s3://customerPurchases/London/") ) ``` 1. John allows his analyst, Diego, to create tables in database `John_DB`. ``` GrantDataLakePrivileges( 123456789012/datalake, Diego, John_DB, [CREATE_TABLE], [] ) ``` 1. Diego creates a table in `John_DB` at `s3://customerPurchases/London/` and automatically gets `ALTER`, `DROP`, `SELECT`, `INSERT`, and `DELETE` permissions. ``` CreateTable( 123456789012/datalake, John_DB, Diego_Table ) ``` # Data filtering and cell-level security in Lake Formation Data filtering and cell-level security When you grant Lake Formation permissions on a Data Catalog table, you can include data filtering specifications to restrict access to certain data in query results and engines integrated with Lake Formation. Lake Formation uses data filtering to achieve column-level security, row-level security, and cell-level security. You can define and apply data filters on nested columns if your source data contains nested structures. With the data filtering capabilities of Lake Formation, you can implement the following levels of data security. **Column-level security** Granting permissions on a Data Catalog table with column-level security (column filtering) allows users to view only specific columns and nested columns that they have access to in the table. Consider a `persons` table that is used in multiple applications for a large multi-region communications company. Granting permissions on Data Catalog tables with column filtering can restrict users who don't work in the HR department from seeing personally identifiable information (PII) such as a social security number or birth date. You can also define security policies and grant access to only partial sub-structures of nested columns. **Row-level security** Granting permissions on a Data Catalog table with row-level security (row filtering) allows users to view only specific rows of data that they have access to in the table. The filtering is based on the values of one or more columns. You can include nested column structures when defining row-filter expressions. For example, if different regional offices of the communications company have their own HR departments, you can limit the person records that HR employees can see to only records for employees in their region. **Cell-level security** Cell-level security combines row filtering and column filtering for a highly flexible permissions model. If you view the rows and columns of a table as a grid, by using cell-level security, you can restrict access to individual elements (cells) of the grid anywhere in the two dimensions. That is, you can restrict access to different columns depending on the row. This is illustrated by the following diagram, in which restricted columns are shaded. ![\[A grid is shown with 5 rows and 6 columns. The rows and columns have headers like Col1, Col2, Row1, Row2, and so on. The grid cells with the following coordinates are shaded: R3,C1; R3,C2; R3,C3; R5,C1; R5;C2; R5,C5; R5,C6.\]](http://docs.aws.amazon.com/lake-formation/latest/dg/images/cells-diagram.png) Continuing the example of the persons table, you can create a *data filter* at the cell-level that restricts access to the street address column if the row has the country column set to "UK", but allows access to the street address column if the row has the country column set to "US". Filters apply only to read operations. Therefore, you can grant only the `SELECT` Lake Formation permission with filters. **Cell-level security on nested columns** Lake Formation allows you to define and apply data filters with cell-level security on nested columns. However, the integrated analytical engines such as Amazon Athena, Amazon EMR, and Amazon Redshift Spectrum support executing queries against Lake Formation managed nested tables with row and column-level security. For limitations, see [Data filtering limitations](data-filtering-notes.md). **Topics** + [ ## Data filters in Lake Formation ](#data-filters-about) + [ # PartiQL support in row filter expressions ](partiql-support.md) + [ # Permissions required for querying tables with cell-level filtering ](row-filtering-prereqs.md) + [ # Managing data filters ](managing-filters.md) ## Data filters in Lake Formation Data filters You can implement column-level, row-level, and cell-level security by creating *data filters*. You select a data filter when you grant the `SELECT` Lake Formation permission on tables. If your table contains nested column structures, you can define a data filter by including or excluding the child columns and define row-level filter expressions on nested attributes. Each data filter belongs to a specific table in your Data Catalog. A data filter includes the following information: + Filter name + The Catalog IDs of the table associated with the filter + Table name + Name of the database that contains the table + Column specification – a list of columns and nested columns (with `struct` datatypes) to include or exclude in query results. + Row filter expression – an expression that specifies the rows to include in query results. With some restrictions, the expression has the syntax of a `WHERE` clause in the PartiQL language. To specify all rows, choose **Access to all rows** under **Row-level access** in the console or use `AllRowsWildcard` in API calls. For more information about what is supported in row filter expressions, see [PartiQL support in row filter expressions](partiql-support.md). The level of filtering that you get depends on how you populate the data filter. + When you specify the "all columns" wildcard and provide a row filter expression, you are establishing row-level security (row filtering) only. + When you include or exclude specific columns and nested columns, and specify "all rows" using the all-rows wildcard, you are establishing column-level security (column filtering) only. + When you include or exclude specific columns and also provide a row filter expression, you are establishing cell-level security (cell filtering). The following screenshot from the Lake Formation console shows a data filter that performs cell-level filtering. For queries against the `orders` table, it restricts access to the `customer_name` column and the query results return only rows where the `product_type` column contains 'pharma'. ![\[The data filter window contains these fields, arranged vertically: Data filter name; Target database; Target table; Option button group with the options Access to all columns, Include columns, and Exclude columns; Select columns (drop-down list); Row filter expression (multi-line text box). The Exclude columns option is selected, the customer_name column is selected for exclusion, and the Row filter expression field contains 'product_type='pharma'.\]](http://docs.aws.amazon.com/lake-formation/latest/dg/images/data-filter-sample-pharma.png) Note the use of single quotes to enclose the string literal, `'pharma'`. You can use the Lake Formation console to create this data filter, or you can supply the following request object to the `CreateDataCellsFilter` API operation. ``` { "Name": "restrict-pharma", "DatabaseName": "sales", "TableName": "orders", "TableCatalogId": "111122223333", "RowFilter": {"FilterExpression": "product_type='pharma'"}, "ColumnWildcard": { "ExcludedColumnNames": ["customer_name"] } } ``` You can create as many data filters as you need for a table. In order to do so, you require `SELECT` permission with the grant option on a table. Data Lake Administrators by default have the permission to create *data filters* on all tables in that account. You typically only use a subset of the possible data filters when granting permissions on the table to a principal. For example, you could create a second data filter for the `orders` table that is a row-security-only data filter. Referring to the preceding screenshot, you could choose the **Access to all columns** option and include a row filter expression of `product_type<>pharma`. The name of this data filter could be `no-pharma`. It restricts access to all rows that have the `product_type` column set to 'pharma'. The request object for the `CreateDataCellsFilter` API operation for this data filter is the following. ``` { "Name": "no-pharma", "DatabaseName": "sales", "TableName": "orders", "TableCatalogId": "111122223333", "RowFilter": {"FilterExpression": "product_type<>'pharma'"}, "ColumnNames": ["customer_id", "customer_name", "order_num" "product_id", "purchase_date", "product_type", "product_manufacturer", "quantity", "price"] } ``` You could then grant `SELECT` on the `orders` table with the `restrict-pharma` data filter to an administrative user, and `SELECT` on the `orders` table with the `no-pharma` data filter to non-administrative users. For users in the healthcare sector, you would grant `SELECT` on the `orders` table with full access to all rows and columns (no data filter), or perhaps with yet another data filter that restricts access to pricing information. You can include or exclude nested columns when specifying column-level and row-level security within a data filter. In the following example, access to the `product.offer` field is specified using qualified column names (wrapped in double quotes). This is important for nested fields in order to avoid errors occurring when column names contain special characters, and to maintain backward compatibility with top level column-level security definitions. ``` { "Name": "example_dcf", "DatabaseName": "example_db", "TableName": "example_table", "TableCatalogId": "111122223333", "RowFilter": { "FilterExpression": "customer.customerName <> 'John'" }, "ColumnNames": ["customer", "\"product\".\"offer\""] } ``` **See also** [Managing data filters](managing-filters.md) # PartiQL support in row filter expressions You can construct row filter expressions using a subset of PartiQL data types, operators, and aggregations. Lake Formation does not allow any user defined or standard partiQL functions in the filter expression. You can use comparison operators to compare columns with constants (for example, `views >= 10000`), but you can't compare columns with other columns. A Row filter expression may be a simple expression or a composite expression. Total length of the expression must be less than 2048 characters. **Simple expression** A simple expression will be of the format:` ` + **Column name** It can either a top level data column, a partition column, or a nested column present in the table schema and must belong to the [Supported data types](#row-filter-supported-datatypes) listed below. + **Comparison operator** The following are the supported operators: `=, >, <, >=, <=, <>,!=, BETWEEN, IN, LIKE, NOT, IS [NOT] NULL` + All string comparisons and `LIKE` pattern matches are case-sensitive. You can't use IS [NOT] NULL operator on partition columns. + **Column value** The Column value must match the data type of the column name. **Composite expression** A composite expression will be of the format: `( ) ()`. Composite expressions can be further combined using logical operators `AND/OR`. ## Supported data types Row filters that refer to an AWS Glue Data Catalog table that contains an unsupported data types will result in an error. The following are the supported data types for table columns and constants, which are mapped to Amazon Redshift data types: + `STRING, CHAR, VARCHAR` + `INT, LONG, BIGINT, FLOAT, DECIMAL, DOUBLE` + `BOOLEAN` + `STRUCT` For more information about data types in Amazon Redshift, see [Data types](https://docs.aws.amazon.com/redshift/latest/dg/c_Supported_data_types.html) in *Amazon Redshift Database Developer Guide*. ## Row filter expressions **Example** The following are examples of valid row filter expressions for a table with columns: ` country (String), id (Long), year (partition column of type Integer), month (partition column of type Integer)` + `year > 2010 and country != 'US'` + `(year > 2010 and country = 'US') or (month < 8 and id > 23)` + `(country between 'Z' and 'U') and (year = 2018)` + `(country like '%ited%') and (year > 2000)` **Example** The following is a valid examples of row filter expressions for a table with nested columns: `year > 2010 and customer.customerId <> 1 ` Nested fields under partition columns should not be referenced when defining nested row-level expressions. String constants must be enclosed in single-quotes. ## Reserved keywords If your row filter expression contains PartiQL keywords, you will receive a parsing error as column names may conflict with the keywords. When this happens, escape the column names by using double quotes. Some examples of reserved keywords are “first”, “last”, “asc”, “missing”. See PartiQL specification for a list of reserved keywords. ## PartiQL reference For more information about PartiQL, see [https://partiql.org/](https://partiql.org/). # Permissions required for querying tables with cell-level filtering The following AWS Identity and Access Management (IAM) permissions are required to run queries against tables with cell-level filtering. ------ #### [ JSON ] **** ``` { "Version":"2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "lakeformation:StartQueryPlanning", "lakeformation:GetQueryState", "lakeformation:GetWorkUnits", "lakeformation:GetWorkUnitResults" ], "Resource": "*" } ] } ``` ------ For more information about Lake Formation permissions, see [Lake Formation personas and IAM permissions reference](permissions-reference.md). # Managing data filters To implement column-level, row-level, and cell-level security, you can create and maintain data filters. Each data filter belongs to a Data Catalog table. You can create multiple data filters for a table, and then use one or more of them when granting permissions on the table. You can also define and apply data filters on nested columns that have `struct` datatypes allowing users to access only sub-structures of nested columns. You require `SELECT` permission with the grant option to create or view a data filter. To allow principals in your account to view and use a data filter, you can grant the `DESCRIBE` permission on it. **Note** Lake Formation doesn't support granting `Describe` permission on a data filter, which is shared from another account. You can manage data filters by using the AWS Lake Formation console, the API, or the AWS Command Line Interface (AWS CLI). For information about data filters, see [Data filters in Lake Formation](data-filtering.md#data-filters-about) # Creating a data filter You can create one or more data filters for each Data Catalog table. **To create a data filter for a Data Catalog table (console)** 1. Open the Lake Formation console at [https://console.aws.amazon.com/lakeformation/](https://console.aws.amazon.com/lakeformation/). Sign as a data lake administrator, the target table owner, or a principal who has a Lake Formation permission on the target table. 1. In the navigation pane, under **Data catalog**, choose **Data filters**. 1. On the **Data filters** page, choose **Create new filter**. 1. In the **Create data filter** dialog box, enter the following information: + Data filter name + Target database – Specify the database that contains the table. + Target table + Column-level access – Leave this set to **Access to all columns** to specify row filtering only. Choose **Include columns** or **Exclude columns** to specify column or cell filtering, and then specify the columns to include or exclude. Nested columns – If you're applying the filter on a table that contains nested columns, you can explicitly specify sub-structures of the nested struct columns within a data filter. When you grant SELECT permission to a principal on this filer, the principal executing the following query, will only see the data for `customer.customerName` and not `customer.customerId`. ``` SELECT "customer" FROM "example_db"."example_table"; ``` ![\[Column-level access settings with options to include specific columns and filter rows.\]](http://docs.aws.amazon.com/lake-formation/latest/dg/images/nested-column-filter.png) When you grant permissions to the `customer` column, the principal receives the access to the column and the nested fields under the column (`customerName` and `customerID`). + Row filter expression – Enter a filter expression to specify row or cell filtering. For supported data types and operators, see [PartiQL support in row filter expressions](partiql-support.md). Choose **Access to all rows** to grant access to all . You can include partial column structs from nested columns in a row filter expression to filter rows that contain specific value. When a principal is granted permissions to a table with a row filter expression `Select * from example_nestedtable where customer.customerName <>'John'`, and **Column-level** access is set to **Access to all columns**, the query results shows only rows where `customerName <>'John'` evaluates to true. The following screenshot shows a data filter that implements cell filtering. In queries against the `orders` table, it denies access to the `customer_name` column and shows only rows that have 'pharma' in the `product_type` column. ![\[The data filter window contains these fields, arranged vertically: Data filter name; Target database; Target table; Option button group with the options Access to all columns, Include columns, and Exclude columns; Select columns (drop-down list); Row filter expression (multi-line text box). The Exclude columns option is selected, the customer_name column is selected for exclusion, and the Row filter expression field contains 'product_type='pharma'.\]](http://docs.aws.amazon.com/lake-formation/latest/dg/images/data-filter-sample-pharma.png) 1. Choose **Create filter**. **To create a data filter with cell-filter policies on a nested field** This section uses the following sample schema to show how to create a data cells filter: ``` [ { name: "customer", type: "struct" }, { name: "customerApplication", type: "struct" }, { name: "product", type: "struct,type:string>" }, { name: "purchaseId", type: "string" }, ] ``` 1. On the **Create a data filter**, page enter a name for the data filter. 1. Next, use the drop-down to choose a database name and table name. 1. In the **Column-level access** section, choose Included columns, and select a nested column (`customer.customerName`). 1. In the **Row-level access** section, choose the **Access to all rows** option. 1. Choose **Create filter**. When you grant `SELECT` permission on this filter, the principal gets access to all rows in the `customerName` column. 1. Next, define another data filter for the same database/table. 1. In the **Column-level access** section, choose Included columns, and select another nested column (`customer.customerid`). 1. In the **Row-level access** section, choose **Filter rows**, and enter a **Row filter expression** (`customer.customerid <> 5`). 1. Choose **Create filter**. When you grant `SELECT` permission on this filter, the principal receives access to all rows in the `customerName`, and `customerId` fields except the cell where the value is 5 in the `customerId` column. # Granting data filter permissions You can grant the `SELECT`, `DESCRIBE` and `DROP` Lake Formation permissions on data filters to principals. At first, only you can view the data filters that you create for a table. To enable another principal to view a data filter and grant Data Catalog permissions with the data filter, you must either: + Grant `SELECT` on a table to the principal with the grant option, and apply the data filter to the grant. + Grant the `DESCRIBE` or `DROP` permission on the data filter to the principal. You can grant the `SELECT` permission to an external AWS account. A data lake administrator in that account can then grant that permission to other principals in the account. When granting to an external account, you must include the grant option so that administrator of the external account can further cascade the permission to other users in his/her account. When granting to a principal in your account, granting with the grant option is optional. You can grant and revoke permissions on data filters by using the AWS Lake Formation console, the API, or the AWS Command Line Interface (AWS CLI). ------ #### [ Console ] 1. Sign in to the AWS Management Console and open the Lake Formation console at [https://console.aws.amazon.com/lakeformation/](https://console.aws.amazon.com/lakeformation/). 1. In the navigation pane, under **Permissions**, choose **Data lake permissions**. 1. On the **Permissions** page, in the **Data permissions** section, choose **Grant**. 1. On the **Grant data permissions** page, choose the principals to grant the permissions to. 1. In the LF-Tags or catalog resources section, choose **Named data catalog resources**. Then choose the database, table, and data filter for which you want to grant permissions. ![\[\]](http://docs.aws.amazon.com/lake-formation/latest/dg/images/grant-data-filter-perms-step2.png) 1. In the **Data filter permissions** section, choose the permissions you want to grant to the selected principals. ![\[\]](http://docs.aws.amazon.com/lake-formation/latest/dg/images/grant-perms-on-filters.png) ------ #### [ AWS CLI ] + Enter a `grant-permissions` command. Specify `DataCellsFilter` for the `resource` argument, and specify `DESCRIBE` or `DROP` for the `Permissions` argument and, optionally, for the `PermissionsWithGrantOption` argument. The following example grants `DESCRIBE` with the grant option to user `datalake_user1` on the data filter `restrict-pharma`, which belongs to the `orders` table in the `sales` database in AWS account 1111-2222-3333. ``` aws lakeformation grant-permissions --cli-input-json file://grant-params.json ``` The following are the contents of file `grant-params.json`. ``` { "Principal": {"DataLakePrincipalIdentifier": "arn:aws:iam::111122223333:user/datalake_user1"}, "Resource": { "DataCellsFilter": { "TableCatalogId": "111122223333", "DatabaseName": "sales", "TableName": "orders", "Name": "restrict-pharma" } }, "Permissions": ["DESCRIBE"], "PermissionsWithGrantOption": ["DESCRIBE"] } ``` ------ # Granting data permissions provided by data filters Data filters represent a subset of data within a table. To provide data access to principals, `SELECT` permissions need to be granted to those principals. With this permission the principals can: + View the actual table name in list of tables shared with their account. + Create data filters on the shared table and grant permissions to their users on those data filters. ------ #### [ Console ] **To grant SELECT permissions** 1. Go to the **Permissions** page in the Lake Formation console, and then choose **Grant**. ![\[\]](http://docs.aws.amazon.com/lake-formation/latest/dg/images/permissions-grant-action.png) 1. Select the principals you want to provide access to, and select **Named data catalog resources**. ![\[\]](http://docs.aws.amazon.com/lake-formation/latest/dg/images/grant-data-filter-perms-step2.png) 1. To provide access to the data that the filter represents, choose **Select** under **Data filter permissions**. ![\[\]](http://docs.aws.amazon.com/lake-formation/latest/dg/images/grant-data-filter-perms-step3.png) ------ #### [ CLI ] Enter a `grant-permissions` command. Specify `DataCellsFilter` for the resource argument, and specify `SELECT` for the Permissions argument. The following example grants `SELECT` with the grant option to user `datalake_user1` on the data filter `restrict-pharma`, which belongs to the `orders` table in the `sales` database in AWS account `1111-2222-3333`. ``` aws lakeformation grant-permissions --cli-input-json file://grant-params.json ``` The following are the contents of file `grant-params.json`. ``` { "Principal": { "DataLakePrincipalIdentifier": "arn:aws:iam::111122223333:user/datalake_user1" }, "Resource": { "DataCellsFilter": { "TableCatalogId": "111122223333", "DatabaseName": "sales", "TableName": "orders", "Name": "restrict-pharma" } }, "Permissions": ["SELECT"] } ``` ------ # Viewing data filters You can use the Lake Formation console, AWS CLI, or the Lake Formation API to view data filters. To view data filters, you must be a Data Lake administrator or have the required permissions on the data filters. ------ #### [ Console ] 1. Sign in to the AWS Management Console and open the Lake Formation console at [https://console.aws.amazon.com/lakeformation/](https://console.aws.amazon.com/lakeformation/). 1. In the navigation pane, under **Data catalog**, choose **Data filters**. The page displays the data filters you have access to. ![\[The Data filters page displays the available data filters with the following columns: Filter name, Table, Database, and Table catalog ID. The screenshot shows a single data filter with the following values: test-df, cloudtrailtest_cloudtrail, lakeformation_cloudtrail, redacted account ID. Above the table there are four buttons (from left to right): Refresh/reload, View (grayed out), Delete (grayed out), and "Create new filter". There is also a search field, which is empty.\]](http://docs.aws.amazon.com/lake-formation/latest/dg/images/list-data-filters.jpg) 1. To view the data filter details, choose the data filter, and then choose View. A new window appears with the data filter detailed information. ![\[The "View data filter" window shows additional information about the selected data filter. The information displayed includes the name, database, table, column-level access setting, row filter expression, and the columns.\]](http://docs.aws.amazon.com/lake-formation/latest/dg/images/list-data-filters-details.jpg) ------ #### [ AWS CLI ] Enter a `list-data-cells-filter` command and specify a table resource. The following example lists the data filters for the `cloudtrailtest_cloudtrail` table. ``` aws lakeformation list-data-cells-filter --table '{ "CatalogId":"123456789012", "DatabaseName":"lakeformation_cloudtrail", "Name":"cloudtrailtest_cloudtrail"}' ``` ------ #### [ API/SDK ] Use the `ListDataCellsFilter` API and specify a table resource. The following example uses Python to list the first 20 data filters for the `myTable` table. ``` response = client.list_data_cells_filter( Table = { 'CatalogId': '111122223333', 'DatabaseName': 'mydb', 'Name': 'myTable' }, MaxResults=20 ) ``` ------ # Listing data filter permissions You can use the Lake Formation console to view the permissions granted on data filters. To view permissions on a data filter, you must be a Data Lake administrator or have the required permissions on the data filter. ------ #### [ Console ] 1. Sign in to the AWS Management Console and open the Lake Formation console at [https://console.aws.amazon.com/lakeformation/](https://console.aws.amazon.com/lakeformation/). 1. In the navigation pane, under **Permissions**, choose **Data permissions**. 1. On the **Data Permissions** page, click or tap in the search field, and on the **Properties** menu, choose **Resource type**. 1. On the **Resource type** menu, choose **Resource type: Data cell filter**. The data filters that you have permissions on are listed. You might have to scroll horizontally to see the **Permissions** and **Grantable** columns. ![\[The Data Permissions page displays a table of permissions with the following columns: Principal, Resource type, Database, Table, Resource, Catalog, and Permissions. The Resource type column shows "Data cell filter" in all four rows. The permissions for the first and second rows are Describe, Drop, and Select. The permissions for the third row is Describe. Above the table is a Clear filter button and a tile indicating that the current search is for Resource type: Data cell filter. Above those is a search (text) field, and above that are Refresh, Revoke, and Grant buttons.\]](http://docs.aws.amazon.com/lake-formation/latest/dg/images/data-permissions-cell-filters.png) ------ #### [ AWS CLI ] + Enter a `list-permissions` command. Specify `DataCellsFilter` for the `resource` argument, and specify `DESCRIBE` or `DROP` for the `Permissions` argument and, optionally, for the `PermissionsWithGrantOption` argument. The following example lists `DESCRIBE` permissions with the grant option on the data filter `restrict-pharma`. The results are limited to permissions granted for the principal `datalake_user1` and the `orders` table in the `sales` database in AWS account 1111-2222-3333. ``` aws lakeformation list-permissions --cli-input-json file://list-params.json ``` The following are the contents of file `grant-params.json`. ``` { "Principal": {"DataLakePrincipalIdentifier": "arn:aws:iam::111122223333:user/datalake_user1"}, "Resource": { "DataCellsFilter": { "TableCatalogId": "111122223333", "DatabaseName": "sales", "TableName": "orders", "Name": "restrict-pharma" } }, "Permissions": ["DESCRIBE"], "PermissionsWithGrantOption": ["DESCRIBE"] } ``` ------ # Viewing database and table permissions in Lake Formation Viewing Database and Table Permissions You can view the Lake Formation permissions that are granted on a Data Catalog database or table. You can do so by using the Lake Formation console, the API, or the AWS Command Line Interface (AWS CLI). Using the console, you can view permissions starting from the **Databases** or **Tables** pages, or from the **Data permissions** page. **Note** If you're not a database administrator or resource owner, you can view permissions that other principals have on the resource only if you have a Lake Formation permission on the resource with the grant option. In addition to the required Lake Formation permissions, you need the AWS Identity and Access Management (IAM) permissions `glue:GetDatabases`, `glue:GetDatabase`, `glue:GetTables`, `glue:GetTable`, and `lakeformation:ListPermissions`. **To view permissions on a database (console, starting from the Databases page)** 1. Open the Lake Formation console at [https://console.aws.amazon.com/lakeformation/](https://console.aws.amazon.com/lakeformation/). Sign in as a data lake administrator, the database creator, or as a user who has any Lake Formation permission on the database with the grant option. 1. In the navigation pane, choose **Databases**. 1. Choose a database, and on the **Actions** menu, choose **View permissions**. **Note** If you choose a database resource link, Lake Formation displays the permissions on the resource link, not on the target database of the resource link. The **Data permissions** page lists all Lake Formation permissions for the database. The database name and catalog ID (AWS account ID) of the database owner appear as labels under the search box. The tiles indicate that a filter has been applied to list permissions only for that database. You can adjust the filter by closing a tile or choosing **Clear filter**. ![\[The Data permissions page displays a search box at the top, with two tiles underneath. The tiles are labeled Database:logs and Catalog ID:111122223333. Next to the tiles is a Clear filter button. Below is the list of databases and their permissions. This example has only one row in the list. It's for the logs database, and the permissions Alter, Create table, and Drop are granted to IAM user Administrator with the grant option. The list includes an Owner account ID column, and the one row has 11112222333 in that column.\]](http://docs.aws.amazon.com/lake-formation/latest/dg/images/permissions-page-database.png) **To view permissions on a database (console, starting from the Data permissions page)** 1. Open the Lake Formation console at [https://console.aws.amazon.com/lakeformation/](https://console.aws.amazon.com/lakeformation/). Sign in as a data lake administrator, the database creator, or as a user who has any Lake Formation permission on the database with the grant option. 1. In the navigation pane, choose **Data permissions**. 1. Position the cursor in the search box at the top of the page, and on the **Properties** menu that appears, choose **Database**. 1. On the **Databases** menu that appears, choose a database. **Note** If you choose a database resource link, Lake Formation displays the permissions on the resource link, not on the target database of the resource link. The **Data permissions** page lists all Lake Formation permissions for the database. The database name appears as a tile under the search box. The tile indicates that a filter has been applied to list permissions only for that database. You can remove the filter by closing the tile or choosing **Clear filter**. **To view permissions on a table (console, starting from the Tables page)** 1. Open the Lake Formation console at [https://console.aws.amazon.com/lakeformation/](https://console.aws.amazon.com/lakeformation/). Sign in as a data lake administrator, the table creator, or as a user who has any Lake Formation permission on the table with the grant option. 1. In the navigation pane, choose **Tables**. 1. Choose a table, and on the **Actions** menu, choose **View permissions**. **Note** If you choose a table resource link, Lake Formation displays the permissions on the resource link, not on the target table of the resource link. The **Data permissions** page lists all Lake Formation permissions for the table. The table name, the database name of the database that contains the table, and the catalog ID (AWS account ID) of the table owner appear as labels under the search box. The labels indicate that a filter has been applied to list permissions only for that table. You can adjust the filter by closing a label or choosing **Clear filter**. ![\[The Data permissions page displays a search field at the top, with three tiles underneath. The tiles are labeled Database:logs, Table:alexa-logs, and Catalog ID:111122223333, going from left to right. Next to the tiles is a Clear filter button. Below is the list of tables and their permissions. This example has only one row in the list. It's for the alexa-logs table, and the Super permissions is granted to IAM user Administrator with the grant option. The list includes an Owner account ID column, and the one row has 11112222333 in that column.\]](http://docs.aws.amazon.com/lake-formation/latest/dg/images/permissions-page-table.png) **To view permissions on a table (console, starting from the Data permissions page)** 1. Open the Lake Formation console at [https://console.aws.amazon.com/lakeformation/](https://console.aws.amazon.com/lakeformation/). Sign in as a data lake administrator, the table creator, or as a user who has any Lake Formation permission on the table with the grant option. 1. In the navigation pane, choose **Data permissions**. 1. Position the cursor in the search box at the top of the page, and on the **Properties** menu that appears, choose **Database**. 1. On the **Databases** menu that appears, choose a database. **Important** If you want to view permissions on a table that was shared with your AWS account from an external account, you must choose the database in the external account that contains the table, not a resource link to the database. The **Data permissions** page lists all Lake Formation permissions for the database. 1. Position the cursor in the search box again, and on the **Properties** menu that appears, choose **Table**. 1. On the **Tables** menu that appears, choose a table. The **Data permissions** page lists all Lake Formation permissions for the table. The table name and the database name of the database that contains the table appear as tiles under the search box. The tiles indicate that a filter has been applied to list permissions only for that table. You can adjust the filter by closing a tile or choosing **Clear filter**. **To view permissions on a table (AWS CLI)** + Enter a `list-permissions` command. The following example lists permissions on a table shared from an external account. The `CatalogId` property is the AWS account ID of the external account, and the database name refers to the database in the external account that contains the table. ``` aws lakeformation list-permissions --resource-type TABLE --resource '{ "Table": {"DatabaseName":"logs", "Name":"alexa-logs", "CatalogId":"123456789012"}}' ``` # Revoking permission using the Lake Formation console Revoking permissions using the console You can use the console to revoke all types of Lake Formation permissions—Data Catalog permissions, policy tag permissions, data filter permissions, and location permissions. **To revoke Lake Formation permissions on a resource (console)** 1. Open the Lake Formation console at [https://console.aws.amazon.com/lakeformation/](https://console.aws.amazon.com/lakeformation/). Sign in as a data lake administrator or as a user who has been granted permissions with the grant option on the resource. 1. In the navigation pane, under **Permissions**, choose **Data lake permissions**, **LF-Tags and permissions**, or **Data locations**. 1. Select the permission or location, and then choose **Revoke**. 1. In the dialog box that opens, choose **Revoke**. # Cross-account data sharing in Lake Formation Cross-account data sharing Lake Formation cross-account capabilities allow users to securely share distributed data lakes across multiple AWS accounts, AWS organizations or directly with IAM principals in another account providing fine-grained access to the Data Catalog metadata and underlying data. Large enterprises typically use multiple AWS accounts, and many of those accounts might need access to a data lake managed by a single AWS account. Users and AWS Glue extract, transform, and load (ETL) jobs can query and join tables across multiple accounts and still take advantage of Lake Formation table-level and column-level data protections. When you grant Lake Formation permissions on a Data Catalog resource to an external account or directly to an IAM principal in another account, Lake Formation uses the AWS Resource Access Manager (AWS RAM) service to share the resource. If the grantee account is in the same organization as the grantor account, the shared resource is available immediately to the grantee. If the grantee account is not in the same organization, AWS RAM sends an invitation to the grantee account to accept or reject the resource grant. Then, to make the shared resource available, the data lake administrator in the grantee account must use the AWS RAM console or AWS CLI to accept the invitation. Lake Formation supports sharing Data Catalog resources with external accounts in hybrid access mode. Hybrid access mode provides the flexibility to selectively enable Lake Formation permissions for databases and tables in your AWS Glue Data Catalog.
 With the Hybrid access mode, you now have an incremental path that allows you to set Lake Formation permissions for a specific set of users without interrupting the permission policies of other existing users or workloads. For more information, see [Hybrid access mode](hybrid-access-mode.md). **Direct cross-account share** Authorized principals can share resources explicitly with an IAM principal in an external account. This feature is useful when an account owner wants to have control over who in the external account can access the resources. The permissions the IAM principal receives will be a union of direct grants and the account level grants that is cascaded down to the principals. Only the recipient of the permission grant can view the direct cross-account grants. The principal who receives the resource share cannot share the resource with other principals. **Methods for sharing Data Catalog resources** With a single Lake Formation grant operation, you can grant cross-account permissions on the following Data Catalog resources. + A database + An individual table (with optional column filtering) + A few selected tables + All tables in a database (by using the All Tables wildcard) There are two options for sharing your databases and tables with another AWS account or IAM principals in another account. + Lake Formation tag-based access control (LF-TBAC) (recommended) Lake Formation tag-based access control is an authorization strategy that defines permissions based on attributes. You can use tag-based access control to share Data Catalog resources (databases, tables, and columns) with external IAM principals, AWS accounts, Organizations and organizational units (OUs). In Lake Formation, these attributes are called LF-tags. For more information, see [Managing a data lake using Lake Formation tag-based access control](https://docs.aws.amazon.com/lake-formation/latest/dg/managing-dl-tutorial.html). **Note** The LF-TBAC method of granting Data Catalog permissions use AWS Resource Access Manager for cross-account grants. Lake Formation now supports granting cross-account permissions to Organizations and organizational units using LF-TBAC method. To enable this capability, you need to update the **Cross account version settings** to **Version 3** or above. For more information, see [Updating cross-account data sharing version settings](optimize-ram.md). + Lake Formation named resources The Lake Formation cross-account data sharing using named resource method allows you to grant Lake Formation permissions with a grant option on Data Catalog tables and databases to external AWS accounts, IAM principals, organizations, or organizational units. The grant operation automatically shares those resources. **Note** You can also allow the AWS Glue crawler to access a data store in a different account using Lake Formation credentials. For more information, see [Cross-account crawling](https://docs.aws.amazon.com/glue/latest/dg/crawler-configuration.html#cross-account-crawling) in AWS Glue Developer Guide. Integrated services such as Athena and Amazon Redshift Spectrum require resource links to be able to include shared resources in queries. For more information about resource links, see [How resource links work in Lake Formation](resource-links-about.md). For considerations and limitation, see [Cross-account data sharing best practices and considerations](cross-account-notes.md). **Topics** + [ # Prerequisites ](cross-account-prereqs.md) + [ # Updating cross-account data sharing version settings ](optimize-ram.md) + [ # Sharing Data Catalog tables and databases across AWS accounts or IAM principals from external accounts ](cross-account-data-share-steps.md) + [ # Granting permissions on a database or table shared with your account ](regranting-shared-resources.md) + [ # Granting resource link permissions ](granting-link-permissions.md) + [ # Accessing the underlying data of a shared table ](cross-account-read-data.md) + [ # Cross-account CloudTrail logging ](cross-account-logging.md) + [ # Managing cross-account permissions using both AWS Glue and Lake Formation ](hybrid-cross-account.md) + [ # Viewing all cross-account grants using the GetResourceShares API operation ](cross-account-getresourcepolicies.md) **Related topics** [Overview of Lake Formation permissions](lf-permissions-overview.md) [Accessing and viewing shared Data Catalog tables and databases](viewing-shared-resources.md) [Creating resource links](creating-resource-links.md) [Troubleshooting cross-account access](troubleshooting.md#trouble-cross-account) # Prerequisites Before your AWS account can share Data Catalog resources (catalogs, databases and tables) with another account or principals in another account, and before you can access the resources shared with your account, the following prerequisites must be met. **General cross-account data sharing requirements** + To share Data Catalog databases and tables in hybrid access mode and share objects in the federated catalogs, you need to update the **Cross account version settings** to **Version 4**. + Before granting cross-account permissions on a Data Catalog resource, you must revoke all Lake Formation permissions from the `IAMAllowedPrincipals` group for the resource. If the calling principal has cross account permissions to access a resource and the `IAMAllowedPrincipals` permission exists on the resource, then Lake Formation throws `AccessDeniedException`. This requirement is applicable only when you register the underlying data location in Lake Formation mode. If you register the data location in hybrid mode, the `IAMAllowedPrincipals` group permissions can exist on the shared database or table. + For databases that contain tables that you intend to share, you must prevent new tables from having a default grant of `Super` to `IAMAllowedPrincipals`. On the Lake Formation console, edit the database and turn off **Use only IAM access control for new tables in this database** or enter the following AWS CLI command, replacing `database` with the name of the database. If the underlying data location is registered in hybrid access mode, you don't need to change this default setting. In hybrid access mode, Lake Formation allows you to selectively enforce Lake Formation permissions and IAM permissions policies for Amazon S3 and AWS Glue on the same resource. ``` aws glue update-database --name database --database-input '{"Name":"database","CreateTableDefaultPermissions":[]}' ``` + To grant cross-account permissions, the grantor must have the required AWS Identity and Access Management (IAM) permissions on AWS Glue and AWS RAM service. The AWS managed policy `AWSLakeFormationCrossAccountManager` grants the required permissions. Data lake administrators in accounts that receive resource shares using AWS RAM must have the following additional policy. It allows the administrator to accept AWS RAM resource share invitations. It also allows the administrator to enable resource sharing with organizations. ------ #### [ JSON ] **** ``` { "Version":"2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "ram:AcceptResourceShareInvitation", "ram:RejectResourceShareInvitation", "ec2:DescribeAvailabilityZones", "ram:EnableSharingWithAwsOrganization" ], "Resource": "*" } ] } ``` ------ + If you want to share Data Catalog resources with AWS Organizations or organizational units, sharing with organizations must be enabled in AWS RAM. For information on how to enable sharing with organizations, see [Enable sharing with AWS organizations](https://docs.aws.amazon.com/ram/latest/userguide/getting-started-sharing.html#getting-started-sharing-orgs) in the *AWS RAM User Guide*. You must have the `ram:EnableSharingWithAwsOrganization` permission to enable sharing with organizations. + To share resources directly with an IAM principal in another account, you need to update the **Cross account version settings** to **Version 3**. This setting is available on the **Data catalog settings** page. If you are using **Version 1**, see instructions to update the setting [Updating cross-account data sharing version settings](optimize-ram.md). + You cannot share Data Catalog resources encrypted with AWS Glue service managed key with another account. You can share only Data Catalog resources encrypted with customer's encryption key, and the account receiving the resource share must have permissions on the Data Catalog encryption key to decrypt the objects. **Cross-account data sharing using LF-TBAC requirements** + To share Data Catalog resources with AWS Organizations and organizational units (OUs), you need to update the **Cross account version settings** to **Version 3**or above. + To share Data Catalog resources with version 3 of the **Cross account version settings**, the grantor requires to have the IAM permissions defined in the AWS managed policy `AWSLakeFormationCrossAccountManager` in your account. + If you are using version 1 or version 2 of the **Cross account version settings**, you must have a Data Catalog resource policy (`glue:PutResourcePolicy`) that enables LF-TBAC. For more information, see [Managing cross-account permissions using both AWS Glue and Lake Formation](hybrid-cross-account.md). + If you're currently using an AWS Glue Data Catalog resource policy to share resources, and you want to grant cross-account permissions using version 3 of the **Cross account version settings**, you must add the `glue:ShareResource` permission in the Data Catalog Settings using the `glue:PutResourcePolicy` API operation as shown in the [Managing cross-account permissions using both AWS Glue and Lake Formation](hybrid-cross-account.md) section. This policy is not required if your account has made no cross-account grants using the AWS Glue Data Catalog resource policy (version 1 and version 2 use `glue:PutResourcePolicy` permission) to grant cross-account access. ``` { "Effect": "Allow", "Action": [ "glue:ShareResource" ], "Principal": {"Service": [ "ram.amazonaws.com" ]}, "Resource": [ "arn:aws:glue:::table/*/*", "arn:aws:glue:::database/*", "arn:aws:glue:::catalog" ] } ``` + If your account has made cross-account shares using AWS Glue Data Catalog resource policy, and you are currently using named resource method or LF-TBAC with **Cross account settings** version 3 to share resources, which uses AWS RAM to share resources, you must set the `EnableHybrid` argument to `'true'` when you invoke the `glue:PutResourcePolicy` API operation. For more information, see [Managing cross-account permissions using both AWS Glue and Lake Formation](hybrid-cross-account.md). **Setup required in each account that accesses the shared resource** + If you are sharing resources with AWS accounts, at least one user in the consumer account must be a data lake administrator to view shared resources. For information on how to create a data lake administrator, see [Create a data lake administrator](initial-lf-config.md#create-data-lake-admin). The data lake administrator can grant Lake Formation permissions on the shared resources to other principals in the account. Other principals can't access shared resources until the data lake administrator grants them permissions on the resources. + Integrated services such as Athena and Redshift Spectrum require resource links to be able to include shared resources in queries. Principals need to create a resource link in their Data Catalog to a shared resource from another AWS account. For more information about resource links, see [How resource links work in Lake Formation](resource-links-about.md). + When a resource is shared directly with an IAM principal, to query the table using Athena, the principal needs to create a resource link. To create a resource link, the principal needs the Lake Formation `CREATE_TABLE` or `CREATE_DATABASE` permission, and the `glue:CreateTable` or `glue:CreateDatabase` IAM permission. If the producer account shares a different table under the same database with the same or another principal, that principal can immediately query the table. **Note** For the data lake administrator and for principals whom the data lake administrator has granted permissions to, shared resources appear in the Data Catalog as if they are local (owned) resources. Extract, transform, and load (ETL) jobs can access the underlying data of shared resources. For shared resources, the **Tables** and **Databases** pages on the Lake Formation console display the owner's account ID. When the underlying data of a shared resource is accessed, CloudTrail log events are generated in both the shared resource recipient's account and the resource owner's account. The CloudTrail events can contain the ARN of the principal that accessed the data, but only if the recipient account opts in to include the principal ARN in the logs. For more information, see [Cross-account CloudTrail logging](cross-account-logging.md). # Updating cross-account data sharing version settings From time to time, AWS Lake Formation updates the cross-account data sharing settings to distinguish the changes made to the AWS RAM usage and to support updates made to the cross-account data sharing feature. When Lake Formation does this, it creates a new version of the **Cross account version settings**. ## Main differences between cross-account version settings For more information about how cross-account data sharing works under different **Cross account version settings**, see the following sections. **Note** To share data with another account, the grantor must have `AWSLakeFormationCrossAccountManager` managed IAM policy permissions. This is a prerequisite for all versions. Updating the **Cross account version settings** does not impact the permissions the recipient has on shared resources. This is applicable when updating from version 1 to version 2, version 2 to version 3, and version 1 to version 3. See the considerations listed below when updating versions. **Version 1** *Named resource method: *Maps each cross-account Lake Formation permission grant to one AWS RAM resource share. User (grantor role or principal) does not require additional permissions. *LF-TBAC method: *Cross-account Lake Formation permission grants don't use AWS RAM to share data. User must have `glue:PutResourcePolicy` permission. *Benefits from updating versions: *Initial version - not applicable. *Considerations when updating versions: *Initial version - not applicable **Version 2** *Named resource method: * Optimizes the number of AWS RAM resource shares by mapping multiple cross-account permission grants with one AWS RAM resource share. User does not require additional permissions. *LF-TBAC method: *Cross-account Lake Formation permission grants don't use AWS RAM to share data. User must have `glue:PutResourcePolicy` permission. *Benefits from updating versions: *Scalable cross-account setup by optimal utilization of AWS RAM capacity. *Considerations when updating versions: *Users who want to grant cross-account Lake Formation permissions must have the permissions in the `AWSLakeFormationCrossAccountManager` AWS managed policy. Otherwise, you need to have `ram:AssociateResourceShare` and `ram:DisassociateResourceShare` permissions to successfully share resources with another account. **Version 3** *Named resource method: * Optimizes the number of AWS RAM resource shares by mapping multiple cross-account permission grants with one AWS RAM resource share. User does not require additional permissions. *LF-TBAC method: * Lake Formation uses AWS RAM for cross-account grants. User must add glue:ShareResource statement to the `glue:PutResourcePolicy` permission. The recipient must accept resource share invitations from AWS RAM. *Benefits from updating versions: *Supports the following capabilities: + Allows sharing resources explicitly with an IAM principal in an external account. For more information, see [Granting permissions on Data Catalog resources](granting-catalog-permissions.md). + Enables cross-account shares using LF-TBAC method to Organizations or organizational units (OUs). + Removes the overhead of maintaining additional AWS Glue policies for cross-account grants. *Considerations when updating versions:* When you use LF-TBAC method to share resources, if the grantor uses a version lower than version 3, and the recipient is using version 3 or higher, the grantor receives the following error message: "Invalid cross account grant request. Consumer account has opt-in to cross account version: v3. Please update `CrossAccountVersion` in `DataLakeSetting` to minimal version v3 (Service: AmazonDataCatalog; Status Code: 400; Error Code: InvalidInputException)". However, if the grantor uses version 3 and the recipient is using version 1 or version 2, the cross-account grants using LF-Tags go through successfully. Cross-account grants made using the named resource method are compatible across different versions. Even if the grantor account is using an older version (version 1 or 2) and the recipient account is using a newer version (version 3 or higher), the cross-account access functionality operates seamlessly without any compatibility issues or errors. To share resources directly with IAM principals in another account, only the grantor needs to use version 3. Cross-account grants made using LF-TBAC method require users to have an AWS Glue Data Catalog resource policy in the account. When you update to version 3, LF-TBAC grants uses AWS RAM. To allow AWS RAM based cross-account grants to succeed, you must add the `glue:ShareResource` statement to your existing Data Catalog resource policies as shown in the [Managing cross-account permissions using both AWS Glue and Lake Formation](hybrid-cross-account.md) section. **Version 4** The grantor needs version 4 or higher to share Data Catalog resources in hybrid access mode or share objects in a federated catalog. **Version 5** Cross Account Version 5 enhances cross-account resource sharing enabling you to share unlimited number of tables to another account, eliminating previous resource association limits per resource type. To get started, upgrade to cross-account version 5 through the Lake Formation console or API. Any new cross-account permission grants will automatically use wildcard patterns in the resource share instead of individual resource associations. All existing cross-account shares continue to function, and all existing Lake Formation APIs remain compatible. *Benefits from updating versions: *Cross-account v5 enhances cross-account sharing, allowing you to share hundreds of thousands of tables across accounts. *Considerations when updating versions: *New grants after version 5 upgrade will add wildcard resource patterns to existing AWS Resource Manager resource shares or create new shares with wildcard patterns. Once upgraded to version 5, downgrade is not supported. ## Optimize AWS RAM resource shares New versions (version 2 and above) of cross-account grants optimally utilize AWS RAM capacity to maximize cross account usage. When you share a resource with an external AWS account or an IAM principal, Lake Formation may create a new resource share or associate the resource with an existing share. By associating with existing shares, Lake Formation reduces the number of resource share invitations a consumer needs to accept. Version 5 further optimizes RAM usage by using wildcard-based resource patterns instead of individual resource associations, thereby significantly reducing resource associations per resource share. ## Enable AWS RAM shares via TBAC or share resources directly to principals To share resources directly with IAM principals in another account or to enable TBAC cross-account shares to Organizations or organizational units, you need to update the **Cross account version settings** to version 3. For more information about AWS RAM resource limits, see [Cross-account data sharing best practices and considerations](cross-account-notes.md). ### Required permissions for updating cross-account version settings If a cross-account permission grantor has `AWSLakeFormationCrossAccountManager` managed IAM policy permissions, then there is no extra permission setup required for the cross-account permission grantor role or principal. However, if the cross-account grantor is not using the managed policy, then the grantor role or principal should have following IAM permissions granted for the new version of the cross-account grant to be successful. ------ #### [ JSON ] **** ``` { "Version":"2012-10-17", "Statement": [ { "Sid": "VisualEditor1", "Effect": "Allow", "Action": [ "ram:AssociateResourceShare", "ram:DisassociateResourceShare", "ram:GetResourceShares" ], "Resource": "*", "Condition": { "StringLike": { "ram:ResourceShareName": "LakeFormation*" } } } ] } ``` ------ ## To enable the new version Follow these steps to update **Cross account version settings** through the AWS Lake Formation console or the AWS CLI. ------ #### [ Console ] 1. Choose **Version 2**, **Version 3**, **Version 4**, or **Version 5** under **Cross account version settings** on the **Data catalog settings** page. If you select **Version 1**, Lake Formation will use the default resource sharing mode. ![\[The screen shows the permissions for all LF-Tags in the account.\]](http://docs.aws.amazon.com/lake-formation/latest/dg/images/cross-account-version-setting.png) 1. Choose **Save**. ------ #### [ AWS Command Line Interface (AWS CLI) ] Use the `put-data-lake-settings` AWS CLI command to set the `CROSS_ACCOUNT_VERSION` parameter. Accepted values are 1, 2, 3, 4, and 5. ``` aws lakeformation put-data-lake-settings --region us-east-1 --data-lake-settings file://settings { "DataLakeAdmins": [ { "DataLakePrincipalIdentifier": "arn:aws:iam::111122223333:user/test" } ], "CreateDatabaseDefaultPermissions": [], "CreateTableDefaultPermissions": [], "Parameters": { "CROSS_ACCOUNT_VERSION": "3" } } ``` ------ **Important** Once you choose **Version 2** or **Version 3**, all new **named resource** grants will go through the new cross-account grant mode. To optimally use AWS RAM capacity for your existing cross-account shares, we recommend you to revoke the grants that were made with the older version, and re-grant in the new mode. # Sharing Data Catalog tables and databases across AWS accounts or IAM principals from external accounts This section includes instructions on how to grant cross-account permissions on Data Catalog resources to an external AWS account, IAM principal, AWS organization, or organizational unit. The grant operation automatically shares those resources. **Topics** + [ # Data sharing using tag-based access control ](cross-account-TBAC.md) + [ # Cross-account data sharing using the named resource method ](cross-account-named-resource.md) # Data sharing using tag-based access control AWS Lake Formation tag-based access control (LF-TBAC) is an authorization strategy that defines permissions based on attributes. The following steps explain how to grant cross-account permissions by using LF-Tags. **Set up required on the producer/grantor account** 1. Add LF-Tags. 1. Sign in to Lake Formation console as a data lake administrator or a LF-Tag creator. 1. In the left navigation bar, choose **Permissions**, and **LF-Tags and permissions**. 1. Choose **Add LF-Tag**. For detailed instructions to create LF-Tags, see [Creating LF-Tags](TBAC-creating-tags.md). 1. Grant **Describe** and/or **Associate** permissions **LF-Tag key-value** pairs to IAM principals in your account or external accounts. Granting permissions on **LF-Tag key-value** pairs enables the principals to view the LF-Tags, and assign them to Data Catalog resources (databases, tables, and columns). 1. Next, the data lake administrator or an IAM principal with **Associate** permission can assign the LF-Tag to databases, tables, or columns. For more information, see [Assigning LF-Tags to Data Catalog resources](TBAC-assigning-tags.md). 1. Next, grant data permission to external accounts using LF-Tag expressions. This enables the grantee or recipient of the permissions to access the Data Catalog resource(s) that are tagged with the same key(s) and value(s). 1. In the navigation pane, choose **Permissions** and **Data permissions**. 1. Choose **Grant**. 1. On the **Grant permissions** page, for **Principals**, choose **External accounts**, and enter the grantee AWS account ID or the IAM role of the principal or the Amazon Resource Name (ARN) for the principal (principal ARN) if making a direct cross-account grant to an external principal. You need to press **Enter** after entering the account ID. ![\[The grant permission screen with external account and LF-Tag key-value pairs specified.\]](http://docs.aws.amazon.com/lake-formation/latest/dg/images/cross-acct-grant-tags.png) 1. For **LF-Tags or catalog resources**, choose **Resources matched by LF-Tags (recommended)**. 1. Choose the option **LF-Tag key-value pairs** or **Saved LF-Tag expressions** . 1. **If you choose, **LF-Tag key-value pairs**, enter the key and value(s)** of the **LF-Tag** that is associated with the Data Catalog resource being shared with the grantee account. The grantee is granted permissions on the Data Catalog resources that were assigned a matching LF-Tag in the LF-Tag expression. If the LF-Tag expression specifies multiple values per tag key, any one of the tag values can be a match. 1. Choose the database-level or table-level permissions to grant on resources that match the LF-Tag expression. **Important** Because the data lake administrator must grant permissions on shared resources to the principals in the grantee account, you must always grant cross-account permissions with the grant option. For more information, see [Granting LF-Tag permissions using the console](TBAC-granting-tags-console.md). **Note** Principals who receive direct cross-account grants will not have the **Grantable permissions** option. **Set up required on the receiving/grantee account** 1. Sign in to Lake Formation console as a data lake administrator of the consumer account. 1. Next, receive the resource share in the consumer account. 1. Open the AWS RAM console. 1. In the navigation pane, under **Shared with me**, choose **Resource shares**. 1. Select the resource shares, choose **Accept resource share**. 1. When you share a resource with another account, the resource still belongs to the producer account and is not visible within the Athena console. To make the resource visible in the Athena console, you need to create a resource link pointing to the shared resource. For instructions on creating a resource link, see [Creating a resource link to a shared Data Catalog table](create-resource-link-table.md) and [Creating a resource link to a shared Data Catalog database](create-resource-link-database.md) 1. Choose **Databases** or **Tables** under the Data Catalog. 1. On the Databases/Tables page, choose **Create**, **Resource link** . 1. Enter the following information for a database resource link: + **Resource link name** – A unique name for the resource link. + **Destination catalog** – The catalog where you're creating the resource link. + **Shared database Region** – The Region of the database shared with you if you are creating the resource link in a different Region. + **Shared database** – Choose the shared database. + **Shared database’s catalog ID** – Enter the catalog ID for the shared database. 1. Choose **Create**. You can see the newly created resource link in the databases list. Similarly, you can create a resource link to a shared table. 1. Now grant **Describe** permission on the resource link to the IAM principals that you are sharing the resource. 1. On the **Databases/Tables** page, select the resource link, and on the **Actions** menu, choose **Grant**. 1. In the **Grant permissions** section, select **IAM users and roles**. 1. Choose the IAM role that you want to grant access to the resource link. 1. In the **Resource link** permissions section, select **Describe**. 1. Choose **Grant**. 1. Next, grant **LF-Tag key-value permissions** to the principals in the consumer account. You should be able to find the LF-Tags that are shared with you in the consumer account on the Lake Formation console, under **Permissions**, **LF-Tags and permissions**. You can associate tags shared from grantor on resources shared from grantor account that includes: databases, tables, and columns. You can further grant permissions on the resources to other principals. ![\[The screen shows the permissions for LF-Tags in the account.\]](http://docs.aws.amazon.com/lake-formation/latest/dg/images/lf-tag-permissions.png) 1. In the navigation pane, under **Permissions**, **Data permissions**, choose **Grant**. 1. On the **Grant permissions** page, choose **IAM users and roles**. 1. Next, choose the IAM users and roles in your account to grant access to the shared databases/tables. 1. Next, for **LF-Tags or catalog resources**, choose **Resources matched by LF-Tags**. 1. Next, choose the key and values of the LF-Tag that is shared with you. 1. Next, choose the database and table permissions that you want to grant to the IAM users and roles. You can also choose **Grantable permissions** that enables the IAM users and roles to grant permissions to other users/roles. 1. Choose **Grant**. 1. You can view the permission grants under **Data permissions** on the Lake Formation console. # Cross-account data sharing using the named resource method You can grant permissions to directly to principals in the another AWS account, or to external AWS accounts or AWS Organizations. Granting Lake Formation permissions to Organizations or organizational units is equivalent to granting the permission to every AWS account in that organization or organizational unit. When you grant permissions to external accounts or organizations, you must include the **Grantable permissions** option. Only the data lake administrator in the external account can access the shared resources until the administrator grants permissions on the shared resources to other principals in the external account. **Note** **Grantable permissions** option is not supported when granting permissions directly to IAM principals from external accounts. Follow instructions in [Granting database permissions using the named resource method](granting-database-permissions.md) to grant cross-account permissions using the named resource method. The following video demonstrates how to share data with an AWS organization using Lake Formation. [![AWS Videos](http://img.youtube.com/vi/https://www.youtube.com/embed/S-Mdcmq6oPM?controls=0&/0.jpg)](http://www.youtube.com/watch?v=https://www.youtube.com/embed/S-Mdcmq6oPM?controls=0&) # Granting permissions on a database or table shared with your account After a Data Catalog resource belonging to another AWS account is shared with your AWS account, as a data lake administrator, you can grant permissions on the shared resource to other principals in your account. You can't, however, grant permissions on the resource to other AWS accounts or organizations. You can use the AWS Lake Formation console, the API, or the AWS Command Line Interface (AWS CLI) to grant the permissions. **To grant permissions on a shared database (named resource method, console)** + Follow the instructions in [Granting database permissions using the named resource method](granting-database-permissions.md). In the **Database** list under **LF-Tags or catalog resources**, ensure that you select the database in the external account, not a resource link for the database. If you don't see the database in the list of databases, ensure that you have accepted the AWS Resource Access Manager (AWS RAM) resource share invitation for the database. For more information, see [Accepting a resource share invitation from AWS RAM](accepting-ram-invite.md). Also, for the `CREATE_TABLE` and `ALTER` permissions, follow the instructions in [Granting data location permissions (same account)](granting-location-permissions-local.md), and be sure to enter the owning account ID in the **Registered account location** field. **To grant permissions on a shared table (named resource method, console)** + Follow the instructions in [Granting table permissions using the named resource method](granting-table-permissions.md). In the **Database** list under **LF-Tags or catalog resources**, ensure that you select the database in the external account, not a resource link for the database. If you don't see the table in the list of tables, ensure that you have accepted the AWS RAM resource share invitation for the table. For more information, see [Accepting a resource share invitation from AWS RAM](accepting-ram-invite.md). Also, for the `ALTER` permission, follow the instructions in [Granting data location permissions (same account)](granting-location-permissions-local.md), and be sure to enter the owning account ID in the **Registered account location** field. **To grant permissions on shared resources (LF-TBAC method, console)** + Follow the instructions in [Granting Data Catalog permissions](granting-catalog-perms-TBAC.md#granting-cat-perms-TBAC-console). In the **LF-Tags or catalog resources** section, grant the exact LF-Tag expression that the external account granted to your account, or a subset of that expression. For example, if an external account granted the LF-Tag expression `module=customers AND environment=production` to your account with the grant option, as a data lake administrator, you can grant that same expression, or `module=customers` or `environment=production` to a principal in your account. You can grant only the same or a subset of the Lake Formation permissions (for example, `SELECT`, `ALTER`, and so on) that were granted on resources through the LF-Tag expression. **To grant permissions on a shared table (named resource method, AWS CLI)** + Enter a command similar to the following. In this example: + Your AWS account ID is 1111-2222-3333. + The account that owns the table and that granted it to your account is 1234-5678-9012. + The `SELECT` permission is being granted on the shared table `pageviews` to user `datalake_user1`. That user is a principal in your account. + The `pageviews` table is in the `analytics` database, which is owned by account 1234-5678-9012. ``` aws lakeformation grant-permissions --principal DataLakePrincipalIdentifier=arn:aws:iam::111122223333:user/datalake_user1 --permissions "SELECT" --resource '{ "Table": {"CatalogId":"123456789012", "DatabaseName":"analytics", "Name":"pageviews"}}' ``` Note that the owning account must be specified in the `CatalogId` property in the `resource` argument. # Granting resource link permissions Follow these steps to grant AWS Lake Formation permissions on one or more resource links to a principal in your AWS account. After you create a resource link, only you can view and access it. (This assumes that **Use only IAM access control for new tables in this database** is not enabled for the database.) To permit other principals in your account to access the resource link, grant at least the `DESCRIBE` permission. **Important** Granting permissions on a resource link doesn't grant permissions on the target (linked) database or table. You must grant permissions on the target separately. You can grant permissions by using the Lake Formation console, the API, or the AWS Command Line Interface (AWS CLI). ------ #### [ console ] **To grant resource link permissions using the Lake Formation console** 1. Do one of the following: + For database resource links, follow the steps in [Granting database permissions using the named resource method](granting-database-permissions.md). to do the following: 1. Select the resource link from the databases list under Data Catalog, **Databases**. 1. Choose **Grant** to open the **Grant permissions** page. 1. Specify principals to grant permissions. 1. The **Catalogs** and **Databases ** fields are populated. + For table resource links, follow the steps in [Granting table permissions using the named resource method](granting-table-permissions.md) to do the following: 1. Select the resource link from the tables list under Data Catalog, **Tables**. 1. Open the **Grant permissions** page. 1. Specify principals. 1. The **Catalogs**, **Databases **, **Tables** fields are populated. 1. Specify principals. 1. Under **Permissions**, select the permissions to grant. Optionally, select grantable permissions. ![\[The Permissions section contains a single tile. The tiles has a group of check boxes for resource link permissions to grant. Check boxes include Drop and Describe. Below that group is another group of the same check boxes for grantable permissions.\]](http://docs.aws.amazon.com/lake-formation/latest/dg/images/grant-resource-link-permissions-TBAC.png) 1. Choose **Grant**. ------ #### [ AWS CLI ] **To grant resource link permissions using AWS CLI** + Run the `grant-permissions` command, specifying a resource link as the resource. **Example** This example grants `DESCRIBE` to user `datalake_user1` on the table resource link `incidents-link` in the database `issues` in AWS account 1111-2222-3333. ``` 1. aws lakeformation grant-permissions --principal DataLakePrincipalIdentifier=arn:aws:iam::111122223333:user/datalake_user1 --permissions "DESCRIBE" --resource '{ "Table": {"DatabaseName":"issues", "Name":"incidents-link"}}' ``` ------ **See Also:** [Creating resource links](creating-resource-links.md) [Lake Formation permissions reference](lf-permissions-reference.md) # Accessing the underlying data of a shared table Assume that AWS account A shares a Data Catalog table with account B—for example, by granting `SELECT` with the grant option on the table to account B. For a principal in account B to be able to read the shared table's underlying data, the following conditions must be met: + The data lake administrator in account B must accept the share. (This isn't necessary if accounts A and B are in the same organization or if the grant was made with the Lake Formation tag-based access control method.) + The data lake administrator must re-grant to the principal the Lake Formation `SELECT` permission that account A granted on the shared table. + The principal must have the following IAM permissions on the table, the database that contains it, and the account A Data Catalog. **Note** In the following IAM policy: Replace ** with the AWS account ID of account A. Replace ** with a valid Region. Replace ** with the name of the database in account A that contains the shared table. Replace ** with the name of the shared table. ------ #### [ JSON ] **** ``` { "Version":"2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "glue:GetTable", "glue:GetTables", "glue:GetPartition", "glue:GetPartitions", "glue:BatchGetPartition", "glue:GetDatabase", "glue:GetDatabases" ], "Resource": [ "arn:aws:glue:us-east-1:111122223333:table//
", "arn:aws:glue:us-east-1:111122223333:database/", "arn:aws:glue:us-east-1:111122223333:catalog" ] }, { "Effect": "Allow", "Action": [ "lakeformation:GetDataAccess" ], "Resource": [ "*" ] } ] } ``` ------ **See Also:** [Accepting a resource share invitation from AWS RAM](accepting-ram-invite.md) # Cross-account CloudTrail logging Lake Formation provides a centralized audit trail of all cross-account access to data in your data lake. When a recipient AWS account accesses data in a shared table, Lake Formation copies the CloudTrail event to the owning account's CloudTrail logs. Copied events include queries against the data by integrated services such as Amazon Athena and Amazon Redshift Spectrum, and data accesses by AWS Glue jobs. CloudTrail events for cross-account operations on Data Catalog resources are similarly copied. As a resource owner, if you enable object-level logging in Amazon S3, you can run queries that join S3 CloudTrail events with Lake Formation CloudTrail events to determine the accounts that have accessed your S3 buckets. **Topics** + [ ## Including principal identities in cross-account CloudTrail logs ](#cross-account-logging-optin) + [ ## Querying CloudTrail logs for Amazon S3 cross-account access ](#cross-account-logging-s3) ## Including principal identities in cross-account CloudTrail logs By default, cross-account CloudTrail events added to the shared resource recipient's logs and copied to resource owner's logs contain only the AWS principal ID of the external account principal—not the human-readable Amazon Resource Name (ARN) of the principal (principal ARN). When sharing resources within trusted boundaries, such as within the same organization or team, you can opt in to include the principal ARN in the CloudTrail events. Resource owner accounts can then track the principals in recipient accounts that access their owned resources. **Important** As a shared resource recipient, to see the principal ARN in events in your own CloudTrail logs, you must opt in to share the principal ARN with the owner account. If the data access occurs through a resource link, two events are logged in the shared resource recipient account: one for the resource link access and one for the target resource access. The event for the resource link access *does* include the principal ARN. The event for the target resource access does not include the principal ARN without the opt-in. The resource link access event is not copied to the owner account. The following is an excerpt from a default cross-account CloudTrail event (without opt-in). The account performing the data access is 1111-2222-3333. This is the log that is shown in both the calling account and the resource owner account. Lake Formation populates logs in both accounts in the cross-account case. ``` { "eventVersion": "1.05", "userIdentity": { "type": "AWSAccount", "principalId": "AROAQGFTBBBGOBWV2EMZA:GlueJobRunnerSession", "accountId": "111122223333" }, "eventSource": "lakeformation.amazonaws.com", "eventName": "GetDataAccess", ... ... "additionalEventData": { "requesterService": "GLUE_JOB", "lakeFormationRoleSessionName": "AWSLF-00-GL-111122223333-G13T0Rmng2" }, ... } ``` As a shared resource consumer, when you opt in to include the principal ARN, the excerpt becomes the following. The `lakeFormationPrincipal` field represents the end role or user performing the query through Amazon Athena, Amazon Redshift Spectrum, or AWS Glue jobs. ``` { "eventVersion": "1.05", "userIdentity": { "type": "AWSAccount", "principalId": "AROAQGFTBBBGOBWV2EMZA:GlueJobRunnerSession", "accountId": "111122223333" }, "eventSource": "lakeformation.amazonaws.com", "eventName": "GetDataAccess", ... ... "additionalEventData": { "requesterService": "GLUE_JOB", "lakeFormationPrincipal": "arn:aws:iam::111122223333:role/ETL-Glue-Role", "lakeFormationRoleSessionName": "AWSLF-00-GL-111122223333-G13T0Rmng2" }, ... } ``` **To opt in to include principal ARNs in cross-account CloudTrail logs** 1. Open the Lake Formation console at [https://console.aws.amazon.com/lakeformation/](https://console.aws.amazon.com/lakeformation/). Sign in as the `Administrator` user, or a user with the `Administrator Access` IAM policy. 1. In the navigation pane, choose **Settings**. 1. On the **Data catalog settings** page, in the **Default permissions for AWS CloudTrail** section, for **Resource owners**, enter one or more AWS resource owner account IDs. Press **Enter** after each account ID. 1. Choose **Save**. Now cross-account CloudTrail events stored in the logs for both the shared resource recipient and the resource owner contain the principal ARN. ## Querying CloudTrail logs for Amazon S3 cross-account access As a shared resource owner, you can query S3 CloudTrail logs to determine the accounts that have accessed your Amazon S3 buckets (provided that you enabled object-level logging in Amazon S3). This applies only to S3 locations that you registered with Lake Formation. If shared resource consumers opt in to include principal ARNs in Lake Formation CloudTrail logs, you can determine the roles or users that accessed the buckets. When running queries with Amazon Athena, you can join Lake Formation CloudTrail events and S3 CloudTrail events on the session name property. Queries can also filter Lake Formation events on `eventName="GetDataAccess"`, and S3 events on `eventName="Get Object"` or `eventName="Put Object"`. The following is an excerpt from a Lake Formation cross-account CloudTrail event where data in a registered S3 location was accessed. ``` { "eventSource": "lakeformation.amazonaws.com", "eventName": "GetDataAccess", .............. .............. "additionalEventData": { "requesterService": "GLUE_JOB", "lakeFormationPrincipal": "arn:aws:iam::111122223333:role/ETL-Glue-Role", "lakeFormationRoleSessionName": "AWSLF-00-GL-111122223333-B8JSAjo5QA" } } ``` The `lakeFormationRoleSessionName` key value, `AWSLF-00-GL-111122223333-B8JSAjo5QA`, can be joined with the session name in the `principalId` key of the S3 CloudTrail event. The following is an excerpt from the S3 CloudTrail event. It shows the location of the session name. ``` { "eventSource": "s3.amazonaws.com", "eventName": "Get Object" .............. .............. "principalId": "AROAQSOX5XXUR7D6RMYLR:AWSLF-00-GL-111122223333-B8JSAjo5QA", "arn": "arn:aws:sets::111122223333:assumed-role/Deformationally/AWSLF-00-GL-111122223333-B8JSAjo5QA", "session Context": { "session Issuer": { "type": "Role", "principalId": "AROAQSOX5XXUR7D6RMYLR", "arn": "arn:aws:iam::111122223333:role/aws-service-role/lakeformation.amazonaws.com/Deformationally", "accountId": "111122223333", "user Name": "Deformationally" }, .............. .............. } ``` The session name is formatted as follows: ``` AWSLF---- ``` **`version-number`** The version of this format, currently `00`. If the session name format changes, the next version will be `01`. **`query-engine-code`** Indicates the entity that accessed the data. Current values are: [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/lake-formation/latest/dg/cross-account-logging.html) **`account-id`** The AWS account ID that requested credentials from Lake Formation. **`suffix`** A randomly generated string. # Managing cross-account permissions using both AWS Glue and Lake Formation It's possible to grant cross-account access to Data Catalog resources and underlying data by using either AWS Glue or AWS Lake Formation. In AWS Glue, you grant cross-account permissions by creating or updating a Data Catalog resource policy. In Lake Formation, you grant cross-account permissions by using the Lake Formation `GRANT/REVOKE` permissions model and the `Grant Permissions` API operation. **Tip** We recommend that rely solely on Lake Formation permissions to secure your data lake. You can view Lake Formation cross-account grants by using the Lake Formation console or the AWS Resource Access Manager (AWS RAM) console. However, those console pages don't show cross-account permissions granted by the AWS Glue Data Catalog resource policy. Similarly, you can view the cross-account grants in the Data Catalog resource policy using the **Settings** page of the AWS Glue console, but that page doesn't show the cross-account permissions granted using Lake Formation. To ensure that you don't miss any grants when viewing and managing cross-account permissions, Lake Formation and AWS Glue require you to perform the following actions to indicate that you are aware of and are permitting cross-account grants by both Lake Formation and AWS Glue. **When granting cross-account permissions using the AWS Glue Data Catalog resource policy** If your account (grantor account or producer account) has made no cross-account grants that uses AWS RAM to share the resources, you can save a Data Catalog resource policy as usual in AWS Glue. However, if grants that involve AWS RAM resource shares have already been made, you must do one of the following to ensure that saving the resource policy succeeds: + When you save the resource policy on the **Settings** page of the AWS Glue console, the console issues an alert stating that the permissions in the policy will be in addition to any permissions granted using the Lake Formation console. You must choose **Proceed** to save the policy. + When you save the resource policy using the `glue:PutResourcePolicy` API operation, you must set the `EnableHybrid` field to '`TRUE`' (type = string). To update an existing resource policy, use the `glue:GetResourcePolicy` API operation to retrieve your current policy first, then modify it as needed before calling `glue:PutResourcePolicy`. **Note** When creating AWS Glue resource policies for cross-account access, grant only the minimum permissions required for your specific use case. For more information, see [PutResourcePolicy Action (Python: put\$1resource\$1policy)](https://docs.aws.amazon.com/glue/latest/dg/aws-glue-api-jobs-security.html#aws-glue-api-jobs-security-PutResourcePolicy) in the *AWS Glue Developer Guide*. **When granting cross-account permissions using the Lake Formation named resources method** If there is no Data Catalog resource policy in your account (producer account), Lake Formation cross-account grants that you make proceed as usual. However, if a Data Catalog resource policy exists, you must add the following statement to it to permit your cross-account grants to succeed if they are made with the named resource method. Replace ** with a valid Region name and ** with your AWS account ID (producer account ID). ``` { "Effect": "Allow", "Action": [ "glue:ShareResource" ], "Principal": {"Service": [ "ram.amazonaws.com" ]}, "Resource": [ "arn:aws:glue:::table/*/*", "arn:aws:glue:::database/*", "arn:aws:glue:::catalog" ] } ``` Without this additional statement, the Lake Formation grant succeeds, but becomes blocked in AWS RAM, and the recipient account can't access the granted resource. **Important** When using the Lake Formation tag-based access control (LF-TBAC) method to make cross-account grants, you must have a Data Catalog resource policy with at least the permissions specified in [Prerequisites](cross-account-prereqs.md). **See Also:** [Metadata access control](access-control-metadata.md) (for a discussion of the named resource method versus the Lake Formation tag-based access control (LF-TBAC) method). [Viewing shared Data Catalog tables and databases](viewing-available-shared-resources.md) [Working with Data Catalog Settings on the AWS Glue Console](https://docs.aws.amazon.com/glue/latest/dg/console-data-catalog-settings.html) in the *AWS Glue Developer Guide* [Granting Cross-Account Access](https://docs.aws.amazon.com/glue/latest/dg/cross-account-access.html) in the *AWS Glue Developer Guide* (for sample Data Catalog resource policies) # Viewing all cross-account grants using the GetResourceShares API operation If your enterprise grants cross-account permissions using both an AWS Glue Data Catalog resource policy and Lake Formation grants, the only way to view all cross-account grants in one place is to use the `glue:GetResourceShares` API operation. When you grant Lake Formation permissions across accounts by using the named resource method, AWS Resource Access Manager (AWS RAM) creates an AWS Identity and Access Management (IAM) resource policy and stores it in your AWS account. The policy grants the permissions required to access the resource. AWS RAM creates a separate resource policy for each cross-account grant. You can view all of these policies by using the `glue:GetResourceShares` API operation. **Note** This operation also returns the Data Catalog resource policy. However, if you enabled meta data encryption in Data Catalog settings, and you don't have permission on the AWS KMS key, the operation won't return the Data Catalog resource policy. **To view all cross-account grants** + Enter the following AWS CLI command. ``` aws glue get-resource-policies ``` The following is an example resource policy that AWS RAM creates and stores when you grant permissions on table `t` in database `db1` to AWS account 1111-2222-3333. ------ #### [ JSON ] **** ``` { "Version":"2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "glue:GetTable", "glue:GetTables", "glue:GetTableVersion", "glue:GetTableVersions", "glue:GetPartition", "glue:GetPartitions", "glue:BatchGetPartition", "glue:SearchTables" ], "Principal": {"AWS": [ "111122223333" ]}, "Resource": [ "arn:aws:glue:us-east-1:111122223333:table/db1/t" ] } ] } ``` ------ **See also:** [GetResourceShares Action (Python: get\$1resource\$1policies)](https://docs.aws.amazon.com/glue/latest/dg/aws-glue-api-jobs-security.html#aws-glue-api-jobs-security-GetResourcePolicies) in the *AWS Glue Developer Guide* # Accessing and viewing shared Data Catalog tables and databases For the data lake administrator and for principals who have been granted permissions, resources that are shared with your AWS account appear in the Data Catalog as if they were resources in your account. The console displays the account that owns the resource. You can view resources that are shared with your account by using the Lake Formation console. You can also use the AWS Resource Access Manager (AWS RAM) console to view both resources that are shared with your account and resources that you've shared with other AWS accounts by using the named resource method. **Important** When someone uses the named resource method to grant cross-account permissions on a Data Catalog resource to your account or AWS organization, Lake Formation uses the AWS Resource Access Manager (AWS RAM) service to share the resource. If your account is in the same AWS organization as the granting account, the shared resource is available to you immediately. However, if your account is not in the same organization, AWS RAM sends an invitation to your account to accept or reject the resource share. Then, to make the shared resource available, the data lake administrator in your account must use the AWS RAM console or CLI to accept the invitation. The Lake Formation console displays an alert if there is an AWS RAM resource share invitation waiting to be accepted. Only users authorized to view AWS RAM invitations receive the alert. **See Also:** [Sharing Data Catalog tables and databases across AWS Accounts](sharing-catalog-resources.md) [Cross-account data sharing in Lake Formation](cross-account-permissions.md) [Accessing the underlying data of a shared table](cross-account-read-data.md) [Metadata access control](access-control-metadata.md) (for information about the named resource method versus the LF-TBAC method for sharing resources.) **Topics** + [ # Accepting a resource share invitation from AWS RAM ](accepting-ram-invite.md) + [ # Viewing shared Data Catalog tables and databases ](viewing-available-shared-resources.md) # Accepting a resource share invitation from AWS RAM Accepting an AWS RAM resource share invitation If a Data Catalog resource is shared with your AWS account and your account is not in the same AWS organization as the sharing account, you do not have access to the shared resource until you accept a resource share invitation from AWS Resource Access Manager (AWS RAM). As a data lake administrator, you must first query AWS RAM for pending invitations and then accept the invitation. You can use the AWS RAM console, API, or AWS Command Line Interface (AWS CLI) to view and accept invitations. **To view and accept a resource share invitation from AWS RAM (console)** 1. Ensure that you have the required AWS Identity and Access Management (IAM) permissions to view and accept resource share invitations. For information about the suggested IAM policies for data lake administrators, see [Data lake administrator permissions](permissions-reference.md#persona-dl-admin). 1. Follow the instructions in [Accepting and Rejecting Invitations](https://docs.aws.amazon.com/ram/latest/userguide/working-with-shared.html#working-with-shared-invitation) in the *AWS RAM User Guide*. **To view and accept a resource share invitation from AWS RAM (AWS CLI)** 1. Ensure that you have the required AWS Identity and Access Management (IAM) permissions to view and accept resource share invitations. For information about the suggested IAM policies for data lake administrators, see [Data lake administrator permissions](permissions-reference.md#persona-dl-admin). 1. Enter the following command to view pending resource share invitations. ``` aws ram get-resource-share-invitations ``` The output should be similar to the following. ``` { "resourceShareInvitations": [ { "resourceShareInvitationArn": "arn:aws:ram:us-east-1:111122223333:resource-share-invitation/a93aa60a-1bd9-46e8-96db-a4e72eec1d9f", "resourceShareName": "111122223333-123456789012-uswuU", "resourceShareArn": "arn:aws:ram:us-east-1:111122223333:resource-share/2a4ab5fb-d859-4751-84f7-8760b35fc1fe", "senderAccountId": "111122223333", "receiverAccountId": "123456789012", "invitationTimestamp": 1589576601.79, "status": "PENDING" } ] } ``` Note the status of `PENDING`. 1. Copy the value of the `resourceShareInvitationArn` key to the clipboard. 1. Paste the value into the following command, replacing **, and enter the command. ``` aws ram accept-resource-share-invitation --resource-share-invitation-arn ``` The output should be similar to the following. ``` { "resourceShareInvitations": [ { "resourceShareInvitationArn": "arn:aws:ram:us-east-1:111122223333:resource-share-invitation/a93aa60a-1bd9-46e8-96db-a4e72eec1d9f", "resourceShareName": "111122223333-123456789012-uswuU", "resourceShareArn": "arn:aws:ram:us-east-1:111122223333:resource-share/2a4ab5fb-d859-4751-84f7-8760b35fc1fe", "senderAccountId": "111122223333", "receiverAccountId": "123456789012", "invitationTimestamp": 1589576601.79, "status": "ACCEPTED" } ] } ``` Note the status of `ACCEPTED`. # Viewing shared Data Catalog tables and databases You can view resources that are shared with your account by using the Lake Formation console or AWS CLI. You can also use the AWS Resource Access Manager (AWS RAM) console or CLI to view both resources that are shared with your account and resources that you've shared with other AWS accounts. **To view shared resources using the Lake Formation console** 1. Open the Lake Formation console at [https://console.aws.amazon.com/lakeformation/](https://console.aws.amazon.com/lakeformation/). Sign in as a data lake administrator or a user who has been granted permissions on a shared table. 1. To view resources that are shared with your AWS account, do one of the following: + To view tables that are shared with your account, in the navigation pane, choose **Tables**. + To view databases that are shared with your account, in the navigation pane, choose **Databases**. The console displays a list of databases or tables both in your account and shared with your account. For resources that are shared with your account, the console displays the owner's AWS account ID under the **Owner account ID** column (the third column in the following screenshot). ![\[The Tables page shows different owner account IDs for the tables.\]](http://docs.aws.amazon.com/lake-formation/latest/dg/images/tables-with-shared.png) 1. To view resources that you shared with other AWS accounts or organizations, in the navigation pane, choose **Data permissions**. Resources that you shared are listed on the **Data permissions** page with the external account number shown in the **Principal** column, as shown in the following image. ![\[The Data permissions page shows that your account granted permissions on a table to an external account. The AWS account ID is under the Principal column.\]](http://docs.aws.amazon.com/lake-formation/latest/dg/images/permissions-with-cross.png) **To view shared resources using the AWS RAM console** 1. Ensure that you have the required AWS Identity and Access Management (IAM) permissions to view shared resources using AWS RAM. At a minimum, you must have the `ram:ListResources` permission.This permission is included in the AWS managed policy `AWSLakeFormationCrossAccountManager`. 1. Sign in to the AWS Management Console and open the AWS RAM console at [https://console.aws.amazon.com/ram/home](https://console.aws.amazon.com/ram/home). 1. Do one of the following: + To see resources that you shared, in the navigation pane, under **Shared by me**, choose **Shared resources**. + To see resources that are shared with you, in the navigation pane, under **Shared with me**, choose **Shared resources**. # Creating resource links Resource links are Data Catalog objects that are links to metadata databases and tables—typically to shared databases and tables from other AWS accounts. They help to enable cross-account access to data in the data lake across all AWS Regions. **Note** Lake Formation supports querying Data Catalog tables across AWS Regions. You can access the Data Catalog databases and tables from any AWS Region by creating resource links in those regions that point to shared databases and tables in different Regions. **Topics** + [ # How resource links work in Lake Formation ](resource-links-about.md) + [ # Creating a resource link to a shared Data Catalog table ](create-resource-link-table.md) + [ # Creating a resource link to a shared Data Catalog database ](create-resource-link-database.md) + [ # Resource link handling in AWS Glue APIs ](resource-links-glue-apis.md) # How resource links work in Lake Formation How resource links work A *resource link* is a Data Catalog object that is a link to a local or shared database or table. After you create a resource link to a database or table, you can use the resource link name wherever you would use the database or table name. Along with tables that you own or tables that are shared with you, table resource links are returned by `glue:GetTables()` and appear as entries on the **Tables** page of the Lake Formation console. Resource links to databases act in a similar manner. Creating a resource link to a database or table enables you to do the following: + Assign a different name to a database or table in your Data Catalog. This is especially useful if different AWS accounts share databases or tables with the same name, or if multiple databases in your account have tables with the same name. + Access the Data Catalog databases and tables from any AWS Region by creating resource links in those regions pointing to the database and tables in another region. You can run queries in any region with these resource links using Athena, Amazon EMR and run AWS Glue ETL Spark jobs, without copying source data nor the metadata in Glue Data Catalog. + Use integrated AWS services such as Amazon Athena and Amazon Redshift Spectrum to run queries that access shared databases or tables. Some integrated services can't directly access databases or tables across accounts. However, they can access resource links in your account to databases and tables in other accounts. **Note** You don't need to create a resource link to reference a shared database or table in AWS Glue extract, transform, and load (ETL) scripts. However, to avoid ambiguity when multiple AWS accounts share a database or table with the same name, you can either create and use a resource link or specify the catalog ID when invoking ETL operations. The following example shows the Lake Formation console **Tables** page, which lists two resource links. Resource link names are always displayed in italics. Each resource link is displayed along with the name and owner of its linked shared resource. In this example, a data lake administrator in AWS account 1111-2222-3333 shared the `inventory` and `incidents` tables with account 1234-5678-9012. A user in that account then created resource links to those shared tables. ![\[The Tables page shows two resource links. The resource link name is shown under the Name column, the shared table name is shown under the Shared resource column, and the account that shared the table is shown under the Shared resource owner column.\]](http://docs.aws.amazon.com/lake-formation/latest/dg/images/tables-with-links.png) The following are notes and restrictions on resource links: + Resource links are required to enable integrated services such as Athena and Redshift Spectrum to query the underlying data of shared tables. Queries in these integrated services are constructed against the resource link names. + Assuming that the setting **Use only IAM access control for new tables in this database** is turned off for the containing database, only the principal who created a resource link can view and access it. To enable other principals in your account to access a resource link, grant the `DESCRIBE` permission on it. To enable others to drop a resource link, grant the `DROP` permission on it. Data lake administrators can access all resource links in the account. To drop a resource link created by another principal, the data lake administrator must first grant themselves the `DROP` permission on the resource link. For more information, see [Lake Formation permissions reference](lf-permissions-reference.md). **Important** Granting permissions on a resource link doesn't grant permissions on the target (linked) database or table. You must grant permissions on the target separately. + To create a resource link, you need the Lake Formation `CREATE_TABLE` or `CREATE_DATABASE` permission, as well as the `glue:CreateTable` or `glue:CreateDatabase` AWS Identity and Access Management (IAM) permission. + You can create resource links to local (owned) Data Catalog resources, as well as to resources shared with your AWS account. + When you create a resource link, no check is performed to see if the target shared resource exists or whether you have cross-account permissions on the resource. This enables you to create the resource link and shared resource in any order. + If you delete a resource link, the linked shared resource is not dropped. If you drop a shared resource, resource links to that resource are not deleted. + It's possible to create resource link chains. However, there is no value in doing so, because the APIs follow only the first resource link. **See also:** [Granting permissions on Data Catalog resources](granting-catalog-permissions.md) # Creating a resource link to a shared Data Catalog table Creating a resource link to a shared table You can create a resource link to a shared table in any AWS Region by using the AWS Lake Formation console, API, or AWS Command Line Interface (AWS CLI). **To create a resource link to a shared table (console)** 1. Open the AWS Lake Formation console at [https://console.aws.amazon.com/lakeformation/](https://console.aws.amazon.com/lakeformation/). Sign in as a principal who has the Lake Formation `CREATE_TABLE` permission on the database to contain the resource link. 1. In the navigation pane, choose **Tables** under Data Catalog, and then choose **Create**, **Resource link**. 1. On the **Create resource link** page, provide the following information: **Resource link name** Enter a name that adheres to the same rules as a table name. The name can be the same as the target shared table. **Database** The database in the local Data Catalog to contain the resource link. **Shared table owner Region** If you are creating the resource link in a different Region, select the region of the target shared table. **Shared table** Select a shared table from the list, or enter a local (owned) or shared table name. The list contains all the tables shared with your account. Note the database and owner account ID that are listed with each table. If you don't see a table that you know was shared with your account, check the following: + If you aren't a data lake administrator, check that the data lake administrator granted you Lake Formation permissions on the table. + If you are a data lake administrator, and your account is not in the same AWS organization as the granting account, ensure that you have accepted the AWS Resource Access Manager (AWS RAM) resource share invitation for the table. For more information, see [Accepting a resource share invitation from AWS RAM](accepting-ram-invite.md). **Shared table's database** If you selected a shared table from the list, this field is populated with the shared table's database in the external account. Otherwise, enter a local database (for a resource link to a local table) or the shared table's database in the external account. **Shared table owner** If you selected a shared table from the list, this field is populated with the shared table's owner account ID. Otherwise, enter your AWS account ID (for a resource link to a local table) or the ID of the AWS account that shared the table. 1. Choose **Create** to create the resource link. You can then view the resource link name under the **Name** column on the **Tables** page. 1. (Optional) Grant the Lake Formation `DESCRIBE` permission on the resource link to principals that must be able to view the link and access the target table. However, granting permissions on a resource link doesn't grant permissions on the target (linked) database or table. You must grant permissions on the target database separately for the table/resource link to be visible in Athena. **To create a resource link to a shared table in the same Region (AWS CLI)** 1. Enter a command similar to the following. ``` aws glue create-table --database-name myissues --table-input '{"Name":"my_customers","TargetTable":{"CatalogId":"111122223333","DatabaseName":"issues","Name":"customers"}}' ``` This command creates a resource link named `my_customers` to the shared table `customers`, which is in the database `issues` in the AWS account 1111-2222-3333. The resource link is stored in the local database `myissues`. 1. (Optional) Grant the Lake Formation `DESCRIBE` permission on the resource link to principals that must be able to view the link and access the target table. However, granting permissions on a resource link doesn't grant permissions on the target (linked) table. You must grant permissions on the target database separately for the table/resource link to be visible in Athena. **To create a resource link to a shared table in a different Region (AWS CLI)** 1. Enter a command similar to the following. ``` aws glue create-table --region eu-west-1 --cli-input-json '{ "CatalogId": "111122223333", "DatabaseName": "ireland_db", "TableInput": { "Name": "rl_useast1salestb_ireland", "TargetTable": { "CatalogId": "444455556666", "DatabaseName": "useast1_salesdb", "Region": "us-east-1", "Name":"useast1_salestb" } } }‘ ``` This command creates a resource link named `rl_useast1salestb_ireland` in the Europe (Ireland) Region to the shared table `useast1_salestb`, which is in the database `useast1_salesdb` in the AWS account 444455556666 in the US East (N. Virginia) Region. The resource link is stored in the local database `ireland_db`. 1. Grant the Lake Formation `DESCRIBE` permission to principals that must be able to view the link and access the link target through the link. However, granting permissions on a resource link doesn't grant permissions on the target (linked) table. You must grant permissions on the target table separately for the table/resource link to be visible in Athena. **See also:** [How resource links work in Lake Formation](resource-links-about.md) [`DESCRIBE`](lf-permissions-reference.md#perm-describe) # Creating a resource link to a shared Data Catalog database Creating a resource link to a shared database You can create a resource link to a shared database by using the AWS Lake Formation console, API, or AWS Command Line Interface (AWS CLI). **To create a resource link to a shared database (console)** 1. Open the AWS Lake Formation console at [https://console.aws.amazon.com/lakeformation/](https://console.aws.amazon.com/lakeformation/). Sign in as a data lake administrator or as a database creator. A database creator is a principal who has been granted the Lake Formation `CREATE_DATABASE` permission. 1. In the navigation pane, choose **Databases**, and then choose **Create**, **Resource link**. 1. On the **Create resource link** page, provide the following information: **Resource link name** Enter a name that adheres to the same rules as a database name. The name can be the same as the target shared database. **Destination catalog** Select the destination catalog for the database resource link. **Shared database owner Region** If you are creating the resource link in a different Region, select the Region of the target shared database. **Shared database** Choose a database from the list, or enter a local (owned) or shared database name. The list contains all the databases shared with your account. Note the owner account ID that is listed with each database. If you don't see a database that you know was shared with your account, check the following: + If you aren't a data lake administrator, check that the data lake administrator granted you Lake Formation permissions on the database. + If you are a data lake administrator, and your account is not in the same AWS organization as the granting account, ensure that you have accepted the AWS Resource Access Manager (AWS RAM) resource share invitation for the database. For more information, see [Accepting a resource share invitation from AWS RAM](accepting-ram-invite.md). **Shared database owner** If you selected a shared database from the list, this field is populated with the shared database's owner account ID. Otherwise, enter your AWS account ID (for a resource link to a local database) or the ID of the AWS account that shared the database. **Shared database's catalog ID** Enter the catalog ID for the shared database. When creating a resource link to a databse that's shared from another AWS account, you need to specify this catalog ID to identify which account's Data Catalog contains the source database. When you select a shared database from the dropdown menu, the system automatically fills in the catalog ID of the account that owns and has shared that database with you. ![\[The Database details dialog box has the Resource link radio button selected, with the following fields filled in: Resource link name, Shared database, Shared database owner ID. Shared database owner ID is disabled (read-only).\]](http://docs.aws.amazon.com/lake-formation/latest/dg/images/create-resource-link-db.png) 1. Choose **Create** to create the resource link. You can then view the resource link name under the **Name** column on the **Databases** page. 1. (Optional) Grant the Lake Formation `DESCRIBE` permission on the resource link to principals from the Europe (Ireland) Region that must be able to view the link and access the target database. However, granting permissions on a resource link doesn't grant permissions on the target (linked) database or table. You must grant permissions on the target database separately for the table/resource link to be visible in Athena. **To create a resource link to a shared database in the same Region(AWS CLI)** 1. Enter a command similar to the following. ``` aws glue create-database --database-input '{"Name":"myissues","TargetDatabase":{"CatalogId":"111122223333","DatabaseName":"issues"}}' ``` This command creates a resource link named `myissues` to the shared database `issues`, which is in the AWS account 1111-2222-3333. 1. (Optional) Grant the Lake Formation `DESCRIBE` permission to principals on the resource link that must be able to view the link and access the target database or table. However, granting permissions on a resource link doesn't grant permissions on the target (linked) database or table. You must grant permissions on the target database separately for the table/resource link to be visible in Athena. **To create a resource link to a shared database in a different Region(AWS CLI)** 1. Enter a command similar to the following. ``` aws glue create-database --region eu-west-1 --cli-input-json '{ "CatalogId": "111122223333", "DatabaseInput": { "Name": "rl_useast1shared_irelanddb", "TargetDatabase": { "CatalogId": "444455556666", "DatabaseName": "useast1shared_db", "Region": "us-east-1" } } }' ``` This command creates a resource link named `rl_useast1shared_irelanddb` in the AWS account 111122223333 in the Europe (Ireland) Region to the shared database `useast1shared_db`, which is in the AWS account 444455556666 in the US East (N. Virginia) Region. 1. Grant the Lake Formation `DESCRIBE` permission to principals from the Europe (Ireland) Region that must be able to view the link and access the link target through the link. **See also:** [How resource links work in Lake Formation](resource-links-about.md) [`DESCRIBE`](lf-permissions-reference.md#perm-describe) # Resource link handling in AWS Glue APIs The following tables explain how the AWS Glue Data Catalog APIs handle database and table resource links. For all `Get*` API operations, only databases and tables that the caller has permissions on get returned. Also, when accessing a target database or table through a resource link, you must have both AWS Identity and Access Management (IAM) and Lake Formation permissions on both the target and the resource link. The Lake Formation permission that is required on resource links is `DESCRIBE`. For more information, see [`DESCRIBE`](lf-permissions-reference.md#perm-describe). **Database API operations** | API operation | Resource link handling | | --- | --- | | CreateDatabase | If the database is a resource link, creates the resource link to the designated target database. | | UpdateDatabase | If the designated database is a resource link, follows the link and updates the target database. If the resource link must be modified to link to a different database, you must delete it and create a new one. | | DeleteDatabase | Deletes the resource link. It doesn't delete the linked (target) database. | | GetDatabase | If the caller has permissions on the target, follows the link to return the target's properties. Otherwise, it returns the properties of the link. | | GetDatabases | Returns a list of databases, including resource links. For each resource link in the result set, the operation follows the link to get the properties of the link target. You must specify ResourceShareType = ALL to see the databases shared with your account. | **Table API operations** | API operation | Resource link handling | | --- | --- | | CreateTable | If the database is a resource link, follows the database link and creates a table in the target database. If the table is a resource link, the operation creates the resource link in the designated database. Creating a table resource link through a database resource link is not supported. | | UpdateTable | If either the table or designated database is a resource link, updates the target table. If both the table and database are resource links, the operation fails. | | DeleteTable | If the designated database is a resource link, follows the link and deletes the table or table resource link in the target database. If the table is a resource link, the operation deletes the table resource link in the designated database. Deleting a table resource link does not delete the target table. | | BatchDeleteTable | Same as DeleteTable. | | GetTable | If the designated database is a resource link, follows the database link and returns the table or table resource link from the target database. Otherwise, if the table is a resource link, the operation follows the link and returns the target table properties. | | GetTables | If the designated database is a resource link, follows the database link and returns the tables and table resource links from the target database. If the target database is a shared database from another AWS account, the operation returns only the shared tables in that database. It doesn't follow the table resource links in the target database. Otherwise, if the designated database is a local (owned) database, the operation returns all the tables in the local database, and follows each table resource link to return target table properties. | | SearchTables | Returns tables and table resource links. It doesn't follow links to return target table properties. You must specify ResourceShareType = ALL to see tables shared with your account. | | GetTableVersion | Same as GetTable. | | GetTableVersions | Same as GetTable. | | DeleteTableVersion | Same as DeleteTable. | | BatchDeleteTableVersion | Same as DeleteTable. | **Partition API operations** | API operation | Resource link handling | | --- | --- | | CreatePartition | If the designated database is a resource link, follows the database link and creates a partition in the designated table in the target database. If the table is a resource link, the operation follows the resource link and creates the partition in the target table. Creating a partition through both a table resource link and database resource link is not supported. | | BatchCreatePartition | Same as CreatePartition. | | UpdatePartition | If the designated database is a resource link, follows the database link and updates the partition in the designated table in the target database. If the table is a resource link, the operation follows the resource link and updates the partition in the target table. Updating a partition through both a table resource link and database resource link is not supported. | | DeletePartition | If the designated database is a resource link, follows the database link and deletes the partition in the designated table in the target database. If the table is a resource link, the operation follows the resource link and deletes the partition in the target table. Deleting a partition through both a table resource link and database resource link is not supported. | | BatchDeletePartition | Same as DeletePartition. | | GetPartition | If the designated database is a resource link, follows the database link and returns partition information from the designated table. Otherwise, if the table is a resource link, the operation follows the link and returns partition information. If both the table and database are resource links, it returns an empty result set. | | GetPartitions | If the designated database is a resource link, follows the database link and returns partition information for all partitions in the designated table. Otherwise, if the table is a resource link, the operation follows the link and returns partition information. If both the table and database are resource links, it returns an empty result set. | | BatchGetPartition | Same as GetPartition. | **User-defined functions API operations** | API operation | Resource Link Handling | | --- | --- | | (All API operations) | If the database is a resource link, follows the resource link and performs the operation on the target database. | **See also:** [How resource links work in Lake Formation](resource-links-about.md) # Accessing tables across Regions Lake Formation supports querying Data Catalog tables across AWS Regions. You can access data in a Region from other Regions using Amazon Athena, Amazon EMR, and AWS Glue ETL by [creating resource links](creating-resource-links.md) in other Regions pointing to the source databases and tables. With cross-Region table access, you can access data across Regions without copying the underlying data or the metadata into the Data Catalog. For example, you can share a database or table in a producer account to a consumer account in Region A. After accepting the resource share invitation in Region A, the data lake administrator of the consumer account can create resource links to the shared resource in Region B. The consumer account administrator can grant permissions on the shared resource to the IAM principals in that account in Region A and can grant resource link permissions in Region B. Using the resource link, the principals in the consumer account can query the shared data from Region B. You can also host the Amazon S3 data source in Region A in a producer account, and register the data location in a central account in Region B. You can create Data Catalog resources in the central account, set up Lake Formation permissions, and share data with consumers in your account or with external accounts in Region B. The cross-Region feature allows users to access these Data Catalog tables from Region C using resource links. Using this feature, you can query federated databases in Apache Hive Metastores across Regions, and also join tables in the local Region with tables in another Region when running queries. Lake Formation supports the following features with cross-Region table access: + LF-Tag based access control + Fine-grained access control permissions + Write operations on the shared database or table with appropriate permissions + Cross-account data sharing at account-level and direct with IAM principals-level Non-administrative users with `Create_Database` and `Create_Table` permissions can create cross-Region resource links. **Note** You can create cross-Region resource links in any Region and access data without applying Lake Formation permissions. For source data in Amazon S3 that isn't registered with Lake Formation, access is determined by IAM permissions policies for Amazon S3 and AWS Glue actions. For limitations, see [Cross-Region data access limitations](x-region-considerations.md). ## Workflows The following diagrams show the workflows for accessing data across AWS Regions from the same AWS account and from an external account. ### Workflow for accessing tables shared within the same AWS account In the diagram below, the data is shared with a user in the same AWS account in the US East (N. Virginia) Region, and the user queries the shared data from the Europe (Ireland) Region. ![\[Diagram showing data sharing between AWS accounts across regions with numbered steps.\]](http://docs.aws.amazon.com/lake-formation/latest/dg/images/cross-region-same-account.png) The data lake administrator performs the following activities (steps 1-2): 1. A data lake administrator sets up an AWS account with the Data Catalog databases and tables and registers an Amazon S3 data location with Lake Formation in the US East (N. Virginia) Region. Grants `Select` permission on a Data Catalog resource (product table in the diagram) to a principal (user) in the same account. 1. Creates a resource link in the Europe (Ireland) Region pointing to the source table in the US East (N. Virginia) Region. Grants `DESCRIBE` permission on the resource link from the Europe (Ireland) Region to the principal. 1. The user queries the table from the Europe (Ireland)Region using Athena. ### Workflow for accessing tables shared with an external AWS account In the diagram below, the producer account (Account A) hosts the Amazon S3 bucket, registers the data location, and shares a Data Catalog table with a consumer account (Account B) in the US East (N. Virginia) Region and a user from the consumer account (Account B) queries the table from the Europe (Ireland) Region. ![\[Diagram showing data sharing between AWS accounts across regions using Amazon S3 and Data Catalog.\]](http://docs.aws.amazon.com/lake-formation/latest/dg/images/cross-region-x-account.png) 1. A data lake administrator sets up an AWS account (producer account) with the Data Catalog resources and an Amazon S3 data location registered with Lake Formation in the US East (N. Virginia) Region. 1. The data lake administrator of the producer account shares a Data Catalog table to a consumer account. 1. The data lake administrator of the consumer account accepts the data share invitation in the US East (N. Virginia) Region and Grants `Select` permission on the shared table to a principal from the same Region. 1. The data lake administrator of the consumer account creates a resource link in the Europe (Ireland) Region pointing to the target shared table in the US East (N. Virginia) Region and grants the user `DESCRIBE` permission on the resource link from Europe (Ireland) Region. 1. The user queries the data from the Europe (Ireland) Region using Athena. # Setting up cross-Region table access To access data from a different Region, you need to first set up the Data Catalog databases and tables in the Region where you register your Amazon S3 data location. You can share the Data Catalog databases and tables with principals in your account or in another account. Then, you need to create data lake administrators who can create resource links pointing to the target shared data location in the Regions where users query the data. **To query data shared within the same account from a different Region** In this section, the target shared table Region is referred to as Region A and users run queries from Region B. 1. **Account setup in Region A (where you create and share the data)** A data lake administrator needs to complete the following actions: 1. Register an Amazon S3 data location. For more information, see [Adding an Amazon S3 location to your data lake](register-data-lake.md). 1. Create databases and tables in the account. This can also be done by a non-administrative user who has permissions to create databases and tables. 1. Grant data permissions on a table to the principals with `Grantable permissions`. For more information see, [Granting permissions on Data Catalog resources](granting-catalog-permissions.md). 1. **Account setup in Region B (where you access the data)** A data lake administrator needs to complete the following actions: 1. Create a resource link in Region B pointing to the target shared table in Region A. Specify the **Shared table owner Region** on the **Create table** screen. ![\[Create table interface showing options for resource link creation and shared table details.\]](http://docs.aws.amazon.com/lake-formation/latest/dg/images/cross-region-resource-link.png) For instructions on creating resource links to databases and tables, see [Creating resource links](creating-resource-links.md). 1. Grant `Describe` permission to IAM principals on the resource link in Region B. For more information on granting permissions on resource links, see [Granting resource link permissions](granting-link-permissions.md). IAM principals in Region B can query the target table through the link using Athena. **To access cross-account data from a different Region** 1. **Producer/grantor account setup** A data lake administrator needs to complete the following actions: 1. Set up the producer/grantor account in Region A. 1. Register an Amazon S3 data location in Region A. 1. Create databases and tables. This can be done by a non-administrative user who has permissions to create tables. 1. Grant data permissions to the consumer/grantee account on a table in Region A with `Grantable permissions`. For more information, see [Sharing Data Catalog tables and databases across AWS accounts or IAM principals from external accounts](cross-account-data-share-steps.md). 1. **Consumer/grantee account setup** A data lake administrator needs to complete the following actions: 1. Accept the resource share invitation from AWS RAM in Region A. 1. Create a resource link in Region B pointing to the shared table. Region B is where users will want to query the table. 1. Grant data permissions on the shared table to IAM principals in Region A. **Note** You must grant permissions to the shared table in the the same Region where the table was shared. 1. Grant permissions to principals on the resource link in Region B. Principals in the consumer account in Region B then query the shared table from Region B using Athena.