Working with AWS managed table buckets - Amazon Simple Storage Service
Permissions to create AWS managed table bucketsQuerying tables in AWS managed table bucketsEncryption for AWS managed table buckets

Working with AWS managed table buckets

AWS managed table buckets are specialized S3 table buckets designed to store AWS managed tables, such as Accelerating data discovery with S3 Metadata journal and live inventory tables. Unlike customer-managed table buckets that you create and manage directly, AWS managed table buckets are automatically provisioned by AWS when you configure features that require AWS managed tables. When managed tables are created, they belong to a predefined namespace based on the source bucket that can't be modified. Each AWS account has one AWS managed table bucket per Region, following the naming convention aws-s3. This bucket serves as a centralized location for all managed tables associated with your account's resources in that Region.

The following table compares AWS managed table buckets with customer-managed table buckets.

Feature AWS managed table buckets Customer-managed table buckets
Creation Automatically created by AWS services You create these manually
Naming Use a standard naming convention (aws-s3) You define your own names
Table creation Only AWS services can create tables You can create tables
Namespace control You can't create or delete namespaces (All tables belong to a fixed namespace) You can create and delete namespaces
Access Read-only access Full access
Encryption You can only change the default encryption (SSE-S3) settings if you encrypted the initial table with a customer manged AWS KMS key. You can set bucket-level default encryption and modify it anytime
Maintenance Managed by AWS services Automated maintenance can be customized at bucket level

Permissions to create AWS managed table buckets

To work with AWS managed table buckets, you need permissions to create AWS managed table buckets and tables, specify encryption settings for AWS managed tables as well as basic read permissions for querying tables.

The following is an example policy that will allow you to create an AWS managed table bucket through service configurations:

{
   "Version":"2012-10-17",
   "Statement":[
      {
         "Sid":"PermissionsToWorkWithMetadataTables",
         "Effect":"Allow",
         "Action":[
             "s3:CreateBucketMetadataTableConfiguration",
             "s3tables:CreateTableBucket",
             "s3tables:CreateNamespace",
             "s3tables:CreateTable",
             "s3tables:GetTable",
             "s3tables:PutTablePolicy"
             "s3tables:PutTableEncryption",
             "kms:DescribeKey"
         ],
         "Resource":[
            "arn:aws:s3:::bucket/amzn-s3-demo-source-bucket",
            "arn:aws:s3tables:region:111122223333:bucket/aws-s3",
            "arn:aws:s3tables:region:111122223333:bucket/aws-s3/table/*"
         ]
       }
    ]
}

The following is an example policy that will allow you to query tables in AWS managed table buckets:

{
   "Version":"2012-10-17",
   "Statement":[
      {
         "Sid":"PermissionsToWorkWithMetadataTables",
         "Effect":"Allow",
         "Action":[
             "s3tables:GetTable",
             "s3tables:GetTableData",
             "s3tables:GetTableMetadataLocation",
             "kms:Decrypt"
         ],
         "Resource":[
            "arn:aws:s3tables:region:111122223333:bucket/aws-s3",
            "arn:aws:s3tables:region:111122223333:bucket/aws-s3/table/*"
         ]
       }
    ]
}

Querying tables in AWS managed table buckets

You can query AWS managed tables in AWS managed table buckets using access methods and engines supported by S3 Tables. The following are some example queries

Using standard SQL

The following example shows how to query AWS managed tables using standard SQL syntax.

SELECT *
FROM "s3tablescatalog/aws-s3"."b_amzn-s3-demo-source-bucket"."inventory"
LIMIT 10;

The following example shows how to join AWS managed tables with your own tables.

SELECT *
FROM "s3tablescatalog/aws-s3"."b_amzn-s3-demo-source-bucket"."inventory" a
JOIN "s3tablescatalog/amzn-s3-demo-table-bucket"."my_namespace"."my_table" b
ON a.key = b.key
LIMIT 10;
Using Spark

The following example shows how to query your table with Spark.

spark.sql("""
    SELECT *
    FROM ice_catalog.inventory a
    JOIN ice_catalog.my_table b
    ON a.key = b.key
""").show(10, true)

The following example shows how to joining your AWS managed table with another table.

SELECT *
FROM inventory a
JOIN my_table b
ON a.key = b.key
LIMIT 10;

Encryption for AWS managed table buckets

By default, AWS managed table buckets are encrypted with server-side encryption using Amazon S3 managed keys (SSE-S3). After your AWS managed table bucket is created, you can use PutTableBucketEncryption to set the bucket's default encryption setting to use server-side encryption with AWS Key Management Service (AWS KMS) keys (SSE-KMS).

During creation of your AWS managed tables, you can choose to encrypt them with SSE-KMS. If you choose to use SSE-KMS, you must provide a customer managed KMS key in the same Region as your AWS managed table bucket. You can set the encryption type for your AWS managed tables only during table creation. After an AWS managed table is created, you can't change its encryption setting.

If you want the AWS managed table bucket and the tables stored in it to use the same KMS key, make sure to use the same KMS key that you used to encrypt your tables to encrypt your table bucket after it's been created. After you've changed the default encryption settings for your table bucket to use SSE-KMS, those encryption settings are used for any future tables that are created in the bucket.