# Data protection in AWS Glue
AWS Glue offers several features that are designed to help protect your data.
**Topics**
+ [Encrypting data at rest](encryption-at-rest.md)
+ [Encrypting data in transit](encryption-in-transit.md)
+ [FIPS compliance](fips-compliance.md)
+ [Key management](key-management.md)
+ [AWS Glue dependency on other AWS services](dependency-on-other-services.md)
+ [Development endpoints](dev-endpoints.md)
# Encrypting data at rest
AWS Glue supports data encryption at rest for [Building visual ETL jobs](author-job-glue.md) and [Developing scripts using development endpoints](dev-endpoint.md). You can configure extract, transform, and load (ETL) jobs and development endpoints to use [AWS Key Management Service (AWS KMS)](https://aws.amazon.com/kms/) keys to write encrypted data at rest. You can also encrypt the metadata stored in the [AWS Glue Data Catalog](components-overview.md#data-catalog-intro) using keys that you manage with AWS KMS. Additionally, you can use AWS KMS keys to encrypt job bookmarks and the logs generated by [crawlers](https://docs.aws.amazon.com/glue/latest/dg/add-crawler.html) and ETL jobs.
You can encrypt metadata objects in your AWS Glue Data Catalog in addition to the data written to Amazon Simple Storage Service (Amazon S3) and Amazon CloudWatch Logs by jobs, crawlers, and development endpoints. When you create jobs, crawlers, and development endpoints in AWS Glue, you can provide encryption settings by attaching a security configuration. Security configurations contain Amazon S3-managed server-side encryption keys (SSE-S3) or customer master keys (CMKs) stored in AWS KMS (SSE-KMS). You can create security configurations using the AWS Glue console.
You can also turn on encryption of the entire Data Catalog in your account. You do so by specifying CMKs stored in AWS KMS.
**Important**
AWS Glue supports only symmetric customer managed keys. For more information, see [Customer Managed Keys (CMKs)](https://docs.aws.amazon.com/kms/latest/developerguide/concepts.html#master_keys) in the *AWS Key Management Service Developer Guide*.
With encryption turned on, when you add Data Catalog objects, run crawlers, run jobs, or start development endpoints, SSE-S3 or SSE-KMS keys are used to write data at rest. In addition, you can configure AWS Glue to only access Java Database Connectivity (JDBC) data stores through a trusted Transport Layer Security (TLS) protocol.
In AWS Glue, you control encryption settings in the following places:
+ The settings of your Data Catalog.
+ The security configurations that you create.
+ The server-side encryption setting (SSE-S3 or SSE-KMS) that is passed as a parameter to your AWS Glue ETL (extract, transform, and load) job.
For more information about how to set up encryption, see [Setting up encryption in AWS Glue](set-up-encryption.md).
**Topics**
+ [Encrypting your Data Catalog](encrypt-glue-data-catalog.md)
+ [Encrypting connection passwords](encrypt-connection-passwords.md)
+ [Encrypting data written by AWS Glue](encryption-security-configuration.md)
# Encrypting your Data Catalog
AWS Glue Data Catalog encryption provides enhanced security for your sensitive data. AWS Glue integrates with AWS Key Management Service (AWS KMS) to encrypt metadata that's stored in the Data Catalog. You can enable or disable encryption settings for resources in the Data Catalog using the AWS Glue console or the AWS CLI.
When you enable encryption for your Data Catalog, all new objects that you create will be encrypted. When you disable encryption, the new objects you create will not be encrypted, but existing encrypted objects will remain encrypted.
You can encrypt your entire Data Catalog using AWS managed encryption keys or customer managed encryption keys. For more information on key types and states, see [AWS Key Management Service concepts](https://docs.aws.amazon.com/kms/latest/developerguide/key-state.html#key-state-cmk-type) in the AWS Key Management Service Developer Guide.
**Note**
When you use the encrypted Data Catalog with a crawler, you must maintain the encryption settings. Removing encryption settings after a crawler has processed an encrypted catalog results in errors. If you need to remove encryption settings, create a new crawler instead of modifying the existing one.
## AWS managed keys
AWS managed keys are KMS keys in your account that are created, managed, and used on your behalf by an AWS service that's integrated with AWS KMS. You can view the AWS managed keys in your account, view their key policies, and audit their use in AWS CloudTrail logs. However, you can't manage these keys or change their permissions.
Encryption at rest automatically integrates with AWS KMS for managing the AWS managed keys for AWS Glue that are used to encrypt your metadata. If an AWS managed key doesn't exist when you enable metadata encryption, AWS KMS automatically creates a new key for you.
For more information, see [AWS managed keys](https://docs.aws.amazon.com/kms/latest/developerguide/concepts.html#aws-managed-cmk).
## Customer managed keys
Customer managed keys are KMS keys in your AWS account that you create, own, and manage. You have full control over these KMS keys. You can:
+ Establish and maintain their key policies, IAM policies, and grants
+ Enable and disable them
+ Rotate their cryptographic material
+ Add tags
+ Create aliases that refer to them
+ Schedule them for deletion
For more information about managing the permissions of a customer managed key, see [Customer managed keys](https://docs.aws.amazon.com/kms/latest/developerguide/concepts.html#customer-cmk).
**Important**
AWS Glue supports only symmetric customer managed keys. The KMS key list displays only symmetric keys. However, if you select **Choose a KMS key ARN**, the console lets you enter an ARN for any key type. Ensure that you enter only ARNs for symmetric keys.
To create a symmetric customer managed key, follow the steps for [creating symmetric customer managed keys](https://docs.aws.amazon.com/kms/latest/developerguide/create-keys.html#create-symmetric-cmk) in the AWS Key Management Service Developer Guide.
When you enable Data Catalog encryption at rest, the following resource types are encrypted using KMS keys:
+ Databases
+ Tables
+ Partitions
+ Table versions
+ Column statistics
+ User-defined functions
+ Data Catalog views
## AWS Glue encryption context
An [encryption context](https://docs.aws.amazon.com/kms/latest/developerguide/concepts.html#encrypt_context) is an optional set of key-value pairs that contain additional contextual information about the data. AWS KMS uses the encryption context as [additional authenticated data](https://docs.aws.amazon.com/crypto/latest/userguide/cryptography-concepts.html#term-aad) to support [authenticated encryption](https://docs.aws.amazon.com/crypto/latest/userguide/cryptography-concepts.html#define-authenticated-encryption). When you include an encryption context in a request to encrypt data, AWS KMS binds the encryption context to the encrypted data. To decrypt data, you include the same encryption context in the request. AWS Glue uses the same encryption context in all AWS KMS cryptographic operations, where the key is `glue_catalog_id` and the value is the `catalogId`.
```
"encryptionContext": {
"glue_catalog_id": "111122223333"
}
```
When you use an AWS managed key or a symmetric customer managed key to encrypt your Data Catalog, you can also use the encryption context in audit records and logs to identify how the key is being used. The encryption context also appears in logs that are generated by AWS CloudTrail or Amazon CloudWatch logs.
## Enabling encryption
You can enable encryption for your AWS Glue Data Catalog objects in the **Data Catalog settings** in the AWS Glue console or by using the AWS CLI.
------
#### [ Console ]
**To enable encryption using the console**
1. Sign in to the AWS Management Console and open the AWS Glue console at [https://console.aws.amazon.com/glue/](https://console.aws.amazon.com/glue/).
1. Choose **Data Catalog** in the navigation pane.
1. On the **Data Catalog settings** page, select the **Metadata encryption** check box, and choose an AWS KMS key.
When you enable encryption, if you don’t specify a customer managed key, the encryption settings use an AWS managed KMS key.
1. (Optional) When you use a customer managed key to encrypt your Data Catalog, the Data Catalog provides an option to register an IAM role to encrypt and decrypt resources. You need to grant your IAM role permissions that AWS Glue can assume on your behalf. This includes AWS KMS permissions to encrypt and decrypt data.
When you create a new resource in the Data Catalog, AWS Glue assumes the IAM role that's provided to encrypt the data. Similarly, when a consumer accesses the resource, AWS Glue assumes the IAM role to decrypt data. If you register an IAM role with the required permissions, the calling principal no longer requires permissions to access the key and decrypt the data.
**Important**
You can delegate KMS operations to an IAM role only when you use a customer managed key to encrypt the Data Catalog resources. KMS role delegation feature doesn't support using AWS managed keys for encrypting Data Catalog resources at this time.
**Warning**
When you enable an IAM role to delegate KMS operations, you can no longer access the Data Catalog resources that were encrypted previously with an AWS managed key.
1. To enable an IAM role that AWS Glue can assume to encrypt and decrypt data on your behalf, select the **Delegate KMS operations to an IAM role** option.
1. Next, choose an IAM role.
To create an IAM role, see [Create an IAM role for AWS Glue](https://docs.aws.amazon.com/glue/latest/dg/create-an-iam-role.html).
The IAM role that AWS Glue assumes to access the Data Catalog must have the permissions to encrypt and decrypt metadata in the Data Catalog. You can create an IAM role, and attach the following inline policies:
+ Add the following policy to include AWS KMS permissions to encrypt and decrypt the Data Catalog.
------
#### [ JSON ]
****
```
{
"Version":"2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"kms:Decrypt",
"kms:Encrypt",
"kms:GenerateDataKey"
],
"Resource": "arn:aws:kms:us-east-1:111122223333:key/"
}
]
}
```
------
+ Next, add the following trust policy to the role for AWS Glue service to assume the IAM role.
------
#### [ JSON ]
****
```
{
"Version":"2012-10-17",
"Statement": [
{
"Sid": "",
"Effect": "Allow",
"Principal": {
"Service": "glue.amazonaws.com"
},
"Action": "sts:AssumeRole"
}
]
}
```
------
+ Next, add the `iam:PassRole` permission to the IAM role.
------
#### [ JSON ]
****
```
{
"Version":"2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"iam:PassRole"
],
"Resource": [
"arn:aws:iam::111122223333:role/"
]
}
]
}
```
------
When you enable encryption, if you haven't specified an IAM role for AWS Glue to assume, the principal accessing the Data Catalog must have permissions to perform the following API operations:
+ `kms:Decrypt`
+ `kms:Encrypt`
+ `kms:GenerateDataKey`
------
#### [ AWS CLI ]
**To enable encryption using the SDK or AWS CLI**
+ Use the `PutDataCatalogEncryptionSettings` API operation. If no key is specified, AWS Glue uses AWS managed encryption key for the customer account to encrypt the Data Catalog.
```
aws glue put-data-catalog-encryption-settings \
--data-catalog-encryption-settings '{
"EncryptionAtRest": {
"CatalogEncryptionMode": "SSE-KMS-WITH-SERVICE-ROLE",
"SseAwsKmsKeyId": "arn:aws:kms:::key/",
"CatalogEncryptionServiceRole":"arn:aws:iam:::role/"
}
}'
```
------
When you enable encryption, all objects that you create in the Data Catalog objects are encrypted. If you clear this setting, the objects you create in the Data Catalog are no longer encrypted. You can continue to access the existing encrypted objects in the Data Catalog with the required KMS permissions.
| |
| --- |
| The AWS KMS key must remain available in the AWS KMS key store for any objects that are encrypted with it in the Data Catalog. If you remove the key, the objects can no longer be decrypted. You might want this in some scenarios to prevent access to Data Catalog metadata. |
## Monitoring your KMS keys for AWS Glue
When you use KMS keys with your Data Catalog resources, you can use AWS CloudTrail or Amazon CloudWatch Logs to track requests that AWS Glue sends to AWS KMS. AWS CloudTrail monitors and records KMS operations that AWS Glue calls to access data that’s encrypted by your KMS keys.
The following examples are AWS CloudTrail events for the `Decrypt` and `GenerateDataKey` operations.
------
#### [ Decrypt ]
```
{
"eventVersion": "1.08",
"userIdentity": {
"type": "AssumedRole",
"principalId": "AROAXPHTESTANDEXAMPLE:Sampleuser01",
"arn": "arn:aws:sts::111122223333:assumed-role/Admin/Sampleuser01",
"accountId": "111122223333",
"accessKeyId": "AKIAIOSFODNN7EXAMPLE",
"sessionContext": {
"sessionIssuer": {
"type": "Role",
"principalId": "AROAXPHTESTANDEXAMPLE",
"arn": "arn:aws:iam::111122223333:role/Admin",
"accountId": "111122223333",
"userName": "Admin"
},
"webIdFederationData": {},
"attributes": {
"creationDate": "2024-01-10T14:33:56Z",
"mfaAuthenticated": "false"
}
},
"invokedBy": "glue.amazonaws.com"
},
"eventTime": "2024-01-10T15:18:11Z",
"eventSource": "kms.amazonaws.com",
"eventName": "Decrypt",
"awsRegion": "eu-west-2",
"sourceIPAddress": "glue.amazonaws.com",
"userAgent": "glue.amazonaws.com",
"requestParameters": {
"encryptionContext": {
"glue_catalog_id": "111122223333"
},
"encryptionAlgorithm": "SYMMETRIC_DEFAULT"
},
"responseElements": null,
"requestID": "43b019aa-34b8-4798-9b98-ee968b2d63df",
"eventID": "d7614763-d3fe-4f84-a1e1-3ca4d2a5bbd5",
"readOnly": true,
"resources": [
{
"accountId": "111122223333",
"type": "AWS::KMS::Key",
"ARN": "arn:aws:kms::111122223333:key/"
}
],
"eventType": "AwsApiCall",
"managementEvent": true,
"recipientAccountId": "111122223333",
"eventCategory": "Management",
"sessionCredentialFromConsole": "true"
}
```
------
#### [ GenerateDataKey ]
```
{
"eventVersion": "1.08",
"userIdentity": {
"type": "AssumedRole",
"principalId": "AROAXPHTESTANDEXAMPLE:V_00_GLUE_KMS_GENERATE_DATA_KEY_111122223333",
"arn": "arn:aws:sts::111122223333:assumed-role/Admin/V_00_GLUE_KMS_GENERATE_DATA_KEY_111122223333",
"accountId": "111122223333",
"accessKeyId": "AKIAIOSFODNN7EXAMPLE",
"sessionContext": {
"sessionIssuer": {
"type": "Role",
"principalId": "AROAXPHTESTANDEXAMPLE",
"arn": "arn:aws:iam::111122223333:role/Admin",
"accountId": "AKIAIOSFODNN7EXAMPLE",
"userName": "Admin"
},
"webIdFederationData": {},
"attributes": {
"creationDate": "2024-01-05T21:15:47Z",
"mfaAuthenticated": "false"
}
},
"invokedBy": "glue.amazonaws.com"
},
"eventTime": "2024-01-05T21:15:47Z",
"eventSource": "kms.amazonaws.com",
"eventName": "GenerateDataKey",
"awsRegion": "eu-west-2",
"sourceIPAddress": "glue.amazonaws.com",
"userAgent": "glue.amazonaws.com",
"requestParameters": {
"keyId": "arn:aws:kms:eu-west-2:AKIAIOSFODNN7EXAMPLE:key/AKIAIOSFODNN7EXAMPLE",
"encryptionContext": {
"glue_catalog_id": "111122223333"
},
"keySpec": "AES_256"
},
"responseElements": null,
"requestID": "64d1783a-4b62-44ba-b0ab-388b50188070",
"eventID": "1c73689b-2ef2-443b-aed7-8c126585ca5e",
"readOnly": true,
"resources": [
{
"accountId": "111122223333",
"type": "AWS::KMS::Key",
"ARN": "arn:aws:kms:eu-west-2:111122223333:key/AKIAIOSFODNN7EXAMPLE"
}
],
"eventType": "AwsApiCall",
"managementEvent": true,
"recipientAccountId": "111122223333",
"eventCategory": "Management"
}
```
------
```
```
# Encrypting connection passwords
You can retrieve connection passwords in the AWS Glue Data Catalog by using the `GetConnection` and `GetConnections` API operations. These passwords are stored in the Data Catalog connection and are used when AWS Glue connects to a Java Database Connectivity (JDBC) data store. When the connection was created or updated, an option in the Data Catalog settings determined whether the password was encrypted, and if so, what AWS Key Management Service (AWS KMS) key was specified.
On the AWS Glue console, you can turn on this option on the **Data catalog settings** page.
**To encrypt connection passwords**
1. Sign in to the AWS Management Console and open the AWS Glue console at [https://console.aws.amazon.com/glue/](https://console.aws.amazon.com/glue/).
1. Choose **Settings** in the navigation pane.
1. On the **Data catalog settings** page, select **Encrypt connection passwords**, and choose an AWS KMS key.
**Important**
AWS Glue supports only symmetric customer master keys (CMKs). The **AWS KMS key** list displays only symmetric keys. However, if you select **Choose a AWS KMS key ARN**, the console lets you enter an ARN for any key type. Ensure that you enter only ARNs for symmetric keys.
For more information, see [Data Catalog settings](console-data-catalog-settings.md).
# Encrypting data written by AWS Glue
A *security configuration* is a set of security properties that can be used by AWS Glue. You can use a security configuration to encrypt data at rest. The following scenarios show some of the ways that you can use a security configuration.
+ Attach a security configuration to an AWS Glue crawler to write encrypted Amazon CloudWatch Logs. For more information about attaching security configurations to crawlers, see [Step 3: Configure security settings](define-crawler-configure-security-settings.md).
+ Attach a security configuration to an extract, transform, and load (ETL) job to write encrypted Amazon Simple Storage Service (Amazon S3) targets and encrypted CloudWatch Logs.
+ Attach a security configuration to an ETL job to write its jobs bookmarks as encrypted Amazon S3 data.
+ Attach a security configuration to a development endpoint to write encrypted Amazon S3 targets.
**Important**
Currently, a security configuration overrides any server-side encryption (SSE-S3) setting that is passed as an ETL job parameter. Thus, if both a security configuration and an SSE-S3 parameter are associated with a job, the SSE-S3 parameter is ignored.
For more information about security configurations, see [Managing security configurations on the AWS Glue console](console-security-configurations.md).
**Topics**
+ [Setting Up AWS Glue to use security configurations](#encryption-setup-Glue)
+ [Creating a route to AWS KMS for VPC jobs and crawlers](#encryption-kms-vpc-endpoint)
+ [Managing security configurations on the AWS Glue console](console-security-configurations.md)
## Setting Up AWS Glue to use security configurations
Follow these steps to set up your AWS Glue environment to use security configurations.
1. Create or update your AWS Key Management Service (AWS KMS) keys to grant AWS KMS permissions to the IAM roles that are passed to AWS Glue crawlers and jobs to encrypt CloudWatch Logs. For more information, see [Encrypt Log Data in CloudWatch Logs Using AWS KMS](https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/encrypt-log-data-kms.html) in the *Amazon CloudWatch Logs User Guide*.
In the following example, *"role1"*, *"role2"*, and *"role3"* are IAM roles that are passed to crawlers and jobs.
```
{
"Effect": "Allow",
"Principal": { "Service": "logs.region.amazonaws.com",
"AWS": [
"role1",
"role2",
"role3"
] },
"Action": [
"kms:Encrypt*",
"kms:Decrypt*",
"kms:ReEncrypt*",
"kms:GenerateDataKey*",
"kms:Describe*"
],
"Resource": "*"
}
```
The `Service` statement, shown as `"Service": "logs.region.amazonaws.com"`, is required if you use the key to encrypt CloudWatch Logs.
1. Ensure that the AWS KMS key is `ENABLED` before it is used.
**Note**
If you are using Iceberg as your data lake framework, Iceberg tables have their own mechanisms to enable server-side encryption. You should enable these configuration in addition to AWS Glue's security configurations. To enable server-side encryption on Iceberg tables, review the guidance from [Iceberg documentation](https://iceberg.apache.org/docs/latest/aws/#s3-server-side-encryption).
## Creating a route to AWS KMS for VPC jobs and crawlers
You can connect directly to AWS KMS through a private endpoint in your virtual private cloud (VPC) instead of connecting over the internet. When you use a VPC endpoint, communication between your VPC and AWS KMS is conducted entirely within the AWS network.
You can create an AWS KMS VPC endpoint within a VPC. Without this step, your jobs or crawlers might fail with a `kms timeout` on jobs or an `internal service exception` on crawlers. For detailed instructions, see [Connecting to AWS KMS Through a VPC Endpoint](https://docs.aws.amazon.com/kms/latest/developerguide/kms-vpc-endpoint.html) in the *AWS Key Management Service Developer Guide*.
As you follow these instructions, on the [VPC console](https://console.aws.amazon.com//vpc), you must do the following:
+ Select **Enable Private DNS name**.
+ Choose the **Security group** (with self-referencing rule) that you use for your job or crawler that accesses Java Database Connectivity (JDBC). For more information about AWS Glue connections, see [Connecting to data](glue-connections.md).
When you add a security configuration to a crawler or job that accesses JDBC data stores, AWS Glue must have a route to the AWS KMS endpoint. You can provide the route with a network address translation (NAT) gateway or with an AWS KMS VPC endpoint. To create a NAT gateway, see [NAT Gateways](https://docs.aws.amazon.com/vpc/latest/userguide/vpc-nat-gateway.html) in the *Amazon VPC User Guide*.
# Managing security configurations on the AWS Glue console
**Warning**
AWS Glue security configurations are not currently supported in Ray jobs.
A *security configuration* in AWS Glue contains the properties that are needed when you write encrypted data. You create security configurations on the AWS Glue console to provide the encryption properties that are used by crawlers, jobs, and development endpoints.
To see a list of all the security configurations that you have created, open the AWS Glue console at [https://console.aws.amazon.com/glue/](https://console.aws.amazon.com/glue/) and choose **Security configurations** in the navigation pane.
The **Security configurations** list displays the following properties about each configuration:
**Name**
The unique name you provided when you created the configuration. The name may contain letters (A-Z), numbers (0-9), hypens (-), or underscores (\$1), and be up to 255 characters long.
**Enable Amazon S3 encryption**
If turned on, the Amazon Simple Storage Service (Amazon S3) encryption mode such as `SSE-KMS` or `SSE-S3` is enabled for metadata store in the data catalog.
**Enable Amazon CloudWatch logs encryption**
If turned on, the Amazon S3 encryption mode such as `SSE-KMS` is enabled when writing logs to Amazon CloudWatch.
**Advanced settings: Enable job bookmark encryption**
If turned on, the Amazon S3 encryption mode such as `CSE-KMS` is enabled when jobs are bookmarked.
You can add or delete configurations in the **Security configurations** section on the console. To see more details for a configuration, choose the configuration name in the list. Details include the information that you defined when you created the configuration.
## Adding a security configuration
To add a security configuration using the AWS Glue console, on the **Security configurations** page, choose **Add security configuration**.
![\[The screenshot shows the Add security configuration page.\]](http://docs.aws.amazon.com/glue/latest/dg/images/add_security_configuration.png)
**Security configuration properties**
Enter a unique security configuration name. The name may contain letters (A-Z), numbers (0-9), hyphens (-), or underscores (\$1), and can be up to 255 characters long.
**Encryption settings**
You can enable at-rest encryption for metadata stored in the Data Catalog in Amazon S3 and logs in Amazon CloudWatch. To set up encryption of data and metadata with AWS Key Management Service (AWS KMS) keys on the AWS Glue console, add a policy to the console user. This policy must specify the allowed resources as key Amazon Resource Names (ARNs) that are used to encrypt Amazon S3 data stores, as in the following example.
------
#### [ JSON ]
****
```
{
"Version":"2012-10-17",
"Statement": {
"Effect": "Allow",
"Action": [
"kms:GenerateDataKey",
"kms:Decrypt",
"kms:Encrypt"
],
"Resource": "arn:aws:kms:us-east-1:111122223333:key/key-id"
}
}
```
------
**Important**
When a security configuration is attached to a crawler or job, the IAM role that is passed must have AWS KMS permissions. For more information, see [Encrypting data written by AWS Glue](encryption-security-configuration.md).
When you define a configuration, you can provide values for the following properties:
**Enable S3 encryption**
When you are writing Amazon S3 data, you use either server-side encryption with Amazon S3 managed keys (SSE-S3) or server-side encryption with AWS KMS managed keys (SSE-KMS). This field is optional. To allow access to Amazon S3, choose an AWS KMS key, or choose **Enter a key ARN** and provide the ARN for the key. Enter the ARN in the form `arn:aws:kms:region:account-id:key/key-id`. You can also provide the ARN as a key alias, such as `arn:aws:kms:region:account-id:alias/alias-name`.
If you enable Spark UI for your job, the Spark UI log file uploaded to Amazon S3 will be applied with the same encryption.
AWS Glue supports only symmetric customer master keys (CMKs). The **AWS KMS key** list displays only symmetric keys. However, if you select **Choose a AWS KMS key ARN**, the console lets you enter an ARN for any key type. Ensure that you enter only ARNs for symmetric keys.
**Enable CloudWatch Logs encryption**
Server-side (SSE-KMS) encryption is used to encrypt CloudWatch Logs. This field is optional. To turn it on, choose an AWS KMS key, or choose **Enter a key ARN** and provide the ARN for the key. Enter the ARN in the form `arn:aws:kms:region:account-id:key/key-id`. You can also provide the ARN as a key alias, such as `arn:aws:kms:region:account-id:alias/alias-name`.
**Advanced settings: Job bookmark encryption**
Client-side (CSE-KMS) encryption is used to encrypt job bookmarks. This field is optional. The bookmark data is encrypted before it is sent to Amazon S3 for storage. To turn it on, choose an AWS KMS key, or choose **Enter a key ARN** and provide the ARN for the key. Enter the ARN in the form `arn:aws:kms:region:account-id:key/key-id`. You can also provide the ARN as a key alias, such as `arn:aws:kms:region:account-id:alias/alias-name`.
For more information, see the following topics in the *Amazon Simple Storage Service User Guide*:
+ For information about `SSE-S3`, see [Protecting Data Using Server-Side Encryption with Amazon S3-Managed Encryption Keys (SSE-S3)](https://docs.aws.amazon.com/AmazonS3/latest/userguide/UsingServerSideEncryption.html).
+ For information about `SSE-KMS`, see [Protecting Data Using Server-Side Encryption with AWS KMS keys](https://docs.aws.amazon.com/AmazonS3/latest/userguide/UsingKMSEncryption.html).
+ For information about `CSE-KMS`, see [ Using a KMS key stored in AWS KMS](https://docs.aws.amazon.com/AmazonS3/latest/userguide/UsingClientSideEncryption.html#client-side-encryption-kms-managed-master-key-intro).
# Encrypting data in transit
AWS provides Transport Layer Security (TLS) encryption for data in motion. You can configure encryption settings for crawlers, ETL jobs, and development endpoints using [security configurations](https://docs.aws.amazon.com/glue/latest/dg/console-security-configurations.html) in AWS Glue. You can turn on AWS Glue Data Catalog encryption via the settings for the Data Catalog.
As of September 4, 2018, AWS KMS (*bring your own key* and *server-side encryption*) for AWS Glue ETL and the AWS Glue Data Catalog is supported.
# FIPS compliance
If you require FIPS 140-2 validated cryptographic modules when accessing AWS through a command line interface or an API, use a FIPS endpoint. For more information about the available FIPS endpoints, see [Federal Information Processing Standard (FIPS) 140-2](https://aws.amazon.com/compliance/fips/).
# Key management
You can use AWS Identity and Access Management (IAM) with AWS Glue to define users, AWS resources, groups, roles and fine-grained policies regarding access, denial, and more.
You can define the access to the metadata using both resource-based and identity-based policies, depending on your organization’s needs. Resource-based policies list the principals that are allowed or denied access to your resources, allowing you to set up policies such as cross-account access. Identity policies are specifically attached to users, groups, and roles within IAM.
For a step-by-step example, see [Restrict access to your AWS Glue Data Catalog with resource-level IAM permissions and resource-based policies](https://aws.amazon.com/blogs/big-data/restrict-access-to-your-aws-glue-data-catalog-with-resource-level-iam-permissions-and-resource-based-policies/) on the AWS Big Data Blog.
The fine-grained access portion of the policy is defined within the `Resource` clause. This portion defines both the AWS Glue Data Catalog object that the action can be performed on, and what resulting objects get returned by that operation.
A *development endpoint* is an environment that you can use to develop and test your AWS Glue scripts. You can add, delete, or rotate the SSH key of a development endpoint.
As of September 4, 2018, AWS KMS (*bring your own key* and *server-side encryption*) for AWS Glue ETL and the AWS Glue Data Catalog is supported.
# AWS Glue dependency on other AWS services
For a user to work with the AWS Glue console, that user must have a minimum set of permissions that allows them to work with the AWS Glue resources for their AWS account. In addition to these AWS Glue permissions, the console requires permissions from the following services:
+ Amazon CloudWatch Logs permissions to display logs.
+ AWS Identity and Access Management (IAM) permissions to list and pass roles.
+ CloudFormation permissions to work with stacks.
+ Amazon Elastic Compute Cloud (Amazon EC2) permissions to list virtual private clouds (VPCs), subnets, security groups, instances, and other objects (to set up Amazon EC2 items such as VPCs when running jobs, crawlers, and creating development endpoints).
+ Amazon Simple Storage Service (Amazon S3) permissions to list buckets and objects, and to retrieve and save scripts.
+ Amazon Redshift permissions to work with clusters.
+ Amazon Relational Database Service (Amazon RDS) permissions to list instances.
# Development endpoints
A development endpoint is an environment that you can use to develop and test your AWS Glue scripts. You can use AWS Glue to create, edit, and delete development endpoints. You can list all the development endpoints that are created. You can add, delete, or rotate the SSH key of a development endpoint. You can also create notebooks that use the development endpoint.
You provide configuration values to provision the development environments. These values tell AWS Glue how to set up the network so that you can access the development endpoint securely, and so that your endpoint can access your data stores. Then, you can create a notebook that connects to the development endpoint. You use your notebook to author and test your ETL script.
Use an AWS Identity and Access Management (IAM) role with permissions similar to the IAM role that you use to run AWS Glue ETL jobs. Use a virtual private cloud (VPC), a subnet, and a security group to create a development endpoint that can connect to your data resources securely. You generate an SSH key pair to connect to the development environment using SSH.
You can create development endpoints for Amazon S3 data and within a VPC that you can use to access datasets using JDBC.
You can install a Jupyter notebook client on your local machine and use it to debug and test ETL scripts on a development endpoint. Or, you can use a Sagemaker notebook to author ETL scripts in JupyterLab on AWS. See [ Use a SageMaker notebook with your development endpoint ](https://docs.aws.amazon.com/glue/latest/dg/dev-endpoint-tutorial-sage.html).
AWS Glue tags Amazon EC2 instances with a name that is prefixed with `aws-glue-dev-endpoint`.
You can set up a notebook server on a development endpoint to run PySpark with AWS Glue extensions.