aws-cdk-lib.aws_glue

AWS Glue Construct Library

This module is part of the AWS Cloud Development Kit project.

AWS Glue is a serverless data integration service that makes it easier to discover, prepare, move, and integrate data from multiple sources for analytics, machine learning (ML), and application development.

The Glue L2 construct has convenience methods working backwards from common use cases and sets required parameters to defaults that align with recommended best practices for each job type. It also provides customers with a balance between flexibility via optional parameter overrides, and opinionated interfaces that discouraging anti-patterns, resulting in reduced time to develop and deploy new resources.

References

• Glue Launch Announcement
• Glue Documentation
• Glue L1 (CloudFormation) Constructs
• Prior version of the aws-cdk-lib/aws-glue module

Migrating from @aws-cdk/aws-glue-alpha

The GA module (aws-cdk-lib/aws-glue) folds in every API change made during GA preparation. If you are coming from an older alpha, you will encounter the union of the changes below in one upgrade. They are grouped by theme; a PR-reference table follows at the end.

Encryption & security

SecurityConfiguration encryption is now built from factories, not object literals — use S3Encryption.s3Managed() / S3Encryption.kms(key?), CloudWatchEncryption.kms(key?), and JobBookmarksEncryption.clientSideKms(key?). The CloudWatchEncryptionMode and JobBookmarksEncryptionMode enums are gone (mode is now implicit); S3EncryptionMode remains.

// Before
new glue.SecurityConfiguration(this, 'Sec', {
  s3Encryption: { mode: glue.S3EncryptionMode.KMS },
});
// After
new glue.SecurityConfiguration(this, 'Sec', {
  s3Encryption: glue.S3Encryption.kms(),
});

S3Table storage & encryption are now value objects — bucket/encryption/ encryptionKey props and the S3Table.encryption/encryptionKey accessors are removed, along with the TableEncryption enum and the deprecated Table/ TableProps (use S3Table).

// Before
new glue.S3Table(this, 'T', { bucket, encryption: glue.TableEncryption.KMS, encryptionKey: key, ... });
// After
new glue.S3Table(this, 'T', {
  storage: glue.S3TableStorage.managedBucket(glue.S3TableEncryption.kms(key)),
  ...
});

New Catalog L2 replaces catalog fields on IDatabase — IDatabase.catalogArn / catalogId are removed in favor of a type-safe ICatalog that carries them.

hasEncryptedData is now a typed prop — supplying a differing or tokenized has_encrypted_data through parameters now throws instead of being silently overridden.

KMS inputs use reference interfaces — S3Table.clientSideEncryptionKey is kms.IKeyRef (was IKey); StorageParameter.writeKmsKeyId takes kms.IKey (was a string).

Jobs

defaultArguments rejects managed & Glue-reserved arguments — each argument now has one source of truth. Set managed args through their dedicated props (continuousLogging, sparkUI, …) instead of defaultArguments. Previously such args were silently honored (Spark/PythonShell) or silently ignored (Ray).

workerType + numberOfWorkers are paired into a workerConfiguration value object for Spark jobs (no longer top-level props). PythonShellJob no longer accepts them (sized by maxCapacity); RayJob no longer accepts workerType (fixed to Z.2X).

// Before
new glue.PySparkEtlJob(this, 'J', { workerType: glue.WorkerType.G_1X, numberOfWorkers: 5, ... });
// After
new glue.PySparkEtlJob(this, 'J', {
  workerConfiguration: { workerType: glue.WorkerType.G_1X, numberOfWorkers: 5 },
  ...
});

Flex jobs default to GlueVersion.V5_0 (was V3_0) — set glueVersion explicitly to keep the old default.

Connections

VPC placement is a value object — subnet / vpc / vpcSubnets are removed; use network: ConnectionNetwork.subnet(...) or ConnectionNetwork.vpc(...).

Tables, schema & data quality

Partition projection variants are internal — the variant fields (integerRange, dateRange, interval, digits, format, intervalUnit, values) are no longer public; DATE projection takes step: { interval, intervalUnit } instead of top-level interval/intervalUnit.

Schema Type is opaque — build column types via the Schema factories or Schema.custom(...) rather than { isPrimitive, inputString } literals.

DataQualityRuleset DQDL is a typed value object — rulesetDqdl: string → dqdl: Dqdl; build with Dqdl.fromString('Rules = [ ... ]').

DataQualityTargetTable is reference-typed — its constructor is removed; use DataQualityTargetTable.fromTable(database, table) / fromTableName(...). DataQualityRulesetProps.clientToken is removed (use the L1 for request-level idempotency).

Workflow triggers (overhauled)

Action and Condition are no longer plain objects — use Action.job(...) / Action.crawler(...) and Condition.job(...) / Condition.crawler(...). Scheduled-trigger helpers collapse into addScheduledTrigger(id, { schedule, ... }) (with TriggerSchedule.daily()/weekly()/cron(...)); addNotifyEventTrigger becomes addEventTrigger; all addXxxTrigger methods return ITriggerRef.

Removed / corrective

• RayJob is not graduated — AWS Glue for Ray is closed to new customers (April 30, 2026). Stay on the alpha if you still need it, or migrate to Amazon EKS with the KubeRay operator.
• OutputFormat.AVRO/ORC type corrected — cross-assignments that relied on the old (buggy) type will now fail to compile in non-TypeScript jsii languages; the intended usage works.

⚠️ Partition index resource replacement

Independent of API changes: partition-index custom resources moved from Custom::AWS to Custom::GluePartitionIndex (AwsCustomResource → Provider-based). You will see resource replacement on the first update after upgrading. Partition indexes are not stateful, so no data is lost. Also, indexes with tokenized key names must now be given an explicit index name.

Create a Glue Job

A Job encapsulates a script that connects to data sources, processes them, and then writes output to a data target. There are three types of Glue Jobs: Spark jobs (with ETL, Streaming, and Flex execution variants), Python Shell jobs, and (deprecated) Ray jobs. Most of the required parameters for these jobs are common across all types, but there are a few differences depending on the languages supported and features provided by each type. For all job types, the L2 defaults to AWS best practice recommendations, such as:

• Use of Secrets Manager for Connection JDBC strings
• Glue job autoscaling
• Default parameter values for Glue job creation

This iteration of the L2 construct introduces breaking changes to the existing glue-alpha-module, but these changes streamline the developer experience, introduce new constants for defaults, and replacing synth-time validations with interface contracts for enforcement of the parameter combinations that Glue supports. As an opinionated construct, the Glue L2 construct does not allow developers to create resources that use non-current versions of Glue or deprecated language dependencies (e.g. deprecated versions of Python). As always, L1s allow you to specify a wider range of parameters if you need or want to use alternative configurations.

Optional and required parameters for each job are enforced via interface rather than validation; see Glue's public documentation for more granular details.

Spark Jobs

ETL Jobs

ETL jobs support pySpark and Scala languages, for which there are separate but similar constructors. ETL jobs default to the G_1X worker type, but you can override this default with any other supported WorkerType (e.g. G_2X, G_4X, G_8X). ETL jobs default to Glue version 4.0, which you can override to any supported GlueVersion (e.g. 3.0, 5.0, 5.1, 6.0). The following ETL features are enabled by default: --enable-metrics, --enable-continuous-cloudwatch-log. The Spark UI (--enable-spark-ui) is off by default; enable it by setting the sparkUI prop. You can find more details about version, worker type and other features in Glue's public documentation.

Note on continuous logging and encryption: Because continuous logging is enabled by default, job driver and executor stdout/stderr are streamed to CloudWatch. Unless you attach a SecurityConfiguration with cloudWatchEncryption, these logs are written to the account-shared, default Glue log group (/aws-glue/jobs/logs-v2/), which is not encrypted with a customer-managed key. Since job logs can contain sensitive runtime data (SQL statements, row values, error stack traces), attach a SecurityConfiguration with cloudWatchEncryption for regulated workloads. The construct emits a synthesis-time warning when continuous logging is on and no SecurityConfiguration is attached.

Reference the pyspark-etl-jobs.test.ts and scalaspark-etl-jobs.test.ts unit tests for examples of required-only and optional job parameters when creating these types of jobs.

For the sake of brevity, examples are shown using the pySpark job variety.

Example with only required parameters:

import * as cdk from 'aws-cdk-lib';
import * as iam from 'aws-cdk-lib/aws-iam';
declare const stack: cdk.Stack;
declare const role: iam.IRole;
declare const script: glue.Code;
new glue.PySparkEtlJob(stack, 'PySparkETLJob', {
  role,
  script,
  jobName: 'PySparkETLJob',
});

import aws_cdk as cdk
import aws_cdk.aws_iam as iam
# stack: cdk.Stack
# role: iam.IRole
# script: glue.Code

glue.PySparkEtlJob(stack, "PySparkETLJob",
    role=role,
    script=script,
    job_name="PySparkETLJob"
)

import software.amazon.awscdk.*;
import software.amazon.awscdk.services.iam.*;
Stack stack;
IRole role;
Code script;

PySparkEtlJob.Builder.create(stack, "PySparkETLJob")
        .role(role)
        .script(script)
        .jobName("PySparkETLJob")
        .build();

using Amazon.CDK;
using Amazon.CDK.AWS.IAM;
Stack stack;
IRole role;
Code script;

new PySparkEtlJob(stack, "PySparkETLJob", new PySparkEtlJobProps {
    Role = role,
    Script = script,
    JobName = "PySparkETLJob"
});

import cdk "github.com/aws/aws-cdk-go/awscdk"
import iam "github.com/aws/aws-cdk-go/awscdk"
var stack Stack
var role IRole
var script Code

glue.NewPySparkEtlJob(stack, jsii.String("PySparkETLJob"), &PySparkEtlJobProps{
	Role: Role,
	Script: Script,
	JobName: jsii.String("PySparkETLJob"),
})

Example with optional override parameters:

import * as cdk from 'aws-cdk-lib';
import * as iam from 'aws-cdk-lib/aws-iam';
declare const stack: cdk.Stack;
declare const role: iam.IRole;
declare const script: glue.Code;
new glue.PySparkEtlJob(stack, 'PySparkETLJob', {
  jobName: 'PySparkETLJobCustomName',
  description: 'This is a description',
  role,
  script,
  glueVersion: glue.GlueVersion.V6_0,
  continuousLogging: { enabled: false },
  workerConfiguration: {
    workerType: glue.WorkerType.G_2X,
    numberOfWorkers: 2,
  },
  maxConcurrentRuns: 100,
  timeout: cdk.Duration.hours(2),
  connections: [glue.Connection.fromConnectionName(stack, 'Connection', 'connectionName')],
  securityConfiguration: glue.SecurityConfiguration.fromSecurityConfigurationName(stack, 'SecurityConfig', 'securityConfigName'),
  tags: {
    FirstTagName: 'FirstTagValue',
    SecondTagName: 'SecondTagValue',
    XTagName: 'XTagValue',
  },
  maxRetries: 2,
});

import aws_cdk as cdk
import aws_cdk.aws_iam as iam
# stack: cdk.Stack
# role: iam.IRole
# script: glue.Code

glue.PySparkEtlJob(stack, "PySparkETLJob",
    job_name="PySparkETLJobCustomName",
    description="This is a description",
    role=role,
    script=script,
    glue_version=glue.GlueVersion.V6_0,
    continuous_logging=glue.ContinuousLoggingProps(enabled=False),
    worker_configuration=glue.WorkerConfiguration(
        worker_type=glue.WorkerType.G_2X,
        number_of_workers=2
    ),
    max_concurrent_runs=100,
    timeout=cdk.Duration.hours(2),
    connections=[glue.Connection.from_connection_name(stack, "Connection", "connectionName")],
    security_configuration=glue.SecurityConfiguration.from_security_configuration_name(stack, "SecurityConfig", "securityConfigName"),
    tags={
        "FirstTagName": "FirstTagValue",
        "SecondTagName": "SecondTagValue",
        "XTagName": "XTagValue"
    },
    max_retries=2
)

import software.amazon.awscdk.*;
import software.amazon.awscdk.services.iam.*;
Stack stack;
IRole role;
Code script;

PySparkEtlJob.Builder.create(stack, "PySparkETLJob")
        .jobName("PySparkETLJobCustomName")
        .description("This is a description")
        .role(role)
        .script(script)
        .glueVersion(GlueVersion.V6_0)
        .continuousLogging(ContinuousLoggingProps.builder().enabled(false).build())
        .workerConfiguration(WorkerConfiguration.builder()
                .workerType(WorkerType.G_2X)
                .numberOfWorkers(2)
                .build())
        .maxConcurrentRuns(100)
        .timeout(Duration.hours(2))
        .connections(List.of(Connection.fromConnectionName(stack, "Connection", "connectionName")))
        .securityConfiguration(SecurityConfiguration.fromSecurityConfigurationName(stack, "SecurityConfig", "securityConfigName"))
        .tags(Map.of(
                "FirstTagName", "FirstTagValue",
                "SecondTagName", "SecondTagValue",
                "XTagName", "XTagValue"))
        .maxRetries(2)
        .build();

using Amazon.CDK;
using Amazon.CDK.AWS.IAM;
Stack stack;
IRole role;
Code script;

new PySparkEtlJob(stack, "PySparkETLJob", new PySparkEtlJobProps {
    JobName = "PySparkETLJobCustomName",
    Description = "This is a description",
    Role = role,
    Script = script,
    GlueVersion = GlueVersion.V6_0,
    ContinuousLogging = new ContinuousLoggingProps { Enabled = false },
    WorkerConfiguration = new WorkerConfiguration {
        WorkerType = WorkerType.G_2X,
        NumberOfWorkers = 2
    },
    MaxConcurrentRuns = 100,
    Timeout = Duration.Hours(2),
    Connections = new [] { Connection.FromConnectionName(stack, "Connection", "connectionName") },
    SecurityConfiguration = SecurityConfiguration.FromSecurityConfigurationName(stack, "SecurityConfig", "securityConfigName"),
    Tags = new Dictionary<string, string> {
        { "FirstTagName", "FirstTagValue" },
        { "SecondTagName", "SecondTagValue" },
        { "XTagName", "XTagValue" }
    },
    MaxRetries = 2
});

import cdk "github.com/aws/aws-cdk-go/awscdk"
import iam "github.com/aws/aws-cdk-go/awscdk"
var stack Stack
var role IRole
var script Code

glue.NewPySparkEtlJob(stack, jsii.String("PySparkETLJob"), &PySparkEtlJobProps{
	JobName: jsii.String("PySparkETLJobCustomName"),
	Description: jsii.String("This is a description"),
	Role: Role,
	Script: Script,
	GlueVersion: glue.GlueVersion_V6_0,
	ContinuousLogging: &ContinuousLoggingProps{
		Enabled: jsii.Boolean(false),
	},
	WorkerConfiguration: &WorkerConfiguration{
		WorkerType: glue.WorkerType_G_2X,
		NumberOfWorkers: jsii.Number(2),
	},
	MaxConcurrentRuns: jsii.Number(100),
	Timeout: cdk.Duration_Hours(jsii.Number(2)),
	Connections: []IConnection{
		glue.Connection_FromConnectionName(stack, jsii.String("Connection"), jsii.String("connectionName")),
	},
	SecurityConfiguration: glue.SecurityConfiguration_FromSecurityConfigurationName(stack, jsii.String("SecurityConfig"), jsii.String("securityConfigName")),
	Tags: map[string]*string{
		"FirstTagName": jsii.String("FirstTagValue"),
		"SecondTagName": jsii.String("SecondTagValue"),
		"XTagName": jsii.String("XTagValue"),
	},
	MaxRetries: jsii.Number(2),
})

Streaming Jobs

Streaming jobs are similar to ETL jobs, except that they perform ETL on data streams using the Apache Spark Structured Streaming framework. Some Spark job features are not available to Streaming ETL jobs. They support Scala and pySpark languages. PySpark streaming jobs run on Python 3. It defaults to the G_1X worker type and Glue 4.0, both of which you can override. The following best practice features are enabled by default: --enable-metrics, --enable-continuous-cloudwatch-log. The Spark UI (--enable-spark-ui) is off by default; enable it by setting the sparkUI prop.

Reference the pyspark-streaming-jobs.test.ts and scalaspark-streaming-jobs.test.ts unit tests for examples of required-only and optional job parameters when creating these types of jobs.

Example with only required parameters:

import * as cdk from 'aws-cdk-lib';
import * as iam from 'aws-cdk-lib/aws-iam';
declare const stack: cdk.Stack;
declare const role: iam.IRole;
declare const script: glue.Code;
new glue.PySparkStreamingJob(stack, 'ImportedJob', { role, script });

import aws_cdk as cdk
import aws_cdk.aws_iam as iam
# stack: cdk.Stack
# role: iam.IRole
# script: glue.Code

glue.PySparkStreamingJob(stack, "ImportedJob", role=role, script=script)

import software.amazon.awscdk.*;
import software.amazon.awscdk.services.iam.*;
Stack stack;
IRole role;
Code script;

PySparkStreamingJob.Builder.create(stack, "ImportedJob").role(role).script(script).build();

using Amazon.CDK;
using Amazon.CDK.AWS.IAM;
Stack stack;
IRole role;
Code script;

new PySparkStreamingJob(stack, "ImportedJob", new PySparkStreamingJobProps { Role = role, Script = script });

import cdk "github.com/aws/aws-cdk-go/awscdk"
import iam "github.com/aws/aws-cdk-go/awscdk"
var stack Stack
var role IRole
var script Code

glue.NewPySparkStreamingJob(stack, jsii.String("ImportedJob"), &PySparkStreamingJobProps{
	Role: Role,
	Script: Script,
})

Example with optional override parameters:

import * as cdk from 'aws-cdk-lib';
import * as iam from 'aws-cdk-lib/aws-iam';
declare const stack: cdk.Stack;
declare const role: iam.IRole;
declare const script: glue.Code;
new glue.PySparkStreamingJob(stack, 'PySparkStreamingJob', {
  jobName: 'PySparkStreamingJobCustomName',
  description: 'This is a description',
  role,
  script,
  glueVersion: glue.GlueVersion.V6_0,
  continuousLogging: { enabled: false },
  workerConfiguration: {
    workerType: glue.WorkerType.G_2X,
    numberOfWorkers: 2,
  },
  maxConcurrentRuns: 100,
  timeout: cdk.Duration.hours(2),
  connections: [glue.Connection.fromConnectionName(stack, 'Connection', 'connectionName')],
  securityConfiguration: glue.SecurityConfiguration.fromSecurityConfigurationName(stack, 'SecurityConfig', 'securityConfigName'),
  tags: {
    FirstTagName: 'FirstTagValue',
    SecondTagName: 'SecondTagValue',
    XTagName: 'XTagValue',
  },
  maxRetries: 2,
});

import aws_cdk as cdk
import aws_cdk.aws_iam as iam
# stack: cdk.Stack
# role: iam.IRole
# script: glue.Code

glue.PySparkStreamingJob(stack, "PySparkStreamingJob",
    job_name="PySparkStreamingJobCustomName",
    description="This is a description",
    role=role,
    script=script,
    glue_version=glue.GlueVersion.V6_0,
    continuous_logging=glue.ContinuousLoggingProps(enabled=False),
    worker_configuration=glue.WorkerConfiguration(
        worker_type=glue.WorkerType.G_2X,
        number_of_workers=2
    ),
    max_concurrent_runs=100,
    timeout=cdk.Duration.hours(2),
    connections=[glue.Connection.from_connection_name(stack, "Connection", "connectionName")],
    security_configuration=glue.SecurityConfiguration.from_security_configuration_name(stack, "SecurityConfig", "securityConfigName"),
    tags={
        "FirstTagName": "FirstTagValue",
        "SecondTagName": "SecondTagValue",
        "XTagName": "XTagValue"
    },
    max_retries=2
)

import software.amazon.awscdk.*;
import software.amazon.awscdk.services.iam.*;
Stack stack;
IRole role;
Code script;

PySparkStreamingJob.Builder.create(stack, "PySparkStreamingJob")
        .jobName("PySparkStreamingJobCustomName")
        .description("This is a description")
        .role(role)
        .script(script)
        .glueVersion(GlueVersion.V6_0)
        .continuousLogging(ContinuousLoggingProps.builder().enabled(false).build())
        .workerConfiguration(WorkerConfiguration.builder()
                .workerType(WorkerType.G_2X)
                .numberOfWorkers(2)
                .build())
        .maxConcurrentRuns(100)
        .timeout(Duration.hours(2))
        .connections(List.of(Connection.fromConnectionName(stack, "Connection", "connectionName")))
        .securityConfiguration(SecurityConfiguration.fromSecurityConfigurationName(stack, "SecurityConfig", "securityConfigName"))
        .tags(Map.of(
                "FirstTagName", "FirstTagValue",
                "SecondTagName", "SecondTagValue",
                "XTagName", "XTagValue"))
        .maxRetries(2)
        .build();

using Amazon.CDK;
using Amazon.CDK.AWS.IAM;
Stack stack;
IRole role;
Code script;

new PySparkStreamingJob(stack, "PySparkStreamingJob", new PySparkStreamingJobProps {
    JobName = "PySparkStreamingJobCustomName",
    Description = "This is a description",
    Role = role,
    Script = script,
    GlueVersion = GlueVersion.V6_0,
    ContinuousLogging = new ContinuousLoggingProps { Enabled = false },
    WorkerConfiguration = new WorkerConfiguration {
        WorkerType = WorkerType.G_2X,
        NumberOfWorkers = 2
    },
    MaxConcurrentRuns = 100,
    Timeout = Duration.Hours(2),
    Connections = new [] { Connection.FromConnectionName(stack, "Connection", "connectionName") },
    SecurityConfiguration = SecurityConfiguration.FromSecurityConfigurationName(stack, "SecurityConfig", "securityConfigName"),
    Tags = new Dictionary<string, string> {
        { "FirstTagName", "FirstTagValue" },
        { "SecondTagName", "SecondTagValue" },
        { "XTagName", "XTagValue" }
    },
    MaxRetries = 2
});

import cdk "github.com/aws/aws-cdk-go/awscdk"
import iam "github.com/aws/aws-cdk-go/awscdk"
var stack Stack
var role IRole
var script Code

glue.NewPySparkStreamingJob(stack, jsii.String("PySparkStreamingJob"), &PySparkStreamingJobProps{
	JobName: jsii.String("PySparkStreamingJobCustomName"),
	Description: jsii.String("This is a description"),
	Role: Role,
	Script: Script,
	GlueVersion: glue.GlueVersion_V6_0,
	ContinuousLogging: &ContinuousLoggingProps{
		Enabled: jsii.Boolean(false),
	},
	WorkerConfiguration: &WorkerConfiguration{
		WorkerType: glue.WorkerType_G_2X,
		NumberOfWorkers: jsii.Number(2),
	},
	MaxConcurrentRuns: jsii.Number(100),
	Timeout: cdk.Duration_Hours(jsii.Number(2)),
	Connections: []IConnection{
		glue.Connection_FromConnectionName(stack, jsii.String("Connection"), jsii.String("connectionName")),
	},
	SecurityConfiguration: glue.SecurityConfiguration_FromSecurityConfigurationName(stack, jsii.String("SecurityConfig"), jsii.String("securityConfigName")),
	Tags: map[string]*string{
		"FirstTagName": jsii.String("FirstTagValue"),
		"SecondTagName": jsii.String("SecondTagValue"),
		"XTagName": jsii.String("XTagValue"),
	},
	MaxRetries: jsii.Number(2),
})

Flex Jobs

The flexible execution class is appropriate for non-urgent jobs such as pre-production jobs, testing, and one-time data loads. Flexible jobs default to Glue version 5.0 and worker type G_1X. The following best practice features are enabled by default: --enable-metrics, --enable-continuous-cloudwatch-log The Spark UI (--enable-spark-ui) is off by default; enable it by setting the sparkUI prop.

Reference the pyspark-flex-etl-jobs.test.ts and scalaspark-flex-etl-jobs.test.ts unit tests for examples of required-only and optional job parameters when creating these types of jobs.

Example with only required parameters:

import * as cdk from 'aws-cdk-lib';
import * as iam from 'aws-cdk-lib/aws-iam';
declare const stack: cdk.Stack;
declare const role: iam.IRole;
declare const script: glue.Code;
new glue.PySparkFlexEtlJob(stack, 'ImportedJob', { role, script });

import aws_cdk as cdk
import aws_cdk.aws_iam as iam
# stack: cdk.Stack
# role: iam.IRole
# script: glue.Code

glue.PySparkFlexEtlJob(stack, "ImportedJob", role=role, script=script)

import software.amazon.awscdk.*;
import software.amazon.awscdk.services.iam.*;
Stack stack;
IRole role;
Code script;

PySparkFlexEtlJob.Builder.create(stack, "ImportedJob").role(role).script(script).build();

using Amazon.CDK;
using Amazon.CDK.AWS.IAM;
Stack stack;
IRole role;
Code script;

new PySparkFlexEtlJob(stack, "ImportedJob", new PySparkFlexEtlJobProps { Role = role, Script = script });

import cdk "github.com/aws/aws-cdk-go/awscdk"
import iam "github.com/aws/aws-cdk-go/awscdk"
var stack Stack
var role IRole
var script Code

glue.NewPySparkFlexEtlJob(stack, jsii.String("ImportedJob"), &PySparkFlexEtlJobProps{
	Role: Role,
	Script: Script,
})

Example with optional override parameters:

import * as cdk from 'aws-cdk-lib';
import * as iam from 'aws-cdk-lib/aws-iam';
declare const stack: cdk.Stack;
declare const role: iam.IRole;
declare const script: glue.Code;
new glue.PySparkFlexEtlJob(stack, 'pySparkFlexEtlJob', {
  jobName: 'pySparkFlexEtlJob',
  description: 'This is a description',
  role,
  script,
  glueVersion: glue.GlueVersion.V6_0,
  continuousLogging: { enabled: false },
  workerConfiguration: {
    workerType: glue.WorkerType.G_2X,
    numberOfWorkers: 2,
  },
  maxConcurrentRuns: 100,
  timeout: cdk.Duration.hours(2),
  connections: [glue.Connection.fromConnectionName(stack, 'Connection', 'connectionName')],
  securityConfiguration: glue.SecurityConfiguration.fromSecurityConfigurationName(stack, 'SecurityConfig', 'securityConfigName'),
  tags: {
    FirstTagName: 'FirstTagValue',
    SecondTagName: 'SecondTagValue',
    XTagName: 'XTagValue',
  },
  maxRetries: 2,
});

import aws_cdk as cdk
import aws_cdk.aws_iam as iam
# stack: cdk.Stack
# role: iam.IRole
# script: glue.Code

glue.PySparkFlexEtlJob(stack, "pySparkFlexEtlJob",
    job_name="pySparkFlexEtlJob",
    description="This is a description",
    role=role,
    script=script,
    glue_version=glue.GlueVersion.V6_0,
    continuous_logging=glue.ContinuousLoggingProps(enabled=False),
    worker_configuration=glue.WorkerConfiguration(
        worker_type=glue.WorkerType.G_2X,
        number_of_workers=2
    ),
    max_concurrent_runs=100,
    timeout=cdk.Duration.hours(2),
    connections=[glue.Connection.from_connection_name(stack, "Connection", "connectionName")],
    security_configuration=glue.SecurityConfiguration.from_security_configuration_name(stack, "SecurityConfig", "securityConfigName"),
    tags={
        "FirstTagName": "FirstTagValue",
        "SecondTagName": "SecondTagValue",
        "XTagName": "XTagValue"
    },
    max_retries=2
)

import software.amazon.awscdk.*;
import software.amazon.awscdk.services.iam.*;
Stack stack;
IRole role;
Code script;

PySparkFlexEtlJob.Builder.create(stack, "pySparkFlexEtlJob")
        .jobName("pySparkFlexEtlJob")
        .description("This is a description")
        .role(role)
        .script(script)
        .glueVersion(GlueVersion.V6_0)
        .continuousLogging(ContinuousLoggingProps.builder().enabled(false).build())
        .workerConfiguration(WorkerConfiguration.builder()
                .workerType(WorkerType.G_2X)
                .numberOfWorkers(2)
                .build())
        .maxConcurrentRuns(100)
        .timeout(Duration.hours(2))
        .connections(List.of(Connection.fromConnectionName(stack, "Connection", "connectionName")))
        .securityConfiguration(SecurityConfiguration.fromSecurityConfigurationName(stack, "SecurityConfig", "securityConfigName"))
        .tags(Map.of(
                "FirstTagName", "FirstTagValue",
                "SecondTagName", "SecondTagValue",
                "XTagName", "XTagValue"))
        .maxRetries(2)
        .build();

using Amazon.CDK;
using Amazon.CDK.AWS.IAM;
Stack stack;
IRole role;
Code script;

new PySparkFlexEtlJob(stack, "pySparkFlexEtlJob", new PySparkFlexEtlJobProps {
    JobName = "pySparkFlexEtlJob",
    Description = "This is a description",
    Role = role,
    Script = script,
    GlueVersion = GlueVersion.V6_0,
    ContinuousLogging = new ContinuousLoggingProps { Enabled = false },
    WorkerConfiguration = new WorkerConfiguration {
        WorkerType = WorkerType.G_2X,
        NumberOfWorkers = 2
    },
    MaxConcurrentRuns = 100,
    Timeout = Duration.Hours(2),
    Connections = new [] { Connection.FromConnectionName(stack, "Connection", "connectionName") },
    SecurityConfiguration = SecurityConfiguration.FromSecurityConfigurationName(stack, "SecurityConfig", "securityConfigName"),
    Tags = new Dictionary<string, string> {
        { "FirstTagName", "FirstTagValue" },
        { "SecondTagName", "SecondTagValue" },
        { "XTagName", "XTagValue" }
    },
    MaxRetries = 2
});

import cdk "github.com/aws/aws-cdk-go/awscdk"
import iam "github.com/aws/aws-cdk-go/awscdk"
var stack Stack
var role IRole
var script Code

glue.NewPySparkFlexEtlJob(stack, jsii.String("pySparkFlexEtlJob"), &PySparkFlexEtlJobProps{
	JobName: jsii.String("pySparkFlexEtlJob"),
	Description: jsii.String("This is a description"),
	Role: Role,
	Script: Script,
	GlueVersion: glue.GlueVersion_V6_0,
	ContinuousLogging: &ContinuousLoggingProps{
		Enabled: jsii.Boolean(false),
	},
	WorkerConfiguration: &WorkerConfiguration{
		WorkerType: glue.WorkerType_G_2X,
		NumberOfWorkers: jsii.Number(2),
	},
	MaxConcurrentRuns: jsii.Number(100),
	Timeout: cdk.Duration_Hours(jsii.Number(2)),
	Connections: []IConnection{
		glue.Connection_FromConnectionName(stack, jsii.String("Connection"), jsii.String("connectionName")),
	},
	SecurityConfiguration: glue.SecurityConfiguration_FromSecurityConfigurationName(stack, jsii.String("SecurityConfig"), jsii.String("securityConfigName")),
	Tags: map[string]*string{
		"FirstTagName": jsii.String("FirstTagValue"),
		"SecondTagName": jsii.String("SecondTagValue"),
		"XTagName": jsii.String("XTagValue"),
	},
	MaxRetries: jsii.Number(2),
})

Python Shell Jobs

Python shell jobs support a Python version that depends on the AWS Glue version you use. These can be used to schedule and run tasks that don't require an Apache Spark environment. Python shell jobs default to Python 3.9 and a MaxCapacity of 0.0625. Python 3.9 supports pre-loaded analytics libraries, enabled by default (librarySet: glue.LibrarySet.ANALYTICS). Set librarySet: glue.LibrarySet.NONE when your libraries are custom or conflict with the pre-installed ones.

Reference the python-shell-job.test.ts unit tests for examples of required-only and optional job parameters when creating these types of jobs.

Example with only required parameters:

import * as cdk from 'aws-cdk-lib';
import * as iam from 'aws-cdk-lib/aws-iam';
declare const stack: cdk.Stack;
declare const role: iam.IRole;
declare const script: glue.Code;
new glue.PythonShellJob(stack, 'ImportedJob', { role, script });

import aws_cdk as cdk
import aws_cdk.aws_iam as iam
# stack: cdk.Stack
# role: iam.IRole
# script: glue.Code

glue.PythonShellJob(stack, "ImportedJob", role=role, script=script)

import software.amazon.awscdk.*;
import software.amazon.awscdk.services.iam.*;
Stack stack;
IRole role;
Code script;

PythonShellJob.Builder.create(stack, "ImportedJob").role(role).script(script).build();

using Amazon.CDK;
using Amazon.CDK.AWS.IAM;
Stack stack;
IRole role;
Code script;

new PythonShellJob(stack, "ImportedJob", new PythonShellJobProps { Role = role, Script = script });

import cdk "github.com/aws/aws-cdk-go/awscdk"
import iam "github.com/aws/aws-cdk-go/awscdk"
var stack Stack
var role IRole
var script Code

glue.NewPythonShellJob(stack, jsii.String("ImportedJob"), &PythonShellJobProps{
	Role: Role,
	Script: Script,
})

Example with optional override parameters:

import * as cdk from 'aws-cdk-lib';
import * as iam from 'aws-cdk-lib/aws-iam';
declare const stack: cdk.Stack;
declare const role: iam.IRole;
declare const script: glue.Code;
declare const extraPythonFile: glue.Code;
new glue.PythonShellJob(stack, 'PythonShellJob', {
  jobName: 'PythonShellJobCustomName',
  description: 'This is a description',
  pythonVersion: glue.PythonVersion.THREE_NINE,
  maxCapacity: glue.MaxCapacity.DPU_1,
  role,
  script,
  extraPythonFiles: [extraPythonFile],
  glueVersion: glue.GlueVersion.V3_0,
  continuousLogging: { enabled: false },
  maxConcurrentRuns: 100,
  timeout: cdk.Duration.hours(2),
  connections: [glue.Connection.fromConnectionName(stack, 'Connection', 'connectionName')],
  securityConfiguration: glue.SecurityConfiguration.fromSecurityConfigurationName(stack, 'SecurityConfig', 'securityConfigName'),
  tags: {
    FirstTagName: 'FirstTagValue',
    SecondTagName: 'SecondTagValue',
    XTagName: 'XTagValue',
  },
  maxRetries: 2,
});

import aws_cdk as cdk
import aws_cdk.aws_iam as iam
# stack: cdk.Stack
# role: iam.IRole
# script: glue.Code
# extra_python_file: glue.Code

glue.PythonShellJob(stack, "PythonShellJob",
    job_name="PythonShellJobCustomName",
    description="This is a description",
    python_version=glue.PythonVersion.THREE_NINE,
    max_capacity=glue.MaxCapacity.DPU_1,
    role=role,
    script=script,
    extra_python_files=[extra_python_file],
    glue_version=glue.GlueVersion.V3_0,
    continuous_logging=glue.ContinuousLoggingProps(enabled=False),
    max_concurrent_runs=100,
    timeout=cdk.Duration.hours(2),
    connections=[glue.Connection.from_connection_name(stack, "Connection", "connectionName")],
    security_configuration=glue.SecurityConfiguration.from_security_configuration_name(stack, "SecurityConfig", "securityConfigName"),
    tags={
        "FirstTagName": "FirstTagValue",
        "SecondTagName": "SecondTagValue",
        "XTagName": "XTagValue"
    },
    max_retries=2
)

import software.amazon.awscdk.*;
import software.amazon.awscdk.services.iam.*;
Stack stack;
IRole role;
Code script;
Code extraPythonFile;

PythonShellJob.Builder.create(stack, "PythonShellJob")
        .jobName("PythonShellJobCustomName")
        .description("This is a description")
        .pythonVersion(PythonVersion.THREE_NINE)
        .maxCapacity(MaxCapacity.DPU_1)
        .role(role)
        .script(script)
        .extraPythonFiles(List.of(extraPythonFile))
        .glueVersion(GlueVersion.V3_0)
        .continuousLogging(ContinuousLoggingProps.builder().enabled(false).build())
        .maxConcurrentRuns(100)
        .timeout(Duration.hours(2))
        .connections(List.of(Connection.fromConnectionName(stack, "Connection", "connectionName")))
        .securityConfiguration(SecurityConfiguration.fromSecurityConfigurationName(stack, "SecurityConfig", "securityConfigName"))
        .tags(Map.of(
                "FirstTagName", "FirstTagValue",
                "SecondTagName", "SecondTagValue",
                "XTagName", "XTagValue"))
        .maxRetries(2)
        .build();

using Amazon.CDK;
using Amazon.CDK.AWS.IAM;
Stack stack;
IRole role;
Code script;
Code extraPythonFile;

new PythonShellJob(stack, "PythonShellJob", new PythonShellJobProps {
    JobName = "PythonShellJobCustomName",
    Description = "This is a description",
    PythonVersion = PythonVersion.THREE_NINE,
    MaxCapacity = MaxCapacity.DPU_1,
    Role = role,
    Script = script,
    ExtraPythonFiles = new [] { extraPythonFile },
    GlueVersion = GlueVersion.V3_0,
    ContinuousLogging = new ContinuousLoggingProps { Enabled = false },
    MaxConcurrentRuns = 100,
    Timeout = Duration.Hours(2),
    Connections = new [] { Connection.FromConnectionName(stack, "Connection", "connectionName") },
    SecurityConfiguration = SecurityConfiguration.FromSecurityConfigurationName(stack, "SecurityConfig", "securityConfigName"),
    Tags = new Dictionary<string, string> {
        { "FirstTagName", "FirstTagValue" },
        { "SecondTagName", "SecondTagValue" },
        { "XTagName", "XTagValue" }
    },
    MaxRetries = 2
});

import cdk "github.com/aws/aws-cdk-go/awscdk"
import iam "github.com/aws/aws-cdk-go/awscdk"
var stack Stack
var role IRole
var script Code
var extraPythonFile Code

glue.NewPythonShellJob(stack, jsii.String("PythonShellJob"), &PythonShellJobProps{
	JobName: jsii.String("PythonShellJobCustomName"),
	Description: jsii.String("This is a description"),
	PythonVersion: glue.PythonVersion_THREE_NINE,
	MaxCapacity: glue.MaxCapacity_DPU_1,
	Role: Role,
	Script: Script,
	ExtraPythonFiles: []Code{
		extraPythonFile,
	},
	GlueVersion: glue.GlueVersion_V3_0,
	ContinuousLogging: &ContinuousLoggingProps{
		Enabled: jsii.Boolean(false),
	},
	MaxConcurrentRuns: jsii.Number(100),
	Timeout: cdk.Duration_Hours(jsii.Number(2)),
	Connections: []IConnection{
		glue.Connection_FromConnectionName(stack, jsii.String("Connection"), jsii.String("connectionName")),
	},
	SecurityConfiguration: glue.SecurityConfiguration_FromSecurityConfigurationName(stack, jsii.String("SecurityConfig"), jsii.String("securityConfigName")),
	Tags: map[string]*string{
		"FirstTagName": jsii.String("FirstTagValue"),
		"SecondTagName": jsii.String("SecondTagValue"),
		"XTagName": jsii.String("XTagValue"),
	},
	MaxRetries: jsii.Number(2),
})

Metrics Control

By default, Glue jobs enable CloudWatch metrics (--enable-metrics) and observability metrics (--enable-observability-metrics) for monitoring and debugging. You can disable these metrics to reduce CloudWatch costs:

import * as cdk from 'aws-cdk-lib';
import * as iam from 'aws-cdk-lib/aws-iam';
declare const stack: cdk.Stack;
declare const role: iam.IRole;
declare const script: glue.Code;

// Disable both metrics for cost optimization
new glue.PySparkEtlJob(stack, 'CostOptimizedJob', {
  role,
  script,
  enableMetrics: false,
  enableObservabilityMetrics: false,
});

// Selective control - keep observability, disable profiling
new glue.PySparkEtlJob(stack, 'SelectiveJob', {
  role,
  script,
  enableMetrics: false,
  // enableObservabilityMetrics defaults to true
});

import aws_cdk as cdk
import aws_cdk.aws_iam as iam
# stack: cdk.Stack
# role: iam.IRole
# script: glue.Code


# Disable both metrics for cost optimization
glue.PySparkEtlJob(stack, "CostOptimizedJob",
    role=role,
    script=script,
    enable_metrics=False,
    enable_observability_metrics=False
)

# Selective control - keep observability, disable profiling
glue.PySparkEtlJob(stack, "SelectiveJob",
    role=role,
    script=script,
    enable_metrics=False
)

import software.amazon.awscdk.*;
import software.amazon.awscdk.services.iam.*;
Stack stack;
IRole role;
Code script;


// Disable both metrics for cost optimization
// Disable both metrics for cost optimization
PySparkEtlJob.Builder.create(stack, "CostOptimizedJob")
        .role(role)
        .script(script)
        .enableMetrics(false)
        .enableObservabilityMetrics(false)
        .build();

// Selective control - keep observability, disable profiling
// Selective control - keep observability, disable profiling
PySparkEtlJob.Builder.create(stack, "SelectiveJob")
        .role(role)
        .script(script)
        .enableMetrics(false)
        .build();

using Amazon.CDK;
using Amazon.CDK.AWS.IAM;
Stack stack;
IRole role;
Code script;


// Disable both metrics for cost optimization
// Disable both metrics for cost optimization
new PySparkEtlJob(stack, "CostOptimizedJob", new PySparkEtlJobProps {
    Role = role,
    Script = script,
    EnableMetrics = false,
    EnableObservabilityMetrics = false
});

// Selective control - keep observability, disable profiling
// Selective control - keep observability, disable profiling
new PySparkEtlJob(stack, "SelectiveJob", new PySparkEtlJobProps {
    Role = role,
    Script = script,
    EnableMetrics = false
});

import cdk "github.com/aws/aws-cdk-go/awscdk"
import iam "github.com/aws/aws-cdk-go/awscdk"
var stack Stack
var role IRole
var script Code


// Disable both metrics for cost optimization
// Disable both metrics for cost optimization
glue.NewPySparkEtlJob(stack, jsii.String("CostOptimizedJob"), &PySparkEtlJobProps{
	Role: Role,
	Script: Script,
	EnableMetrics: jsii.Boolean(false),
	EnableObservabilityMetrics: jsii.Boolean(false),
})

// Selective control - keep observability, disable profiling
// Selective control - keep observability, disable profiling
glue.NewPySparkEtlJob(stack, jsii.String("SelectiveJob"), &PySparkEtlJobProps{
	Role: Role,
	Script: Script,
	EnableMetrics: jsii.Boolean(false),
})

This feature is available for all Spark job types (ETL, Streaming, Flex).

Job Arguments

Glue jobs are configured through a map of name-value arguments (DefaultArguments). This construct manages several of these arguments on your behalf and exposes each one through a dedicated, strongly-typed prop:

Managed argument(s)	Prop
--enable-continuous-cloudwatch-log, --continuous-log-*	continuousLogging
--enable-metrics	enableMetrics
--enable-observability-metrics	enableObservabilityMetrics
--enable-spark-ui, --spark-event-logs-path	sparkUI
--job-language, --class	job class / className
--extra-jars, --user-jars-first, --extra-py-files, --extra-files	extraJars, extraJarsFirst, extraPythonFiles, extraFiles
library-set	librarySet (Python Shell)

The defaultArguments prop is the escape hatch for arguments this construct does not model. Use it for any argument without a dedicated prop:

import * as cdk from 'aws-cdk-lib';
import * as iam from 'aws-cdk-lib/aws-iam';
declare const stack: cdk.Stack;
declare const role: iam.IRole;
declare const script: glue.Code;

new glue.PySparkEtlJob(stack, 'PySparkETLJob', {
  role,
  script,
  defaultArguments: {
    // an argument this construct does not manage
    '--enable-glue-datacatalog': 'true',
  },
});

import aws_cdk as cdk
import aws_cdk.aws_iam as iam
# stack: cdk.Stack
# role: iam.IRole
# script: glue.Code


glue.PySparkEtlJob(stack, "PySparkETLJob",
    role=role,
    script=script,
    default_arguments={
        # an argument this construct does not manage
        "--enable-glue-datacatalog": "true"
    }
)

import software.amazon.awscdk.*;
import software.amazon.awscdk.services.iam.*;
Stack stack;
IRole role;
Code script;


PySparkEtlJob.Builder.create(stack, "PySparkETLJob")
        .role(role)
        .script(script)
        .defaultArguments(Map.of(
                // an argument this construct does not manage
                "--enable-glue-datacatalog", "true"))
        .build();

using Amazon.CDK;
using Amazon.CDK.AWS.IAM;
Stack stack;
IRole role;
Code script;


new PySparkEtlJob(stack, "PySparkETLJob", new PySparkEtlJobProps {
    Role = role,
    Script = script,
    DefaultArguments = new Dictionary<string, string> {
        // an argument this construct does not manage
        { "--enable-glue-datacatalog", "true" }
    }
});

import cdk "github.com/aws/aws-cdk-go/awscdk"
import iam "github.com/aws/aws-cdk-go/awscdk"
var stack Stack
var role IRole
var script Code


glue.NewPySparkEtlJob(stack, jsii.String("PySparkETLJob"), &PySparkEtlJobProps{
	Role: Role,
	Script: Script,
	DefaultArguments: map[string]*string{
		// an argument this construct does not manage
		"--enable-glue-datacatalog": jsii.String("true"),
	},
})

To keep a single, unambiguous way to express each intent, setting a construct-managed argument (any argument in the table above) or a Glue-reserved argument (--debug, --mode, --JOB_NAME, --endpoint) through defaultArguments throws at synthesis time. This holds even when the feature is turned off — for example, enableMetrics: false combined with defaultArguments: { '--enable-metrics': '' } throws rather than silently re-enabling metrics. Configure managed arguments through their dedicated prop instead — for example, use continuousLogging: { enabled: false } rather than defaultArguments: { '--enable-continuous-cloudwatch-log': 'false' }.

Enable Job Run Queuing

AWS Glue job queuing monitors your account level quotas and limits. If quotas or limits are insufficient to start a Glue job run, AWS Glue will automatically queue the job and wait for limits to free up. Once limits become available, AWS Glue will retry the job run. Glue jobs will queue for limits like max concurrent job runs per account, max concurrent Data Processing Units (DPU), and resource unavailable due to IP address exhaustion in Amazon Virtual Private Cloud (Amazon VPC).

Enable job run queuing by setting the jobRunQueuingEnabled property to true.

import * as cdk from 'aws-cdk-lib';
import * as iam from 'aws-cdk-lib/aws-iam';
declare const stack: cdk.Stack;
declare const role: iam.IRole;
declare const script: glue.Code;
new glue.PySparkEtlJob(stack, 'PySparkETLJob', {
  role,
  script,
  jobName: 'PySparkETLJob',
  jobRunQueuingEnabled: true
});

import aws_cdk as cdk
import aws_cdk.aws_iam as iam
# stack: cdk.Stack
# role: iam.IRole
# script: glue.Code

glue.PySparkEtlJob(stack, "PySparkETLJob",
    role=role,
    script=script,
    job_name="PySparkETLJob",
    job_run_queuing_enabled=True
)

import software.amazon.awscdk.*;
import software.amazon.awscdk.services.iam.*;
Stack stack;
IRole role;
Code script;

PySparkEtlJob.Builder.create(stack, "PySparkETLJob")
        .role(role)
        .script(script)
        .jobName("PySparkETLJob")
        .jobRunQueuingEnabled(true)
        .build();

using Amazon.CDK;
using Amazon.CDK.AWS.IAM;
Stack stack;
IRole role;
Code script;

new PySparkEtlJob(stack, "PySparkETLJob", new PySparkEtlJobProps {
    Role = role,
    Script = script,
    JobName = "PySparkETLJob",
    JobRunQueuingEnabled = true
});

import cdk "github.com/aws/aws-cdk-go/awscdk"
import iam "github.com/aws/aws-cdk-go/awscdk"
var stack Stack
var role IRole
var script Code

glue.NewPySparkEtlJob(stack, jsii.String("PySparkETLJob"), &PySparkEtlJobProps{
	Role: Role,
	Script: Script,
	JobName: jsii.String("PySparkETLJob"),
	JobRunQueuingEnabled: jsii.Boolean(true),
})

Uploading scripts from the CDK app repository to S3

Similar to other L2 constructs, the Glue L2 automates uploading local scripts to S3. Use glue.Code.fromAsset(path) to point at a script in your local file structure; it is uploaded to the CDK-managed asset bucket. To reference a script that already exists in S3, use glue.Code.fromBucket(bucket, key), which performs no upload. A script is required for every job.

Reference the unit tests for examples of repo and S3 code target examples.

Workflow Triggers

You can use Glue workflows to create and visualize complex extract, transform, and load (ETL) activities involving multiple crawlers, jobs, and triggers. Standalone triggers are an anti-pattern, so you must create triggers from within a workflow using the L2 construct.

Within a workflow object, there are functions to create different types of triggers with actions and predicates. You add triggers to the workflow, and each trigger references the jobs or crawlers it runs as its actions.

startOnCreation applies to scheduled triggers (and, via ConditionalTriggerOptions, conditional triggers) only. It defaults to false, but you can override it if you prefer for your trigger to start on creation.

Reference the workflow-triggers.test.ts unit tests for examples of creating workflows and triggers.

import * as cdk from 'aws-cdk-lib';
import * as iam from 'aws-cdk-lib/aws-iam';
declare const stack: cdk.Stack;
declare const role: iam.IRole;
declare const script: glue.Code;

// Create a job to run from the workflow
const job = new glue.PySparkEtlJob(stack, 'Job', { role, script });

// Create a workflow and add a trigger that runs the job
const workflow = new glue.Workflow(stack, 'Workflow');
workflow.addOnDemandTrigger('OnDemandTrigger', {
  actions: [glue.Action.job(job)],
});

import aws_cdk as cdk
import aws_cdk.aws_iam as iam
# stack: cdk.Stack
# role: iam.IRole
# script: glue.Code


# Create a job to run from the workflow
job = glue.PySparkEtlJob(stack, "Job", role=role, script=script)

# Create a workflow and add a trigger that runs the job
workflow = glue.Workflow(stack, "Workflow")
workflow.add_on_demand_trigger("OnDemandTrigger",
    actions=[glue.Action.job(job)]
)

import software.amazon.awscdk.*;
import software.amazon.awscdk.services.iam.*;
Stack stack;
IRole role;
Code script;


// Create a job to run from the workflow
PySparkEtlJob job = PySparkEtlJob.Builder.create(stack, "Job").role(role).script(script).build();

// Create a workflow and add a trigger that runs the job
Workflow workflow = new Workflow(stack, "Workflow");
workflow.addOnDemandTrigger("OnDemandTrigger", OnDemandTriggerOptions.builder()
        .actions(List.of(Action.job(job)))
        .build());

using Amazon.CDK;
using Amazon.CDK.AWS.IAM;
Stack stack;
IRole role;
Code script;


// Create a job to run from the workflow
var job = new PySparkEtlJob(stack, "Job", new PySparkEtlJobProps { Role = role, Script = script });

// Create a workflow and add a trigger that runs the job
var workflow = new Workflow(stack, "Workflow");
workflow.AddOnDemandTrigger("OnDemandTrigger", new OnDemandTriggerOptions {
    Actions = new [] { Action.Job(job) }
});

import cdk "github.com/aws/aws-cdk-go/awscdk"
import iam "github.com/aws/aws-cdk-go/awscdk"
var stack Stack
var role IRole
var script Code


// Create a job to run from the workflow
job := glue.NewPySparkEtlJob(stack, jsii.String("Job"), &PySparkEtlJobProps{
	Role: Role,
	Script: Script,
})

// Create a workflow and add a trigger that runs the job
workflow := glue.NewWorkflow(stack, jsii.String("Workflow"))
workflow.AddOnDemandTrigger(jsii.String("OnDemandTrigger"), &OnDemandTriggerOptions{
	Actions: []Action{
		glue.Action_Job(job),
	},
})

1. On-Demand Triggers

On-demand triggers can start glue jobs or crawlers. This construct provides convenience functions to create on-demand crawler or job triggers. The constructor takes an optional description parameter, but abstracts the requirement of an actions list using the job or crawler objects using conditional types.

2. Scheduled Triggers

Use addScheduledTrigger with a TriggerSchedule to fire on a cron schedule. TriggerSchedule.daily() and TriggerSchedule.weekly() are convenience factories; TriggerSchedule.cron(...) lets you build any schedule from the existing event Schedule class without writing raw cron expressions. The L2 extracts the expression that Glue requires from the TriggerSchedule.

import * as cdk from 'aws-cdk-lib';
import * as iam from 'aws-cdk-lib/aws-iam';
declare const stack: cdk.Stack;
declare const role: iam.IRole;
declare const script: glue.Code;
const job = new glue.PySparkEtlJob(stack, 'Job', { role, script });
const workflow = new glue.Workflow(stack, 'Workflow');

workflow.addScheduledTrigger('WeeklyTrigger', {
  actions: [glue.Action.job(job)],
  schedule: glue.TriggerSchedule.weekly(),
});

import aws_cdk as cdk
import aws_cdk.aws_iam as iam
# stack: cdk.Stack
# role: iam.IRole
# script: glue.Code

job = glue.PySparkEtlJob(stack, "Job", role=role, script=script)
workflow = glue.Workflow(stack, "Workflow")

workflow.add_scheduled_trigger("WeeklyTrigger",
    actions=[glue.Action.job(job)],
    schedule=glue.TriggerSchedule.weekly()
)

import software.amazon.awscdk.*;
import software.amazon.awscdk.services.iam.*;
Stack stack;
IRole role;
Code script;

PySparkEtlJob job = PySparkEtlJob.Builder.create(stack, "Job").role(role).script(script).build();
Workflow workflow = new Workflow(stack, "Workflow");

workflow.addScheduledTrigger("WeeklyTrigger", ScheduledTriggerOptions.builder()
        .actions(List.of(Action.job(job)))
        .schedule(TriggerSchedule.weekly())
        .build());

using Amazon.CDK;
using Amazon.CDK.AWS.IAM;
Stack stack;
IRole role;
Code script;

var job = new PySparkEtlJob(stack, "Job", new PySparkEtlJobProps { Role = role, Script = script });
var workflow = new Workflow(stack, "Workflow");

workflow.AddScheduledTrigger("WeeklyTrigger", new ScheduledTriggerOptions {
    Actions = new [] { Action.Job(job) },
    Schedule = TriggerSchedule.Weekly()
});

import cdk "github.com/aws/aws-cdk-go/awscdk"
import iam "github.com/aws/aws-cdk-go/awscdk"
var stack Stack
var role IRole
var script Code

job := glue.NewPySparkEtlJob(stack, jsii.String("Job"), &PySparkEtlJobProps{
	Role: Role,
	Script: Script,
})
workflow := glue.NewWorkflow(stack, jsii.String("Workflow"))

workflow.AddScheduledTrigger(jsii.String("WeeklyTrigger"), &ScheduledTriggerOptions{
	Actions: []Action{
		glue.Action_Job(job),
	},
	Schedule: glue.TriggerSchedule_Weekly(),
})

3. Event Triggers

Use addEventTrigger for EventBridge event-based triggers. There are two types: batching and non-batching. For batching triggers, you must specify batchSize. For non-batching triggers, batchSize defaults to 1. For both, batchWindow defaults to 900 seconds, but you can override the window to align with your workload's requirements.

4. Conditional Triggers

Conditional triggers have a predicate and actions associated with them. The trigger actions are executed when the predicateCondition is true.

Connection Properties

A Connection allows Glue jobs, crawlers and development endpoints to access certain types of data stores.

• Secrets Management Manage JDBC connection credentials in Secrets Manager and pass the secret to the connection via the secret property (see the example below), rather than embedding credentials in properties.

• Networking - the CDK determines the best fit subnet for Glue connection configuration Configure VPC placement through the network property, built with ConnectionNetwork.subnet(subnet) to pin a specific subnet, or ConnectionNetwork.vpc(vpc, vpcSubnets?) to let the L2 select one via the existing EC2 Subnet Selection library. A Glue connection targets a single subnet, so the first subnet of the selection is used. The two factories are mutually exclusive, so a subnet and a VPC can never be combined.

Pin the connection to a specific subnet:

declare const securityGroup: ec2.SecurityGroup;
declare const subnet: ec2.Subnet;
new glue.Connection(this, 'MyConnection', {
  type: glue.ConnectionType.NETWORK,
  // The security groups granting AWS Glue inbound access to the data source within the VPC
  securityGroups: [securityGroup],
  // The VPC subnet which contains the data source
  network: glue.ConnectionNetwork.subnet(subnet),
});

# security_group: ec2.SecurityGroup
# subnet: ec2.Subnet

glue.Connection(self, "MyConnection",
    type=glue.ConnectionType.NETWORK,
    # The security groups granting AWS Glue inbound access to the data source within the VPC
    security_groups=[security_group],
    # The VPC subnet which contains the data source
    network=glue.ConnectionNetwork.subnet(subnet)
)

SecurityGroup securityGroup;
Subnet subnet;

Connection.Builder.create(this, "MyConnection")
        .type(ConnectionType.NETWORK)
        // The security groups granting AWS Glue inbound access to the data source within the VPC
        .securityGroups(List.of(securityGroup))
        // The VPC subnet which contains the data source
        .network(ConnectionNetwork.subnet(subnet))
        .build();

SecurityGroup securityGroup;
Subnet subnet;

new Connection(this, "MyConnection", new ConnectionProps {
    Type = ConnectionType.NETWORK,
    // The security groups granting AWS Glue inbound access to the data source within the VPC
    SecurityGroups = new [] { securityGroup },
    // The VPC subnet which contains the data source
    Network = ConnectionNetwork.Subnet(subnet)
});

var securityGroup SecurityGroup
var subnet Subnet

glue.NewConnection(this, jsii.String("MyConnection"), &ConnectionProps{
	Type: glue.ConnectionType_NETWORK(),
	// The security groups granting AWS Glue inbound access to the data source within the VPC
	SecurityGroups: []ISecurityGroup{
		securityGroup,
	},
	// The VPC subnet which contains the data source
	Network: glue.ConnectionNetwork_Subnet(subnet),
})

Or let the CDK select a subnet from a VPC:

declare const securityGroup: ec2.SecurityGroup;
declare const vpc: ec2.Vpc;
new glue.Connection(this, 'MyConnection', {
  type: glue.ConnectionType.NETWORK,
  securityGroups: [securityGroup],
  // vpcSubnets is optional - defaults to private subnets
  network: glue.ConnectionNetwork.vpc(vpc, { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }),
});

# security_group: ec2.SecurityGroup
# vpc: ec2.Vpc

glue.Connection(self, "MyConnection",
    type=glue.ConnectionType.NETWORK,
    security_groups=[security_group],
    # vpcSubnets is optional - defaults to private subnets
    network=glue.ConnectionNetwork.vpc(vpc, subnet_type=ec2.SubnetType.PRIVATE_WITH_EGRESS)
)

SecurityGroup securityGroup;
Vpc vpc;

Connection.Builder.create(this, "MyConnection")
        .type(ConnectionType.NETWORK)
        .securityGroups(List.of(securityGroup))
        // vpcSubnets is optional - defaults to private subnets
        .network(ConnectionNetwork.vpc(vpc, SubnetSelection.builder().subnetType(SubnetType.PRIVATE_WITH_EGRESS).build()))
        .build();

SecurityGroup securityGroup;
Vpc vpc;

new Connection(this, "MyConnection", new ConnectionProps {
    Type = ConnectionType.NETWORK,
    SecurityGroups = new [] { securityGroup },
    // vpcSubnets is optional - defaults to private subnets
    Network = ConnectionNetwork.Vpc(vpc, new SubnetSelection { SubnetType = SubnetType.PRIVATE_WITH_EGRESS })
});

var securityGroup SecurityGroup
var vpc Vpc

glue.NewConnection(this, jsii.String("MyConnection"), &ConnectionProps{
	Type: glue.ConnectionType_NETWORK(),
	SecurityGroups: []ISecurityGroup{
		securityGroup,
	},
	// vpcSubnets is optional - defaults to private subnets
	Network: glue.ConnectionNetwork_Vpc(vpc, &SubnetSelection{
		SubnetType: ec2.SubnetType_PRIVATE_WITH_EGRESS,
	}),
})

For RDS Connection by JDBC, it is recommended to manage credentials using AWS Secrets Manager. Pass the secret via the secret property: Glue reads the credentials at runtime through the connection's SECRET_ID, so the secret value never enters the template. Note that in this case, the subnet must have a route to the AWS Secrets Manager VPC endpoint or to the AWS Secrets Manager endpoint through a NAT gateway.

declare const securityGroup: ec2.SecurityGroup;
declare const subnet: ec2.Subnet;
declare const db: rds.DatabaseCluster;
new glue.Connection(this, "RdsConnection", {
  type: glue.ConnectionType.JDBC,
  securityGroups: [securityGroup],
  network: glue.ConnectionNetwork.subnet(subnet),
  secret: db.secret,
  properties: {
    JDBC_CONNECTION_URL: `jdbc:mysql://${db.clusterEndpoint.socketAddress}/databasename`,
    JDBC_ENFORCE_SSL: "false",
  },
});

# security_group: ec2.SecurityGroup
# subnet: ec2.Subnet
# db: rds.DatabaseCluster

glue.Connection(self, "RdsConnection",
    type=glue.ConnectionType.JDBC,
    security_groups=[security_group],
    network=glue.ConnectionNetwork.subnet(subnet),
    secret=db.secret,
    properties={
        "JDBC_CONNECTION_URL": f"jdbc:mysql://{db.clusterEndpoint.socketAddress}/databasename",
        "JDBC_ENFORCE_SSL": "false"
    }
)

SecurityGroup securityGroup;
Subnet subnet;
DatabaseCluster db;

Connection.Builder.create(this, "RdsConnection")
        .type(ConnectionType.JDBC)
        .securityGroups(List.of(securityGroup))
        .network(ConnectionNetwork.subnet(subnet))
        .secret(db.getSecret())
        .properties(Map.of(
                "JDBC_CONNECTION_URL", String.format("jdbc:mysql://%s/databasename", db.getClusterEndpoint().getSocketAddress()),
                "JDBC_ENFORCE_SSL", "false"))
        .build();

SecurityGroup securityGroup;
Subnet subnet;
DatabaseCluster db;

new Connection(this, "RdsConnection", new ConnectionProps {
    Type = ConnectionType.JDBC,
    SecurityGroups = new [] { securityGroup },
    Network = ConnectionNetwork.Subnet(subnet),
    Secret = db.Secret,
    Properties = new Dictionary<string, string> {
        { "JDBC_CONNECTION_URL", $"jdbc:mysql://{db.clusterEndpoint.socketAddress}/databasename" },
        { "JDBC_ENFORCE_SSL", "false" }
    }
});

var securityGroup SecurityGroup
var subnet Subnet
var db DatabaseCluster

glue.NewConnection(this, jsii.String("RdsConnection"), &ConnectionProps{
	Type: glue.ConnectionType_JDBC(),
	SecurityGroups: []ISecurityGroup{
		securityGroup,
	},
	Network: glue.ConnectionNetwork_Subnet(subnet),
	Secret: db.Secret,
	Properties: map[string]*string{
		"JDBC_CONNECTION_URL": fmt.Sprintf("jdbc:mysql://%v/databasename", db.clusterEndpoint.socketAddress),
		"JDBC_ENFORCE_SSL": jsii.String("false"),
	},
})

Prefer the secret property over placing credentials in properties. Connection properties are emitted verbatim into the CloudFormation template, so any credential placed there in plaintext is stored in plaintext in the template, cdk.out, and source control. If a property key looks like a credential (for example PASSWORD, SECRET, or TOKEN) and holds a plaintext literal, the construct emits a synthesis-time warning.

If you need to use a connection type that doesn't exist as a static member on ConnectionType, you can instantiate a ConnectionType object, e.g: new glue.ConnectionType('NEW_TYPE').

See Adding a Connection to Your Data Store and Connection Structure documentation for more information on the supported data stores and their configurations.

SecurityConfiguration

A SecurityConfiguration is a set of security properties that can be used by AWS Glue to encrypt data at rest.

Each encryption config is built with a factory that pairs the encryption mode with its key, so illegal combinations (such as an S3-managed encryption carrying a KMS key) cannot be expressed:

new glue.SecurityConfiguration(this, 'MySecurityConfiguration', {
  cloudWatchEncryption: glue.CloudWatchEncryption.kms(),
  jobBookmarksEncryption: glue.JobBookmarksEncryption.clientSideKms(),
  s3Encryption: glue.S3Encryption.kms(),
});

glue.SecurityConfiguration(self, "MySecurityConfiguration",
    cloud_watch_encryption=glue.CloudWatchEncryption.kms(),
    job_bookmarks_encryption=glue.JobBookmarksEncryption.client_side_kms(),
    s3_encryption=glue.S3Encryption.kms()
)

SecurityConfiguration.Builder.create(this, "MySecurityConfiguration")
        .cloudWatchEncryption(CloudWatchEncryption.kms())
        .jobBookmarksEncryption(JobBookmarksEncryption.clientSideKms())
        .s3Encryption(S3Encryption.kms())
        .build();

new SecurityConfiguration(this, "MySecurityConfiguration", new SecurityConfigurationProps {
    CloudWatchEncryption = CloudWatchEncryption.Kms(),
    JobBookmarksEncryption = JobBookmarksEncryption.ClientSideKms(),
    S3Encryption = S3Encryption.Kms()
});

glue.NewSecurityConfiguration(this, jsii.String("MySecurityConfiguration"), &SecurityConfigurationProps{
	CloudWatchEncryption: glue.CloudWatchEncryption_Kms(),
	JobBookmarksEncryption: glue.JobBookmarksEncryption_ClientSideKms(),
	S3Encryption: glue.S3Encryption_Kms(),
})

By default, a shared KMS key is created for use with the encryption configurations that require one. You can also supply your own key to any factory, for example, for CloudWatch encryption:

declare const key: kms.Key;
new glue.SecurityConfiguration(this, 'MySecurityConfiguration', {
  cloudWatchEncryption: glue.CloudWatchEncryption.kms(key),
});

# key: kms.Key

glue.SecurityConfiguration(self, "MySecurityConfiguration",
    cloud_watch_encryption=glue.CloudWatchEncryption.kms(key)
)

Key key;

SecurityConfiguration.Builder.create(this, "MySecurityConfiguration")
        .cloudWatchEncryption(CloudWatchEncryption.kms(key))
        .build();

Key key;

new SecurityConfiguration(this, "MySecurityConfiguration", new SecurityConfigurationProps {
    CloudWatchEncryption = CloudWatchEncryption.Kms(key)
});

var key Key

glue.NewSecurityConfiguration(this, jsii.String("MySecurityConfiguration"), &SecurityConfigurationProps{
	CloudWatchEncryption: glue.CloudWatchEncryption_Kms(key),
})

Use glue.S3Encryption.s3Managed() for S3-managed (SSE-S3) encryption, which takes no key.

See documentation for more info for Glue encrypting data written by Crawlers, Jobs, and Development Endpoints.

Catalog

The Glue Data Catalog is a persistent metadata store for your data assets. Every account has an implicit, account-wide catalog that always exists, and you can also create additional catalogs as AWS::Glue::Catalog resources (for example, to federate to another metastore).

A catalog's encryption is fixed when the catalog is created: a catalog either carries encryption settings or it does not. This keeps its configuration easy to reason about — there are no mutation methods that change encryption after the fact.

The account-wide catalog

Use Catalog.forAccount(scope) to obtain the implicit account catalog. It is not a CloudFormation resource — it always exists. Repeated calls within the same stack return the same instance:

const catalog = glue.Catalog.forAccount(this);

catalog = glue.Catalog.for_account(self)

ICatalog catalog = Catalog.forAccount(this);

var catalog = Catalog.ForAccount(this);

catalog := glue.Catalog_ForAccount(this)

To configure Data Catalog encryption for the account, use Catalog.encryptAccount(scope, options):

declare const key: kms.Key;
glue.Catalog.encryptAccount(this, {
  encryptionAtRest: glue.DataCatalogEncryptionAtRest.kms(key),
});

# key: kms.Key

glue.Catalog.encrypt_account(self,
    encryption_at_rest=glue.DataCatalogEncryptionAtRest.kms(key)
)

Key key;

Catalog.encryptAccount(this, CatalogEncryptionOptions.builder()
        .encryptionAtRest(DataCatalogEncryptionAtRest.kms(key))
        .build());

Key key;

Catalog.EncryptAccount(this, new CatalogEncryptionOptions {
    EncryptionAtRest = DataCatalogEncryptionAtRest.Kms(key)
});

var key Key

glue.Catalog_EncryptAccount(this, &CatalogEncryptionOptions{
	EncryptionAtRest: glue.DataCatalogEncryptionAtRest_Kms(key),
})

Because encryption is fixed at construction, encryptAccount must be called before the account catalog is first used in the stack — before any Catalog.forAccount(this) call, and before any Database that uses the account catalog. Calling it after the account catalog has been materialized throws.

The account catalog's encryption is an account- and region-wide setting, managed through the singleton PutDataCatalogEncryptionSettings API. Configure it in exactly one stack. Configuring it from multiple stacks in the same account and region makes those stacks overwrite one another at deploy time, and the result is order-dependent. Unlike duplicate settings within a single stack (which CloudFormation rejects), this cross-stack conflict is not caught at synthesis time, because each stack synthesizes to its own template.

Creating a catalog

To create a new catalog resource, use the Catalog constructor. Encryption is configured through the encryptionAtRest and connectionPasswordEncryption props:

new glue.Catalog(this, 'MyCatalog', {
  catalogName: 'my-catalog',
  description: 'my catalog description',
});

glue.Catalog(self, "MyCatalog",
    catalog_name="my-catalog",
    description="my catalog description"
)

Catalog.Builder.create(this, "MyCatalog")
        .catalogName("my-catalog")
        .description("my catalog description")
        .build();

new Catalog(this, "MyCatalog", new CatalogProps {
    CatalogName = "my-catalog",
    Description = "my catalog description"
});

glue.NewCatalog(this, jsii.String("MyCatalog"), &CatalogProps{
	CatalogName: jsii.String("my-catalog"),
	Description: jsii.String("my catalog description"),
})

Encryption at rest

Configure Data Catalog encryption at rest through the encryptionAtRest option (on Catalog.encryptAccount or the Catalog constructor). It accepts a DataCatalogEncryptionAtRest describing the mode:

declare const key: kms.Key;

// SSE-KMS with a customer-managed key
glue.Catalog.encryptAccount(this, {
  encryptionAtRest: glue.DataCatalogEncryptionAtRest.kms(key),
});

// SSE-KMS with an AWS-managed key (omit the key)
new glue.Catalog(this, 'ManagedKeyCatalog', {
  catalogName: 'managed-key-catalog',
  encryptionAtRest: glue.DataCatalogEncryptionAtRest.kms(),
});

// Disable encryption at rest
new glue.Catalog(this, 'PlaintextCatalog', {
  catalogName: 'plaintext-catalog',
  encryptionAtRest: glue.DataCatalogEncryptionAtRest.disabled(),
});

# key: kms.Key


# SSE-KMS with a customer-managed key
glue.Catalog.encrypt_account(self,
    encryption_at_rest=glue.DataCatalogEncryptionAtRest.kms(key)
)

# SSE-KMS with an AWS-managed key (omit the key)
glue.Catalog(self, "ManagedKeyCatalog",
    catalog_name="managed-key-catalog",
    encryption_at_rest=glue.DataCatalogEncryptionAtRest.kms()
)

# Disable encryption at rest
glue.Catalog(self, "PlaintextCatalog",
    catalog_name="plaintext-catalog",
    encryption_at_rest=glue.DataCatalogEncryptionAtRest.disabled()
)

Key key;


// SSE-KMS with a customer-managed key
Catalog.encryptAccount(this, CatalogEncryptionOptions.builder()
        .encryptionAtRest(DataCatalogEncryptionAtRest.kms(key))
        .build());

// SSE-KMS with an AWS-managed key (omit the key)
// SSE-KMS with an AWS-managed key (omit the key)
Catalog.Builder.create(this, "ManagedKeyCatalog")
        .catalogName("managed-key-catalog")
        .encryptionAtRest(DataCatalogEncryptionAtRest.kms())
        .build();

// Disable encryption at rest
// Disable encryption at rest
Catalog.Builder.create(this, "PlaintextCatalog")
        .catalogName("plaintext-catalog")
        .encryptionAtRest(DataCatalogEncryptionAtRest.disabled())
        .build();

Key key;


// SSE-KMS with a customer-managed key
Catalog.EncryptAccount(this, new CatalogEncryptionOptions {
    EncryptionAtRest = DataCatalogEncryptionAtRest.Kms(key)
});

// SSE-KMS with an AWS-managed key (omit the key)
// SSE-KMS with an AWS-managed key (omit the key)
new Catalog(this, "ManagedKeyCatalog", new CatalogProps {
    CatalogName = "managed-key-catalog",
    EncryptionAtRest = DataCatalogEncryptionAtRest.Kms()
});

// Disable encryption at rest
// Disable encryption at rest
new Catalog(this, "PlaintextCatalog", new CatalogProps {
    CatalogName = "plaintext-catalog",
    EncryptionAtRest = DataCatalogEncryptionAtRest.Disabled()
});

var key Key


// SSE-KMS with a customer-managed key
glue.Catalog_EncryptAccount(this, &CatalogEncryptionOptions{
	EncryptionAtRest: glue.DataCatalogEncryptionAtRest_Kms(key),
})

// SSE-KMS with an AWS-managed key (omit the key)
// SSE-KMS with an AWS-managed key (omit the key)
glue.NewCatalog(this, jsii.String("ManagedKeyCatalog"), &CatalogProps{
	CatalogName: jsii.String("managed-key-catalog"),
	EncryptionAtRest: glue.DataCatalogEncryptionAtRest_*Kms(),
})

// Disable encryption at rest
// Disable encryption at rest
glue.NewCatalog(this, jsii.String("PlaintextCatalog"), &CatalogProps{
	CatalogName: jsii.String("plaintext-catalog"),
	EncryptionAtRest: glue.DataCatalogEncryptionAtRest_Disabled(),
})

When you use SSE-KMS-WITH-SERVICE-ROLE, AWS Glue accesses the KMS key through a service role you provide. If you pass a customer-managed key, the role is automatically granted the permissions it needs to encrypt and decrypt catalog data:

import * as iam from 'aws-cdk-lib/aws-iam';
declare const key: kms.Key;
declare const role: iam.IRole;
glue.Catalog.encryptAccount(this, {
  encryptionAtRest: glue.DataCatalogEncryptionAtRest.kmsWithServiceRole(role, key),
});

import aws_cdk.aws_iam as iam
# key: kms.Key
# role: iam.IRole

glue.Catalog.encrypt_account(self,
    encryption_at_rest=glue.DataCatalogEncryptionAtRest.kms_with_service_role(role, key)
)

import software.amazon.awscdk.services.iam.*;
Key key;
IRole role;

Catalog.encryptAccount(this, CatalogEncryptionOptions.builder()
        .encryptionAtRest(DataCatalogEncryptionAtRest.kmsWithServiceRole(role, key))
        .build());

using Amazon.CDK.AWS.IAM;
Key key;
IRole role;

Catalog.EncryptAccount(this, new CatalogEncryptionOptions {
    EncryptionAtRest = DataCatalogEncryptionAtRest.KmsWithServiceRole(role, key)
});

import iam "github.com/aws/aws-cdk-go/awscdk"
var key Key
var role IRole

glue.Catalog_EncryptAccount(this, &CatalogEncryptionOptions{
	EncryptionAtRest: glue.DataCatalogEncryptionAtRest_KmsWithServiceRole(role, key),
})

The customer-managed key, when configured, is exposed on the catalog as encryptionKey (and the connection-password key as connectionPasswordKey), so you can reference it to grant additional access. It is undefined when encryption is disabled or an AWS-managed key is used.

Connection password encryption

Independently from encryption at rest, the Data Catalog can encrypt the passwords stored in connection properties. Configure it through the connectionPasswordEncryption option:

declare const key: kms.Key;
glue.Catalog.encryptAccount(this, {
  connectionPasswordEncryption: {
    kmsKey: key,
    // Whether GetConnection/GetConnections return the password encrypted (default: true)
    returnConnectionPasswordEncrypted: true,
  },
});

# key: kms.Key

glue.Catalog.encrypt_account(self,
    connection_password_encryption=glue.ConnectionPasswordEncryption(
        kms_key=key,
        # Whether GetConnection/GetConnections return the password encrypted (default: true)
        return_connection_password_encrypted=True
    )
)

Key key;

Catalog.encryptAccount(this, CatalogEncryptionOptions.builder()
        .connectionPasswordEncryption(ConnectionPasswordEncryption.builder()
                .kmsKey(key)
                // Whether GetConnection/GetConnections return the password encrypted (default: true)
                .returnConnectionPasswordEncrypted(true)
                .build())
        .build());

Key key;

Catalog.EncryptAccount(this, new CatalogEncryptionOptions {
    ConnectionPasswordEncryption = new ConnectionPasswordEncryption {
        KmsKey = key,
        // Whether GetConnection/GetConnections return the password encrypted (default: true)
        ReturnConnectionPasswordEncrypted = true
    }
});

var key Key

glue.Catalog_EncryptAccount(this, &CatalogEncryptionOptions{
	ConnectionPasswordEncryption: &ConnectionPasswordEncryption{
		KmsKey: key,
		// Whether GetConnection/GetConnections return the password encrypted (default: true)
		ReturnConnectionPasswordEncrypted: jsii.Boolean(true),
	},
})

The two encryption blocks are independent: enabling one does not require the other, and each may use a different KMS key. The customer-managed key for connection passwords is exposed as connectionPasswordKey.

Importing a catalog

You can import an existing catalog by ARN or by id. An imported catalog is a pure identity handle — it emits no resources and does not manage the catalog's encryption:

const byId = glue.Catalog.fromCatalogId(this, 'ById', 'my-catalog-id');
const byArn = glue.Catalog.fromCatalogArn(this, 'ByArn', 'arn:aws:glue:us-east-1:123456789012:catalog/my-catalog-id');

by_id = glue.Catalog.from_catalog_id(self, "ById", "my-catalog-id")
by_arn = glue.Catalog.from_catalog_arn(self, "ByArn", "arn:aws:glue:us-east-1:123456789012:catalog/my-catalog-id")

ICatalog byId = Catalog.fromCatalogId(this, "ById", "my-catalog-id");
ICatalog byArn = Catalog.fromCatalogArn(this, "ByArn", "arn:aws:glue:us-east-1:123456789012:catalog/my-catalog-id");

var byId = Catalog.FromCatalogId(this, "ById", "my-catalog-id");
var byArn = Catalog.FromCatalogArn(this, "ByArn", "arn:aws:glue:us-east-1:123456789012:catalog/my-catalog-id");

byId := glue.Catalog_FromCatalogId(this, jsii.String("ById"), jsii.String("my-catalog-id"))
byArn := glue.Catalog_FromCatalogArn(this, jsii.String("ByArn"), jsii.String("arn:aws:glue:us-east-1:123456789012:catalog/my-catalog-id"))

To manage the Data Catalog encryption of a catalog you did not create in this stack, add a CfnDataCatalogEncryptionSettings resource targeting its id directly. Do this from exactly one stack: like the account catalog, a catalog has a single encryption configuration, so two settings resources targeting the same id race to overwrite one another at deploy time. Within a single stack this is caught by CloudFormation template validation (E3019, duplicate primary identifiers); across stacks it is not, since each stack synthesizes to its own template.

import { CfnDataCatalogEncryptionSettings } from 'aws-cdk-lib/aws-glue';

new CfnDataCatalogEncryptionSettings(this, 'Encryption', {
  catalogId: 'my-catalog-id',
  dataCatalogEncryptionSettings: {
    encryptionAtRest: { catalogEncryptionMode: 'SSE-KMS' },
  },
});

from aws_cdk.aws_glue import CfnDataCatalogEncryptionSettings


CfnDataCatalogEncryptionSettings(self, "Encryption",
    catalog_id="my-catalog-id",
    data_catalog_encryption_settings=CfnDataCatalogEncryptionSettings.DataCatalogEncryptionSettingsProperty(
        encryption_at_rest=CfnDataCatalogEncryptionSettings.EncryptionAtRestProperty(catalog_encryption_mode="SSE-KMS")
    )
)

import software.amazon.awscdk.services.glue.CfnDataCatalogEncryptionSettings;


CfnDataCatalogEncryptionSettings.Builder.create(this, "Encryption")
        .catalogId("my-catalog-id")
        .dataCatalogEncryptionSettings(DataCatalogEncryptionSettingsProperty.builder()
                .encryptionAtRest(EncryptionAtRestProperty.builder().catalogEncryptionMode("SSE-KMS").build())
                .build())
        .build();

using Amazon.CDK.AWS.Glue;


new CfnDataCatalogEncryptionSettings(this, "Encryption", new CfnDataCatalogEncryptionSettingsProps {
    CatalogId = "my-catalog-id",
    DataCatalogEncryptionSettings = new DataCatalogEncryptionSettingsProperty {
        EncryptionAtRest = new EncryptionAtRestProperty { CatalogEncryptionMode = "SSE-KMS" }
    }
});

import "github.com/aws/aws-cdk-go/awscdk"


awscdk.NewCfnDataCatalogEncryptionSettings(this, jsii.String("Encryption"), &CfnDataCatalogEncryptionSettingsProps{
	CatalogId: jsii.String("my-catalog-id"),
	DataCatalogEncryptionSettings: &DataCatalogEncryptionSettingsProperty{
		EncryptionAtRest: &EncryptionAtRestProperty{
			CatalogEncryptionMode: jsii.String("SSE-KMS"),
		},
	},
})

Database

A Database is a logical grouping of Tables in the Glue Catalog.

new glue.Database(this, 'MyDatabase', {
  databaseName: 'my_database',
  description: 'my_database_description',
});

glue.Database(self, "MyDatabase",
    database_name="my_database",
    description="my_database_description"
)

Database.Builder.create(this, "MyDatabase")
        .databaseName("my_database")
        .description("my_database_description")
        .build();

new Database(this, "MyDatabase", new DatabaseProps {
    DatabaseName = "my_database",
    Description = "my_database_description"
});

glue.NewDatabase(this, jsii.String("MyDatabase"), &DatabaseProps{
	DatabaseName: jsii.String("my_database"),
	Description: jsii.String("my_database_description"),
})

Because a database is a container for tables and their metadata, it is retained by default when removed from the stack, to avoid accidental data loss. Set removalPolicy to RemovalPolicy.DESTROY to have it deleted instead:

import { RemovalPolicy } from 'aws-cdk-lib';

new glue.Database(this, 'MyDatabase', {
  databaseName: 'my_database',
  removalPolicy: RemovalPolicy.DESTROY,
});

from aws_cdk import RemovalPolicy


glue.Database(self, "MyDatabase",
    database_name="my_database",
    removal_policy=RemovalPolicy.DESTROY
)

import software.amazon.awscdk.RemovalPolicy;


Database.Builder.create(this, "MyDatabase")
        .databaseName("my_database")
        .removalPolicy(RemovalPolicy.DESTROY)
        .build();

using Amazon.CDK;


new Database(this, "MyDatabase", new DatabaseProps {
    DatabaseName = "my_database",
    RemovalPolicy = RemovalPolicy.DESTROY
});

import "github.com/aws/aws-cdk-go/awscdk"


glue.NewDatabase(this, jsii.String("MyDatabase"), &DatabaseProps{
	DatabaseName: jsii.String("my_database"),
	RemovalPolicy: awscdk.RemovalPolicy_DESTROY,
})

Table

A Glue table describes a table of data in S3: its structure (column names and types), location of data (S3 objects with a common prefix in a S3 bucket), and format for the files (Json, Avro, Parquet, etc.):

declare const myDatabase: glue.Database;
new glue.S3Table(this, 'MyTable', {
  database: myDatabase,
  columns: [{
    name: 'col1',
    type: glue.Schema.STRING,
  }, {
    name: 'col2',
    type: glue.Schema.array(glue.Schema.STRING),
    comment: 'col2 is an array of strings' // comment is optional
  }],
  dataFormat: glue.DataFormat.JSON,
});

# my_database: glue.Database

glue.S3Table(self, "MyTable",
    database=my_database,
    columns=[glue.Column(
        name="col1",
        type=glue.Schema.STRING
    ), glue.Column(
        name="col2",
        type=glue.Schema.array(glue.Schema.STRING),
        comment="col2 is an array of strings"
    )],
    data_format=glue.DataFormat.JSON
)

Database myDatabase;

S3Table.Builder.create(this, "MyTable")
        .database(myDatabase)
        .columns(List.of(Column.builder()
                .name("col1")
                .type(Schema.STRING)
                .build(), Column.builder()
                .name("col2")
                .type(Schema.array(Schema.STRING))
                .comment("col2 is an array of strings")
                .build()))
        .dataFormat(DataFormat.JSON)
        .build();

Database myDatabase;

new S3Table(this, "MyTable", new S3TableProps {
    Database = myDatabase,
    Columns = new [] { new Column {
        Name = "col1",
        Type = Schema.STRING
    }, new Column {
        Name = "col2",
        Type = Schema.Array(Schema.STRING),
        Comment = "col2 is an array of strings"
    } },
    DataFormat = DataFormat.JSON
});

var myDatabase Database

glue.NewS3Table(this, jsii.String("MyTable"), &S3TableProps{
	Database: myDatabase,
	Columns: []Column{
		&Column{
			Name: jsii.String("col1"),
			Type: glue.Schema_STRING(),
		},
		&Column{
			Name: jsii.String("col2"),
			Type: glue.Schema_Array(glue.Schema_STRING()),
			Comment: jsii.String("col2 is an array of strings"),
		},
	},
	DataFormat: glue.DataFormat_JSON(),
})

By default, a S3 bucket will be created to store the table's data but you can bring your own with S3TableStorage.fromBucket and set an s3Prefix:

declare const myBucket: s3.Bucket;
declare const myDatabase: glue.Database;
new glue.S3Table(this, 'MyTable', {
  storage: glue.S3TableStorage.fromBucket(myBucket),
  s3Prefix: 'my-table/',
  // ...
  database: myDatabase,
  columns: [{
    name: 'col1',
    type: glue.Schema.STRING,
  }],
  dataFormat: glue.DataFormat.JSON,
});

# my_bucket: s3.Bucket
# my_database: glue.Database

glue.S3Table(self, "MyTable",
    storage=glue.S3TableStorage.from_bucket(my_bucket),
    s3_prefix="my-table/",
    # ...
    database=my_database,
    columns=[glue.Column(
        name="col1",
        type=glue.Schema.STRING
    )],
    data_format=glue.DataFormat.JSON
)

Bucket myBucket;
Database myDatabase;

S3Table.Builder.create(this, "MyTable")
        .storage(S3TableStorage.fromBucket(myBucket))
        .s3Prefix("my-table/")
        // ...
        .database(myDatabase)
        .columns(List.of(Column.builder()
                .name("col1")
                .type(Schema.STRING)
                .build()))
        .dataFormat(DataFormat.JSON)
        .build();

Bucket myBucket;
Database myDatabase;

new S3Table(this, "MyTable", new S3TableProps {
    Storage = S3TableStorage.FromBucket(myBucket),
    S3Prefix = "my-table/",
    // ...
    Database = myDatabase,
    Columns = new [] { new Column {
        Name = "col1",
        Type = Schema.STRING
    } },
    DataFormat = DataFormat.JSON
});

var myBucket Bucket
var myDatabase Database

glue.NewS3Table(this, jsii.String("MyTable"), &S3TableProps{
	Storage: glue.S3TableStorage_FromBucket(myBucket),
	S3Prefix: jsii.String("my-table/"),
	// ...
	Database: myDatabase,
	Columns: []Column{
		&Column{
			Name: jsii.String("col1"),
			Type: glue.Schema_STRING(),
		},
	},
	DataFormat: glue.DataFormat_JSON(),
})

Glue tables can be configured to contain user-defined properties, to describe the physical storage of table data, through the storageParameters property:

declare const myDatabase: glue.Database;
new glue.S3Table(this, 'MyTable', {
  storageParameters: [
    glue.StorageParameter.skipHeaderLineCount(1),
    glue.StorageParameter.compressionType(glue.CompressionType.GZIP),
    glue.StorageParameter.custom('separatorChar', ',')
  ],
  // ...
  database: myDatabase,
  columns: [{
    name: 'col1',
    type: glue.Schema.STRING,
  }],
  dataFormat: glue.DataFormat.JSON,
});

# my_database: glue.Database

glue.S3Table(self, "MyTable",
    storage_parameters=[
        glue.StorageParameter.skip_header_line_count(1),
        glue.StorageParameter.compression_type(glue.CompressionType.GZIP),
        glue.StorageParameter.custom("separatorChar", ",")
    ],
    # ...
    database=my_database,
    columns=[glue.Column(
        name="col1",
        type=glue.Schema.STRING
    )],
    data_format=glue.DataFormat.JSON
)

Database myDatabase;

S3Table.Builder.create(this, "MyTable")
        .storageParameters(List.of(StorageParameter.skipHeaderLineCount(1), StorageParameter.compressionType(CompressionType.GZIP), StorageParameter.custom("separatorChar", ",")))
        // ...
        .database(myDatabase)
        .columns(List.of(Column.builder()
                .name("col1")
                .type(Schema.STRING)
                .build()))
        .dataFormat(DataFormat.JSON)
        .build();

Database myDatabase;

new S3Table(this, "MyTable", new S3TableProps {
    StorageParameters = new [] { StorageParameter.SkipHeaderLineCount(1), StorageParameter.CompressionType(CompressionType.GZIP), StorageParameter.Custom("separatorChar", ",") },
    // ...
    Database = myDatabase,
    Columns = new [] { new Column {
        Name = "col1",
        Type = Schema.STRING
    } },
    DataFormat = DataFormat.JSON
});

var myDatabase Database

glue.NewS3Table(this, jsii.String("MyTable"), &S3TableProps{
	StorageParameters: []StorageParameter{
		glue.StorageParameter_SkipHeaderLineCount(jsii.Number(1)),
		glue.StorageParameter_CompressionType(glue.CompressionType_GZIP),
		glue.StorageParameter_Custom(jsii.String("separatorChar"), jsii.String(",")),
	},
	// ...
	Database: myDatabase,
	Columns: []Column{
		&Column{
			Name: jsii.String("col1"),
			Type: glue.Schema_STRING(),
		},
	},
	DataFormat: glue.DataFormat_JSON(),
})

Glue tables can also be configured to contain user-defined table properties through the parameters property:

declare const myDatabase: glue.Database;
new glue.S3Table(this, 'MyTable', {
  parameters: {
    key1: 'val1',
    key2: 'val2',
  },
  database: myDatabase,
  columns: [{
    name: 'col1',
    type: glue.Schema.STRING,
  }],
  dataFormat: glue.DataFormat.JSON,
});

# my_database: glue.Database

glue.S3Table(self, "MyTable",
    parameters={
        "key1": "val1",
        "key2": "val2"
    },
    database=my_database,
    columns=[glue.Column(
        name="col1",
        type=glue.Schema.STRING
    )],
    data_format=glue.DataFormat.JSON
)

Database myDatabase;

S3Table.Builder.create(this, "MyTable")
        .parameters(Map.of(
                "key1", "val1",
                "key2", "val2"))
        .database(myDatabase)
        .columns(List.of(Column.builder()
                .name("col1")
                .type(Schema.STRING)
                .build()))
        .dataFormat(DataFormat.JSON)
        .build();

Database myDatabase;

new S3Table(this, "MyTable", new S3TableProps {
    Parameters = new Dictionary<string, string> {
        { "key1", "val1" },
        { "key2", "val2" }
    },
    Database = myDatabase,
    Columns = new [] { new Column {
        Name = "col1",
        Type = Schema.STRING
    } },
    DataFormat = DataFormat.JSON
});

var myDatabase Database

glue.NewS3Table(this, jsii.String("MyTable"), &S3TableProps{
	Parameters: map[string]*string{
		"key1": jsii.String("val1"),
		"key2": jsii.String("val2"),
	},
	Database: myDatabase,
	Columns: []Column{
		&Column{
			Name: jsii.String("col1"),
			Type: glue.Schema_STRING(),
		},
	},
	DataFormat: glue.DataFormat_JSON(),
})

Partition Keys

To improve query performance, a table can specify partitionKeys on which data is stored and queried separately. For example, you might partition a table by year and month to optimize queries based on a time window:

declare const myDatabase: glue.Database;
new glue.S3Table(this, 'MyTable', {
  database: myDatabase,
  columns: [{
    name: 'col1',
    type: glue.Schema.STRING,
  }],
  partitionKeys: [{
    name: 'year',
    type: glue.Schema.SMALL_INT,
  }, {
    name: 'month',
    type: glue.Schema.SMALL_INT,
  }],
  dataFormat: glue.DataFormat.JSON,
});

# my_database: glue.Database

glue.S3Table(self, "MyTable",
    database=my_database,
    columns=[glue.Column(
        name="col1",
        type=glue.Schema.STRING
    )],
    partition_keys=[glue.Column(
        name="year",
        type=glue.Schema.SMALL_INT
    ), glue.Column(
        name="month",
        type=glue.Schema.SMALL_INT
    )],
    data_format=glue.DataFormat.JSON
)

Database myDatabase;

S3Table.Builder.create(this, "MyTable")
        .database(myDatabase)
        .columns(List.of(Column.builder()
                .name("col1")
                .type(Schema.STRING)
                .build()))
        .partitionKeys(List.of(Column.builder()
                .name("year")
                .type(Schema.SMALL_INT)
                .build(), Column.builder()
                .name("month")
                .type(Schema.SMALL_INT)
                .build()))
        .dataFormat(DataFormat.JSON)
        .build();

Database myDatabase;

new S3Table(this, "MyTable", new S3TableProps {
    Database = myDatabase,
    Columns = new [] { new Column {
        Name = "col1",
        Type = Schema.STRING
    } },
    PartitionKeys = new [] { new Column {
        Name = "year",
        Type = Schema.SMALL_INT
    }, new Column {
        Name = "month",
        Type = Schema.SMALL_INT
    } },
    DataFormat = DataFormat.JSON
});

var myDatabase Database

glue.NewS3Table(this, jsii.String("MyTable"), &S3TableProps{
	Database: myDatabase,
	Columns: []Column{
		&Column{
			Name: jsii.String("col1"),
			Type: glue.Schema_STRING(),
		},
	},
	PartitionKeys: []Column{
		&Column{
			Name: jsii.String("year"),
			Type: glue.Schema_SMALL_INT(),
		},
		&Column{
			Name: jsii.String("month"),
			Type: glue.Schema_SMALL_INT(),
		},
	},
	DataFormat: glue.DataFormat_JSON(),
})

Partition Indexes

Another way to improve query performance is to specify partition indexes. If no partition indexes are present on the table, AWS Glue loads all partitions of the table and filters the loaded partitions using the query expression. The query takes more time to run as the number of partitions increase. With an index, the query will try to fetch a subset of the partitions instead of loading all partitions of the table.

The keys of a partition index must be a subset of the partition keys of the table. You can have a maximum of 3 partition indexes per table. To specify a partition index, you can use the partitionIndexes property:

declare const myDatabase: glue.Database;
new glue.S3Table(this, 'MyTable', {
  database: myDatabase,
  columns: [{
    name: 'col1',
    type: glue.Schema.STRING,
  }],
  partitionKeys: [{
    name: 'year',
    type: glue.Schema.SMALL_INT,
  }, {
    name: 'month',
    type: glue.Schema.SMALL_INT,
  }],
  partitionIndexes: [{
    indexName: 'my-index', // optional
    keyNames: ['year'],
  }], // supply up to 3 indexes
  dataFormat: glue.DataFormat.JSON,
});

# my_database: glue.Database

glue.S3Table(self, "MyTable",
    database=my_database,
    columns=[glue.Column(
        name="col1",
        type=glue.Schema.STRING
    )],
    partition_keys=[glue.Column(
        name="year",
        type=glue.Schema.SMALL_INT
    ), glue.Column(
        name="month",
        type=glue.Schema.SMALL_INT
    )],
    partition_indexes=[glue.PartitionIndex(
        index_name="my-index",  # optional
        key_names=["year"]
    )],  # supply up to 3 indexes
    data_format=glue.DataFormat.JSON
)

Database myDatabase;

S3Table.Builder.create(this, "MyTable")
        .database(myDatabase)
        .columns(List.of(Column.builder()
                .name("col1")
                .type(Schema.STRING)
                .build()))
        .partitionKeys(List.of(Column.builder()
                .name("year")
                .type(Schema.SMALL_INT)
                .build(), Column.builder()
                .name("month")
                .type(Schema.SMALL_INT)
                .build()))
        .partitionIndexes(List.of(PartitionIndex.builder()
                .indexName("my-index") // optional
                .keyNames(List.of("year"))
                .build())) // supply up to 3 indexes
        .dataFormat(DataFormat.JSON)
        .build();

Database myDatabase;

new S3Table(this, "MyTable", new S3TableProps {
    Database = myDatabase,
    Columns = new [] { new Column {
        Name = "col1",
        Type = Schema.STRING
    } },
    PartitionKeys = new [] { new Column {
        Name = "year",
        Type = Schema.SMALL_INT
    }, new Column {
        Name = "month",
        Type = Schema.SMALL_INT
    } },
    PartitionIndexes = new [] { new PartitionIndex {
        IndexName = "my-index",  // optional
        KeyNames = new [] { "year" }
    } },  // supply up to 3 indexes
    DataFormat = DataFormat.JSON
});

var myDatabase Database

glue.NewS3Table(this, jsii.String("MyTable"), &S3TableProps{
	Database: myDatabase,
	Columns: []Column{
		&Column{
			Name: jsii.String("col1"),
			Type: glue.Schema_STRING(),
		},
	},
	PartitionKeys: []Column{
		&Column{
			Name: jsii.String("year"),
			Type: glue.Schema_SMALL_INT(),
		},
		&Column{
			Name: jsii.String("month"),
			Type: glue.Schema_SMALL_INT(),
		},
	},
	PartitionIndexes: []PartitionIndex{
		&PartitionIndex{
			IndexName: jsii.String("my-index"),
			 // optional
			KeyNames: []*string{
				jsii.String("year"),
			},
		},
	},
	 // supply up to 3 indexes
	DataFormat: glue.DataFormat_JSON(),
})

Alternatively, you can call the addPartitionIndex() function on a table:

declare const myTable: glue.S3Table;
myTable.addPartitionIndex({
  indexName: 'my-index',
  keyNames: ['year'],
});

# my_table: glue.S3Table

my_table.add_partition_index(
    index_name="my-index",
    key_names=["year"]
)

S3Table myTable;

myTable.addPartitionIndex(PartitionIndex.builder()
        .indexName("my-index")
        .keyNames(List.of("year"))
        .build());

S3Table myTable;

myTable.AddPartitionIndex(new PartitionIndex {
    IndexName = "my-index",
    KeyNames = new [] { "year" }
});

var myTable S3Table

myTable.AddPartitionIndex(&PartitionIndex{
	IndexName: jsii.String("my-index"),
	KeyNames: []*string{
		jsii.String("year"),
	},
})

Partition Filtering

If you have a table with a large number of partitions that grows over time, consider using AWS Glue partition indexing and filtering.

declare const myDatabase: glue.Database;
new glue.S3Table(this, 'MyTable', {
  database: myDatabase,
  columns: [{
      name: 'col1',
      type: glue.Schema.STRING,
  }],
  partitionKeys: [{
      name: 'year',
      type: glue.Schema.SMALL_INT,
  }, {
      name: 'month',
      type: glue.Schema.SMALL_INT,
  }],
  dataFormat: glue.DataFormat.JSON,
  enablePartitionFiltering: true,
});

# my_database: glue.Database

glue.S3Table(self, "MyTable",
    database=my_database,
    columns=[glue.Column(
        name="col1",
        type=glue.Schema.STRING
    )],
    partition_keys=[glue.Column(
        name="year",
        type=glue.Schema.SMALL_INT
    ), glue.Column(
        name="month",
        type=glue.Schema.SMALL_INT
    )],
    data_format=glue.DataFormat.JSON,
    enable_partition_filtering=True
)

Database myDatabase;

S3Table.Builder.create(this, "MyTable")
        .database(myDatabase)
        .columns(List.of(Column.builder()
                .name("col1")
                .type(Schema.STRING)
                .build()))
        .partitionKeys(List.of(Column.builder()
                .name("year")
                .type(Schema.SMALL_INT)
                .build(), Column.builder()
                .name("month")
                .type(Schema.SMALL_INT)
                .build()))
        .dataFormat(DataFormat.JSON)
        .enablePartitionFiltering(true)
        .build();

Database myDatabase;

new S3Table(this, "MyTable", new S3TableProps {
    Database = myDatabase,
    Columns = new [] { new Column {
        Name = "col1",
        Type = Schema.STRING
    } },
    PartitionKeys = new [] { new Column {
        Name = "year",
        Type = Schema.SMALL_INT
    }, new Column {
        Name = "month",
        Type = Schema.SMALL_INT
    } },
    DataFormat = DataFormat.JSON,
    EnablePartitionFiltering = true
});

var myDatabase Database

glue.NewS3Table(this, jsii.String("MyTable"), &S3TableProps{
	Database: myDatabase,
	Columns: []Column{
		&Column{
			Name: jsii.String("col1"),
			Type: glue.Schema_STRING(),
		},
	},
	PartitionKeys: []Column{
		&Column{
			Name: jsii.String("year"),
			Type: glue.Schema_SMALL_INT(),
		},
		&Column{
			Name: jsii.String("month"),
			Type: glue.Schema_SMALL_INT(),
		},
	},
	DataFormat: glue.DataFormat_JSON(),
	EnablePartitionFiltering: jsii.Boolean(true),
})

Partition Projection

Partition projection allows Athena to automatically add new partitions as new data arrives, without requiring ALTER TABLE ADD PARTITION statements. This improves query performance and reduces management overhead by eliminating the need to manually manage partition metadata.

For more information, see the AWS documentation on partition projection.

INTEGER Projection

For partition keys with sequential numeric values:

declare const myDatabase: glue.Database;
new glue.S3Table(this, 'MyTable', {
  database: myDatabase,
  columns: [{
    name: 'data',
    type: glue.Schema.STRING,
  }],
  partitionKeys: [{
    name: 'year',
    type: glue.Schema.INTEGER,
  }],
  dataFormat: glue.DataFormat.JSON,
  partitionProjection: {
    year: glue.PartitionProjectionConfiguration.integer({
      min: 2020,
      max: 2023,
      interval: 1,  // optional, defaults to 1
      digits: 4,    // optional, pads with leading zeros
    }),
  },
});

# my_database: glue.Database

glue.S3Table(self, "MyTable",
    database=my_database,
    columns=[glue.Column(
        name="data",
        type=glue.Schema.STRING
    )],
    partition_keys=[glue.Column(
        name="year",
        type=glue.Schema.INTEGER
    )],
    data_format=glue.DataFormat.JSON,
    partition_projection={
        "year": glue.PartitionProjectionConfiguration.integer(
            min=2020,
            max=2023,
            interval=1,  # optional, defaults to 1
            digits=4
        )
    }
)

Database myDatabase;

S3Table.Builder.create(this, "MyTable")
        .database(myDatabase)
        .columns(List.of(Column.builder()
                .name("data")
                .type(Schema.STRING)
                .build()))
        .partitionKeys(List.of(Column.builder()
                .name("year")
                .type(Schema.INTEGER)
                .build()))
        .dataFormat(DataFormat.JSON)
        .partitionProjection(Map.of(
                "year", PartitionProjectionConfiguration.integer(IntegerPartitionProjectionConfigurationProps.builder()
                        .min(2020)
                        .max(2023)
                        .interval(1) // optional, defaults to 1
                        .digits(4)
                        .build())))
        .build();

Database myDatabase;

new S3Table(this, "MyTable", new S3TableProps {
    Database = myDatabase,
    Columns = new [] { new Column {
        Name = "data",
        Type = Schema.STRING
    } },
    PartitionKeys = new [] { new Column {
        Name = "year",
        Type = Schema.INTEGER
    } },
    DataFormat = DataFormat.JSON,
    PartitionProjection = new Dictionary<string, PartitionProjectionConfiguration> {
        { "year", PartitionProjectionConfiguration.Integer(new IntegerPartitionProjectionConfigurationProps {
            Min = 2020,
            Max = 2023,
            Interval = 1,  // optional, defaults to 1
            Digits = 4
        }) }
    }
});

var myDatabase Database

glue.NewS3Table(this, jsii.String("MyTable"), &S3TableProps{
	Database: myDatabase,
	Columns: []Column{
		&Column{
			Name: jsii.String("data"),
			Type: glue.Schema_STRING(),
		},
	},
	PartitionKeys: []Column{
		&Column{
			Name: jsii.String("year"),
			Type: glue.Schema_INTEGER(),
		},
	},
	DataFormat: glue.DataFormat_JSON(),
	PartitionProjection: map[string]PartitionProjectionConfiguration{
		"year": glue.PartitionProjectionConfiguration_integer(&IntegerPartitionProjectionConfigurationProps{
			"min": jsii.Number(2020),
			"max": jsii.Number(2023),
			"interval": jsii.Number(1),
			 // optional, defaults to 1
			"digits": jsii.Number(4),
		}),
	},
})

DATE Projection

For partition keys with date or timestamp values. Supports both fixed dates and relative dates using NOW:

declare const myDatabase: glue.Database;
new glue.S3Table(this, 'MyTable', {
  database: myDatabase,
  columns: [{
    name: 'data',
    type: glue.Schema.STRING,
  }],
  partitionKeys: [{
    name: 'date',
    type: glue.Schema.STRING,
  }],
  dataFormat: glue.DataFormat.JSON,
  partitionProjection: {
    date: glue.PartitionProjectionConfiguration.date({
      min: '2020-01-01',
      max: '2023-12-31',
      format: 'yyyy-MM-dd',
      // `step` bundles interval + unit (supply both or neither). Optional at day
      // precision or coarser; required when the format is sub-day (e.g. hours).
      step: { interval: 1, intervalUnit: glue.DateIntervalUnit.DAYS },
    }),
  },
});

# my_database: glue.Database

glue.S3Table(self, "MyTable",
    database=my_database,
    columns=[glue.Column(
        name="data",
        type=glue.Schema.STRING
    )],
    partition_keys=[glue.Column(
        name="date",
        type=glue.Schema.STRING
    )],
    data_format=glue.DataFormat.JSON,
    partition_projection={
        "date": glue.PartitionProjectionConfiguration.date(
            min="2020-01-01",
            max="2023-12-31",
            format="yyyy-MM-dd",
            # `step` bundles interval + unit (supply both or neither). Optional at day
            # precision or coarser; required when the format is sub-day (e.g. hours).
            step=glue.DateProjectionStep(interval=1, interval_unit=glue.DateIntervalUnit.DAYS)
        )
    }
)

Database myDatabase;

S3Table.Builder.create(this, "MyTable")
        .database(myDatabase)
        .columns(List.of(Column.builder()
                .name("data")
                .type(Schema.STRING)
                .build()))
        .partitionKeys(List.of(Column.builder()
                .name("date")
                .type(Schema.STRING)
                .build()))
        .dataFormat(DataFormat.JSON)
        .partitionProjection(Map.of(
                "date", PartitionProjectionConfiguration.date(DatePartitionProjectionConfigurationProps.builder()
                        .min("2020-01-01")
                        .max("2023-12-31")
                        .format("yyyy-MM-dd")
                        // `step` bundles interval + unit (supply both or neither). Optional at day
                        // precision or coarser; required when the format is sub-day (e.g. hours).
                        .step(DateProjectionStep.builder().interval(1).intervalUnit(DateIntervalUnit.DAYS).build())
                        .build())))
        .build();

Database myDatabase;

new S3Table(this, "MyTable", new S3TableProps {
    Database = myDatabase,
    Columns = new [] { new Column {
        Name = "data",
        Type = Schema.STRING
    } },
    PartitionKeys = new [] { new Column {
        Name = "date",
        Type = Schema.STRING
    } },
    DataFormat = DataFormat.JSON,
    PartitionProjection = new Dictionary<string, PartitionProjectionConfiguration> {
        { "date", PartitionProjectionConfiguration.Date(new DatePartitionProjectionConfigurationProps {
            Min = "2020-01-01",
            Max = "2023-12-31",
            Format = "yyyy-MM-dd",
            // `step` bundles interval + unit (supply both or neither). Optional at day
            // precision or coarser; required when the format is sub-day (e.g. hours).
            Step = new DateProjectionStep { Interval = 1, IntervalUnit = DateIntervalUnit.DAYS }
        }) }
    }
});

var myDatabase Database

glue.NewS3Table(this, jsii.String("MyTable"), &S3TableProps{
	Database: myDatabase,
	Columns: []Column{
		&Column{
			Name: jsii.String("data"),
			Type: glue.Schema_STRING(),
		},
	},
	PartitionKeys: []Column{
		&Column{
			Name: jsii.String("date"),
			Type: glue.Schema_STRING(),
		},
	},
	DataFormat: glue.DataFormat_JSON(),
	PartitionProjection: map[string]PartitionProjectionConfiguration{
		"date": glue.PartitionProjectionConfiguration_date(&DatePartitionProjectionConfigurationProps{
			"min": jsii.String("2020-01-01"),
			"max": jsii.String("2023-12-31"),
			"format": jsii.String("yyyy-MM-dd"),
			// `step` bundles interval + unit (supply both or neither). Optional at day
			// precision or coarser; required when the format is sub-day (e.g. hours).
			"step": &DateProjectionStep{
				"interval": jsii.Number(1),
				"intervalUnit": glue.DateIntervalUnit_DAYS,
			},
		}),
	},
})

You can also use relative dates with NOW:

declare const myDatabase: glue.Database;
new glue.S3Table(this, 'MyTable', {
  database: myDatabase,
  columns: [{
    name: 'data',
    type: glue.Schema.STRING,
  }],
  partitionKeys: [{
    name: 'date',
    type: glue.Schema.STRING,
  }],
  dataFormat: glue.DataFormat.JSON,
  partitionProjection: {
    date: glue.PartitionProjectionConfiguration.date({
      min: 'NOW-3YEARS',
      max: 'NOW',
      format: 'yyyy-MM-dd',
    }),
  },
});

# my_database: glue.Database

glue.S3Table(self, "MyTable",
    database=my_database,
    columns=[glue.Column(
        name="data",
        type=glue.Schema.STRING
    )],
    partition_keys=[glue.Column(
        name="date",
        type=glue.Schema.STRING
    )],
    data_format=glue.DataFormat.JSON,
    partition_projection={
        "date": glue.PartitionProjectionConfiguration.date(
            min="NOW-3YEARS",
            max="NOW",
            format="yyyy-MM-dd"
        )
    }
)

Database myDatabase;

S3Table.Builder.create(this, "MyTable")
        .database(myDatabase)
        .columns(List.of(Column.builder()
                .name("data")
                .type(Schema.STRING)
                .build()))
        .partitionKeys(List.of(Column.builder()
                .name("date")
                .type(Schema.STRING)
                .build()))
        .dataFormat(DataFormat.JSON)
        .partitionProjection(Map.of(
                "date", PartitionProjectionConfiguration.date(DatePartitionProjectionConfigurationProps.builder()
                        .min("NOW-3YEARS")
                        .max("NOW")
                        .format("yyyy-MM-dd")
                        .build())))
        .build();

Database myDatabase;

new S3Table(this, "MyTable", new S3TableProps {
    Database = myDatabase,
    Columns = new [] { new Column {
        Name = "data",
        Type = Schema.STRING
    } },
    PartitionKeys = new [] { new Column {
        Name = "date",
        Type = Schema.STRING
    } },
    DataFormat = DataFormat.JSON,
    PartitionProjection = new Dictionary<string, PartitionProjectionConfiguration> {
        { "date", PartitionProjectionConfiguration.Date(new DatePartitionProjectionConfigurationProps {
            Min = "NOW-3YEARS",
            Max = "NOW",
            Format = "yyyy-MM-dd"
        }) }
    }
});

var myDatabase Database

glue.NewS3Table(this, jsii.String("MyTable"), &S3TableProps{
	Database: myDatabase,
	Columns: []Column{
		&Column{
			Name: jsii.String("data"),
			Type: glue.Schema_STRING(),
		},
	},
	PartitionKeys: []Column{
		&Column{
			Name: jsii.String("date"),
			Type: glue.Schema_STRING(),
		},
	},
	DataFormat: glue.DataFormat_JSON(),
	PartitionProjection: map[string]PartitionProjectionConfiguration{
		"date": glue.PartitionProjectionConfiguration_date(&DatePartitionProjectionConfigurationProps{
			"min": jsii.String("NOW-3YEARS"),
			"max": jsii.String("NOW"),
			"format": jsii.String("yyyy-MM-dd"),
		}),
	},
})

ENUM Projection

For partition keys with a known set of values:

declare const myDatabase: glue.Database;
new glue.S3Table(this, 'MyTable', {
  database: myDatabase,
  columns: [{
    name: 'data',
    type: glue.Schema.STRING,
  }],
  partitionKeys: [{
    name: 'region',
    type: glue.Schema.STRING,
  }],
  dataFormat: glue.DataFormat.JSON,
  partitionProjection: {
    region: glue.PartitionProjectionConfiguration.enum({
      values: ['us-east-1', 'us-west-2', 'eu-west-1'],
    }),
  },
});

# my_database: glue.Database

glue.S3Table(self, "MyTable",
    database=my_database,
    columns=[glue.Column(
        name="data",
        type=glue.Schema.STRING
    )],
    partition_keys=[glue.Column(
        name="region",
        type=glue.Schema.STRING
    )],
    data_format=glue.DataFormat.JSON,
    partition_projection={
        "region": glue.PartitionProjectionConfiguration.enum(
            values=["us-east-1", "us-west-2", "eu-west-1"]
        )
    }
)

Database myDatabase;

S3Table.Builder.create(this, "MyTable")
        .database(myDatabase)
        .columns(List.of(Column.builder()
                .name("data")
                .type(Schema.STRING)
                .build()))
        .partitionKeys(List.of(Column.builder()
                .name("region")
                .type(Schema.STRING)
                .build()))
        .dataFormat(DataFormat.JSON)
        .partitionProjection(Map.of(
                "region", PartitionProjectionConfiguration.enum(EnumPartitionProjectionConfigurationProps.builder()
                        .values(List.of("us-east-1", "us-west-2", "eu-west-1"))
                        .build())))
        .build();

Database myDatabase;

new S3Table(this, "MyTable", new S3TableProps {
    Database = myDatabase,
    Columns = new [] { new Column {
        Name = "data",
        Type = Schema.STRING
    } },
    PartitionKeys = new [] { new Column {
        Name = "region",
        Type = Schema.STRING
    } },
    DataFormat = DataFormat.JSON,
    PartitionProjection = new Dictionary<string, PartitionProjectionConfiguration> {
        { "region", PartitionProjectionConfiguration.Enum(new EnumPartitionProjectionConfigurationProps {
            Values = new [] { "us-east-1", "us-west-2", "eu-west-1" }
        }) }
    }
});

var myDatabase Database

glue.NewS3Table(this, jsii.String("MyTable"), &S3TableProps{
	Database: myDatabase,
	Columns: []Column{
		&Column{
			Name: jsii.String("data"),
			Type: glue.Schema_STRING(),
		},
	},
	PartitionKeys: []Column{
		&Column{
			Name: jsii.String("region"),
			Type: glue.Schema_STRING(),
		},
	},
	DataFormat: glue.DataFormat_JSON(),
	PartitionProjection: map[string]PartitionProjectionConfiguration{
		"region": glue.PartitionProjectionConfiguration_enum(&EnumPartitionProjectionConfigurationProps{
			"values": []*string{
				jsii.String("us-east-1"),
				jsii.String("us-west-2"),
				jsii.String("eu-west-1"),
			},
		}),
	},
})

INJECTED Projection

For custom partition values injected at query time:

declare const myDatabase: glue.Database;
new glue.S3Table(this, 'MyTable', {
  database: myDatabase,
  columns: [{
    name: 'data',
    type: glue.Schema.STRING,
  }],
  partitionKeys: [{
    name: 'custom',
    type: glue.Schema.STRING,
  }],
  dataFormat: glue.DataFormat.JSON,
  partitionProjection: {
    custom: glue.PartitionProjectionConfiguration.injected(),
  },
});

# my_database: glue.Database

glue.S3Table(self, "MyTable",
    database=my_database,
    columns=[glue.Column(
        name="data",
        type=glue.Schema.STRING
    )],
    partition_keys=[glue.Column(
        name="custom",
        type=glue.Schema.STRING
    )],
    data_format=glue.DataFormat.JSON,
    partition_projection={
        "custom": glue.PartitionProjectionConfiguration.injected()
    }
)

Database myDatabase;

S3Table.Builder.create(this, "MyTable")
        .database(myDatabase)
        .columns(List.of(Column.builder()
                .name("data")
                .type(Schema.STRING)
                .build()))
        .partitionKeys(List.of(Column.builder()
                .name("custom")
                .type(Schema.STRING)
                .build()))
        .dataFormat(DataFormat.JSON)
        .partitionProjection(Map.of(
                "custom", PartitionProjectionConfiguration.injected()))
        .build();

Database myDatabase;

new S3Table(this, "MyTable", new S3TableProps {
    Database = myDatabase,
    Columns = new [] { new Column {
        Name = "data",
        Type = Schema.STRING
    } },
    PartitionKeys = new [] { new Column {
        Name = "custom",
        Type = Schema.STRING
    } },
    DataFormat = DataFormat.JSON,
    PartitionProjection = new Dictionary<string, PartitionProjectionConfiguration> {
        { "custom", PartitionProjectionConfiguration.Injected() }
    }
});

var myDatabase Database

glue.NewS3Table(this, jsii.String("MyTable"), &S3TableProps{
	Database: myDatabase,
	Columns: []Column{
		&Column{
			Name: jsii.String("data"),
			Type: glue.Schema_STRING(),
		},
	},
	PartitionKeys: []Column{
		&Column{
			Name: jsii.String("custom"),
			Type: glue.Schema_STRING(),
		},
	},
	DataFormat: glue.DataFormat_JSON(),
	PartitionProjection: map[string]PartitionProjectionConfiguration{
		"custom": glue.PartitionProjectionConfiguration_injected(),
	},
})

Multiple Partition Projections

You can configure partition projection for multiple partition keys:

declare const myDatabase: glue.Database;
new glue.S3Table(this, 'MyTable', {
  database: myDatabase,
  columns: [{
    name: 'data',
    type: glue.Schema.STRING,
  }],
  partitionKeys: [
    {
      name: 'year',
      type: glue.Schema.INTEGER,
    },
    {
      name: 'month',
      type: glue.Schema.INTEGER,
    },
    {
      name: 'region',
      type: glue.Schema.STRING,
    },
  ],
  dataFormat: glue.DataFormat.JSON,
  partitionProjection: {
    year: glue.PartitionProjectionConfiguration.integer({
      min: 2020,
      max: 2023,
    }),
    month: glue.PartitionProjectionConfiguration.integer({
      min: 1,
      max: 12,
      digits: 2,
    }),
    region: glue.PartitionProjectionConfiguration.enum({
      values: ['us-east-1', 'us-west-2'],
    }),
  },
});

# my_database: glue.Database

glue.S3Table(self, "MyTable",
    database=my_database,
    columns=[glue.Column(
        name="data",
        type=glue.Schema.STRING
    )],
    partition_keys=[glue.Column(
        name="year",
        type=glue.Schema.INTEGER
    ), glue.Column(
        name="month",
        type=glue.Schema.INTEGER
    ), glue.Column(
        name="region",
        type=glue.Schema.STRING
    )
    ],
    data_format=glue.DataFormat.JSON,
    partition_projection={
        "year": glue.PartitionProjectionConfiguration.integer(
            min=2020,
            max=2023
        ),
        "month": glue.PartitionProjectionConfiguration.integer(
            min=1,
            max=12,
            digits=2
        ),
        "region": glue.PartitionProjectionConfiguration.enum(
            values=["us-east-1", "us-west-2"]
        )
    }
)

Database myDatabase;

S3Table.Builder.create(this, "MyTable")
        .database(myDatabase)
        .columns(List.of(Column.builder()
                .name("data")
                .type(Schema.STRING)
                .build()))
        .partitionKeys(List.of(Column.builder()
                .name("year")
                .type(Schema.INTEGER)
                .build(), Column.builder()
                .name("month")
                .type(Schema.INTEGER)
                .build(), Column.builder()
                .name("region")
                .type(Schema.STRING)
                .build()))
        .dataFormat(DataFormat.JSON)
        .partitionProjection(Map.of(
                "year", PartitionProjectionConfiguration.integer(IntegerPartitionProjectionConfigurationProps.builder()
                        .min(2020)
                        .max(2023)
                        .build()),
                "month", PartitionProjectionConfiguration.integer(IntegerPartitionProjectionConfigurationProps.builder()
                        .min(1)
                        .max(12)
                        .digits(2)
                        .build()),
                "region", PartitionProjectionConfiguration.enum(EnumPartitionProjectionConfigurationProps.builder()
                        .values(List.of("us-east-1", "us-west-2"))
                        .build())))
        .build();

Database myDatabase;

new S3Table(this, "MyTable", new S3TableProps {
    Database = myDatabase,
    Columns = new [] { new Column {
        Name = "data",
        Type = Schema.STRING
    } },
    PartitionKeys = new [] { new Column {
        Name = "year",
        Type = Schema.INTEGER
    }, new Column {
        Name = "month",
        Type = Schema.INTEGER
    }, new Column {
        Name = "region",
        Type = Schema.STRING
    } },
    DataFormat = DataFormat.JSON,
    PartitionProjection = new Dictionary<string, PartitionProjectionConfiguration> {
        { "year", PartitionProjectionConfiguration.Integer(new IntegerPartitionProjectionConfigurationProps {
            Min = 2020,
            Max = 2023
        }) },
        { "month", PartitionProjectionConfiguration.Integer(new IntegerPartitionProjectionConfigurationProps {
            Min = 1,
            Max = 12,
            Digits = 2
        }) },
        { "region", PartitionProjectionConfiguration.Enum(new EnumPartitionProjectionConfigurationProps {
            Values = new [] { "us-east-1", "us-west-2" }
        }) }
    }
});

var myDatabase Database

glue.NewS3Table(this, jsii.String("MyTable"), &S3TableProps{
	Database: myDatabase,
	Columns: []Column{
		&Column{
			Name: jsii.String("data"),
			Type: glue.Schema_STRING(),
		},
	},
	PartitionKeys: []Column{
		&Column{
			Name: jsii.String("year"),
			Type: glue.Schema_INTEGER(),
		},
		&Column{
			Name: jsii.String("month"),
			Type: glue.Schema_INTEGER(),
		},
		&Column{
			Name: jsii.String("region"),
			Type: glue.Schema_STRING(),
		},
	},
	DataFormat: glue.DataFormat_JSON(),
	PartitionProjection: map[string]PartitionProjectionConfiguration{
		"year": glue.PartitionProjectionConfiguration_integer(&IntegerPartitionProjectionConfigurationProps{
			"min": jsii.Number(2020),
			"max": jsii.Number(2023),
		}),
		"month": glue.PartitionProjectionConfiguration_integer(&IntegerPartitionProjectionConfigurationProps{
			"min": jsii.Number(1),
			"max": jsii.Number(12),
			"digits": jsii.Number(2),
		}),
		"region": glue.PartitionProjectionConfiguration_enum(&EnumPartitionProjectionConfigurationProps{
			"values": []*string{
				jsii.String("us-east-1"),
				jsii.String("us-west-2"),
			},
		}),
	},
})

Glue Connections

Glue connections allow external data connections to third party databases and data warehouses. However, these connections can also be assigned to Glue Tables, allowing you to query external data sources using the Glue Data Catalog.

Whereas S3Table will point to (and if needed, create) a bucket to store the tables' data, ExternalTable will point to an existing table in a data source. For example, to create a table in Glue that points to a table in Redshift:

declare const myConnection: glue.Connection;
declare const myDatabase: glue.Database;
new glue.ExternalTable(this, 'MyTable', {
  connection: myConnection,
  externalDataLocation: 'default_db_public_example', // A table in Redshift
  // ...
  database: myDatabase,
  columns: [{
    name: 'col1',
    type: glue.Schema.STRING,
  }],
  dataFormat: glue.DataFormat.JSON,
});

# my_connection: glue.Connection
# my_database: glue.Database

glue.ExternalTable(self, "MyTable",
    connection=my_connection,
    external_data_location="default_db_public_example",  # A table in Redshift
    # ...
    database=my_database,
    columns=[glue.Column(
        name="col1",
        type=glue.Schema.STRING
    )],
    data_format=glue.DataFormat.JSON
)

Connection myConnection;
Database myDatabase;

ExternalTable.Builder.create(this, "MyTable")
        .connection(myConnection)
        .externalDataLocation("default_db_public_example") // A table in Redshift
        // ...
        .database(myDatabase)
        .columns(List.of(Column.builder()
                .name("col1")
                .type(Schema.STRING)
                .build()))
        .dataFormat(DataFormat.JSON)
        .build();

Connection myConnection;
Database myDatabase;

new ExternalTable(this, "MyTable", new ExternalTableProps {
    Connection = myConnection,
    ExternalDataLocation = "default_db_public_example",  // A table in Redshift
    // ...
    Database = myDatabase,
    Columns = new [] { new Column {
        Name = "col1",
        Type = Schema.STRING
    } },
    DataFormat = DataFormat.JSON
});

var myConnection Connection
var myDatabase Database

glue.NewExternalTable(this, jsii.String("MyTable"), &ExternalTableProps{
	Connection: myConnection,
	ExternalDataLocation: jsii.String("default_db_public_example"),
	 // A table in Redshift
	// ...
	Database: myDatabase,
	Columns: []Column{
		&Column{
			Name: jsii.String("col1"),
			Type: glue.Schema_STRING(),
		},
	},
	DataFormat: glue.DataFormat_JSON(),
})

Data Quality Ruleset

A DataQualityRuleset defines a set of data quality rules — authored in Glue's Data Quality Definition Language (DQDL) — that are evaluated against a table in the Data Catalog.

declare const database: glue.IDatabase;
new glue.DataQualityRuleset(this, 'MyRuleset', {
  rulesetName: 'my_ruleset',
  dqdl: glue.Dqdl.fromString('Rules = [ RowCount > 100, IsComplete "order_id" ]'),
  targetTable: glue.DataQualityTargetTable.fromTableName(database, 'my_table'),
});

# database: glue.IDatabase

glue.DataQualityRuleset(self, "MyRuleset",
    ruleset_name="my_ruleset",
    dqdl=glue.Dqdl.from_string("Rules = [ RowCount > 100, IsComplete \"order_id\" ]"),
    target_table=glue.DataQualityTargetTable.from_table_name(database, "my_table")
)

IDatabase database;

DataQualityRuleset.Builder.create(this, "MyRuleset")
        .rulesetName("my_ruleset")
        .dqdl(Dqdl.fromString("Rules = [ RowCount > 100, IsComplete \"order_id\" ]"))
        .targetTable(DataQualityTargetTable.fromTableName(database, "my_table"))
        .build();

IDatabase database;

new DataQualityRuleset(this, "MyRuleset", new DataQualityRulesetProps {
    RulesetName = "my_ruleset",
    Dqdl = Dqdl.FromString("Rules = [ RowCount > 100, IsComplete \"order_id\" ]"),
    TargetTable = DataQualityTargetTable.FromTableName(database, "my_table")
});

var database IDatabase

glue.NewDataQualityRuleset(this, jsii.String("MyRuleset"), &DataQualityRulesetProps{
	RulesetName: jsii.String("my_ruleset"),
	Dqdl: glue.Dqdl_FromString(jsii.String("Rules = [ RowCount > 100, IsComplete \"order_id\" ]")),
	TargetTable: glue.DataQualityTargetTable_FromTableName(database, jsii.String("my_table")),
})

Build the DQDL document with Dqdl.fromString(...). Glue parses and validates the DQDL when the ruleset is deployed; see the DQDL reference for the full rule syntax.

Encryption

When the table creates its own S3 bucket (S3TableStorage.managedBucket, the default), that bucket enforces SSL: a bucket policy denies any request made over plain HTTP. If you bring your own bucket with S3TableStorage.fromBucket, enabling enforceSSL on it is your responsibility.

Server-side encryption applies only to a bucket the table manages. Choose it with storage: glue.S3TableStorage.managedBucket(...):

• S3Managed - (default) Server side encryption (SSE-S3) with an Amazon S3-managed key.

declare const myDatabase: glue.Database;
new glue.S3Table(this, 'MyTable', {
  storage: glue.S3TableStorage.managedBucket(glue.S3TableEncryption.s3Managed()),
  // ...
  database: myDatabase,
  columns: [{
    name: 'col1',
    type: glue.Schema.STRING,
  }],
  dataFormat: glue.DataFormat.JSON,
});

# my_database: glue.Database

glue.S3Table(self, "MyTable",
    storage=glue.S3TableStorage.managed_bucket(glue.S3TableEncryption.s3_managed()),
    # ...
    database=my_database,
    columns=[glue.Column(
        name="col1",
        type=glue.Schema.STRING
    )],
    data_format=glue.DataFormat.JSON
)

Database myDatabase;

S3Table.Builder.create(this, "MyTable")
        .storage(S3TableStorage.managedBucket(S3TableEncryption.s3Managed()))
        // ...
        .database(myDatabase)
        .columns(List.of(Column.builder()
                .name("col1")
                .type(Schema.STRING)
                .build()))
        .dataFormat(DataFormat.JSON)
        .build();

Database myDatabase;

new S3Table(this, "MyTable", new S3TableProps {
    Storage = S3TableStorage.ManagedBucket(S3TableEncryption.S3Managed()),
    // ...
    Database = myDatabase,
    Columns = new [] { new Column {
        Name = "col1",
        Type = Schema.STRING
    } },
    DataFormat = DataFormat.JSON
});

var myDatabase Database

glue.NewS3Table(this, jsii.String("MyTable"), &S3TableProps{
	Storage: glue.S3TableStorage_ManagedBucket(glue.S3TableEncryption_S3Managed()),
	// ...
	Database: myDatabase,
	Columns: []Column{
		&Column{
			Name: jsii.String("col1"),
			Type: glue.Schema_STRING(),
		},
	},
	DataFormat: glue.DataFormat_JSON(),
})

• Kms - Server-side encryption (SSE-KMS) with an AWS KMS Key managed by the account owner.

declare const myDatabase: glue.Database;
// KMS key is created automatically
new glue.S3Table(this, 'MyTable', {
  storage: glue.S3TableStorage.managedBucket(glue.S3TableEncryption.kms()),
  // ...
  database: myDatabase,
  columns: [{
    name: 'col1',
    type: glue.Schema.STRING,
  }],
  dataFormat: glue.DataFormat.JSON,
});

// with an explicit KMS key
new glue.S3Table(this, 'MyTable', {
  storage: glue.S3TableStorage.managedBucket(glue.S3TableEncryption.kms(new kms.Key(this, 'MyKey'))),
  // ...
  database: myDatabase,
  columns: [{
    name: 'col1',
    type: glue.Schema.STRING,
  }],
  dataFormat: glue.DataFormat.JSON,
});

# my_database: glue.Database

# KMS key is created automatically
glue.S3Table(self, "MyTable",
    storage=glue.S3TableStorage.managed_bucket(glue.S3TableEncryption.kms()),
    # ...
    database=my_database,
    columns=[glue.Column(
        name="col1",
        type=glue.Schema.STRING
    )],
    data_format=glue.DataFormat.JSON
)

# with an explicit KMS key
glue.S3Table(self, "MyTable",
    storage=glue.S3TableStorage.managed_bucket(glue.S3TableEncryption.kms(kms.Key(self, "MyKey"))),
    # ...
    database=my_database,
    columns=[glue.Column(
        name="col1",
        type=glue.Schema.STRING
    )],
    data_format=glue.DataFormat.JSON
)

Database myDatabase;

// KMS key is created automatically
// KMS key is created automatically
S3Table.Builder.create(this, "MyTable")
        .storage(S3TableStorage.managedBucket(S3TableEncryption.kms()))
        // ...
        .database(myDatabase)
        .columns(List.of(Column.builder()
                .name("col1")
                .type(Schema.STRING)
                .build()))
        .dataFormat(DataFormat.JSON)
        .build();

// with an explicit KMS key
// with an explicit KMS key
S3Table.Builder.create(this, "MyTable")
        .storage(S3TableStorage.managedBucket(S3TableEncryption.kms(new Key(this, "MyKey"))))
        // ...
        .database(myDatabase)
        .columns(List.of(Column.builder()
                .name("col1")
                .type(Schema.STRING)
                .build()))
        .dataFormat(DataFormat.JSON)
        .build();

Database myDatabase;

// KMS key is created automatically
// KMS key is created automatically
new S3Table(this, "MyTable", new S3TableProps {
    Storage = S3TableStorage.ManagedBucket(S3TableEncryption.Kms()),
    // ...
    Database = myDatabase,
    Columns = new [] { new Column {
        Name = "col1",
        Type = Schema.STRING
    } },
    DataFormat = DataFormat.JSON
});

// with an explicit KMS key
// with an explicit KMS key
new S3Table(this, "MyTable", new S3TableProps {
    Storage = S3TableStorage.ManagedBucket(S3TableEncryption.Kms(new Key(this, "MyKey"))),
    // ...
    Database = myDatabase,
    Columns = new [] { new Column {
        Name = "col1",
        Type = Schema.STRING
    } },
    DataFormat = DataFormat.JSON
});

var myDatabase Database

// KMS key is created automatically
// KMS key is created automatically
glue.NewS3Table(this, jsii.String("MyTable"), &S3TableProps{
	Storage: glue.S3TableStorage_ManagedBucket(glue.S3TableEncryption_Kms()),
	// ...
	Database: myDatabase,
	Columns: []Column{
		&Column{
			Name: jsii.String("col1"),
			Type: glue.Schema_STRING(),
		},
	},
	DataFormat: glue.DataFormat_JSON(),
})

// with an explicit KMS key
// with an explicit KMS key
glue.NewS3Table(this, jsii.String("MyTable"), &S3TableProps{
	Storage: glue.S3TableStorage_*ManagedBucket(glue.S3TableEncryption_*Kms(kms.NewKey(this, jsii.String("MyKey")))),
	// ...
	Database: myDatabase,
	Columns: []Column{
		&Column{
			Name: jsii.String("col1"),
			Type: glue.Schema_STRING(),
		},
	},
	DataFormat: glue.DataFormat_JSON(),
})

• KmsManaged - Server-side encryption (SSE-KMS), like Kms, except with an AWS KMS Key managed by the AWS Key Management Service.

declare const myDatabase: glue.Database;
new glue.S3Table(this, 'MyTable', {
  storage: glue.S3TableStorage.managedBucket(glue.S3TableEncryption.kmsManaged()),
  // ...
  database: myDatabase,
  columns: [{
    name: 'col1',
    type: glue.Schema.STRING,
  }],
  dataFormat: glue.DataFormat.JSON,
});

# my_database: glue.Database

glue.S3Table(self, "MyTable",
    storage=glue.S3TableStorage.managed_bucket(glue.S3TableEncryption.kms_managed()),
    # ...
    database=my_database,
    columns=[glue.Column(
        name="col1",
        type=glue.Schema.STRING
    )],
    data_format=glue.DataFormat.JSON
)

Database myDatabase;

S3Table.Builder.create(this, "MyTable")
        .storage(S3TableStorage.managedBucket(S3TableEncryption.kmsManaged()))
        // ...
        .database(myDatabase)
        .columns(List.of(Column.builder()
                .name("col1")
                .type(Schema.STRING)
                .build()))
        .dataFormat(DataFormat.JSON)
        .build();

Database myDatabase;

new S3Table(this, "MyTable", new S3TableProps {
    Storage = S3TableStorage.ManagedBucket(S3TableEncryption.KmsManaged()),
    // ...
    Database = myDatabase,
    Columns = new [] { new Column {
        Name = "col1",
        Type = Schema.STRING
    } },
    DataFormat = DataFormat.JSON
});

var myDatabase Database

glue.NewS3Table(this, jsii.String("MyTable"), &S3TableProps{
	Storage: glue.S3TableStorage_ManagedBucket(glue.S3TableEncryption_KmsManaged()),
	// ...
	Database: myDatabase,
	Columns: []Column{
		&Column{
			Name: jsii.String("col1"),
			Type: glue.Schema_STRING(),
		},
	},
	DataFormat: glue.DataFormat_JSON(),
})

Client-side encryption (CSE-KMS) is independent of the bucket's server-side encryption and works with either a managed or an existing bucket. Configure it with clientSideEncryption:

declare const myDatabase: glue.Database;
// KMS key is created automatically
new glue.S3Table(this, 'MyTable', {
  clientSideEncryption: glue.TableClientSideEncryption.kms(),
  // ...
  database: myDatabase,
  columns: [{
    name: 'col1',
    type: glue.Schema.STRING,
  }],
  dataFormat: glue.DataFormat.JSON,
});

// with an explicit KMS key
new glue.S3Table(this, 'MyTable', {
  clientSideEncryption: glue.TableClientSideEncryption.kms(new kms.Key(this, 'MyKey')),
  // ...
  database: myDatabase,
  columns: [{
    name: 'col1',
    type: glue.Schema.STRING,
  }],
  dataFormat: glue.DataFormat.JSON,
});

# my_database: glue.Database

# KMS key is created automatically
glue.S3Table(self, "MyTable",
    client_side_encryption=glue.TableClientSideEncryption.kms(),
    # ...
    database=my_database,
    columns=[glue.Column(
        name="col1",
        type=glue.Schema.STRING
    )],
    data_format=glue.DataFormat.JSON
)

# with an explicit KMS key
glue.S3Table(self, "MyTable",
    client_side_encryption=glue.TableClientSideEncryption.kms(kms.Key(self, "MyKey")),
    # ...
    database=my_database,
    columns=[glue.Column(
        name="col1",
        type=glue.Schema.STRING
    )],
    data_format=glue.DataFormat.JSON
)

Database myDatabase;

// KMS key is created automatically
// KMS key is created automatically
S3Table.Builder.create(this, "MyTable")
        .clientSideEncryption(TableClientSideEncryption.kms())
        // ...
        .database(myDatabase)
        .columns(List.of(Column.builder()
                .name("col1")
                .type(Schema.STRING)
                .build()))
        .dataFormat(DataFormat.JSON)
        .build();

// with an explicit KMS key
// with an explicit KMS key
S3Table.Builder.create(this, "MyTable")
        .clientSideEncryption(TableClientSideEncryption.kms(new Key(this, "MyKey")))
        // ...
        .database(myDatabase)
        .columns(List.of(Column.builder()
                .name("col1")
                .type(Schema.STRING)
                .build()))
        .dataFormat(DataFormat.JSON)
        .build();

Database myDatabase;

// KMS key is created automatically
// KMS key is created automatically
new S3Table(this, "MyTable", new S3TableProps {
    ClientSideEncryption = TableClientSideEncryption.Kms(),
    // ...
    Database = myDatabase,
    Columns = new [] { new Column {
        Name = "col1",
        Type = Schema.STRING
    } },
    DataFormat = DataFormat.JSON
});

// with an explicit KMS key
// with an explicit KMS key
new S3Table(this, "MyTable", new S3TableProps {
    ClientSideEncryption = TableClientSideEncryption.Kms(new Key(this, "MyKey")),
    // ...
    Database = myDatabase,
    Columns = new [] { new Column {
        Name = "col1",
        Type = Schema.STRING
    } },
    DataFormat = DataFormat.JSON
});

var myDatabase Database

// KMS key is created automatically
// KMS key is created automatically
glue.NewS3Table(this, jsii.String("MyTable"), &S3TableProps{
	ClientSideEncryption: glue.TableClientSideEncryption_Kms(),
	// ...
	Database: myDatabase,
	Columns: []Column{
		&Column{
			Name: jsii.String("col1"),
			Type: glue.Schema_STRING(),
		},
	},
	DataFormat: glue.DataFormat_JSON(),
})

// with an explicit KMS key
// with an explicit KMS key
glue.NewS3Table(this, jsii.String("MyTable"), &S3TableProps{
	ClientSideEncryption: glue.TableClientSideEncryption_*Kms(kms.NewKey(this, jsii.String("MyKey"))),
	// ...
	Database: myDatabase,
	Columns: []Column{
		&Column{
			Name: jsii.String("col1"),
			Type: glue.Schema_STRING(),
		},
	},
	DataFormat: glue.DataFormat_JSON(),
})

To store the table's data in an existing bucket, use glue.S3TableStorage.fromBucket(bucket). CDK does not manage that bucket's server-side encryption, so an encryption choice can never be paired with a provided bucket — but client-side encryption still applies.

Marking table data as encrypted

Both S3Table and ExternalTable set the has_encrypted_data table parameter, which Athena reads when querying client-side (CSE-KMS) encrypted datasets. It defaults to true. Set hasEncryptedData to false when the underlying data is not encrypted:

declare const myDatabase: glue.Database;
new glue.S3Table(this, 'MyTable', {
  hasEncryptedData: false,
  database: myDatabase,
  columns: [{
    name: 'col1',
    type: glue.Schema.STRING,
  }],
  dataFormat: glue.DataFormat.JSON,
});

# my_database: glue.Database

glue.S3Table(self, "MyTable",
    has_encrypted_data=False,
    database=my_database,
    columns=[glue.Column(
        name="col1",
        type=glue.Schema.STRING
    )],
    data_format=glue.DataFormat.JSON
)

Database myDatabase;

S3Table.Builder.create(this, "MyTable")
        .hasEncryptedData(false)
        .database(myDatabase)
        .columns(List.of(Column.builder()
                .name("col1")
                .type(Schema.STRING)
                .build()))
        .dataFormat(DataFormat.JSON)
        .build();

Database myDatabase;

new S3Table(this, "MyTable", new S3TableProps {
    HasEncryptedData = false,
    Database = myDatabase,
    Columns = new [] { new Column {
        Name = "col1",
        Type = Schema.STRING
    } },
    DataFormat = DataFormat.JSON
});

var myDatabase Database

glue.NewS3Table(this, jsii.String("MyTable"), &S3TableProps{
	HasEncryptedData: jsii.Boolean(false),
	Database: myDatabase,
	Columns: []Column{
		&Column{
			Name: jsii.String("col1"),
			Type: glue.Schema_STRING(),
		},
	},
	DataFormat: glue.DataFormat_JSON(),
})

Do not set has_encrypted_data through the free-form parameters map as well - a value there that conflicts with hasEncryptedData is rejected at synthesis time.

Types

A table's schema is a collection of columns, each of which have a name and a type. Types are recursive structures, consisting of primitive and complex types:

declare const myDatabase: glue.Database;
new glue.S3Table(this, 'MyTable', {
  columns: [{
    name: 'primitive_column',
    type: glue.Schema.STRING,
  }, {
    name: 'array_column',
    type: glue.Schema.array(glue.Schema.INTEGER),
    comment: 'array<integer>',
  }, {
    name: 'map_column',
    type: glue.Schema.map(
      glue.Schema.STRING,
      glue.Schema.TIMESTAMP),
    comment: 'map<string,timestamp>',
  }, {
    name: 'struct_column',
    type: glue.Schema.struct([{
      name: 'nested_column',
      type: glue.Schema.DATE,
      comment: 'nested comment',
    }]),
    comment: "struct<nested_column:date COMMENT 'nested comment'>",
  }],
  // ...
  database: myDatabase,
  dataFormat: glue.DataFormat.JSON,
});

# my_database: glue.Database

glue.S3Table(self, "MyTable",
    columns=[glue.Column(
        name="primitive_column",
        type=glue.Schema.STRING
    ), glue.Column(
        name="array_column",
        type=glue.Schema.array(glue.Schema.INTEGER),
        comment="array<integer>"
    ), glue.Column(
        name="map_column",
        type=glue.Schema.map(glue.Schema.STRING, glue.Schema.TIMESTAMP),
        comment="map<string,timestamp>"
    ), glue.Column(
        name="struct_column",
        type=glue.Schema.struct([
            name="nested_column",
            type=glue.Schema.DATE,
            comment="nested comment"
        ]),
        comment="struct<nested_column:date COMMENT 'nested comment'>"
    )],
    # ...
    database=my_database,
    data_format=glue.DataFormat.JSON
)

Database myDatabase;

S3Table.Builder.create(this, "MyTable")
        .columns(List.of(Column.builder()
                .name("primitive_column")
                .type(Schema.STRING)
                .build(), Column.builder()
                .name("array_column")
                .type(Schema.array(Schema.INTEGER))
                .comment("array<integer>")
                .build(), Column.builder()
                .name("map_column")
                .type(Schema.map(Schema.STRING, Schema.TIMESTAMP))
                .comment("map<string,timestamp>")
                .build(), Column.builder()
                .name("struct_column")
                .type(Schema.struct(List.of(Column.builder()
                        .name("nested_column")
                        .type(Schema.DATE)
                        .comment("nested comment")
                        .build())))
                .comment("struct<nested_column:date COMMENT 'nested comment'>")
                .build()))
        // ...
        .database(myDatabase)
        .dataFormat(DataFormat.JSON)
        .build();

Database myDatabase;

new S3Table(this, "MyTable", new S3TableProps {
    Columns = new [] { new Column {
        Name = "primitive_column",
        Type = Schema.STRING
    }, new Column {
        Name = "array_column",
        Type = Schema.Array(Schema.INTEGER),
        Comment = "array<integer>"
    }, new Column {
        Name = "map_column",
        Type = Schema.Map(Schema.STRING, Schema.TIMESTAMP),
        Comment = "map<string,timestamp>"
    }, new Column {
        Name = "struct_column",
        Type = Schema.Struct(new [] { new Column {
            Name = "nested_column",
            Type = Schema.DATE,
            Comment = "nested comment"
        } }),
        Comment = "struct<nested_column:date COMMENT 'nested comment'>"
    } },
    // ...
    Database = myDatabase,
    DataFormat = DataFormat.JSON
});

var myDatabase Database

glue.NewS3Table(this, jsii.String("MyTable"), &S3TableProps{
	Columns: []Column{
		&Column{
			Name: jsii.String("primitive_column"),
			Type: glue.Schema_STRING(),
		},
		&Column{
			Name: jsii.String("array_column"),
			Type: glue.Schema_Array(glue.Schema_INTEGER()),
			Comment: jsii.String("array<integer>"),
		},
		&Column{
			Name: jsii.String("map_column"),
			Type: glue.Schema_Map(glue.Schema_STRING(), glue.Schema_TIMESTAMP()),
			Comment: jsii.String("map<string,timestamp>"),
		},
		&Column{
			Name: jsii.String("struct_column"),
			Type: glue.Schema_Struct([]Column{
				&Column{
					Name: jsii.String("nested_column"),
					Type: glue.Schema_DATE(),
					Comment: jsii.String("nested comment"),
				},
			}),
			Comment: jsii.String("struct<nested_column:date COMMENT 'nested comment'>"),
		},
	},
	// ...
	Database: myDatabase,
	DataFormat: glue.DataFormat_JSON(),
})

For a type the Schema factories don't model, use glue.Schema.custom('...'), which takes the raw Glue input string.

Public FAQ

What are we launching today?

We’re launching new features to an AWS CDK Glue L2 Construct to provide best-practice defaults and convenience methods to create Glue Jobs, Connections, Triggers, Workflows, and the underlying permissions and configuration.

Why should I use this Construct?

Developers should use this Construct to reduce the amount of boilerplate code and complexity each individual has to navigate, and make it easier to create best-practice Glue resources.

What’s not in scope?

Glue Crawlers and other resources that are now managed by the AWS LakeFormation team are not in scope for this effort. Developers should use existing methods to create these resources, and the new Glue L2 construct assumes they already exist as inputs. While best practice is for application and infrastructure code to be as close as possible for teams using fully-implemented DevOps mechanisms, in practice these ETL scripts are likely managed by a data science team who know Python or Scala and don’t necessarily own or manage their own infrastructure deployments. We want to meet developers where they are, and not assume that all of the code resides in the same repository, Developers can automate this themselves via the CDK, however, if they do own both.

Validating Glue version and feature use per AWS region at synth time is also not in scope. AWS’ intention is for all features to eventually be propagated to all Global regions, so the complexity involved in creating and updating region- specific configuration to match shifting feature sets does not out-weigh the likelihood that a developer will use this construct to deploy resources to a region without a particular new feature to a region that doesn’t yet support it without researching or manually attempting to use that feature before developing it via IaC. The developer will, of course, still get feedback from the underlying Glue APIs as CloudFormation deploys the resources similar to the current CDK L1 Glue experience.

Types

Name	Kind	Summary
Catalog	Construct (L2)	A Glue Data Catalog.
CatalogBase	Construct (L2)	Base class for all ICatalog implementations.
Connection	Construct (L2)	An AWS Glue connection to a data source.
Database	Construct (L2)	A Glue database.
DataQualityRuleset	Construct (L2)	A Glue Data Quality ruleset.
ExternalTable	Construct (L2)	A Glue table that targets an external data location (e.g. A table in a Redshift Cluster).
Job	Construct (L2)	A Glue Job.
JobBase	Construct (L2)	A base class is needed to be able to import existing Jobs into a CDK app to reference as part of a larger stack or construct.
PySparkEtlJob	Construct (L2)	PySpark ETL Jobs class.
PySparkFlexEtlJob	Construct (L2)	Flex Jobs class.
PySparkStreamingJob	Construct (L2)	Python Spark Streaming Jobs class.
PythonShellJob	Construct (L2)	Python Shell Jobs class.
S3Table	Construct (L2)	A Glue table that targets a S3 dataset.
ScalaSparkEtlJob	Construct (L2)	Spark ETL Jobs class.
ScalaSparkFlexEtlJob	Construct (L2)	Scala Spark Flex ETL Jobs class.
ScalaSparkStreamingJob	Construct (L2)	Scala Streaming Jobs class.
SecurityConfiguration	Construct (L2)	A security configuration is a set of security properties that can be used by AWS Glue to encrypt data at rest.
SparkJob	Construct (L2)	Base class for different types of Spark Jobs.
TableBase	Construct (L2)	A Glue table.
Workflow	Construct (L2)	This module defines a construct for creating and managing AWS Glue Workflows and Triggers.
WorkflowBase	Construct (L2)	Base abstract class for Workflow.
CfnBlueprint	Construct (L1)	Resource Type definition for AWS::Glue::Blueprint.
CfnCatalog	Construct (L1)	Creates a catalog in the Glue Data Catalog.
CfnClassifier	Construct (L1)	The AWS::Glue::Classifier resource creates an AWS Glue classifier that categorizes data sources and specifies schemas.
CfnConnection	Construct (L1)	The AWS::Glue::Connection resource specifies an AWS Glue connection to a data source.
CfnConnectionType	Construct (L1)	Registers a custom connection type in Glue based on the configuration provided.
CfnCrawler	Construct (L1)	The AWS::Glue::Crawler resource specifies an AWS Glue crawler.
CfnCustomEntityType	Construct (L1)	Creates a custom pattern that is used to detect sensitive data across the columns and rows of your structured data.
CfnDatabase	Construct (L1)	The AWS::Glue::Database resource specifies a logical grouping of tables in AWS Glue .
CfnDataCatalogEncryptionSettings	Construct (L1)	Sets the security configuration for a specified catalog.
CfnDataQualityRuleset	Construct (L1)	The AWS::Glue::DataQualityRuleset resource specifies a data quality ruleset with DQDL rules applied to a specified AWS Glue table.
CfnDevEndpoint	Construct (L1)	The AWS::Glue::DevEndpoint resource specifies a development endpoint where a developer can remotely debug ETL scripts for AWS Glue .
CfnIdentityCenterConfiguration	Construct (L1)	Creates a new AWS Glue Identity Center configuration to enable integration between AWS Glue and AWS IAM Identity Center for authentication and authorization.
CfnIntegration	Construct (L1)	The AWS::Glue::Integration resource specifies an AWS Glue zero-ETL integration from a data source to a target.
CfnIntegrationResourceProperty	Construct (L1)	The AWS::Glue::IntegrationResourceProperty resource type can be used to setup ResourceProperty of the AWS Glue connection (for the SaaS source), DynamoDB Database (for DynamoDB source), or AWS Glue database ARN (for the target).
CfnJob	Construct (L1)	The AWS::Glue::Job resource specifies an AWS Glue job in the data catalog.
CfnMLTransform	Construct (L1)	The AWS::Glue::MLTransform is an AWS Glue resource type that manages machine learning transforms.
CfnPartition	Construct (L1)	The AWS::Glue::Partition resource creates an AWS Glue partition, which represents a slice of table data.
CfnRegistry	Construct (L1)	The AWS::Glue::Registry is an AWS Glue resource type that manages registries of schemas in the AWS Glue Schema Registry.
CfnSchema	Construct (L1)	The AWS::Glue::Schema is an AWS Glue resource type that manages schemas in the AWS Glue Schema Registry.
CfnSchemaVersion	Construct (L1)	The AWS::Glue::SchemaVersion is an AWS Glue resource type that manages schema versions of schemas in the AWS Glue Schema Registry.
CfnSchemaVersionMetadata	Construct (L1)	The AWS::Glue::SchemaVersionMetadata is an AWS Glue resource type that defines the metadata key-value pairs for a schema version in AWS Glue Schema Registry.
CfnSecurityConfiguration	Construct (L1)	Creates a new security configuration.
CfnSession	Construct (L1)	Resource Type definition for AWS::Glue::Session.
CfnTable	Construct (L1)	The AWS::Glue::Table resource specifies tabular data in the AWS Glue data catalog.
CfnTableOptimizer	Construct (L1)	An AWS Glue resource for enabling table optimizers to improve read performance for open table formats.
CfnTrigger	Construct (L1)	The AWS::Glue::Trigger resource specifies triggers that run AWS Glue jobs.
CfnUsageProfile	Construct (L1)	Creates an AWS Glue usage profile.
CfnUserDefinedFunction	Construct (L1)	Represents a user-defined function (UDF) definition in the AWS Glue Data Catalog.
CfnWorkflow	Construct (L1)	The AWS::Glue::Workflow is an AWS Glue resource type that manages AWS Glue workflows.
Action	Class	An action initiated by a trigger.
AssetCode	Class	Job Code from a local file.
CatalogEncryptionMode	Enum	The encryption-at-rest mode for a Glue Data Catalog.
CatalogEncryptionOptions	Struct	Encryption configuration for a Glue Data Catalog.
CatalogProps	Struct	Construction properties for a Catalog.
CfnBlueprintProps	Struct	Properties for defining a CfnBlueprint.
CfnCatalog.CatalogPropertiesProperty	Struct	A structure that specifies data lake access properties and other custom properties.
CfnCatalog.DataLakeAccessPropertiesProperty	Struct	Data lake access properties for the catalog.
CfnCatalog.DataLakePrincipalProperty	Struct	The Lake Formation principal.
CfnCatalog.FederatedCatalogProperty	Struct	A FederatedCatalog structure that references an entity outside the Glue Data Catalog.
CfnCatalog.PrincipalPermissionsProperty	Struct	Permissions granted to a principal.
CfnCatalog.TargetRedshiftCatalogProperty	Struct	A structure that describes a target catalog for resource linking.
CfnCatalogProps	Struct	Properties for defining a CfnCatalog.
CfnClassifier.CsvClassifierProperty	Struct	A classifier for custom CSV content.
CfnClassifier.GrokClassifierProperty	Struct	A classifier that uses grok patterns.
CfnClassifier.JsonClassifierProperty	Struct	A classifier for JSON content.
CfnClassifier.XMLClassifierProperty	Struct	A classifier for XML content.
CfnClassifierProps	Struct	Properties for defining a CfnClassifier.
CfnConnection.AuthenticationConfigurationProperty	Struct
CfnConnection.AuthorizationCodePropertiesProperty	Struct	The set of properties required for the the OAuth2 AUTHORIZATION_CODE grant type workflow.
CfnConnection.BasicAuthenticationCredentialsProperty	Struct	For supplying basic auth credentials when not providing a SecretArn value.
CfnConnection.ConnectionInputProperty	Struct	A structure that is used to specify a connection to create or update.
CfnConnection.OAuth2ClientApplicationProperty	Struct	The OAuth2 client app used for the connection.
CfnConnection.OAuth2CredentialsProperty	Struct	The credentials used when the authentication type is OAuth2 authentication.
CfnConnection.OAuth2PropertiesProperty	Struct	A structure containing the authentication credentials in the CreateConnection request.
CfnConnection.PhysicalConnectionRequirementsProperty	Struct	The OAuth client app in GetConnection response.
CfnConnectionProps	Struct	Properties for defining a CfnConnection.
CfnConnectionType.BasicAuthenticationPropertiesProperty	Struct	Basic authentication configuration.
CfnConnectionType.BetweenConfigurationProperty	Struct	Configuration that defines how BETWEEN range filter operations are translated into REST API request parameters.
CfnConnectionType.ClientCredentialsPropertiesProperty	Struct	OAuth2 client credentials configuration.
CfnConnectionType.ConnectionPropertiesConfigurationProperty	Struct	Configuration that defines the base URL and additional request parameters needed during connection creation.
CfnConnectionType.ConnectorAuthenticationConfigurationProperty	Struct	Configuration that defines supported authentication types and required properties.
CfnConnectionType.ConnectorAuthorizationCodePropertiesProperty	Struct	OAuth2 authorization code configuration.
CfnConnectionType.ConnectorOAuth2PropertiesProperty	Struct	OAuth2 configuration container.
CfnConnectionType.ConnectorPropertyProperty	Struct	Defines a property configuration for connection types.
CfnConnectionType.CursorConfigurationProperty	Struct	Cursor-based pagination configuration.
CfnConnectionType.CustomAuthenticationPropertiesProperty	Struct	Custom authentication configuration.
CfnConnectionType.EntityConfigurationProperty	Struct	Configuration for interacting with a specific data entity.
CfnConnectionType.ExtractedParameterProperty	Struct	Parameter extraction configuration.
CfnConnectionType.FieldDefinitionProperty	Struct	Defines a field in an entity schema.
CfnConnectionType.FilterConfigurationProperty	Struct	Configuration that defines how filter predicates are applied to REST API requests, supporting both query parameter and filter string strategies.
CfnConnectionType.FilterOverridesProperty	Struct	Configuration that defines per-field overrides for filter behavior, allowing individual fields to customize how filter operations are applied.
CfnConnectionType.FilterStringConfigurationProperty	Struct	Configuration for constructing filter expression strings when using the FILTER_STRING filter mode.
CfnConnectionType.JWTBearerPropertiesProperty	Struct	JWT bearer token configuration.
CfnConnectionType.OffsetConfigurationProperty	Struct	Offset-based pagination configuration.
CfnConnectionType.PaginationConfigurationProperty	Struct	Configuration for handling paginated responses.
CfnConnectionType.ResponseConfigurationProperty	Struct	Configuration for parsing JSON responses from REST API calls.
CfnConnectionType.ResponseExtractionMappingProperty	Struct	Defines how to extract values from HTTP responses.
CfnConnectionType.RestConfigurationProperty	Struct	Configuration for HTTP request and response handling.
CfnConnectionType.SecretConnectorPropertyProperty	Struct	Defines a secret property configuration.
CfnConnectionType.SourceConfigurationProperty	Struct	Configuration that defines how to make requests to endpoints.
CfnConnectionType.ValidationEndpointConfigurationProperty	Struct	Configuration for the validation endpoint.
CfnConnectionTypeProps	Struct	Properties for defining a CfnConnectionType.
CfnCrawler.CatalogTargetProperty	Struct	Specifies an AWS Glue Data Catalog target.
CfnCrawler.DeltaTargetProperty	Struct	Specifies a Delta data store to crawl one or more Delta tables.
CfnCrawler.DynamoDBTargetProperty	Struct	Specifies an Amazon DynamoDB table to crawl.
CfnCrawler.HudiTargetProperty	Struct	Specifies an Apache Hudi data source.
CfnCrawler.IcebergTargetProperty	Struct	Specifies Apache Iceberg data store targets.
CfnCrawler.JdbcTargetProperty	Struct	Specifies a JDBC data store to crawl.
CfnCrawler.LakeFormationConfigurationProperty	Struct	Specifies AWS Lake Formation configuration settings for the crawler.
CfnCrawler.MongoDBTargetProperty	Struct	Specifies an Amazon DocumentDB or MongoDB data store to crawl.
CfnCrawler.RecrawlPolicyProperty	Struct	When crawling an Amazon S3 data source after the first crawl is complete, specifies whether to crawl the entire dataset again or to crawl only folders that were added since the last crawler run.
CfnCrawler.S3TargetProperty	Struct	Specifies a data store in Amazon Simple Storage Service (Amazon S3).
CfnCrawler.ScheduleProperty	Struct	A scheduling object using a cron statement to schedule an event.
CfnCrawler.SchemaChangePolicyProperty	Struct	The policy that specifies update and delete behaviors for the crawler.
CfnCrawler.TargetsProperty	Struct	Specifies data stores to crawl.
CfnCrawlerProps	Struct	Properties for defining a CfnCrawler.
CfnCustomEntityTypeProps	Struct	Properties for defining a CfnCustomEntityType.
CfnDatabase.DatabaseIdentifierProperty	Struct	A structure that describes a target database for resource linking.
CfnDatabase.DatabaseInputProperty	Struct	The structure used to create or update a database.
CfnDatabase.DataLakePrincipalProperty	Struct	The AWS Lake Formation principal.
CfnDatabase.FederatedDatabaseProperty	Struct	A FederatedDatabase structure that references an entity outside the AWS Glue Data Catalog .
CfnDatabase.PrincipalPrivilegesProperty	Struct	the permissions granted to a principal.
CfnDatabaseProps	Struct	Properties for defining a CfnDatabase.
CfnDataCatalogEncryptionSettings.ConnectionPasswordEncryptionProperty	Struct	The data structure used by the Data Catalog to encrypt the password as part of CreateConnection or UpdateConnection and store it in the ENCRYPTED_PASSWORD field in the connection properties.
CfnDataCatalogEncryptionSettings.DataCatalogEncryptionSettingsProperty	Struct	Contains configuration information for maintaining Data Catalog security.
CfnDataCatalogEncryptionSettings.EncryptionAtRestProperty	Struct	Specifies the encryption-at-rest configuration for the Data Catalog.
CfnDataCatalogEncryptionSettingsProps	Struct	Properties for defining a CfnDataCatalogEncryptionSettings.
CfnDataQualityRuleset.DataQualityTargetTableProperty	Struct	An object representing an AWS Glue table.
CfnDataQualityRulesetProps	Struct	Properties for defining a CfnDataQualityRuleset.
CfnDevEndpointProps	Struct	Properties for defining a CfnDevEndpoint.
CfnIdentityCenterConfigurationProps	Struct	Properties for defining a CfnIdentityCenterConfiguration.
CfnIntegration.IntegrationConfigProperty	Struct	Properties associated with the integration.
CfnIntegrationProps	Struct	Properties for defining a CfnIntegration.
CfnIntegrationResourceProperty.SourceProcessingPropertiesProperty	Struct	The structure used to define the resource properties associated with the integration source.
CfnIntegrationResourceProperty.TargetProcessingPropertiesProperty	Struct	The structure used to define the resource properties associated with the integration target.
CfnIntegrationResourcePropertyProps	Struct	Properties for defining a CfnIntegrationResourceProperty.
CfnJob.ConnectionsListProperty	Struct	Specifies the connections used by a job.
CfnJob.ExecutionPropertyProperty	Struct	An execution property of a job.
CfnJob.JobCommandProperty	Struct	Specifies code executed when a job is run.
CfnJob.NotificationPropertyProperty	Struct	Specifies configuration properties of a notification.
CfnJobProps	Struct	Properties for defining a CfnJob.
CfnMLTransform.FindMatchesParametersProperty	Struct	The parameters to configure the find matches transform.
CfnMLTransform.GlueTablesProperty	Struct	The database and table in the AWS Glue Data Catalog that is used for input or output data.
CfnMLTransform.InputRecordTablesProperty	Struct	A list of AWS Glue table definitions used by the transform.
CfnMLTransform.MLUserDataEncryptionProperty	Struct	The encryption-at-rest settings of the transform that apply to accessing user data.
CfnMLTransform.TransformEncryptionProperty	Struct	The encryption-at-rest settings of the transform that apply to accessing user data.
CfnMLTransform.TransformParametersProperty	Struct	The algorithm-specific parameters that are associated with the machine learning transform.
CfnMLTransformProps	Struct	Properties for defining a CfnMLTransform.
CfnPartition.ColumnProperty	Struct	A column in a Table .
CfnPartition.OrderProperty	Struct	Specifies the sort order of a sorted column.
CfnPartition.PartitionInputProperty	Struct	The structure used to create and update a partition.
CfnPartition.SchemaIdProperty	Struct	A structure that contains schema identity fields.
CfnPartition.SchemaReferenceProperty	Struct	An object that references a schema stored in the AWS Glue Schema Registry.
CfnPartition.SerdeInfoProperty	Struct	Information about a serialization/deserialization program (SerDe) that serves as an extractor and loader.
CfnPartition.SkewedInfoProperty	Struct	Specifies skewed values in a table.
CfnPartition.StorageDescriptorProperty	Struct	Describes the physical storage of table data.
CfnPartitionProps	Struct	Properties for defining a CfnPartition.
CfnRegistryProps	Struct	Properties for defining a CfnRegistry.
CfnSchema.RegistryProperty	Struct	Specifies a registry in the AWS Glue Schema Registry.
CfnSchema.SchemaVersionProperty	Struct	Specifies the version of a schema.
CfnSchemaProps	Struct	Properties for defining a CfnSchema.
CfnSchemaVersion.SchemaProperty	Struct	A wrapper structure to contain schema identity fields.
CfnSchemaVersionMetadataProps	Struct	Properties for defining a CfnSchemaVersionMetadata.
CfnSchemaVersionProps	Struct	Properties for defining a CfnSchemaVersion.
CfnSecurityConfiguration.CloudWatchEncryptionProperty	Struct	Specifies how Amazon CloudWatch data should be encrypted.
CfnSecurityConfiguration.EncryptionConfigurationProperty	Struct	Specifies an encryption configuration.
CfnSecurityConfiguration.JobBookmarksEncryptionProperty	Struct	Specifies how job bookmark data should be encrypted.
CfnSecurityConfiguration.S3EncryptionProperty	Struct	Specifies how Amazon Simple Storage Service (Amazon S3) data should be encrypted.
CfnSecurityConfigurationProps	Struct	Properties for defining a CfnSecurityConfiguration.
CfnSession.ConnectionsListProperty	Struct	Specifies the connections used by the session.
CfnSession.SessionCommandProperty	Struct	The SessionCommand that runs the job.
CfnSessionProps	Struct	Properties for defining a CfnSession.
CfnTable.ColumnProperty	Struct	A column in a Table .
CfnTable.IcebergInputProperty	Struct	Specifies an input structure that defines an Apache Iceberg metadata table.
CfnTable.IcebergPartitionFieldProperty	Struct	A partition field in an Iceberg partition spec.
CfnTable.IcebergPartitionSpecProperty	Struct	The partition spec for an Iceberg table.
CfnTable.IcebergSchemaProperty	Struct	The schema for an Iceberg table.
CfnTable.IcebergSortFieldProperty	Struct	A sort field in an Iceberg sort order.
CfnTable.IcebergSortOrderProperty	Struct	The sort order for an Iceberg table.
CfnTable.IcebergStructFieldProperty	Struct	A field in an Iceberg schema.
CfnTable.IcebergTableInputProperty	Struct	Specifies the Iceberg table configuration.
CfnTable.OpenTableFormatInputProperty	Struct	Specifies an OpenTableFormatInput structure when creating an open format table.
CfnTable.OrderProperty	Struct	Specifies the sort order of a sorted column.
CfnTable.SchemaIdProperty	Struct	A structure that contains schema identity fields.
CfnTable.SchemaReferenceProperty	Struct	An object that references a schema stored in the AWS Glue Schema Registry.
CfnTable.SerdeInfoProperty	Struct	Information about a serialization/deserialization program (SerDe) that serves as an extractor and loader.
CfnTable.SkewedInfoProperty	Struct	Specifies skewed values in a table.
CfnTable.StorageDescriptorProperty	Struct	Describes the physical storage of table data.
CfnTable.TableIdentifierProperty	Struct	A structure that describes a target table for resource linking.
CfnTable.TableInputProperty	Struct	A structure used to define a table.
CfnTable.ViewDefinitionProperty	Struct	A structure that defines the view.
CfnTable.ViewRepresentationProperty	Struct	A structure that defines a view representation.
CfnTableOptimizer.CompactionConfigurationProperty	Struct
CfnTableOptimizer.IcebergCompactionConfigurationProperty	Struct
CfnTableOptimizer.IcebergConfigurationProperty	Struct	IcebergConfiguration is a property type within the AWS::Glue::TableOptimizer resource in AWS CloudFormation.
CfnTableOptimizer.IcebergRetentionConfigurationProperty	Struct
CfnTableOptimizer.OrphanFileDeletionConfigurationProperty	Struct	Configuration for removing files that are are not tracked by the Iceberg table metadata, and are older than your configured age limit.
CfnTableOptimizer.RetentionConfigurationProperty	Struct	The configuration for a snapshot retention optimizer for Apache Iceberg tables.
CfnTableOptimizer.TableOptimizerConfigurationProperty	Struct	Specifies configuration details of a table optimizer.
CfnTableOptimizer.VpcConfigurationProperty	Struct	An object that describes the VPC configuration for a table optimizer.
CfnTableOptimizerProps	Struct	Properties for defining a CfnTableOptimizer.
CfnTableProps	Struct	Properties for defining a CfnTable.
CfnTrigger.ActionProperty	Struct	Defines an action to be initiated by a trigger.
CfnTrigger.ConditionProperty	Struct	Defines a condition under which a trigger fires.
CfnTrigger.EventBatchingConditionProperty	Struct	Batch condition that must be met (specified number of events received or batch time window expired) before EventBridge event trigger fires.
CfnTrigger.NotificationPropertyProperty	Struct	Specifies configuration properties of a job run notification.
CfnTrigger.PredicateProperty	Struct	Defines the predicate of the trigger, which determines when it fires.
CfnTriggerProps	Struct	Properties for defining a CfnTrigger.
CfnUsageProfile.ConfigurationObjectProperty	Struct	Specifies the values that an admin sets for each job or session parameter configured in a AWS Glue usage profile.
CfnUsageProfile.ProfileConfigurationProperty	Struct	Specifies the job and session values that an admin configures in an AWS Glue usage profile.
CfnUsageProfileProps	Struct	Properties for defining a CfnUsageProfile.
CfnUserDefinedFunction.ResourceUriProperty	Struct	The URIs for function resources.
CfnUserDefinedFunctionProps	Struct	Properties for defining a CfnUserDefinedFunction.
CfnWorkflowProps	Struct	Properties for defining a CfnWorkflow.
ClassificationString	Class	Classification string given to tables with this data format.
CloudWatchEncryption	Class	CloudWatch Logs encryption configuration for a SecurityConfiguration.
Code	Class	Represents a Glue Job's Code assets (an asset can be a scripts, a jar, a python file or any other file).
CodeConfig	Struct	Result of binding Code into a Job.
Column	Struct	A column of a table.
ColumnCountMismatchHandlingAction	Enum	Identifies if the file contains less or more values for a row than the number of columns specified in the external table definition.
CompressionType	Enum	The compression type.
Condition	Class	A condition that determines when a conditional trigger fires.
ConditionalTriggerOptions	Struct	Properties for configuring a Condition (Predicate) based Glue Trigger.
ConditionLogicalOperator	Enum	Represents the logical operator for evaluating a single condition in the Glue Trigger API.
ConditionOptions	Struct	Options shared by all trigger conditions.
ConnectionNetwork	Class	VPC network placement for a Glue Connection.
ConnectionOptions	Struct	Base Connection Options.
ConnectionPasswordEncryption	Struct	Connection-password encryption configuration for a Glue Data Catalog.
ConnectionProps	Struct	Construction properties for Connection.
ConnectionType	Class	The type of the glue connection.
ContinuousLoggingProps	Struct	Properties for enabling Continuous Logging for Glue Jobs.
CrawlerActionOptions	Struct	Options for the execution of a crawler.
CrawlerState	Enum	Represents the state of a crawler for a condition in the Glue Trigger API.
DatabaseProps	Struct
DataCatalogEncryptionAtRest	Class	Encryption-at-rest configuration for a Glue Data Catalog.
DataFormat	Class	Defines the input/output formats and ser/de for a single DataFormat.
DataFormatProps	Struct	Properties of a DataFormat instance.
DataQualityRulesetProps	Struct	Construction properties for DataQualityRuleset.
DataQualityTargetTable	Class	The Glue table a DataQualityRuleset evaluates.
DateIntervalUnit	Enum	Date interval unit for partition projection.
DatePartitionProjectionConfigurationProps	Struct	Properties for DATE partition projection configuration.
DateProjectionStep	Struct	A required-together interval step for DATE partition projection.
Dqdl	Class	The Data Quality Definition Language (DQDL) document for a DataQualityRuleset.
EnumPartitionProjectionConfigurationProps	Struct	Properties for ENUM partition projection configuration.
EventBatchingCondition	Struct	Represents event trigger batch condition.
EventTriggerOptions	Struct	Properties for configuring an Event Bridge based Glue Trigger.
ExecutionClass	Enum	The ExecutionClass whether the job is run with a standard or flexible execution class.
ExternalTableProps	Struct
GlueVersion	Enum	AWS Glue version determines the versions of Apache Spark and Python that are available to the job.
ICatalog	Interface	A Glue Data Catalog, either the implicit account-wide catalog or one created as an AWS::Glue::Catalog resource.
IConnection	Interface	Interface representing a created or an imported Connection.
IDatabase	Interface
IDataQualityRuleset	Interface
IJob	Interface	Interface representing a new or an imported Glue Job.
InputFormat	Class	Absolute class name of the Hadoop InputFormat to use when reading table files.
IntegerPartitionProjectionConfigurationProps	Struct	Properties for INTEGER partition projection configuration.
InvalidCharHandlingAction	Enum	Specifies the action to perform when query results contain invalid UTF-8 character values.
ISecurityConfiguration	Interface	Interface representing a created or an imported SecurityConfiguration.
ITable	Interface
IWorkflow	Interface	The base interface for Glue Workflow.
JobActionOptions	Struct	Options for the execution of a job.
JobAttributes	Struct	A subset of Job attributes are required for importing an existing job into a CDK project.
JobBookmarksEncryption	Class	Job bookmarks encryption configuration for a SecurityConfiguration.
JobLanguage	Enum	Runtime language of the Glue job.
JobProps	Struct	JobProps will be used to create new Glue Jobs using this L2 Construct.
JobState	Enum	Job states emitted by Glue to CloudWatch Events.
JobType	Enum	The job type.
LibrarySet	Enum	The set of pre-installed Python libraries available to a Python shell job running Python 3.9.
MaxCapacity	Enum	The number of AWS Glue data processing units (DPUs) that can be allocated when this job runs.
MetricType	Enum	The Glue CloudWatch metric type.
NumericOverflowHandlingAction	Enum	Specifies the action to perform when ORC data contains an integer (for example, BIGINT or int64) that is larger than the column definition (for example, SMALLINT or int16).
OnDemandTriggerOptions	Struct	Properties for configuring an on-demand Glue Trigger.
OrcColumnMappingType	Enum	Specifies how to map columns when the table uses ORC data format.
OutputFormat	Class	Absolute class name of the Hadoop OutputFormat to use when writing table files.
PartitionIndex	Struct	Properties of a Partition Index.
PartitionProjectionConfiguration	Class	Factory class for creating partition projection configurations.
PartitionProjectionType	Enum	Partition projection type.
Predicate	Struct	Represents a trigger predicate.
PredicateLogical	Enum
PySparkEtlJobProps	Struct	Properties for creating a Python Spark ETL job.
PySparkFlexEtlJobProps	Struct	Properties for PySparkFlexEtlJob.
PySparkStreamingJobProps	Struct	Properties for creating a Python Spark ETL job.
PythonShellJobProps	Struct	Properties for creating a Python Shell job.
PythonVersion	Enum	Python version.
S3Code	Class	Glue job Code from an S3 bucket.
S3Encryption	Class	S3 encryption configuration for a SecurityConfiguration.
S3EncryptionMode	Enum	Encryption mode for S3.
S3TableEncryption	Class	Server-side encryption for the S3 bucket that a managed S3Table creates.
S3TableProps	Struct
S3TableStorage	Class	Where an S3Table stores its data.
ScalaSparkEtlJobProps	Struct	Properties for creating a Scala Spark ETL job.
ScalaSparkFlexEtlJobProps	Struct	Properties for a ScalaSparkFlexEtlJob.
ScalaSparkStreamingJobProps	Struct	Properties for creating a Scala Spark ETL job.
ScheduledTriggerOptions	Struct	Properties for configuring a scheduled Glue Trigger.
Schema	Class
SecurityConfigurationProps	Struct	Constructions properties of SecurityConfiguration.
SerializationLibrary	Class	Serialization library to use when serializing/deserializing (SerDe) table records.
SparkExtraCodeProps	Struct	Code props for different {@link Code} assets used by different types of Spark jobs.
SparkJobProps	Struct	Common properties for different types of Spark jobs.
SparkUILoggingLocation	Struct	The Spark UI logging location.
SparkUIProps	Struct	Properties for enabling Spark UI monitoring feature for Spark-based Glue jobs.
StartableTriggerOptions	Struct	Base options for triggers that can be started when they are created.
StorageParameter	Class	A storage parameter. The list of storage parameters available is not exhaustive and other keys may be used.
StorageParameters	Enum	The storage parameter keys that are currently known, this list is not exhaustive and other keys may be used.
SurplusBytesHandlingAction	Enum	Specifies how to handle data being loaded that exceeds the length of the data type defined for columns containing VARBYTE data.
SurplusCharHandlingAction	Enum	Specifies how to handle data being loaded that exceeds the length of the data type defined for columns containing VARCHAR, CHAR, or string data.
TableAttributes	Struct
TableBaseProps	Struct
TableClientSideEncryption	Class	Client-side encryption for an S3Table's data.
TriggerOptions	Struct	Properties for configuring a Glue Trigger.
TriggerSchedule	Class	Represents a trigger schedule.
Type	Class	The type of a column in a table schema.
WorkerConfiguration	Struct	The worker configuration for a Spark job.
WorkerType	Enum	The type of predefined worker that is allocated when a job runs.
WorkflowAttributes	Struct	Properties for importing a Workflow using its attributes.
WorkflowProps	Struct	Properties for defining a Workflow.
WriteParallel	Enum	Specifies how to handle data being loaded that exceeds the length of the data type defined for columns containing VARCHAR, CHAR, or string data.