AWS CDK Reference

Appearance

v2.274.0

WorkerConfiguration Struct ​

The worker configuration for a Spark job.

The worker type and the number of workers are set together: providing this configuration requires both values, so a Spark job can never be given one without the other.

Source
lib/jobs/spark-job.ts
Type last changed
v2.270.0-alpha.0 · 18 Sep 2026

Example ​

ts
import * as cdk from 'aws-cdk-lib';
import * as iam from 'aws-cdk-lib/aws-iam';
declare const stack: cdk.Stack;
declare const role: iam.IRole;
declare const script: glue.Code;
new glue.PySparkEtlJob(stack, 'PySparkETLJob', {
  jobName: 'PySparkETLJobCustomName',
  description: 'This is a description',
  role,
  script,
  glueVersion: glue.GlueVersion.V5_1,
  continuousLogging: { enabled: false },
  workerConfiguration: {
    workerType: glue.WorkerType.G_2X,
    numberOfWorkers: 2,
  },
  maxConcurrentRuns: 100,
  timeout: cdk.Duration.hours(2),
  connections: [glue.Connection.fromConnectionName(stack, 'Connection', 'connectionName')],
  securityConfiguration: glue.SecurityConfiguration.fromSecurityConfigurationName(stack, 'SecurityConfig', 'securityConfigName'),
  tags: {
    FirstTagName: 'FirstTagValue',
    SecondTagName: 'SecondTagValue',
    XTagName: 'XTagValue',
  },
  maxRetries: 2,
});

Properties ​

numberOfWorkersnumbernumber_of_workersint | floatnumberOfWorkersNumberNumberOfWorkersdoubleNumberOfWorkers*float64required#
The number of workers of the given `workerType` that are allocated when a job runs.
workerTypeWorkerTypeworker_typeWorkerTypeworkerTypeWorkerTypeWorkerTypeWorkerTypeWorkerTypeWorkerTyperequired#
The type of predefined worker that is allocated when a job runs.

Enum options: Standard, G_1X, G_2X, G_025X, G_4X, G_8X, Z_2X