Package software.amazon.awscdk.services.glue

Class CfnJob

java.lang.Object
software.amazon.jsii.JsiiObject
software.constructs.Construct
software.amazon.awscdk.core.Construct
software.amazon.awscdk.core.CfnElement
software.amazon.awscdk.core.CfnRefElement
software.amazon.awscdk.core.CfnResource
software.amazon.awscdk.services.glue.CfnJob
All Implemented Interfaces:
IConstruct, IDependable, IInspectable, software.amazon.jsii.JsiiSerializable, software.constructs.IConstruct
@Generated(value="jsii-pacmak/1.84.0 (build 5404dcf)", date="2023-06-19T16:30:42.398Z") @Stability(Stable) public class CfnJob extends CfnResource implements IInspectable
A CloudFormation AWS::Glue::Job.

The AWS::Glue::Job resource specifies an AWS Glue job in the data catalog. For more information, see Adding Jobs in AWS Glue and Job Structure in the AWS Glue Developer Guide.

Example: 

 // The code below shows an example of how to instantiate this type.
 // The values are placeholders you should change.
 import software.amazon.awscdk.services.glue.*;
 Object defaultArguments;
 Object nonOverridableArguments;
 Object tags;
 CfnJob cfnJob = CfnJob.Builder.create(this, "MyCfnJob")
         .command(JobCommandProperty.builder()
                 .name("name")
                 .pythonVersion("pythonVersion")
                 .runtime("runtime")
                 .scriptLocation("scriptLocation")
                 .build())
         .role("role")
         // the properties below are optional
         .allocatedCapacity(123)
         .connections(ConnectionsListProperty.builder()
                 .connections(List.of("connections"))
                 .build())
         .defaultArguments(defaultArguments)
         .description("description")
         .executionClass("executionClass")
         .executionProperty(ExecutionPropertyProperty.builder()
                 .maxConcurrentRuns(123)
                 .build())
         .glueVersion("glueVersion")
         .logUri("logUri")
         .maxCapacity(123)
         .maxRetries(123)
         .name("name")
         .nonOverridableArguments(nonOverridableArguments)
         .notificationProperty(NotificationPropertyProperty.builder()
                 .notifyDelayAfter(123)
                 .build())
         .numberOfWorkers(123)
         .securityConfiguration("securityConfiguration")
         .tags(tags)
         .timeout(123)
         .workerType("workerType")
         .build();

  • Field Details

    • CFN_RESOURCE_TYPE_NAME

      @Stability(Stable) public static final String CFN_RESOURCE_TYPE_NAME
      The CloudFormation resource type name for this resource class.

  • Constructor Details

    • CfnJob

      protected CfnJob(software.amazon.jsii.JsiiObjectRef objRef)

    • CfnJob

      protected CfnJob(software.amazon.jsii.JsiiObject.InitializationMode initializationMode)

    • CfnJob

      @Stability(Stable) public CfnJob(@NotNull Construct scope, @NotNull String id, @NotNull CfnJobProps props)
      Create a new AWS::Glue::Job.

      Parameters:
      scope -
      • scope in which this resource is defined.
      This parameter is required.
      id -
      • scoped id of the resource.
      This parameter is required.
      props -
      • resource properties.
      This parameter is required.

  • Method Details

    • inspect

      @Stability(Stable) public void inspect(@NotNull TreeInspector inspector)
      Examines the CloudFormation resource and discloses attributes.

      Specified by:
      inspect in interface IInspectable
      Parameters:
      inspector -
      • tree inspector to collect and process attributes.
      This parameter is required.

    • renderProperties

      @Stability(Stable) @NotNull protected Map<String,Object> renderProperties(@NotNull Map<String,Object> props)
      Overrides:
      renderProperties in class CfnResource
      Parameters:
      props - This parameter is required.

    • getCfnProperties

      @Stability(Stable) @NotNull protected Map<String,Object> getCfnProperties()
      Overrides:
      getCfnProperties in class CfnResource

    • getTags

      @Stability(Stable) @NotNull public TagManager getTags()
      The tags to use with this job.

    • getCommand

      @Stability(Stable) @NotNull public Object getCommand()
      The code that executes a job.

    • setCommand

      @Stability(Stable) public void setCommand(@NotNull IResolvable value)
      The code that executes a job.

    • setCommand

      @Stability(Stable) public void setCommand(@NotNull CfnJob.JobCommandProperty value)
      The code that executes a job.

    • getDefaultArguments

      @Stability(Stable) @NotNull public Object getDefaultArguments()
      The default arguments for this job, specified as name-value pairs.

      You can specify arguments here that your own job-execution script consumes, in addition to arguments that AWS Glue itself consumes.

      For information about how to specify and consume your own job arguments, see Calling AWS Glue APIs in Python in the AWS Glue Developer Guide .

      For information about the key-value pairs that AWS Glue consumes to set up your job, see Special Parameters Used by AWS Glue in the AWS Glue Developer Guide .

    • setDefaultArguments

      @Stability(Stable) public void setDefaultArguments(@NotNull Object value)
      The default arguments for this job, specified as name-value pairs.

      You can specify arguments here that your own job-execution script consumes, in addition to arguments that AWS Glue itself consumes.

      For information about how to specify and consume your own job arguments, see Calling AWS Glue APIs in Python in the AWS Glue Developer Guide .

      For information about the key-value pairs that AWS Glue consumes to set up your job, see Special Parameters Used by AWS Glue in the AWS Glue Developer Guide .

    • getNonOverridableArguments

      @Stability(Stable) @NotNull public Object getNonOverridableArguments()
      Non-overridable arguments for this job, specified as name-value pairs.

    • setNonOverridableArguments

      @Stability(Stable) public void setNonOverridableArguments(@NotNull Object value)
      Non-overridable arguments for this job, specified as name-value pairs.

    • getRole

      @Stability(Stable) @NotNull public String getRole()
      The name or Amazon Resource Name (ARN) of the IAM role associated with this job.

    • setRole

      @Stability(Stable) public void setRole(@NotNull String value)
      The name or Amazon Resource Name (ARN) of the IAM role associated with this job.

    • getAllocatedCapacity

      @Stability(Stable) @Nullable public Number getAllocatedCapacity()
      This parameter is no longer supported. Use MaxCapacity instead.

      The number of capacity units that are allocated to this job.

    • setAllocatedCapacity

      @Stability(Stable) public void setAllocatedCapacity(@Nullable Number value)
      This parameter is no longer supported. Use MaxCapacity instead.

      The number of capacity units that are allocated to this job.

    • getConnections

      @Stability(Stable) @Nullable public Object getConnections()
      The connections used for this job.

    • setConnections

      @Stability(Stable) public void setConnections(@Nullable IResolvable value)
      The connections used for this job.

    • setConnections

      @Stability(Stable) public void setConnections(@Nullable CfnJob.ConnectionsListProperty value)
      The connections used for this job.

    • getDescription

      @Stability(Stable) @Nullable public String getDescription()
      A description of the job.

    • setDescription

      @Stability(Stable) public void setDescription(@Nullable String value)
      A description of the job.

    • getExecutionClass

      @Stability(Stable) @Nullable public String getExecutionClass()
      Indicates whether the job is run with a standard or flexible execution class.

      The standard execution class is ideal for time-sensitive workloads that require fast job startup and dedicated resources.

      The flexible execution class is appropriate for time-insensitive jobs whose start and completion times may vary.

      Only jobs with AWS Glue version 3.0 and above and command type glueetl will be allowed to set ExecutionClass to FLEX . The flexible execution class is available for Spark jobs.

    • setExecutionClass

      @Stability(Stable) public void setExecutionClass(@Nullable String value)
      Indicates whether the job is run with a standard or flexible execution class.

      The standard execution class is ideal for time-sensitive workloads that require fast job startup and dedicated resources.

      The flexible execution class is appropriate for time-insensitive jobs whose start and completion times may vary.

      Only jobs with AWS Glue version 3.0 and above and command type glueetl will be allowed to set ExecutionClass to FLEX . The flexible execution class is available for Spark jobs.

    • getExecutionProperty

      @Stability(Stable) @Nullable public Object getExecutionProperty()
      The maximum number of concurrent runs that are allowed for this job.

    • setExecutionProperty

      @Stability(Stable) public void setExecutionProperty(@Nullable IResolvable value)
      The maximum number of concurrent runs that are allowed for this job.

    • setExecutionProperty

      @Stability(Stable) public void setExecutionProperty(@Nullable CfnJob.ExecutionPropertyProperty value)
      The maximum number of concurrent runs that are allowed for this job.

    • getGlueVersion

      @Stability(Stable) @Nullable public String getGlueVersion()
      Glue version determines the versions of Apache Spark and Python that AWS Glue supports.

      The Python version indicates the version supported for jobs of type Spark.

      For more information about the available AWS Glue versions and corresponding Spark and Python versions, see Glue version in the developer guide.

      Jobs that are created without specifying a Glue version default to Glue 0.9.

    • setGlueVersion

      @Stability(Stable) public void setGlueVersion(@Nullable String value)
      Glue version determines the versions of Apache Spark and Python that AWS Glue supports.

      The Python version indicates the version supported for jobs of type Spark.

      For more information about the available AWS Glue versions and corresponding Spark and Python versions, see Glue version in the developer guide.

      Jobs that are created without specifying a Glue version default to Glue 0.9.

    • getLogUri

      @Stability(Stable) @Nullable public String getLogUri()
      This field is reserved for future use.

    • setLogUri

      @Stability(Stable) public void setLogUri(@Nullable String value)
      This field is reserved for future use.

    • getMaxCapacity

      @Stability(Stable) @Nullable public Number getMaxCapacity()
      The number of AWS Glue data processing units (DPUs) that can be allocated when this job runs.

      A DPU is a relative measure of processing power that consists of 4 vCPUs of compute capacity and 16 GB of memory.

      Do not set Max Capacity if using WorkerType and NumberOfWorkers .

      The value that can be allocated for MaxCapacity depends on whether you are running a Python shell job or an Apache Spark ETL job:

      • When you specify a Python shell job ( JobCommand.Name ="pythonshell"), you can allocate either 0.0625 or 1 DPU. The default is 0.0625 DPU.
      • When you specify an Apache Spark ETL job ( JobCommand.Name ="glueetl"), you can allocate from 2 to 100 DPUs. The default is 10 DPUs. This job type cannot have a fractional DPU allocation.

    • setMaxCapacity

      @Stability(Stable) public void setMaxCapacity(@Nullable Number value)
      The number of AWS Glue data processing units (DPUs) that can be allocated when this job runs.

      A DPU is a relative measure of processing power that consists of 4 vCPUs of compute capacity and 16 GB of memory.

      Do not set Max Capacity if using WorkerType and NumberOfWorkers .

      The value that can be allocated for MaxCapacity depends on whether you are running a Python shell job or an Apache Spark ETL job:

      • When you specify a Python shell job ( JobCommand.Name ="pythonshell"), you can allocate either 0.0625 or 1 DPU. The default is 0.0625 DPU.
      • When you specify an Apache Spark ETL job ( JobCommand.Name ="glueetl"), you can allocate from 2 to 100 DPUs. The default is 10 DPUs. This job type cannot have a fractional DPU allocation.

    • getMaxRetries

      @Stability(Stable) @Nullable public Number getMaxRetries()
      The maximum number of times to retry this job after a JobRun fails.

    • setMaxRetries

      @Stability(Stable) public void setMaxRetries(@Nullable Number value)
      The maximum number of times to retry this job after a JobRun fails.

    • getName

      @Stability(Stable) @Nullable public String getName()
      The name you assign to this job definition.

    • setName

      @Stability(Stable) public void setName(@Nullable String value)
      The name you assign to this job definition.

    • getNotificationProperty

      @Stability(Stable) @Nullable public Object getNotificationProperty()
      Specifies configuration properties of a notification.

    • setNotificationProperty

      @Stability(Stable) public void setNotificationProperty(@Nullable IResolvable value)
      Specifies configuration properties of a notification.

    • setNotificationProperty

      @Stability(Stable) public void setNotificationProperty(@Nullable CfnJob.NotificationPropertyProperty value)
      Specifies configuration properties of a notification.

    • getNumberOfWorkers

      @Stability(Stable) @Nullable public Number getNumberOfWorkers()
      The number of workers of a defined workerType that are allocated when a job runs.

      The maximum number of workers you can define are 299 for G.1X , and 149 for G.2X .

    • setNumberOfWorkers

      @Stability(Stable) public void setNumberOfWorkers(@Nullable Number value)
      The number of workers of a defined workerType that are allocated when a job runs.

      The maximum number of workers you can define are 299 for G.1X , and 149 for G.2X .

    • getSecurityConfiguration

      @Stability(Stable) @Nullable public String getSecurityConfiguration()
      The name of the SecurityConfiguration structure to be used with this job.

    • setSecurityConfiguration

      @Stability(Stable) public void setSecurityConfiguration(@Nullable String value)
      The name of the SecurityConfiguration structure to be used with this job.

    • getTimeout

      @Stability(Stable) @Nullable public Number getTimeout()
      The job timeout in minutes.

      This is the maximum time that a job run can consume resources before it is terminated and enters TIMEOUT status. The default is 2,880 minutes (48 hours).

    • setTimeout

      @Stability(Stable) public void setTimeout(@Nullable Number value)
      The job timeout in minutes.

      This is the maximum time that a job run can consume resources before it is terminated and enters TIMEOUT status. The default is 2,880 minutes (48 hours).

    • getWorkerType

      @Stability(Stable) @Nullable public String getWorkerType()
      The type of predefined worker that is allocated when a job runs.

      Accepts a value of Standard, G.1X, or G.2X.

      • For the Standard worker type, each worker provides 4 vCPU, 16 GB of memory and a 50GB disk, and 2 executors per worker.
      • For the G.1X worker type, each worker maps to 1 DPU (4 vCPU, 16 GB of memory, 64 GB disk), and provides 1 executor per worker. We recommend this worker type for memory-intensive jobs.
      • For the G.2X worker type, each worker maps to 2 DPU (8 vCPU, 32 GB of memory, 128 GB disk), and provides 1 executor per worker. We recommend this worker type for memory-intensive jobs.

    • setWorkerType

      @Stability(Stable) public void setWorkerType(@Nullable String value)
      The type of predefined worker that is allocated when a job runs.

      Accepts a value of Standard, G.1X, or G.2X.

      • For the Standard worker type, each worker provides 4 vCPU, 16 GB of memory and a 50GB disk, and 2 executors per worker.
      • For the G.1X worker type, each worker maps to 1 DPU (4 vCPU, 16 GB of memory, 64 GB disk), and provides 1 executor per worker. We recommend this worker type for memory-intensive jobs.
      • For the G.2X worker type, each worker maps to 2 DPU (8 vCPU, 32 GB of memory, 128 GB disk), and provides 1 executor per worker. We recommend this worker type for memory-intensive jobs.