Package software.amazon.awscdk.services.glue.alpha

Class ScalaSparkFlexEtlJob.Builder

java.lang.Object
software.amazon.awscdk.services.glue.alpha.ScalaSparkFlexEtlJob.Builder
All Implemented Interfaces:
software.amazon.jsii.Builder<ScalaSparkFlexEtlJob>
Enclosing class:
ScalaSparkFlexEtlJob
@Stability(Experimental) public static final class ScalaSparkFlexEtlJob.Builder extends Object implements software.amazon.jsii.Builder<ScalaSparkFlexEtlJob>
(experimental) A fluent builder for ScalaSparkFlexEtlJob.

  • Method Details

    • create

      @Stability(Experimental) public static ScalaSparkFlexEtlJob.Builder create(software.constructs.Construct scope, String id)
      Parameters:
      scope - This parameter is required.
      id - This parameter is required.
      Returns:
      a new instance of ScalaSparkFlexEtlJob.Builder.

    • role

      @Stability(Experimental) public ScalaSparkFlexEtlJob.Builder role(IRole role)
      (experimental) IAM Role (required) IAM Role to use for Glue job execution Must be specified by the developer because the L2 doesn't have visibility into the actions the script(s) takes during the job execution The role must trust the Glue service principal (glue.amazonaws.com) and be granted sufficient permissions.

      Parameters:
      role - IAM Role (required) IAM Role to use for Glue job execution Must be specified by the developer because the L2 doesn't have visibility into the actions the script(s) takes during the job execution The role must trust the Glue service principal (glue.amazonaws.com) and be granted sufficient permissions. This parameter is required.
      Returns:
      this
      See Also:

    • script

      @Stability(Experimental) public ScalaSparkFlexEtlJob.Builder script(Code script)
      (experimental) Script Code Location (required) Script to run when the Glue job executes.

      Can be uploaded from the local directory structure using fromAsset or referenced via S3 location using fromBucket

      Parameters:
      script - Script Code Location (required) Script to run when the Glue job executes. This parameter is required.
      Returns:
      this

    • connections

      @Stability(Experimental) public ScalaSparkFlexEtlJob.Builder connections(List<? extends IConnection> connections)
      (experimental) Connections (optional) List of connections to use for this Glue job Connections are used to connect to other AWS Service or resources within a VPC.

      Default: [] - no connections are added to the job

      Parameters:
      connections - Connections (optional) List of connections to use for this Glue job Connections are used to connect to other AWS Service or resources within a VPC. This parameter is required.
      Returns:
      this

    • continuousLogging

      @Stability(Experimental) public ScalaSparkFlexEtlJob.Builder continuousLogging(ContinuousLoggingProps continuousLogging)
      (experimental) Enables continuous logging with the specified props.

      Default: - continuous logging is enabled.

      Parameters:
      continuousLogging - Enables continuous logging with the specified props. This parameter is required.
      Returns:
      this
      See Also:

    • defaultArguments

      @Stability(Experimental) public ScalaSparkFlexEtlJob.Builder defaultArguments(Map<String,String> defaultArguments)
      (experimental) Default Arguments (optional) The default arguments for every run of this Glue job, specified as name-value pairs.

      Default: - no arguments

      Parameters:
      defaultArguments - Default Arguments (optional) The default arguments for every run of this Glue job, specified as name-value pairs. This parameter is required.
      Returns:
      this
      See Also:

    • description

      @Stability(Experimental) public ScalaSparkFlexEtlJob.Builder description(String description)
      (experimental) Description (optional) Developer-specified description of the Glue job.

      Default: - no value

      Parameters:
      description - Description (optional) Developer-specified description of the Glue job. This parameter is required.
      Returns:
      this

    • enableProfilingMetrics

      @Stability(Experimental) public ScalaSparkFlexEtlJob.Builder enableProfilingMetrics(Boolean enableProfilingMetrics)
      (experimental) Enables the collection of metrics for job profiling.

      Default: - no profiling metrics emitted.

      Parameters:
      enableProfilingMetrics - Enables the collection of metrics for job profiling. This parameter is required.
      Returns:
      this
      See Also:

    • glueVersion

      @Stability(Experimental) public ScalaSparkFlexEtlJob.Builder glueVersion(GlueVersion glueVersion)
      (experimental) Glue Version The version of Glue to use to execute this job.

      Default: 3.0 for ETL

      Parameters:
      glueVersion - Glue Version The version of Glue to use to execute this job. This parameter is required.
      Returns:
      this

    • jobName

      @Stability(Experimental) public ScalaSparkFlexEtlJob.Builder jobName(String jobName)
      (experimental) Name of the Glue job (optional) Developer-specified name of the Glue job.

      Default: - a name is automatically generated

      Parameters:
      jobName - Name of the Glue job (optional) Developer-specified name of the Glue job. This parameter is required.
      Returns:
      this

    • maxConcurrentRuns

      @Stability(Experimental) public ScalaSparkFlexEtlJob.Builder maxConcurrentRuns(Number maxConcurrentRuns)
      (experimental) Max Concurrent Runs (optional) The maximum number of runs this Glue job can concurrently run.

      An error is returned when this threshold is reached. The maximum value you can specify is controlled by a service limit.

      Default: 1

      Parameters:
      maxConcurrentRuns - Max Concurrent Runs (optional) The maximum number of runs this Glue job can concurrently run. This parameter is required.
      Returns:
      this

    • maxRetries

      @Stability(Experimental) public ScalaSparkFlexEtlJob.Builder maxRetries(Number maxRetries)
      (experimental) Max Retries (optional) Maximum number of retry attempts Glue performs if the job fails.

      Default: 0

      Parameters:
      maxRetries - Max Retries (optional) Maximum number of retry attempts Glue performs if the job fails. This parameter is required.
      Returns:
      this

    • numberOfWorkers

      @Stability(Experimental) public ScalaSparkFlexEtlJob.Builder numberOfWorkers(Number numberOfWorkers)
      (experimental) Number of Workers (optional) Number of workers for Glue to use during job execution.

      Default: 10

      Parameters:
      numberOfWorkers - Number of Workers (optional) Number of workers for Glue to use during job execution. This parameter is required.
      Returns:
      this

    • securityConfiguration

      @Stability(Experimental) public ScalaSparkFlexEtlJob.Builder securityConfiguration(ISecurityConfiguration securityConfiguration)
      (experimental) Security Configuration (optional) Defines the encryption options for the Glue job.

      Default: - no security configuration.

      Parameters:
      securityConfiguration - Security Configuration (optional) Defines the encryption options for the Glue job. This parameter is required.
      Returns:
      this

    • tags

      @Stability(Experimental) public ScalaSparkFlexEtlJob.Builder tags(Map<String,String> tags)
      (experimental) Tags (optional) A list of key:value pairs of tags to apply to this Glue job resources.

      Default: {} - no tags

      Parameters:
      tags - Tags (optional) A list of key:value pairs of tags to apply to this Glue job resources. This parameter is required.
      Returns:
      this

    • timeout

      @Stability(Experimental) public ScalaSparkFlexEtlJob.Builder timeout(Duration timeout)
      (experimental) Timeout (optional) The maximum time that a job run can consume resources before it is terminated and enters TIMEOUT status.

      Specified in minutes.

      Default: 2880 (2 days for non-streaming)

      Parameters:
      timeout - Timeout (optional) The maximum time that a job run can consume resources before it is terminated and enters TIMEOUT status. This parameter is required.
      Returns:
      this

    • workerType

      @Stability(Experimental) public ScalaSparkFlexEtlJob.Builder workerType(WorkerType workerType)
      (experimental) Worker Type (optional) Type of Worker for Glue to use during job execution Enum options: Standard, G_1X, G_2X, G_025X.

      G_4X, G_8X, Z_2X

      Default: WorkerType.G_1X

      Parameters:
      workerType - Worker Type (optional) Type of Worker for Glue to use during job execution Enum options: Standard, G_1X, G_2X, G_025X. This parameter is required.
      Returns:
      this

    • sparkUi

      @Stability(Experimental) public ScalaSparkFlexEtlJob.Builder sparkUi(SparkUIProps sparkUi)
      (experimental) Enables the Spark UI debugging and monitoring with the specified props.

      Default: - Spark UI debugging and monitoring is disabled.

      Parameters:
      sparkUi - Enables the Spark UI debugging and monitoring with the specified props. This parameter is required.
      Returns:
      this
      See Also:

    • className

      @Stability(Experimental) public ScalaSparkFlexEtlJob.Builder className(String className)
      (experimental) The fully qualified Scala class name that serves as the entry point for the job.

      Parameters:
      className - The fully qualified Scala class name that serves as the entry point for the job. This parameter is required.
      Returns:
      this
      See Also:

    • extraFiles

      @Stability(Experimental) public ScalaSparkFlexEtlJob.Builder extraFiles(List<? extends Code> extraFiles)
      (experimental) Additional files, such as configuration files that AWS Glue copies to the working directory of your script before executing it.

      Default: - no extra files specified.

      Parameters:
      extraFiles - Additional files, such as configuration files that AWS Glue copies to the working directory of your script before executing it. This parameter is required.
      Returns:
      this
      See Also:

    • extraJars

      @Stability(Experimental) public ScalaSparkFlexEtlJob.Builder extraJars(List<? extends Code> extraJars)
      (experimental) Extra Jars S3 URL (optional) S3 URL where additional jar dependencies are located.

      Default: - no extra jar files

      Parameters:
      extraJars - Extra Jars S3 URL (optional) S3 URL where additional jar dependencies are located. This parameter is required.
      Returns:
      this

    • extraJarsFirst

      @Stability(Experimental) public ScalaSparkFlexEtlJob.Builder extraJarsFirst(Boolean extraJarsFirst)
      (experimental) Setting this value to true prioritizes the customer's extra JAR files in the classpath.

      Default: false - priority is not given to user-provided jars

      Parameters:
      extraJarsFirst - Setting this value to true prioritizes the customer's extra JAR files in the classpath. This parameter is required.
      Returns:
      this
      See Also:

    • notifyDelayAfter

      @Stability(Experimental) public ScalaSparkFlexEtlJob.Builder notifyDelayAfter(Duration notifyDelayAfter)
      (experimental) Specifies configuration properties of a notification (optional).

      After a job run starts, the number of minutes to wait before sending a job run delay notification.

      Default: - undefined

      Parameters:
      notifyDelayAfter - Specifies configuration properties of a notification (optional). This parameter is required.
      Returns:
      this

    • build

      @Stability(Experimental) public ScalaSparkFlexEtlJob build()
      Specified by:
      build in interface software.amazon.jsii.Builder<ScalaSparkFlexEtlJob>
      Returns:
      a newly built instance of ScalaSparkFlexEtlJob.