EmrContainersStartJobRunProps

class aws_cdk.aws_stepfunctions_tasks.EmrContainersStartJobRunProps(*, comment=None, query_language=None, state_name=None, credentials=None, heartbeat=None, heartbeat_timeout=None, integration_pattern=None, task_timeout=None, timeout=None, assign=None, input_path=None, output_path=None, outputs=None, result_path=None, result_selector=None, job_driver, release_label, virtual_cluster, application_config=None, execution_role=None, job_name=None, monitoring=None, tags=None)

Bases: TaskStateBaseProps

The props for a EMR Containers StartJobRun Task.

Parameters:

comment (Optional[str]) – A comment describing this state. Default: No comment
query_language (Optional[QueryLanguage]) – The name of the query language used by the state. If the state does not contain a queryLanguage field, then it will use the query language specified in the top-level queryLanguage field. Default: - JSONPath
state_name (Optional[str]) – Optional name for this state. Default: - The construct ID will be used as state name
credentials (Union[Credentials, Dict[str, Any], None]) – Credentials for an IAM Role that the State Machine assumes for executing the task. This enables cross-account resource invocations. Default: - None (Task is executed using the State Machine’s execution role)
heartbeat (Optional[Duration]) – (deprecated) Timeout for the heartbeat. Default: - None
heartbeat_timeout (Optional[Timeout]) – Timeout for the heartbeat. [disable-awslint:duration-prop-type] is needed because all props interface in aws-stepfunctions-tasks extend this interface Default: - None
integration_pattern (Optional[IntegrationPattern]) – AWS Step Functions integrates with services directly in the Amazon States Language. You can control these AWS services using service integration patterns. Depending on the AWS Service, the Service Integration Pattern availability will vary. Default: - IntegrationPattern.REQUEST_RESPONSE for most tasks. IntegrationPattern.RUN_JOB for the following exceptions: BatchSubmitJob, EmrAddStep, EmrCreateCluster, EmrTerminationCluster, and EmrContainersStartJobRun.
task_timeout (Optional[Timeout]) – Timeout for the task. [disable-awslint:duration-prop-type] is needed because all props interface in aws-stepfunctions-tasks extend this interface Default: - None
timeout (Optional[Duration]) – (deprecated) Timeout for the task. Default: - None
assign (Optional[Mapping[str, Any]]) – Workflow variables to store in this step. Using workflow variables, you can store data in a step and retrieve that data in future steps. Default: - Not assign variables
input_path (Optional[str]) – JSONPath expression to select part of the state to be the input to this state. May also be the special value JsonPath.DISCARD, which will cause the effective input to be the empty object {}. Default: $
output_path (Optional[str]) – JSONPath expression to select part of the state to be the output to this state. May also be the special value JsonPath.DISCARD, which will cause the effective output to be the empty object {}. Default: $
outputs (Any) – Used to specify and transform output from the state. When specified, the value overrides the state output default. The output field accepts any JSON value (object, array, string, number, boolean, null). Any string value, including those inside objects or arrays, will be evaluated as JSONata if surrounded by {% %} characters. Output also accepts a JSONata expression directly. Default: - $states.result or $states.errorOutput
result_path (Optional[str]) – JSONPath expression to indicate where to inject the state’s output. May also be the special value JsonPath.DISCARD, which will cause the state’s input to become its output. Default: $
result_selector (Optional[Mapping[str, Any]]) – The JSON that will replace the state’s raw result and become the effective result before ResultPath is applied. You can use ResultSelector to create a payload with values that are static or selected from the state’s raw result. Default: - None
job_driver (Union[JobDriver, Dict[str, Any]]) – The job driver for the job run.
release_label (ReleaseLabel) – The Amazon EMR release version to use for the job run.
virtual_cluster (VirtualClusterInput) – The ID of the virtual cluster where the job will be run.
application_config (Optional[Sequence[Union[ApplicationConfiguration, Dict[str, Any]]]]) – The configurations for the application running in the job run. Maximum of 100 items Default: - No application config
execution_role (Optional[IRole]) – The execution role for the job run. If virtualClusterId is from a JSON input path, an execution role must be provided. If an execution role is provided, follow the documentation to update the role trust policy. Default: - Automatically generated only when the provided virtualClusterId is not an encoded JSON path
job_name (Optional[str]) – The name of the job run. Default: - No job run name
monitoring (Union[Monitoring, Dict[str, Any], None]) – Configuration for monitoring the job run. Default: - logging enabled and resources automatically generated if monitoring.logging is set to true
tags (Optional[Mapping[str, str]]) – The tags assigned to job runs. Default: - None

ExampleMetadata:

infused

Example:

tasks.EmrContainersStartJobRun(self, "EMR Containers Start Job Run",
    virtual_cluster=tasks.VirtualClusterInput.from_virtual_cluster_id("de92jdei2910fwedz"),
    release_label=tasks.ReleaseLabel.EMR_6_2_0,
    job_name="EMR-Containers-Job",
    job_driver=tasks.JobDriver(
        spark_submit_job_driver=tasks.SparkSubmitJobDriver(
            entry_point=sfn.TaskInput.from_text("local:///usr/lib/spark/examples/src/main/python/pi.py")
        )
    ),
    application_config=[tasks.ApplicationConfiguration(
        classification=tasks.Classification.SPARK_DEFAULTS,
        properties={
            "spark.executor.instances": "1",
            "spark.executor.memory": "512M"
        }
    )]
)

Attributes

application_config

The configurations for the application running in the job run.

Maximum of 100 items

Default:

No application config

See:

https://docs.aws.amazon.com/emr-on-eks/latest/APIReference/API_Configuration.html

assign

Workflow variables to store in this step.

Using workflow variables, you can store data in a step and retrieve that data in future steps.

Default:

Not assign variables

See:

https://docs.aws.amazon.com/step-functions/latest/dg/workflow-variables.html

comment

A comment describing this state.

Default:: No comment

credentials

Credentials for an IAM Role that the State Machine assumes for executing the task.

This enables cross-account resource invocations.

Default:

None (Task is executed using the State Machine’s execution role)

See:

https://docs.aws.amazon.com/step-functions/latest/dg/concepts-access-cross-acct-resources.html

execution_role

The execution role for the job run.

If virtualClusterId is from a JSON input path, an execution role must be provided. If an execution role is provided, follow the documentation to update the role trust policy.

Default:

Automatically generated only when the provided virtualClusterId is not an encoded JSON path

See:

https://docs.aws.amazon.com/emr/latest/EMR-on-EKS-DevelopmentGuide/setting-up-trust-policy.html

heartbeat

(deprecated) Timeout for the heartbeat.

Default:

None

Deprecated:

use heartbeatTimeout

Stability:

deprecated

heartbeat_timeout

Timeout for the heartbeat.

[disable-awslint:duration-prop-type] is needed because all props interface in aws-stepfunctions-tasks extend this interface

Default:

None

input_path

JSONPath expression to select part of the state to be the input to this state.

May also be the special value JsonPath.DISCARD, which will cause the effective input to be the empty object {}.

Default:: $

integration_pattern

AWS Step Functions integrates with services directly in the Amazon States Language.

You can control these AWS services using service integration patterns.

Depending on the AWS Service, the Service Integration Pattern availability will vary.

Default:

IntegrationPattern.REQUEST_RESPONSE for most tasks.

IntegrationPattern.RUN_JOB for the following exceptions: BatchSubmitJob, EmrAddStep, EmrCreateCluster, EmrTerminationCluster, and EmrContainersStartJobRun.

See:: https://docs.aws.amazon.com/step-functions/latest/dg/connect-supported-services.html

job_driver

The job driver for the job run.

See:: https://docs.aws.amazon.com/emr-on-eks/latest/APIReference/API_JobDriver.html

job_name

The name of the job run.

Default:

No job run name

monitoring

Configuration for monitoring the job run.

Default:

logging enabled and resources automatically generated if monitoring.logging is set to true

See:

https://docs.aws.amazon.com/emr-on-eks/latest/APIReference/API_MonitoringConfiguration.html

output_path

JSONPath expression to select part of the state to be the output to this state.

May also be the special value JsonPath.DISCARD, which will cause the effective output to be the empty object {}.

Default:: $

outputs

Used to specify and transform output from the state.

When specified, the value overrides the state output default. The output field accepts any JSON value (object, array, string, number, boolean, null). Any string value, including those inside objects or arrays, will be evaluated as JSONata if surrounded by {% %} characters. Output also accepts a JSONata expression directly.

Default:

$states.result or $states.errorOutput

See:

https://docs.aws.amazon.com/step-functions/latest/dg/concepts-input-output-filtering.html

query_language

The name of the query language used by the state.

If the state does not contain a queryLanguage field, then it will use the query language specified in the top-level queryLanguage field.

Default:

JSONPath

release_label: The Amazon EMR release version to use for the job run.

result_path

JSONPath expression to indicate where to inject the state’s output.

May also be the special value JsonPath.DISCARD, which will cause the state’s input to become its output.

Default:: $

result_selector

The JSON that will replace the state’s raw result and become the effective result before ResultPath is applied.

You can use ResultSelector to create a payload with values that are static or selected from the state’s raw result.

Default:

None

See:

https://docs.aws.amazon.com/step-functions/latest/dg/input-output-inputpath-params.html#input-output-resultselector

state_name

Optional name for this state.

Default:

The construct ID will be used as state name

tags

The tags assigned to job runs.

Default:

None

task_timeout

Timeout for the task.

[disable-awslint:duration-prop-type] is needed because all props interface in aws-stepfunctions-tasks extend this interface

Default:

None

timeout

(deprecated) Timeout for the task.

Default:

None

Deprecated:

use taskTimeout

Stability:

deprecated

virtual_cluster: The ID of the virtual cluster where the job will be run.