# Using job templates A job template stores values that can be shared across `StartJobRun` API invocations when starting a job run. It supports two use cases: + To prevent repetitive recurring `StartJobRun` API request values. + To enforce a rule that certain values must be provided via `StartJobRun` API requests. Job templates enable you to define a reusable template for job runs to apply additional customization, for example: + Configuring executor and driver compute capacity + Setting security and governance properties such as IAM roles + Customizing a docker image to use across multiple applications and data pipelines The following topics provide detailed information on using templates, including how to use them to start a job run and how to change template parameters. **Topics** + [ # Create and using a job template to start a job run ](create-job-template.md) + [ # Defining job template parameters ](use-job-template-parameters.md) + [ # Controlling access to job templates ](iam-job-template.md) # Create and using a job template to start a job run This section describes creating a job template and using the template to start a job run with the AWS Command Line Interface (AWS CLI). **To create a job template** 1. Create a `create-job-template-request.json` file and specify the required parameters for your job template, as shown in the following example JSON file. For information about all available parameters, see the [CreateJobTemplate](https://docs.aws.amazon.com/emr-on-eks/latest/APIReference/Welcome.html) API. Most values that are required for the `StartJobRun` API are also required for `jobTemplateData`. If you want to use placeholders for any parameters and provide values when invoking StartJobRun using a job template, please see the next section on job template parameters. ``` { "name": "mytemplate", "jobTemplateData": { "executionRoleArn": "iam_role_arn_for_job_execution", "releaseLabel": "emr-6.7.0-latest", "jobDriver": { "sparkSubmitJobDriver": { "entryPoint": "entryPoint_location", "entryPointArguments": [ "argument1","argument2",...], "sparkSubmitParameters": "--class --conf spark.executor.instances=2 --conf spark.executor.memory=2G --conf spark.executor.cores=2 --conf spark.driver.cores=1" } }, "configurationOverrides": { "applicationConfiguration": [ { "classification": "spark-defaults", "properties": { "spark.driver.memory":"2G" } } ], "monitoringConfiguration": { "persistentAppUI": "ENABLED", "cloudWatchMonitoringConfiguration": { "logGroupName": "my_log_group", "logStreamNamePrefix": "log_stream_prefix" }, "s3MonitoringConfiguration": { "logUri": "s3://my_s3_log_location/" } } } } } ``` 1. Use the `create-job-template` command with a path to the `create-job-template-request.json` file stored locally. ``` aws emr-containers create-job-template \ --cli-input-json file://./create-job-template-request.json ``` **To start a job run using a job template** Supply the virtual cluster id, job template id, and job name in the `StartJobRun` command, as shown in the following example. ``` aws emr-containers start-job-run \ --virtual-cluster-id 123456 \ --name myjob \ --job-template-id 1234abcd ``` # Defining job template parameters Job template parameters allow you to specify variables in the job template. Values for these parameter variables will need to be specified when starting a job run using that job template. Job template parameters are specified in `${parameterName}` format. You can choose to specify any value in a `jobTemplateData` field as a job template parameter. For each of the job template parameter variables, specify its data type (`STRING` or `NUMBER`) and optionally a default value. The example below shows how you can specify job template parameters for entry point location, main class, and S3 log location values. **To specify entry point location, main class, and Amazon S3 log location as job template parameters** 1. Create a `create-job-template-request.json` file and specify the required parameters for your job template, as shown in the following example JSON file. For more information about the parameters, see the [CreateJobTemplate](https://docs.aws.amazon.com/emr-on-eks/latest/APIReference/Welcome.html) API. ``` { "name": "mytemplate", "jobTemplateData": { "executionRoleArn": "iam_role_arn_for_job_execution", "releaseLabel": "emr-6.7.0-latest", "jobDriver": { "sparkSubmitJobDriver": { "entryPoint": "${EntryPointLocation}", "entryPointArguments": [ "argument1","argument2",...], "sparkSubmitParameters": "--class ${MainClass} --conf spark.executor.instances=2 --conf spark.executor.memory=2G --conf spark.executor.cores=2 --conf spark.driver.cores=1" } }, "configurationOverrides": { "applicationConfiguration": [ { "classification": "spark-defaults", "properties": { "spark.driver.memory":"2G" } } ], "monitoringConfiguration": { "persistentAppUI": "ENABLED", "cloudWatchMonitoringConfiguration": { "logGroupName": "my_log_group", "logStreamNamePrefix": "log_stream_prefix" }, "s3MonitoringConfiguration": { "logUri": "${LogS3BucketUri}" } } }, "parameterConfiguration": { "EntryPointLocation": { "type": "STRING" }, "MainClass": { "type": "STRING", "defaultValue":"Main" }, "LogS3BucketUri": { "type": "STRING", "defaultValue":"s3://my_s3_log_location/" } } } } ``` 1. Use the `create-job-template` command with a path to the `create-job-template-request.json` file stored locally or in Amazon S3. ``` aws emr-containers create-job-template \ --cli-input-json file://./create-job-template-request.json ``` **To start a job run using job template with job template parameters** To start a job run with a job template containing job template parameters, specify the job template id as well as values for job template parameters in the `StartJobRun` API request as shown below. ``` aws emr-containers start-job-run \ --virtual-cluster-id 123456 \ --name myjob \ --job-template-id 1234abcd \ --job-template-parameters '{"EntryPointLocation": "entry_point_location","MainClass": "ExampleMainClass","LogS3BucketUri": "s3://example_s3_bucket/"}' ``` # Controlling access to job templates `StartJobRun` policy lets you enforce that a user or a role can only run jobs using job templates that you specify and cannot run `StartJobRun` operations without using the specified job templates. To achieve this, first ensure that you give the user or role a read permission to the specified job templates as shown below. ------ #### [ JSON ] **** ``` { "Version":"2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "emr-containers:DescribeJobRun" ], "Resource": [ "arn:aws:emr-containers:*:*:jobtemplate/job_template_1_id", "arn:aws:emr-containers:*:*:jobtemplate/job_template_2_id" ], "Sid": "AllowEMRCONTAINERSDescribejobtemplate" } ] } ``` ------ To enforce that a user or role is able to invoke `StartJobRun` operation only when using specified job templates, you can assign the following `StartJobRun` policy permission to a given user or role. ------ #### [ JSON ] **** ``` { "Version":"2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "emr-containers:StartJobRun" ], "Resource": [ "arn:aws:emr-containers:*:*:/virtualclusters/virtual_cluster_id" ], "Condition": { "ArnLike": { "emr-containers:JobTemplateArn": [ "arn:aws:emr-containers:*:*:jobtemplate/job_template_1_id", "arn:aws:emr-containers:*:*:jobtemplate/job_template_2_id" ] } }, "Sid": "AllowEMRCONTAINERSStartjobrun" } ] } ``` ------ If the job template specifies a job template parameter inside the execution role ARN field, then the user will be able to provide a value for this parameter and thus be able to invoke `StartJobRun` using an arbitrary execution role. To restrict the execution roles the user can provide, see **Controlling access to the execution role** in [Using job execution roles with Amazon EMR on EKS](iam-execution-role.md). If no condition is specified in the above `StartJobRun` action policy for a given user or a role, the user or the role will be allowed to invoke `StartJobRun` action on the specified virtual cluster using an arbitrary job template that they have read access to or using an arbitrary execution role.