Set environment variables in a queue environment

Many applications and frameworks use environment variables to control feature settings, logging levels, and display configuration. You can use Open Job Description (OpenJD) environments to set environment variables that every task command within their scope inherits.

Environment variable scope

AWS Deadline Cloud applies environment variables from queue environments that you attach to a queue. Within a job template, you can also define environment variables at the job and step levels using OpenJD environments. Variables defined at a narrower scope override variables with the same name from a broader scope.

Setting variables in a queue environment

You can set environment variables in a queue environment using a variables map for fixed values, or using a script with an onEnter action for dynamic values.

The following queue environment template uses a variables map to set the QT_QPA_PLATFORM variable to offscreen, which allows applications that use the Qt Framework to run on worker hosts without an interactive display.

specificationVersion: 'environment-2023-09'
environment:
  name: QtOffscreen
  variables:
    QT_QPA_PLATFORM: offscreen

For dynamic values, such as modifying PATH or activating virtual environments, use a script that prints lines in the format openjd_env: VAR=value to stdout. The openjd_env: prefix is required. Using echo, export, or other shell mechanisms without the prefix does not propagate variables to jobs and tasks.

The following queue environment template sets the QT_QPA_PLATFORM variable using a script.

specificationVersion: 'environment-2023-09'
environment:
  name: QtOffscreen
  script:
    actions:
      onEnter:
        command: bash
        args:
        - "{{Env.File.Enter}}"
    embeddedFiles:
    - name: Enter
      type: TEXT
      data: |
        #!/bin/env bash
        set -euo pipefail
        echo "openjd_env: QT_QPA_PLATFORM=offscreen"

To attach a queue environment to your queue, use the Deadline Cloud console or the AWS CLI. For more information, see Create a queue environment in the AWS Deadline Cloud User Guide. The following AWS CLI command creates a queue environment from a template file.

aws deadline create-queue-environment \
    --farm-id FARM_ID \
    --queue-id QUEUE_ID \
    --priority 1 \
    --template-type YAML \
    --template file://my-queue-env.yaml

For more complex examples, such as creating and activating conda virtual environments, see the Deadline Cloud queue environment samples on GitHub.

Setting variables in a job template

In a job template, add a variables map to a jobEnvironments or stepEnvironments entry. Each entry is a key-value pair where the key is the variable name and the value is the variable value.

The following job template sets the QT_QPA_PLATFORM environment variable to offscreen, which allows applications that use the Qt Framework to run on worker hosts without an interactive display.

specificationVersion: 'jobtemplate-2023-09'
name: MyJob
jobEnvironments:
- name: JobEnv
  variables:
    QT_QPA_PLATFORM: offscreen

You can set multiple variables in a single environment definition.

jobEnvironments:
- name: JobEnv
  variables:
    JOB_VERBOSITY: MEDIUM
    JOB_PROJECT_ID: my-project-id
    JOB_ENDPOINT_URL: https://my-host-name/my/path
    QT_QPA_PLATFORM: offscreen

You can reference job parameters in variable values by using the {{Param.ParameterName}} syntax.

jobEnvironments:
- name: JobEnv
  variables:
    JOB_EXAMPLE_PARAM: "{{Param.ExampleParam}}"

To override a job-level variable for a specific step, define a stepEnvironments entry with the same variable name. The following example defines JOB_PROJECT_ID at the job level with the value project-12, and then overrides the value at the step level with step-project-12. Tasks in the step use the step-level value.

specificationVersion: 'jobtemplate-2023-09'
name: MyJob
jobEnvironments:
- name: JobEnv
  variables:
    JOB_PROJECT_ID: project-12
steps:
- name: MyStep
  stepEnvironments:
  - name: StepEnv
    variables:
      JOB_PROJECT_ID: step-project-12

Try it: Running the environment variable sample

The Deadline Cloud samples repository includes a job bundle that demonstrates setting and viewing environment variables. The sample job template defines variables at both the job and step levels, then runs a task that prints the merged result. Use the following procedure to run the sample and inspect the results.

Prerequisites

Running the sample

Comparing session actions with their definitions

The log view shows three session actions. Open the file job_env_vars/template.yaml in a text editor to compare each action with its definition in the job template.