Changing a DAG's timezone on Amazon MWAA - Amazon Managed Workflows for Apache Airflow
VersionPrerequisitesPermissionsCreate a plugin to change the timezone in Airflow logsCreate a plugins.zipCode sampleWhat's next?

Changing a DAG's timezone on Amazon MWAA

Apache Airflow schedules your directed acyclic graph (DAG) in UTC+0 by default. The following steps present how you can change the timezone in which Amazon MWAA runs your DAGs with Pendulum. Optionally, this topic demonstrates how you can create a custom plugin to change the timezone for your environment's Apache Airflow logs.

Version

You can use the code example on this page with Apache Airflow v2 in Python 3.10 and Apache Airflow v3 in Python 3.11.

Prerequisites

To use the sample code on this page, you'll need the following:

Permissions

No additional permissions are required to use the code example on this page.

Create a plugin to change the timezone in Airflow logs

Apache Airflow run the Python files in the plugins directory at start-up. With the following plugin, you can override the executor's timezone, which modifies the timezone in which Apache Airflow writes logs.

  1. Create a directory named plugins for your custom plugin, and navigate to the directory. For example:

    $ mkdir plugins
$ cd plugins

  2. Copy the contents of the following code sample and save locally as dag-timezone-plugin.py in the plugins folder.

    import time
import os

os.environ['TZ'] = 'America/Los_Angeles'
time.tzset()

  3. In the plugins directory, create an empty Python file named __init__.py. Your plugins directory should be similar to the following:

    plugins/
 |-- __init__.py
 |-- dag-timezone-plugin.py

Create a plugins.zip

The following steps explain how to create plugins.zip. The content of this example can be combined with other plugins and binaries into a single plugins.zip file.

  1. In your command prompt, navigate to the plugins directory from the previous step. For example:

    cd plugins

  2. Zip the contents within your plugins directory.

    zip -r ../plugins.zip ./

  3. Upload plugins.zip to your S3 bucket

    aws s3 cp plugins.zip s3://your-mwaa-bucket/

Code sample

To change the default timezone (UTC+0) in which the DAG runs, we'll use a library called Pendulum, a Python library for working with timezone-aware datetime.

  1. In your command prompt, navigate to the directory where your DAGs are stored. For example:

    cd dags

  2. Copy the content of the following example and save as tz-aware-dag.py.

    from airflow import DAG
from airflow.operators.bash_operator import BashOperator
from datetime import datetime, timedelta
# Import the Pendulum library.
import pendulum

# Instantiate Pendulum and set your timezone.
local_tz = pendulum.timezone("America/Los_Angeles")

with DAG(
    dag_id = "tz_test",
    schedule_interval="0 12 * * *",
    catchup=False,
    start_date=datetime(2022, 1, 1, tzinfo=local_tz)
) as dag:
    bash_operator_task = BashOperator(
        task_id="tz_aware_task",
        dag=dag,
        bash_command="date"
    )

  3. Run the following AWS CLI command to copy the DAG to your environment's bucket, then trigger the DAG using the Apache Airflow UI. 

    aws s3 cp your-dag.py s3://your-environment-bucket/dags/

  4. If successful, you'll output similar to the following in the task logs for the tz_aware_task in the tz_test DAG:

    [2022-08-01, 12:00:00 PDT] {{subprocess.py:74}} INFO - Running command: ['bash', '-c', 'date']
[2022-08-01, 12:00:00 PDT] {{subprocess.py:85}} INFO - Output:
[2022-08-01, 12:00:00 PDT] {{subprocess.py:89}} INFO - Mon Aug  1 12:00:00 PDT 2022
[2022-08-01, 12:00:00 PDT] {{subprocess.py:93}} INFO - Command exited with return code 0
[2022-08-01, 12:00:00 PDT] {{taskinstance.py:1280}} INFO - Marking task as SUCCESS. dag_id=tz_test, task_id=tz_aware_task, execution_date=20220801T190033, start_date=20220801T190035, end_date=20220801T190035
[2022-08-01, 12:00:00 PDT] {{local_task_job.py:154}} INFO - Task exited with return code 0
[2022-08-01, 12:00:00 PDT] {{local_task_job.py:264}} INFO - 0 downstream tasks scheduled from follow-on schedule check

What's next?