Preparing and uploading lifecycle scripts - Amazon SageMaker AI
Prepare the lifecycle scriptsCreate configuration fileVerify files in Amazon S3 bucket

Preparing and uploading lifecycle scripts

After creating all the required resources, you'll need to set up lifecycle scripts for your SageMaker HyperPod cluster. These lifecycle scripts provide a base configuration you can use to create a basic HyperPod Slurm cluster.

Prepare the lifecycle scripts

Follow these steps to get the lifecycle scripts.

  1. Download the lifecycle scripts from the GitHub repository to your machine.

  2. Upload the lifecycle scripts to the Amazon S3 bucket you created in Provision basic resources, using the cp CLI command.

    aws s3 cp --recursive LifeCycleScripts/base-config s3://${ROOT_BUCKET_NAME}/LifeCycleScripts/base-config

Create configuration file

Follow these steps to create the configuration file and upload it to the same Amazon S3 bucket where you store the lifecycle scripts.

  1. Create a configuration file named provisioning_parameters.json with the following configuration. Note that slurm_sns_arn is optional. If not provided, HyperPod will not set up the Amazon SNS notifications.

    cat <<EOF > /tmp/provisioning_parameters.json
{
  "version": "1.0.0",
  "workload_manager": "slurm",
  "controller_group": "$CONTOLLER_IG_NAME",
  "login_group": "my-login-group",
  "worker_groups": [
    {
      "instance_group_name": "$COMPUTE_IG_NAME",
      "partition_name": "dev"
    }
  ],
  "fsx_dns_name": "$SLURM_FSX_DNS_NAME",
  "fsx_mountname": "$SLURM_FSX_MOUNT_NAME",
  "slurm_configurations": {
    "slurm_database_secret_arn": "$SLURM_DB_SECRET_ARN",
    "slurm_database_endpoint": "$SLURM_DB_ENDPOINT_ADDRESS",
    "slurm_shared_directory": "/fsx",
    "slurm_database_user": "$DB_USER_NAME",
    "slurm_sns_arn": "$SLURM_SNS_FAILOVER_TOPIC_ARN"
  }
}
EOF

  2. Upload the provisioning_parameters.json file to the same Amazon S3 bucket where you store the lifecycle scripts.

    aws s3 cp /tmp/provisioning_parameters.json s3://${ROOT_BUCKET_NAME}/LifeCycleScripts/base-config/provisioning_parameters.json
    Note

    If you are using API-driven configuration, the provisioning_parameters.json file is not required. With API-driven configuration, you define Slurm node types, partitions, and FSx mounting directly in the CreateCluster API payload. For details, see Getting started with SageMaker HyperPod using the AWS CLI.

Verify files in Amazon S3 bucket

After you upload all the lifecycle scripts and the provisioning_parameters.json file, your Amazon S3 bucket should look like the following.

Image showing all the lifecycle scripts uploaded to the Amazon S3 bucket in the Amazon Simple Storage Service console.

For more information, see Start with base lifecycle scripts provided by HyperPod.