SageMaker HyperPod references

Find more information and references about using SageMaker HyperPod in the following topics.

SageMaker HyperPod pricing

The following topics provide information about SageMaker HyperPod pricing. To find more details on price per hour for using SageMaker HyperPod instances, see also Amazon SageMaker Pricing.

Capacity requests

You can allocate on-demand or reserved compute capacity with SageMaker AI for use on SageMaker HyperPod. On-demand cluster creation allocates available capacity from the SageMaker AI on-demand capacity pool. Alternatively, you can request reserved capacity to ensure access by submitting a ticket for a quota increase. Inbound capacity requests are prioritized by SageMaker AI and you receive an estimated time for capacity allocation.

Service billing

When you provision a compute capacity on SageMaker HyperPod, you are billed for the duration of the capacity allocation. SageMaker HyperPod billing appears in your anniversary bills with a line item for the type of capacity allocation (on-demand, reserved), the instance type, and the time spent on using the instance.

To submit a ticket for a quota increase, see SageMaker HyperPod quotas.

SageMaker HyperPod APIs

The following list is a full set of SageMaker HyperPod APIs for submitting action requests in JSON format to SageMaker AI through AWS CLI or AWS SDK for Python (Boto3).

SageMaker HyperPod Slurm configuration

HyperPod supports two approaches for configuring Slurm on your cluster. Choose the approach that best fits your needs.

API-driven Slurm configuration (Recommended)

With API-driven configuration, you define Slurm node types, partition assignments, and filesystem mounts directly in the CreateCluster and UpdateCluster API requests. This approach provides:

SlurmConfig (per instance group)

Add SlurmConfig to each instance group to define the Slurm node type and partition assignment.

"SlurmConfig": {
    "NodeType": "Controller | Login | Compute",
    "PartitionNames": ["string"]
}

Parameters:

Example:

{
    "InstanceGroupName": "gpu-compute",
    "InstanceType": "ml.p4d.24xlarge",
    "InstanceCount": 8,
    "SlurmConfig": {
        "NodeType": "Compute",
        "PartitionNames": ["gpu-training"]
    },
    "LifeCycleConfig": {
        "SourceS3Uri": "s3://sagemaker-bucket/lifecycle/src/",
        "OnCreate": "on_create.sh"
    },
    "ExecutionRole": "arn:aws:iam::111122223333:role/HyperPodRole"
}

Orchestrator.Slurm (cluster level)

Add Orchestrator.Slurm to the cluster configuration to specify how HyperPod manages the slurm.conf file.

"Orchestrator": {
    "Slurm": {
        "SlurmConfigStrategy": "Managed | Overwrite | Merge"
    }
}

Parameters:

Example:

{
    "ClusterName": "my-hyperpod-cluster",
    "InstanceGroups": [...],
    "Orchestrator": {
        "Slurm": {
            "SlurmConfigStrategy": "Managed"
        }
    }
}

SlurmConfigStrategy comparison

FSx configuration via InstanceStorageConfigs

With API-driven configuration, you can configure FSx filesystems per instance group using InstanceStorageConfigs. This allows different instance groups to mount different filesystems.

Prerequisites:

FsxLustreConfig

Configure FSx for Lustre filesystem mounting for an instance group.

"InstanceStorageConfigs": [
    {
        "FsxLustreConfig": {
            "DnsName": "string",
            "MountPath": "string",
            "MountName": "string"
        }
    }
]

Parameters:

FsxOpenZfsConfig

Configure FSx for OpenZFS filesystem mounting for an instance group.

"InstanceStorageConfigs": [
    {
        "FsxOpenZfsConfig": {
            "DnsName": "string",
            "MountPath": "string"
        }
    }
]

Parameters:

Example with multiple filesystems:

{
    "InstanceGroupName": "gpu-compute",
    "InstanceType": "ml.p4d.24xlarge",
    "InstanceCount": 4,
    "SlurmConfig": {
        "NodeType": "Compute",
        "PartitionNames": ["gpu-training"]
    },
    "InstanceStorageConfigs": [
        {
            "FsxLustreConfig": {
                "DnsName": "fs-0abc123def456789.fsx.us-west-2.amazonaws.com",
                "MountPath": "/fsx",
                "MountName": "abcdefgh"
            }
        },
        {
            "FsxOpenZfsConfig": {
                "DnsName": "fs-0xyz987654321.fsx.us-west-2.amazonaws.com",
                "MountPath": "/shared"
            }
        },
        {
            "EbsVolumeConfig": {
                "VolumeSizeInGB": 500
            }
        }
    ],
    "LifeCycleConfig": {
        "SourceS3Uri": "s3://sagemaker-bucket/lifecycle/src/",
        "OnCreate": "on_create.sh"
    },
    "ExecutionRole": "arn:aws:iam::111122223333:role/HyperPodRole"
}

Complete API-driven configuration example

The following example shows a complete CreateCluster request using API-driven Slurm configuration:

{
    "ClusterName": "ml-training-cluster",
    "InstanceGroups": [
        {
            "InstanceGroupName": "controller",
            "InstanceType": "ml.c5.xlarge",
            "InstanceCount": 1,
            "SlurmConfig": {
                "NodeType": "Controller"
            },
            "LifeCycleConfig": {
                "SourceS3Uri": "s3://sagemaker-us-west-2-111122223333/lifecycle/src/",
                "OnCreate": "on_create.sh"
            },
            "ExecutionRole": "arn:aws:iam::111122223333:role/HyperPodRole",
            "ThreadsPerCore": 2
        },
        {
            "InstanceGroupName": "login",
            "InstanceType": "ml.m5.xlarge",
            "InstanceCount": 1,
            "SlurmConfig": {
                "NodeType": "Login"
            },
            "LifeCycleConfig": {
                "SourceS3Uri": "s3://sagemaker-us-west-2-111122223333/lifecycle/src/",
                "OnCreate": "on_create.sh"
            },
            "ExecutionRole": "arn:aws:iam::111122223333:role/HyperPodRole",
            "ThreadsPerCore": 2
        },
        {
            "InstanceGroupName": "gpu-compute",
            "InstanceType": "ml.p4d.24xlarge",
            "InstanceCount": 8,
            "SlurmConfig": {
                "NodeType": "Compute",
                "PartitionNames": ["gpu-training"]
            },
            "InstanceStorageConfigs": [
                {
                    "FsxLustreConfig": {
                        "DnsName": "fs-0abc123def456789.fsx.us-west-2.amazonaws.com",
                        "MountPath": "/fsx",
                        "MountName": "abcdefgh"
                    }
                }
            ],
            "LifeCycleConfig": {
                "SourceS3Uri": "s3://sagemaker-us-west-2-111122223333/lifecycle/src/",
                "OnCreate": "on_create.sh"
            },
            "ExecutionRole": "arn:aws:iam::111122223333:role/HyperPodRole",
            "ThreadsPerCore": 2,
            "OnStartDeepHealthChecks": ["InstanceStress", "InstanceConnectivity"]
        },
        {
            "InstanceGroupName": "cpu-compute",
            "InstanceType": "ml.c5.18xlarge",
            "InstanceCount": 4,
            "SlurmConfig": {
                "NodeType": "Compute",
                "PartitionNames": ["cpu-preprocessing"]
            },
            "InstanceStorageConfigs": [
                {
                    "FsxLustreConfig": {
                        "DnsName": "fs-0abc123def456789.fsx.us-west-2.amazonaws.com",
                        "MountPath": "/fsx",
                        "MountName": "abcdefgh"
                    }
                }
            ],
            "LifeCycleConfig": {
                "SourceS3Uri": "s3://sagemaker-us-west-2-111122223333/lifecycle/src/",
                "OnCreate": "on_create.sh"
            },
            "ExecutionRole": "arn:aws:iam::111122223333:role/HyperPodRole",
            "ThreadsPerCore": 2
        }
    ],
    "Orchestrator": {
        "Slurm": {
            "SlurmConfigStrategy": "Managed"
        }
    },
    "VpcConfig": {
        "SecurityGroupIds": ["sg-0abc123def456789a"],
        "Subnets": ["subnet-0abc123def456789a", "subnet-0abc123def456789b"]
    },
    "Tags": [
        {
            "Key": "Project",
            "Value": "ML-Training"
        }
    ]
}

To learn more about using API-driven configuration, see Customizing SageMaker HyperPod clusters using lifecycle scripts.

Legacy configuration: provisioning_parameters.json

With the legacy approach, you create a Slurm configuration file named provisioning_parameters.json and upload it to Amazon S3 as part of your lifecycle scripts. HyperPod reads this file during cluster creation to configure Slurm nodes.

Configuration form for provisioning_parameters.json

The following code is the Slurm configuration form you should prepare to properly set up Slurm nodes on your HyperPod cluster. You should complete this form and upload it as part of a set of lifecycle scripts during cluster creation. To learn how this form should be prepared throughout HyperPod cluster creation processes, see Customizing SageMaker HyperPod clusters using lifecycle scripts.

// Save as provisioning_parameters.json.
{
    "version": "1.0.0",
    "workload_manager": "slurm",
    "controller_group": "string",
    "login_group": "string",
    "worker_groups": [
        {
            "instance_group_name": "string",
            "partition_name": "string"
        }
    ],
    "fsx_dns_name": "string",
    "fsx_mountname": "string"
}

Parameters:

Comparison: API-driven vs. legacy configuration

SageMaker HyperPod DLAMI

SageMaker HyperPod runs a DLAMI based on:

The SageMaker HyperPod DLAMI is bundled with additional packages to support open source tools such as Slurm, Kubernetes, dependencies, and SageMaker HyperPod cluster software packages to support resiliency features such as cluster health check and auto-resume. To follow up with HyperPod software updates that the HyperPod service team distributes through DLAMIs, see Amazon SageMaker HyperPod release notes.

SageMaker HyperPod API permissions reference

When you are setting up access control for allowing to run SageMaker HyperPod API operations and writing a permissions policy that you can attach to IAM users for cloud administrators, use the following table as a reference.

For a complete list of permissions and resource types for SageMaker APIs, see Actions, resources, and condition keys for Amazon SageMaker AI in the AWS Service Authorization Reference.

SageMaker HyperPod commands in AWS CLI

The following are the AWS CLI commands for SageMaker HyperPod to run the core HyperPod API operations.

SageMaker HyperPod Python modules in AWS SDK for Python (Boto3)

The following are the methods of the AWS SDK for Python (Boto3) client for SageMaker AI to run the core HyperPod API operations.