Submitting jobs remotely with the toolkit library

When the toolkit-for-ray-on-sagemaker-ai package is installed, Ray's standard Jobs CLI and Python SDK authenticate through the cluster's secured endpoint using the sagemaker_ray:// address scheme the package registers. You submit and track jobs from a laptop, a CI/CD pipeline, or any environment with AWS credentials, with no kubectl port-forward and no direct network path to the cluster. The package is preinstalled in SageMaker Distribution images.

Prerequisites

Address the cluster

The package adds a sagemaker_ray address scheme. You pass it to any ray job command with --address:

sagemaker_ray://my-cluster/my-namespace

Submit and list jobs from the CLI

List the jobs on a cluster, then submit a working directory with an entry script.

ray job list --address sagemaker_ray://my-cluster/my-namespace

ray job submit \
    --address sagemaker_ray://my-cluster/my-namespace \
    --working-dir ./src \
    -- python my-script.py

The endpoint authenticates the request against your identity, so you reach the cluster without a VPN or a forwarded port.

Submit from Python

To submit from Python, use Ray's standard JobSubmissionClient with the sagemaker_ray address. When the toolkit-for-ray-on-sagemaker-ai package is installed, it registers the address scheme, so no other change is needed.

from ray.job_submission import JobSubmissionClient

client = JobSubmissionClient("sagemaker_ray://my-cluster/my-namespace")
job_id = client.submit_job(
    entrypoint="python my-script.py",
    runtime_env={"working_dir": "./src"},
)
print(job_id)

Tracking a submitted job

The same address manages the job, so a job you submitted remotely is tracked remotely. Each command takes the job ID returned at submission.

ray job status my-job-id --address sagemaker_ray://my-cluster/my-namespace

ray job logs my-job-id --address sagemaker_ray://my-cluster/my-namespace --follow

ray job stop my-job-id --address sagemaker_ray://my-cluster/my-namespace

ray job logs --follow streams output until the job ends. ray job stop requests a graceful stop.

The Ray Dashboard Jobs view lists every job on the cluster with its status, start time, and logs. In Studio, the Tasks tab lists Ray workloads including submitted jobs. For more information, see Managing Ray workloads with Studio.

For the full set of Ray Jobs CLI commands and options, see Quickstart using the Ray Jobs CLI in the Ray documentation.