# CI/CD for Amazon SageMaker Unified Studio
<a name="cicd"></a>

The Amazon SageMaker Unified Studio CI/CD CLI (`aws-smus-cicd-cli`) is an open-source command line tool that deploys multi-service data and AI applications across development, test, and production environments from a single manifest file. The CLI is available at no additional cost — you pay only for the underlying AWS resources provisioned during deployment.

**GitHub repository:** [github.com/aws/CICD-for-SageMakerUnifiedStudio](https://github.com/aws/CICD-for-SageMakerUnifiedStudio) on the GitHub website

When you build data and AI applications in Amazon SageMaker Unified Studio, your applications often combine multiple AWS services into a single workflow. Promoting these applications from development to test and production requires substituting service-specific configurations for each stage and provisioning resources in the correct order. The CI/CD CLI automates this process.

The CLI uses a declarative YAML manifest (`manifest.yaml`) that defines:
+ **Application resources** — Glue ETL jobs, Athena queries, MWAA workflows, Bedrock agents, QuickSight dashboards, and catalog assets
+ **Pipeline stages** — Each stage maps to an independent Amazon SageMaker Unified Studio project, providing full isolation between environments
+ **Stage-specific configurations** — S3 bucket paths, IAM role ARNs, account IDs, and connection strings that are substituted automatically at deploy time

# Key concept
<a name="cicd-key-concepts"></a>

## Manifest
<a name="cicd-manifest"></a>

The manifest file (`manifest.yaml`) is the single source of truth for your application. It declares what resources your application uses, which Amazon SageMaker Unified Studio projects to deploy into, and how configurations differ across stages. Data teams own the manifest. DevOps teams do not need to modify it.

## Stages and targets
<a name="cicd-stages-and-targets"></a>

Each stage in the manifest (for example, `dev`, `test`, `prod`) maps to a separate Amazon SageMaker Unified Studio project and domain. Stages can span different AWS accounts and Regions. The CLI substitutes stage-specific configurations at deploy time, so the same application definition works across all environments.

## Bundles
<a name="cicd-bundles"></a>

A bundle is an immutable, versioned archive produced by reading from a **source target** (typically your development environment). The `bundle` command packages your application code, workflow definitions, and resolved configurations from the source into a self-contained artifact.

The `deploy` command then deploys the bundle contents to a **destination target** (for example, test or production). This separation means:
+ **Source → Bundle:** `aws-smus-cicd-cli bundle` reads from your source(dev) project and creates the artifact
+ **Bundle → Destination:** `aws-smus-cicd-cli deploy` deploys the artifact into the target project

The same bundle is promoted across stages without rebuilding, supporting immutable artifacts, audit trails, and controlled promotion through quality gates.

## Catalog support
<a name="cicd-catalog-support"></a>

The CLI supports catalog asset management as part of the deployment process. You can define business metadata, including glossaries, glossary terms, assets, and data products, in your manifest. During deployment, the CLI automatically searches for assets in the catalog, creates subscription requests for required access, and waits for subscription approval before proceeding.

# CLI commands
<a name="cicd-cli-commands"></a>

The CI/CD CLI provides four core commands that cover the deployment lifecycle:


| Command | Description | 
| --- | --- | 
| aws-smus-cicd-cli describe | Validates the manifest, checks that target projects exist, and confirms the execution role has required permissions. Use --connect to validate against live AWS environments. | 
| aws-smus-cicd-cli bundle | Reads from the source target and packages application code, workflow definitions, and configurations into an immutable, versioned archive. | 
| aws-smus-cicd-cli deploy | Deploys bundle contents to the destination target. Provisions resources in dependency order — for example, uploading scripts to S3 before creating Glue jobs that reference them. Use --dry-run to preview changes without deploying. | 
| aws-smus-cicd-cli test | Runs post-deployment validation to confirm that services are running and ready for workloads in the target environment. | 

Additional commands:


| Command | Description | 
| --- | --- | 
| aws-smus-cicd-cli create | Generates a starter manifest from an existing Amazon SageMaker Unified Studio project. | 
| aws-smus-cicd-cli run | Triggers Airflow workflow execution on MWAA or Airflow Serverless connections. | 
| aws-smus-cicd-cli monitor | Monitors workflow execution status in real time. | 
| aws-smus-cicd-cli logs | Fetches and streams workflow execution logs. | 
| aws-smus-cicd-cli destroy | Removes deployed resources and projects. Use for failure recovery or environment cleanup. | 

# Prerequisites
<a name="cicd-prerequisites"></a>

Before using the CI/CD CLI, you need:
+ **Python 3.8 or later**
+ **AWS credentials** configured with permissions to deploy resources to your target Amazon SageMaker Unified Studio projects. See [Sample IAM policy](cicd-sample-iam-policy.md) for required permissions.
+ **Existing Amazon SageMaker Unified Studio projects** at each target stage. The CLI deploys application workloads into existing projects — it does not create domains or projects. Use AWS CloudFormation, AWS CDK, or Terraform for infrastructure setup.

# Getting started
<a name="cicd-getting-started"></a>

The CLI source code, example manifests, and documentation are available in the [GitHub repository](https://github.com/aws/CICD-for-SageMakerUnifiedStudio) on the GitHub website.

## Install the CLI
<a name="cicd-install-cli"></a>

Install the CLI using pip:

```
pip install aws-smus-cicd-cli
```

Verify the installation:

```
aws-smus-cicd-cli --version
```

## Create a manifest
<a name="cicd-create-manifest"></a>

Clone the repository and start from an example manifest for your application pattern:

```
git clone https://github.com/aws/CICD-for-SageMakerUnifiedStudio.git
cp CICD-for-SageMakerUnifiedStudio/examples/data-notebooks/manifest.yaml manifest.yaml
```

Available example patterns:
+ **Analytics** — Glue ETL \$1 QuickSight dashboards
+ **Data notebooks** — Jupyter notebook workflows
+ **ML training** — SageMaker AI training jobs
+ **ML deployment** — Model endpoints and inference
+ **GenAI** — Bedrock agent applications

Edit the manifest to specify your resources, source locations, and stage-specific configurations.

Alternatively, generate a manifest from an existing project:

```
aws-smus-cicd-cli create --domain-id <domain-id> --dev-project-id <project-id>
```

## Validate your configuration
<a name="cicd-validate-configuration"></a>

Before deploying, you can preview what will happen without making changes:

```
# Validate permissions and connections
aws-smus-cicd-cli describe --manifest manifest.yaml --connect

# Preview deployment without applying changes
aws-smus-cicd-cli deploy --targets test --manifest manifest.yaml --dry-run
```

## Deploy
<a name="cicd-deploy"></a>

```
aws-smus-cicd-cli deploy --targets test --manifest manifest.yaml
```

## Run post-deployment validation
<a name="cicd-post-deployment-validation"></a>

```
aws-smus-cicd-cli test --manifest manifest.yaml --targets test
```

# Deployment modes
<a name="cicd-deployment-modes"></a>

## Direct deployment
<a name="cicd-direct-deployment"></a>

Deploy directly from a local source directory:

```
aws-smus-cicd-cli deploy --manifest manifest.yaml --targets prod
```

## Bundle-based deployment
<a name="cicd-bundle-based-deployment"></a>

Package your application from the source target into an immutable artifact, then promote it to the destination:

```
# Create the bundle (reads from source target)
aws-smus-cicd-cli bundle --manifest manifest.yaml

# Deploy the bundle to production (deploys to destination target)
aws-smus-cicd-cli deploy --manifest app.tar.gz --targets prod
```

# Cross-region and cross-account deployment
<a name="cicd-cross-region-cross-account"></a>

Each deployment target in the manifest maps to an independent Amazon SageMaker Unified Studio project and domain. Targets can be in different AWS Regions and different AWS accounts. Provide AWS credentials for each target through IAM roles, OIDC federation, or environment variables at deploy time.

# CI/CD integration
<a name="cicd-integration"></a>

The CLI runs inside your existing CI/CD systems. Reusable workflow templates are available for:
+ **GitHub Actions** — See [GitHub Actions integration guide](https://github.com/aws/CICD-for-SageMakerUnifiedStudio/blob/main/docs/github-actions-integration.md) on the GitHub website
+ **Jenkins** — Use the CLI commands in your Jenkinsfile pipeline stages
+ **GitLab CI** — Use the CLI commands in your `.gitlab-ci.yml` stages

# Sample IAM policy
<a name="cicd-sample-iam-policy"></a>

The following policy grants the permissions required to deploy applications that include Glue, Athena, QuickSight, and SageMaker AI resources. Scope permissions to specific resources and accounts for production deployments.

```
{
    "Version": "2012-10-17", 		 	 	 
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:PutObject",
                "s3:GetObject",
                "s3:ListBucket",
                "glue:*",
                "quicksight:*",
                "athena:*",
                "lakeformation:*",
                "logs:*",
                "iam:GetRole",
                "iam:PassRole",
                "sts:GetCallerIdentity",
                "datazone:GetProject",
                "datazone:GetDomain",
                "datazone:GetEnvironment",
                "datazone:ListConnections",
                "datazone:GetConnection",
                "datazone:SearchListings",
                "datazone:GetListing"
            ],
            "Resource": "*"
        }
    ]
}
```

# Failure recovery
<a name="cicd-failure-recovery"></a>

The CLI stops at the point of failure and reports the error with a detailed stack trace. Resources provisioned before the failure remain in place. To recover:

1. Fix the issue identified in the error output.

1. Run `aws-smus-cicd-cli describe --connect` to confirm which resources exist and which permissions are missing.

1. Rerun `aws-smus-cicd-cli deploy`, or redeploy a previous bundle for bundle-based deployments.

1. To clean up a failed deployment, use `aws-smus-cicd-cli destroy --targets <target>` to remove deployed resources.

For detailed rollback procedures, see the [Rollback Guide](https://github.com/aws/CICD-for-SageMakerUnifiedStudio/blob/main/docs/rollback-guide.md) on the GitHub website.

# Current limitations
<a name="cicd-current-limitations"></a>
+ Automatic rollback is not supported. Recover by redeploying a previous bundle, deploying a fix, or using the `destroy` command.
+ The CLI does not create Amazon SageMaker Unified Studio domains or projects. Use existing IaC tools for infrastructure setup.
+ Native notebook deployment is not yet supported.

# Additional resources
<a name="cicd-additional-resources"></a>
+ [GitHub repository](https://github.com/aws/CICD-for-SageMakerUnifiedStudio) on the GitHub website
+ [Quick Start Guide](https://github.com/aws/CICD-for-SageMakerUnifiedStudio/blob/main/docs/getting-started/quickstart.md) on the GitHub website
+ [Admin Guide](https://github.com/aws/CICD-for-SageMakerUnifiedStudio/blob/main/docs/getting-started/admin-quickstart.md) on the GitHub website
+ [Manifest Reference](https://github.com/aws/CICD-for-SageMakerUnifiedStudio/blob/main/docs/manifest.md) on the GitHub website
+ [CLI Command Reference](https://github.com/aws/CICD-for-SageMakerUnifiedStudio/blob/main/docs/cli-commands.md) on the GitHub website
+ [Example Applications](https://github.com/aws/CICD-for-SageMakerUnifiedStudio/blob/main/docs/examples-guide.md) on the GitHub website
+ [Rollback Guide](https://github.com/aws/CICD-for-SageMakerUnifiedStudio/blob/main/docs/rollback-guide.md) on the GitHub website