# Model governance to manage permissions and track model performance
Model governance is a framework that gives systematic visibility into machine learning (ML) model development, validation, and usage. Amazon SageMaker AI provides purpose-built ML governance tools for managing control access, activity tracking, and reporting across the ML lifecycle.
Manage least-privilege permissions for ML practitioners using Amazon SageMaker Role Manager, create detailed model documentation using Amazon SageMaker Model Cards, and gain visibility into your models with centralized dashboards using Amazon SageMaker Model Dashboard.
## Amazon SageMaker Role Manager
With Amazon SageMaker Role Manager, administrators can define user permissions with least-privilege permissions for common machine learning activities. Use Amazon SageMaker Role Manager to build and manage persona-based IAM roles specific to your business needs.
For more information, see [Amazon SageMaker Role Manager](role-manager.md).
## Amazon SageMaker Model Cards
Use Amazon SageMaker Model Cards to document, retrieve, and share essential model information from conception to deployment. With model cards, model risk managers, data scientists, and ML engineers can create an immutable record of intended model uses, risk ratings, training details, evaluation results, and more.
For more information, see [Amazon SageMaker Model Cards](model-cards.md).
## Amazon SageMaker Model Dashboard
Amazon SageMaker Model Dashboard is a pre-built, visual overview of all the models in your account. SageMaker Model Dashboard integrates valuable information from Amazon SageMaker Model Monitor, Transform Jobs, Endpoints, ML Lineage Tracking and Amazon CloudWatch so you can access high-level model information and track model performance in one unified view.
For more information, see [Amazon SageMaker Model Dashboard](model-dashboard.md).
## Amazon SageMaker Assets
Amazon SageMaker Assets is a new workflow that streamlines ML governance. It allows users to easily publish, share, and subscribe to ML assets and data assets, such as feature groups and Amazon Redshift tables.
Administrators use Amazon DataZone to set up the databases and the ML infrastructure for users to share assets within Amazon SageMaker Studio. After set up, users can seamlessly share assets with each other without additional administrator overhead. For more information about Amazon SageMaker Assets, see [Controlled access to assets with Amazon SageMaker Assets](sm-assets.md).
# Amazon SageMaker Model Cards
**Important**
Amazon SageMaker Model Card is integrated with SageMaker Model Registry. If you're registering a model within Model Registry, you can use the integration to add auditing information. For more information, see [Update the Details of a Model Version](model-registry-details.md).
Use Amazon SageMaker Model Cards to document critical details about your machine learning (ML) models in a single place for streamlined governance and reporting. Model cards can help you to capture key information about your models throughout their lifecycle and implement responsible AI practices.
Catalog details such as the intended use and risk rating of a model, training details and metrics, evaluation results and observations, and additional call-outs such as considerations, recommendations, and custom information. By creating model cards, you can do the following:
+ Provide guidance on how a model should be used.
+ Support audit activities with detailed descriptions of model training and performance.
+ Communicate how a model is intended to support business goals.
Model cards provide prescriptive guidance on what information to document and include fields for custom information. After creating a model card, you can export it to a PDF or download it to share with relevant stakeholders. Any edits other than an approval status update made to a model card result in additional model card versions in order to have an immutable record of model changes.
**Topics**
+ [Prerequisites](#model-cards-prerequisites)
+ [Intended uses of a model](#model-cards-intended-uses)
+ [Risk ratings](#model-cards-risk-rating)
+ [Model card JSON schema](#model-cards-json-schema)
+ [Create a model card](model-cards-create.md)
+ [Model cards actions](model-cards-manage.md)
+ [Set up cross-account support for Amazon SageMaker Model Cards](model-cards-xaccount.md)
+ [Low-level SageMaker APIs for model cards](model-cards-apis.md)
+ [Model card FAQs](model-cards-faqs.md)
## Prerequisites
To get started with Amazon SageMaker Model Cards, you must have permission to create, edit, view, and export model cards.
## Intended uses of a model
Specifying the intended uses of a model helps ensure that model developers and users have the information they need to train or deploy the model responsibly. The intended uses of a model should describe the scenarios in which the model is appropriate to use as well as the scenarios in which the model is not recommended to use.
We recommend including:
+ The general purpose of the model
+ Use cases for which the model was intended
+ Use cases for which the model was not intended
+ Assumptions made when developing the model
The intended uses of a model go beyond technical details and describe how a model should be used in production, the scenarios in which is appropriate to use a model, and additional considerations such as the type of data to use with the model or any assumptions made during development.
## Risk ratings
Developers create ML models for use cases with varying levels of risk. For example, a model that approves loan applications might be a higher risk model than one that detects the category of an email. Given the varied risk profiles of a model, model cards provide a field for you to categorize a model’s risk rating.
This risk rating can be `unknown`, `low`, `medium`, or `high`. Use these risk rating fields to label unknown, low, medium, or high-risk models and help your organization comply with any existing rules about putting certain models into production.
## Model card JSON schema
Evaluation details for a model card must be provided in JSON format. If you have existing JSON format evaluation reports generated by [SageMaker Clarify](https://docs.aws.amazon.com/sagemaker/latest/dg/clarify-processing-job-run.html) or [SageMaker AI Model Monitor](https://docs.aws.amazon.com/sagemaker/latest/dg/model-monitor-model-quality.html), upload them to Amazon S3 and provide an S3 URI to automatically parse evaluation metrics. For more information and sample reports, see the [example metrics](https://github.com/aws/amazon-sagemaker-examples/tree/main/sagemaker_model_governance/example_metrics) folder in the *Amazon SageMaker Model Governance - Model Cards* example notebook.
When creating a model card using the SageMaker Python SDK, model content must be in the model card JSON schema and provided as a string. Provide model content similar to the following example.
### Model card JSON schema sample file
```
{
"$schema": "http://json-schema.org/draft-07/schema#",
"$id": "http://json-schema.org/draft-07/schema#",
"title": "SageMakerModelCardSchema",
"description": "Internal model card schema for SageMakerRepositoryService without model_package_details",
"version": "0.1.0",
"type": "object",
"additionalProperties": false,
"properties": {
"model_overview": {
"description": "Overview about the model",
"type": "object",
"additionalProperties": false,
"properties": {
"model_description": {
"description": "description of model",
"type": "string",
"maxLength": 1024
},
"model_creator": {
"description": "Creator of model",
"type": "string",
"maxLength": 1024
},
"model_artifact": {
"description": "Location of the model artifact",
"type": "array",
"maxContains": 15,
"items": {
"type": "string",
"maxLength": 1024
}
},
"algorithm_type": {
"description": "Algorithm used to solve the problem",
"type": "string",
"maxLength": 1024
},
"problem_type": {
"description": "Problem being solved with the model",
"type": "string"
},
"model_owner": {
"description": "Owner of model",
"type": "string",
"maxLength": 1024
}
}
},
"intended_uses": {
"description": "Intended usage of model",
"type": "object",
"additionalProperties": false,
"properties": {
"purpose_of_model": {
"description": "Why the model was developed?",
"type": "string",
"maxLength": 2048
},
"intended_uses": {
"description": "intended use cases",
"type": "string",
"maxLength": 2048
},
"factors_affecting_model_efficiency": {
"type": "string",
"maxLength": 2048
},
"risk_rating": {
"description": "Risk rating for model card",
"$ref": "#/definitions/risk_rating"
},
"explanations_for_risk_rating": {
"type": "string",
"maxLength": 2048
}
}
},
"business_details": {
"description": "Business details of model",
"type": "object",
"additionalProperties": false,
"properties": {
"business_problem": {
"description": "What business problem does the model solve?",
"type": "string",
"maxLength": 2048
},
"business_stakeholders": {
"description": "Business stakeholders",
"type": "string",
"maxLength": 2048
},
"line_of_business": {
"type": "string",
"maxLength": 2048
}
}
},
"training_details": {
"description": "Overview about the training",
"type": "object",
"additionalProperties": false,
"properties": {
"objective_function": {
"description": "the objective function the model will optimize for",
"function": {
"$ref": "#/definitions/objective_function"
},
"notes": {
"type": "string",
"maxLength": 1024
}
},
"training_observations": {
"type": "string",
"maxLength": 1024
},
"training_job_details": {
"type": "object",
"additionalProperties": false,
"properties": {
"training_arn": {
"description": "SageMaker Training job arn",
"type": "string",
"maxLength": 1024
},
"training_datasets": {
"description": "Location of the model datasets",
"type": "array",
"maxContains": 15,
"items": {
"type": "string",
"maxLength": 1024
}
},
"training_environment": {
"type": "object",
"additionalProperties": false,
"properties": {
"container_image": {
"description": "SageMaker training image uri",
"type": "array",
"maxContains": 15,
"items": {
"type": "string",
"maxLength": 1024
}
}
}
},
"training_metrics": {
"type": "array",
"items": {
"maxItems": 50,
"$ref": "#/definitions/training_metric"
}
},
"user_provided_training_metrics": {
"type": "array",
"items": {
"maxItems": 50,
"$ref": "#/definitions/training_metric"
}
},
"hyper_parameters": {
"type": "array",
"items": {
"maxItems": 100,
"$ref": "#/definitions/training_hyper_parameter"
}
},
"user_provided_hyper_parameters": {
"type": "array",
"items": {
"maxItems": 100,
"$ref": "#/definitions/training_hyper_parameter"
}
}
}
}
}
},
"evaluation_details": {
"type": "array",
"default": [],
"items": {
"type": "object",
"required": [
"name"
],
"additionalProperties": false,
"properties": {
"name": {
"type": "string",
"pattern": ".{1,63}"
},
"evaluation_observation": {
"type": "string",
"maxLength": 2096
},
"evaluation_job_arn": {
"type": "string",
"maxLength": 256
},
"datasets": {
"type": "array",
"items": {
"type": "string",
"maxLength": 1024
},
"maxItems": 10
},
"metadata": {
"description": "additional attributes associated with the evaluation results",
"type": "object",
"additionalProperties": {
"type": "string",
"maxLength": 1024
}
},
"metric_groups": {
"type": "array",
"default": [],
"items": {
"type": "object",
"required": [
"name",
"metric_data"
],
"properties": {
"name": {
"type": "string",
"pattern": ".{1,63}"
},
"metric_data": {
"type": "array",
"items": {
"anyOf": [
{
"$ref": "#/definitions/simple_metric"
},
{
"$ref": "#/definitions/linear_graph_metric"
},
{
"$ref": "#/definitions/bar_chart_metric"
},
{
"$ref": "#/definitions/matrix_metric"
}
]
}
}
}
}
}
}
}
},
"additional_information": {
"additionalProperties": false,
"type": "object",
"properties": {
"ethical_considerations": {
"description": "Any ethical considerations that the author wants to provide",
"type": "string",
"maxLength": 2048
},
"caveats_and_recommendations": {
"description": "Caveats and recommendations for people who might use this model in their applications.",
"type": "string",
"maxLength": 2048
},
"custom_details": {
"type": "object",
"additionalProperties": {
"$ref": "#/definitions/custom_property"
}
}
}
}
},
"definitions": {
"source_algorithms": {
"type": "array",
"minContains": 1,
"maxContains": 1,
"items": {
"type": "object",
"additionalProperties": false,
"required": [
"algorithm_name"
],
"properties": {
"algorithm_name": {
"description": "The name of an algorithm that was used to create the model package. The algorithm must be either an algorithm resource in your SageMaker account or an algorithm in AWS Marketplace that you are subscribed to.",
"type": "string",
"maxLength": 170
},
"model_data_url": {
"description": "The Amazon S3 path where the model artifacts, which result from model training, are stored.",
"type": "string",
"maxLength": 1024
}
}
}
},
"inference_specification": {
"type": "object",
"additionalProperties": false,
"required": [
"containers"
],
"properties": {
"containers": {
"description": "Contains inference related information which were used to create model package.",
"type": "array",
"minContains": 1,
"maxContains": 15,
"items": {
"type": "object",
"additionalProperties": false,
"required": [
"image"
],
"properties": {
"model_data_url": {
"description": "The Amazon S3 path where the model artifacts, which result from model training, are stored.",
"type": "string",
"maxLength": 1024
},
"image": {
"description": "Inference environment path. The Amazon EC2 Container Registry (Amazon ECR) path where inference code is stored.",
"type": "string",
"maxLength": 255
},
"nearest_model_name": {
"description": "The name of a pre-trained machine learning benchmarked by Amazon SageMaker Inference Recommender model that matches your model.",
"type": "string"
}
}
}
}
}
},
"risk_rating": {
"description": "Risk rating of model",
"type": "string",
"enum": [
"High",
"Medium",
"Low",
"Unknown"
]
},
"custom_property": {
"description": "Additional property in section",
"type": "string",
"maxLength": 1024
},
"objective_function": {
"description": "objective function that training job is optimized for",
"additionalProperties": false,
"properties": {
"function": {
"type": "string",
"enum": [
"Maximize",
"Minimize"
]
},
"facet": {
"type": "string",
"maxLength": 63
},
"condition": {
"type": "string",
"maxLength": 63
}
}
},
"training_metric": {
"description": "training metric data",
"type": "object",
"required": [
"name",
"value"
],
"additionalProperties": false,
"properties": {
"name": {
"type": "string",
"pattern": ".{1,255}"
},
"notes": {
"type": "string",
"maxLength": 1024
},
"value": {
"type": "number"
}
}
},
"training_hyper_parameter": {
"description": "training hyper parameter",
"type": "object",
"required": [
"name"
],
"additionalProperties": false,
"properties": {
"name": {
"type": "string",
"pattern": ".{1,255}"
},
"value": {
"type": "string",
"pattern": ".{0,255}"
}
}
},
"linear_graph_metric": {
"type": "object",
"required": [
"name",
"type",
"value"
],
"additionalProperties": false,
"properties": {
"name": {
"type": "string",
"pattern": ".{1,255}"
},
"notes": {
"type": "string",
"maxLength": 1024
},
"type": {
"type": "string",
"enum": [
"linear_graph"
]
},
"value": {
"anyOf": [
{
"type": "array",
"items": {
"type": "array",
"items": {
"type": "number"
},
"minItems": 2,
"maxItems": 2
},
"minItems": 1
}
]
},
"x_axis_name": {
"$ref": "#/definitions/axis_name_string"
},
"y_axis_name": {
"$ref": "#/definitions/axis_name_string"
}
}
},
"bar_chart_metric": {
"type": "object",
"required": [
"name",
"type",
"value"
],
"additionalProperties": false,
"properties": {
"name": {
"type": "string",
"pattern": ".{1,255}"
},
"notes": {
"type": "string",
"maxLength": 1024
},
"type": {
"type": "string",
"enum": [
"bar_chart"
]
},
"value": {
"anyOf": [
{
"type": "array",
"items": {
"type": "number"
},
"minItems": 1
}
]
},
"x_axis_name": {
"$ref": "#/definitions/axis_name_array"
},
"y_axis_name": {
"$ref": "#/definitions/axis_name_string"
}
}
},
"matrix_metric": {
"type": "object",
"required": [
"name",
"type",
"value"
],
"additionalProperties": false,
"properties": {
"name": {
"type": "string",
"pattern": ".{1,255}"
},
"notes": {
"type": "string",
"maxLength": 1024
},
"type": {
"type": "string",
"enum": [
"matrix"
]
},
"value": {
"anyOf": [
{
"type": "array",
"items": {
"type": "array",
"items": {
"type": "number"
},
"minItems": 1,
"maxItems": 20
},
"minItems": 1,
"maxItems": 20
}
]
},
"x_axis_name": {
"$ref": "#/definitions/axis_name_array"
},
"y_axis_name": {
"$ref": "#/definitions/axis_name_array"
}
}
},
"simple_metric": {
"description": "metric data",
"type": "object",
"required": [
"name",
"type",
"value"
],
"additionalProperties": false,
"properties": {
"name": {
"type": "string",
"pattern": ".{1,255}"
},
"notes": {
"type": "string",
"maxLength": 1024
},
"type": {
"type": "string",
"enum": [
"number",
"string",
"boolean"
]
},
"value": {
"anyOf": [
{
"type": "number"
},
{
"type": "string",
"maxLength": 63
},
{
"type": "boolean"
}
]
},
"x_axis_name": {
"$ref": "#/definitions/axis_name_string"
},
"y_axis_name": {
"$ref": "#/definitions/axis_name_string"
}
}
},
"axis_name_array": {
"type": "array",
"items": {
"type": "string",
"maxLength": 63
}
},
"axis_name_string": {
"type": "string",
"maxLength": 63
}
}
}
```
# Create a model card
**Important**
Custom IAM policies that allow Amazon SageMaker Studio or Amazon SageMaker Studio Classic to create Amazon SageMaker resources must also grant permissions to add tags to those resources. The permission to add tags to resources is required because Studio and Studio Classic automatically tag any resources they create. If an IAM policy allows Studio and Studio Classic to create resources but does not allow tagging, "AccessDenied" errors can occur when trying to create resources. For more information, see [Provide permissions for tagging SageMaker AI resources](security_iam_id-based-policy-examples.md#grant-tagging-permissions).
[AWS managed policies for Amazon SageMaker AI](security-iam-awsmanpol.md) that give permissions to create SageMaker resources already include permissions to add tags while creating those resources.
You can create an Amazon SageMaker Model Card using either the SageMaker AI console or the SageMaker Python SDK. You can also use the API operations directly. For more information about the API operations, see [Low-level SageMaker APIs for model cards](model-cards-apis.md).
## Create a model card using the SageMaker AI console
Go to the Amazon SageMaker AI console. In the navigation pane, under **Governance**, choose **Model cards**. On the upper right-hand corner, choose **Create model card**.
Go through the four steps in the **Create model card** prompt to document details about your model.
### Step 1: Enter model details and intended use
If your model is an AWS resource, specify the exact model name in this field to auto-populate model details. To browse existing model names, see **Models** in the Amazon SageMaker AI console. Each unique model name can have only one associated model card.
If your model is not an AWS resource, provide a unique name for your model. To add a model as an AWS resource, see [Create a model](https://docs.aws.amazon.com/sagemaker/latest/dg/realtime-endpoints-deployment.html#realtime-endpoints-deployment-create-model) in the *Amazon SageMaker AI Developer Guide*. Alternatively, you can add your model as a model package using [SageMaker AI Marketplace](https://docs.aws.amazon.com/sagemaker/latest/dg/sagemaker-marketplace.html) or [SageMaker AI Model Registry](https://docs.aws.amazon.com/sagemaker/latest/dg/model-registry.html).
For more information on intended uses, see [Intended uses of a model](model-cards.md#model-cards-intended-uses). For more information on risk ratings, see [Risk ratings](model-cards.md#model-cards-risk-rating).
#### Step 2: Enter training details
Add any training details, training observations, datasets, hyperparameters, and details about the model's objective function to the model card.
The objective function in a model card can be any function that is optimized during training. This can include, but is not limited to, cost functions, loss functions, or objective metrics. In this section, document the objective function that is most critical to training your model.
We recommend that you catalog the following attributes of your objective function:
+ Optimization direction
+ Metric
+ Description
For example, you might minimize (optimization direction) cross entropy loss (metric) for a binary classification problem (description) or maximize likelihood for logistic regression. Additionally, you can provide notes about why you chose this objective function over others.
#### Step 3: Enter evaluation details
If you have existing evaluation reports generated by SageMaker Clarify or Model Monitor, either provide an S3 URI for those reports or upload them manually to add them to the model card.
For more information on SageMaker Clarify, see [Run SageMaker Clarify Processing Jobs for Bias Analysis and Explainability](https://docs.aws.amazon.com/sagemaker/latest/dg/clarify-processing-job-run.html).
For more information on monitoring drift in model quality metrics using Model Monitor, see [Monitor model quality](https://docs.aws.amazon.com/sagemaker/latest/dg/model-monitor-model-quality.html).
To add your own evaluation report, choose **Generic model card evaluation**. All model card evaluation reports must be in the [Model card JSON schema](model-cards.md#model-cards-json-schema).
#### Step 4: Enter additional details
Add custom model card detail fields for any additional information that you want to address on your model card. For example, you might include the custom field *Line of business* with a value of *Personal finance*.
#### Save model card
After reviewing the information in your model card, choose **Save** in the lower right-hand corner to save your model card.
## Create a model card using the SageMaker Python SDK
Before creating a model card, you must first define the content of your model card. When using the SageMaker Python SDK, model content consists of a model overview, training details, intended uses, evaluation details, and additional information.
You can create model cards for:
+ Models that are hosted within SageMaker AI
+ Model packages (models) within the SageMaker Model Registry
+ Models that are hosted or registered outside of SageMaker AI
You can also create model cards without associating any models to them.
We recommend adding the models that you've trained to the SageMaker Model Registry. The model registry helps you catalog models and track model versions. When you create a model card, the information about the model from the model registry automatically populates the model card. You can edit the model card or add information to it after you create it.
For information about using the model registry, see [Model Registration Deployment with Model Registry](model-registry.md). For information about creating a model card from a model registry, see [Create a model card for your model in the SageMaker Model Registry](#model-cards-sdk-create-model-registry).
**Note**
To use model cards with the SageMaker Python SDK, you first need to establish a SageMaker AI Session. For more information, see [Session](https://sagemaker.readthedocs.io/en/stable/api/utility/session.html) in the SageMaker Python SDK API reference.
To create a model card for models that aren't in the SageMaker Model Registry, see [Create a model that isn't in the model registry](#model-card-sdk-create-model-non-model-registry).
### Create a model that isn't in the model registry
Use the information in the following sections to create a model card for a model that you haven't added to the model registry.
#### Step 1: Define model overview
Define an overview of your model.
```
model_overview = ModelOverview.from_model_name(
model_name=model_name,
sagemaker_session=sagemaker_session,
model_description="A-description-of-your-model",
problem_type="Problem-type", # For example, "Binary Classification"
algorithm_type="Algorithm-type", # For example, "Logistic Regression"
model_creator="Name-of-model-creator",
model_owner="Name-of-model-owner",
)
```
If your model is an AWS resource, then overview information such as the model ARN, inference container URI, and the S3 location of model artifacts is automatically retrievable. Print the associated AWS metadata with the following commands:
```
print(model_overview.model_id)
print(model_overview.inference_environment.container_image)
print(model_overview.model_artifact)
```
#### Step 2: Define training details
To define your model's training details, you must first define its objective function.
```
objective_function = ObjectiveFunction(
function=Function(
function=ObjectiveFunctionEnum.MINIMIZE,
facet=FacetEnum.LOSS,
),
notes="An-explanation-about-objective-function",
)
```
Next, you can define your training details using your existing model overview, session, and objective function. Add any training observations here.
```
training_details = TrainingDetails.from_model_overview(
model_overview=model_overview,
sagemaker_session=sagemaker_session,
objective_function=objective_function,
training_observations="Model-training-observations",
)
```
Once again, if your model is an AWS resource, certain training details are autopopulated. Print the training job ARN, training container URI, and training metrics with the following commands:
```
print(training_details.training_job_details.training_arn)
print(training_details.training_job_details.training_environment.container_image)
print([{"name": i.name, "value": i.value} for i in training_details.training_job_details.training_metrics])
```
#### Define evaluation details
To define your model's evaluation details, you must first define one or more metric groups to describe metrics used for any evaluation jobs.
```
my_metric_group = MetricGroup(
name="binary classification metrics",
metric_data=[Metric(name="accuracy", type=MetricTypeEnum.NUMBER, value=0.5)]
)
```
Next, you can define your evaluation details using evaluation metrics and datasets for each evaluation job. Add any evaluation observations here and give your evaluation job a unique name.
```
evaluation_details = [
EvaluationJob(
name="Example-evaluation-job",
evaluation_observation="Evaluation-observations",
datasets=["s3://path/to/evaluation/data"],
metric_groups=[my_metric_group],
)
]
```
If you have existing evaluation reports generated by [SageMaker AI Clarify](https://docs.aws.amazon.com/sagemaker/latest/dg/clarify-processing-job-run.html) or [SageMaker AI Model Monitor](https://docs.aws.amazon.com/sagemaker/latest/dg/model-monitor-model-quality.html), upload them to Amazon S3 and provide an S3 URI to automatically parse evaluation metrics. To add your own generic model card evaluation report, provide a report in the [evaluation results JSON format](https://docs.aws.amazon.com/sagemaker/latest/dg/model-cards-json-schema.html).
```
report_type = "clarify_bias.json"
example_evaluation_job.add_metric_group_from_json(
f"example_metrics/{report_type}", EvaluationMetricTypeEnum.CLARIFY_BIAS
)
```
#### Step 3: Define intended uses
Define the intended uses of the model, including the general purpose of the model and the use cases for which it was intended. It is also recommended to include any factors that potentially a this model's efficacy in a particular use case and your organization's risk rating of the model. For more information, see [Intended uses of a model](model-cards.md#model-cards-intended-uses)and [Risk ratings](model-cards.md#model-cards-risk-rating).
```
intended_uses = IntendedUses(
purpose_of_model="Purpose-of-the-model",
intended_uses="The-intended-uses-of-this-model",
factors_affecting_model_efficiency="Any-factors-effecting-model-efficacy",
risk_rating=RiskRatingEnum.LOW,
explanations_for_risk_rating="Explanation-for-low-risk-rating",
)
```
#### Define additional information
Lastly, you can add additional custom information to your model card. You can document any ethical considerations, caveats, and recommendations about the model. You can also add any custom details that you would like in the form of key-value pairs.
```
additional_information = AdditionalInformation(
ethical_considerations="Any-ethical-considerations",
caveats_and_recommendations="Any-caveats-and-recommendations",
custom_details={"custom details1": "details-value"},
)
```
#### Step 4: Create model card
Name your model card, define a model card, and then use that definition to create a model card using the SageMaker Python SDK.
```
model_card_name = "my-model-card"
my_card = ModelCard(
name=model_card_name,
status=ModelCardStatusEnum.DRAFT,
model_overview=model_overview,
training_details=training_details,
intended_uses=intended_uses,
evaluation_details=evaluation_details,
additional_information=additional_information,
sagemaker_session=sagemaker_session,
)
my_card.create()
```
### Create a model card for your model in the SageMaker Model Registry
Before you begin creating a model card, make sure that you've created a model package group and a model package. For more information about using model registry, see [Model Registration Deployment with Model Registry](model-registry.md).
**Important**
You must have permissions to use the operations in SageMaker Model Registry. We recommend using `AmazonSageMakerModelRegistryFullAccess` AWS managed policy. For more information about the managed policy, see [AWS Managed Policies for Model Registry](security-iam-awsmanpol-model-registry.md).
Use the SageMaker Python SDK to create a model card for a model package within the SageMaker Model Registry. A model package is a model that you've trained. When you create a model card, Amazon SageMaker Model Cards automatically imports the data from the model package into the model card.
When you create a model card for a model package, Amazon SageMaker Model Card uses the [DescribeModelPackage](https://docs.aws.amazon.com/APIReference/API_DescribeModelPackage.html#API_DescribeModelPackage_ResponseSyntax) operation to add the data from the model package to the model card. The following are examples of the fields that can be imported from a model package into a model card:
+ [ModelDataUrl](https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_ModelPackageContainerDefinition.html#sagemaker-Type-ModelPackageContainerDefinition-ModelDataUrl)
+ [ModelPackageDescription](https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_DescribeModelPackage.html#sagemaker-DescribeModelPackage-response-ModelPackageDescription)
+ [ModelPackageGroupName](https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_DescribeModelPackage.html#sagemaker-DescribeModelPackage-response-ModelPackageGroupName)
+ [ModelPackageStatus](https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_DescribeModelPackage.html#sagemaker-DescribeModelPackage-response-ModelPackageStatus)
+ [ModelPackageVersion](https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_DescribeModelPackage.html#sagemaker-DescribeModelPackage-response-ModelPackageVersion)
Use the following code to define the model package and create a model card from it:
```
mp_details = ModelPackage.from_model_package_arn(
model_package_arn="example_model_package_arn",
sagemaker_session=sagemaker_session,
)
model_card_name = "example-model-card"
my_card = ModelCard(
name=model_card_name,
status=ModelCardStatusEnum.status,
model_package_details=mp_details,
sagemaker_session=sagemaker_session,
)
my_card.create()
```
For `status`, you're specifying the approval status of the model card. If you don't specify a status, SageMaker Model Cards uses the default value of `DRAFT`. If you don't specify a SageMaker AI session, SageMaker Model Cards uses the default SageMaker AI session.
You must specify a name for the model and the Amazon Resource Name (ARN) of the model package. For information about getting the Amazon Resource Name (ARN) for the model package, see [View and Update the Details of a Model Version (Boto3)](model-registry-details.md#model-registry-details-api).
The model card that you've created from the model package might have information that is either missing or inaccurate. You can add information to the model card or edit it. For more information about managing your model cards, see [Model cards actions](model-cards-manage.md).
SageMaker Model Registry supports versioning of your model packages. You can version your model package and create a model card for each version. The information from model cards of preceding versions carry over to model cards created from subsequent versions. For example, you could have version 1, version 2, and version 3 of a model package. Suppose you've already created a model card for version 1, but you haven't created one for version 2. If you create a model card for version 3, Amazon SageMaker Model Cards automatically carries over the information from the model card for version 1 to the model card for version 3.
**Note**
You can also create model cards for model packages that don't use versioning. However, most machine learning workflows involve multiple versions of the same model, so we recommend doing the following:
Creating a version for each model package
Creating a model card for each version of the model package
# Model cards actions
After you've created a model card, you can manage them. Managing model cards include the following actions:
+ Editing a model card
+ Deleting a model card
+ Exporting a model card to a PDF
You can manage using either the Amazon SageMaker AI console or the SageMaker Python SDK. For more information about using the Python SDK see [Amazon SageMaker Model Cards](https://sagemaker.readthedocs.io/en/stable/api/governance/model_card.html) in the SageMaker Python SDK API reference.
For example notebook using the SageMaker Python SDK, see the *[Amazon SageMaker Model Governance - Model Card](https://github.com/aws/amazon-sagemaker-examples/tree/main/sagemaker_model_governance/model_card.ipynb)* example notebook.
**Topics**
+ [Edit a model card](model-cards-console-edit.md)
+ [Export a model card](model-cards-console-export.md)
+ [Delete a model card](model-cards-console-delete.md)
# Edit a model card
To edit a model card, navigate to the model card of your choice by selecting its name in the Amazon SageMaker Model Card console and choose **Edit**.
After you save a model card, you cannot edit the name of that model card. After you save a model card version, you cannot update that version of the model card. Any edits that you need to make are saved as a subsequent version in order to have an immutable record of model changes.
To view different versions of the model card, choose **Actions**, **Select version**, and then choose the version that you want to view.
You can edit a model card using the `model_card.update()` method. Updating a model card creates a new model card version in order to have an immutable record of model changes. You cannot update the name of a model card.
```
my_card.model_overview.model_description = "updated-model-decription"
my_card.update()
```
# Export a model card
Follow these steps to export a model card.
1. Go to the Amazon SageMaker Model Card console.
1. Choose the name of the model card you want to export.
1. In the model card overview, choose **Actions** and then **Export PDF**.
1. Enter an S3 URI or browse available S3 buckets for your model card PDF.
1. If your model card exports successfully, you can either choose **Download PDF** in the resulting banner or download your PDF directly from Amazon S3.
You can export a model card in the SageMaker Python SDK by specifying an S3 output path and export your model card PDF to it with the following commands:
```
s3_output_path = f"s3://{bucket}/{prefix}/export"
pdf_s3_url = my_card.export_pdf(s3_output_path=s3_output_path).delete()
```
# Delete a model card
Follow these steps to permanently delete one or more model cards.
1. Go to the Amazon SageMaker Model Cards console.
1. Check the box to the left of the name of the card(s) you want to delete.
1. Choose **Delete** in the upper right-hand corner.
1. Confirm your request to permanently delete one or more cards..
You can also delete a model card when viewing the model card overview in the console by choosing **Actions** and then **Delete model card**.
Within the SageMaker Python SDK, you can permanently delete a model card with the following command:
```
my_card.delete()
```
# Set up cross-account support for Amazon SageMaker Model Cards
Use cross-account support in Amazon SageMaker Model Cards to share model cards between AWS accounts. The account where the model cards are created is the *model card account*. Users in the model card account share them with the *shared accounts*. The users in a shared account can update the model cards or create PDFs of them.
Users in the model card account share their model cards through AWS Resource Access Manager (AWS RAM). AWS RAM helps you share resources across AWS accounts. For an introduction to AWS RAM, see [What is AWS Resource Access Manager?](https://docs.aws.amazon.com/ram/latest/userguide/what-is.html)
The following is the process to share model cards:
1. A user in the model card account sets up the cross-account model card sharing using AWS Resource Access Manager.
1. If the model cards are encrypted with AWS KMS keys, the user setting up model sharing must also provide users in the shared account with AWS KMS permissions.
1. A user in the shared account accepts the invite to the resource share.
1. A user in the shared account provides the other users with permissions to access the model cards.
If you're a user in the model card account, see the following sections:
+ [Set up cross-account model card sharing](#model-cards-xaccount-set-up)
+ [Set up AWS KMS permissions for the shared account](#model-cards-xaccount-kms)
+ [Get responses to your resource share invitation](#model-cards-xaccount-set-up-responses)
If you're a user in the shared account, see [Set up IAM user permissions in the shared account](#model-cards-xaccount-shared-account-permissions) about setting up permissions for yourself and the other users in the account.
## Set up cross-account model card sharing
Use AWS Resource Access Manager (AWS RAM) to grant users in your AWS account access to view or update model cards created in a different AWS account.
To set up model card sharing, you must create a resource share. A resource share specifies:
+ The resources being shared
+ Who or what has access to the resources
+ Managed permissions for the resources
For more information about resource shares, see [Terms and concepts for AWS RAM](https://docs.aws.amazon.com/ram/latest/userguide/getting-started-terms-and-concepts.html). We recommend taking the time to understand AWS RAM conceptually before you go through the process of creating a resource share.
**Important**
You must have permissions to create a resource share. For more information about permissions, see [How AWS RAM works with IAM](https://docs.aws.amazon.com/ram/latest/userguide/security-iam-policies.html).
For procedures to create a resource share and additional information about them, see [Create a resource share](https://docs.aws.amazon.com/ram/latest/userguide/getting-started-sharing.html#getting-started-sharing-create).
When you go through the procedure of creating a resource share, you specify `sagemaker:ModelCard` as the resource type. You must also specify the Amazon Resource Number (ARN) of the AWS RAM resource-based policy. You can specify either the default policy or the policy that has additional permissions to create a PDF of the model card.
With the default `AWSRAMPermissionSageMakerModelCards` resource-based policy, the users in the shared account have permissions to do the following operations:
+ [DescribeModelCard](https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_DescribeModelCard.html)
+ [ListModelCardVersions](https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_ListModelCardVersions.html)
+ [UpdateModelCard](https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_UpdateModelCard.html)
With the `AWSRAMPermissionSageMakerModelCardsAllowExport` resource-based policy, the users in the shared account have permissions to do all of the preceding actions. They also have permissions to create a model card export job and describe it through the following operations:
+ [CreateModelCardExportJob](https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_CreateModelCardExportJob.html)
+ [DescribeModelCardExportJob](https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_DescribeModelCardExportJob.html)
The users in the shared account can create an export job to generate a PDF of a model card. They can also describe an export job that has been created to find the PDF's Amazon S3 URI.
Model cards and export jobs are resources. The model card account owns the export jobs created by a user in the shared account. For example, a user in account A shares model card X with shared account B. A user in account B creates export job Y for model card X that stores the output in an Amazon S3 location that the user in account B specifies. Even though account B created export job Y, it belongs to account A.
Each AWS account has resource quotas. For information about quotas related to model cards, see [Amazon SageMaker AI endpoints and quotas](https://docs.aws.amazon.com/general/latest/gr/sagemaker.html#limits_sagemaker).
### Set up AWS KMS permissions for the shared account
If the model cards that you're sharing have been encrypted with AWS Key Management Service keys, you also need to share the access to the keys with the shared account. Otherwise, the users in the shared account can't view, update, or export the model cards. For an overview of AWS KMS, see [AWS Key Management Service](https://docs.aws.amazon.com/kms/latest/developerguide/overview.html).
To provide AWS KMS permissions to users in the shared account, update your key policy with the following statement:
```
{
"Effect": "Allow",
"Principal": {
"AWS": [
"arn:aws:iam::shared-account-id::role/example-IAM-role"
]
},
"Action": [
"kms:GenerateDataKey",
"kms:Decrypt",
]
"Resource": "arn:aws:kms:AWS-Region-of-model-card-account:model-card-account-id:key/AWS KMS-key-id"
"Condition": {
"Bool": {"kms:GrantIsForAWSResource": true },
"StringEquals": {
"kms:ViaService": [
"sagemaker.AWS-Region.amazonaws.com",
"s3.AWS-Region.amazonaws.com"
],
},
"StringLike": {
"kms:EncryptionContext:aws:sagemaker:model-card-arn": "arn:aws:sagemaker:AWS-Region:model-card-account-id:model-card/model-card-name"
}
}
}
```
The preceding statement provides users in the shared account with `kms:Decrypt` and `kms:GenerateDataKey` permissions. With `kms:Decrypt`, users can decrypt the model cards. With `kms:GenerateDataKey`, users can encrypt the model cards that they update or the PDFs that they create.
### Get responses to your resource share invitation
After you've created a resource share, the shared accounts that you've specified in the resource share receive an invitation to join it. They must accept the invite to access the resources.
For information about accepting a resource share invite, see [Using shared AWS resources ](https://docs.aws.amazon.com/ram/latest/userguide/getting-started-shared.html)in the *AWS Resource Access Manager User Guide*.
### Set up IAM user permissions in the shared account
The following information assumes that you've accepted the resource share invitation from the model card account. For more information about accepting a resource share invitation, see [Using shared AWS resources ](https://docs.aws.amazon.com/ram/latest/userguide/getting-started-shared.html).
You and the other users in your account use an IAM role to access the model cards shared from the model card account. Use the following template to change the policy of the IAM role. You can modify the template for your own use case.
------
#### [ JSON ]
****
```
{
"Version":"2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"sagemaker:DescribeModelCard",
"sagemaker:UpdateModelCard",
"sagemaker:CreateModelCardExportJob",
"sagemaker:ListModelCardVersions",
"sagemaker:DescribeModelCardExportJob"
],
"Resource": [
"arn:aws:sagemaker:us-east-1:111122223333:model-card/example-model-card-name-0",
"arn:aws:sagemaker:us-east-1:111122223333:model-card/example-model-card-name-1/*"
]
},
{
"Effect": "Allow",
"Action": "s3:PutObject",
"Resource": "arn:aws:s3:::amzn-s3-demo-bucket-storing-the-pdf-of-the-model-card/model-card-name/*"
}
]
}
```
------
To access model cards encrypted using AWS KMS, you must provide users in your account with the following AWS KMS permissions.
```
{
"Effect": "Allow",
"Action": [
"kms:GenerateDataKey",
"kms:Decrypt",
],
"Resource": "arn:aws:kms:AWS-Region:AWS-account-id-where-the-model-card-is-created:key/AWS Key Management Service-key-id"
}
```
# Low-level SageMaker APIs for model cards
You can create an Amazon SageMaker Model Card directly through the SageMaker API or the AWS Command Line Interface (AWS CLI).
**Note**
When creating a model card with the low-level APIs, the content must be in the model card JSON schema and provided as a string. For more information, see [Model card JSON schema](model-cards.md#model-cards-json-schema).
## SageMaker API
Use the following SageMaker API commands to work with Amazon SageMaker Model Cards:
+ [CreateModelCard](https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_CreateModelCard.html)
+ [DescribeModelCard](https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_DescribeModelCard.html)
+ [ListModelCards](https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_ListModelCards.html)
+ [ListModelCardVersions](https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_ListModelCardVersions.html)
+ [UpdateModelCard](https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_UpdateModelCard.html)
+ [CreateModelCardExportJob](https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_CreateModelCardExportJob.html)
+ [DescribeModelCardExportJob](https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_DescribeModelCardExportJob.html)
+ [ListModelCardExportJobs](https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_ListModelCardExportJobs.html)
+ [DeleteModelCard](https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_DeleteModelCard.html)
## AWS CLI
Use the following AWS CLI commands to work with Amazon SageMaker Model Cards:
+ [create-model-card](https://docs.aws.amazon.com/cli/latest/reference/sagemaker/create-model-card.html)
+ [describe-model-card](https://docs.aws.amazon.com/cli/latest/reference/sagemaker/describe-model-card.html)
+ [list-model-cards](https://docs.aws.amazon.com/cli/latest/reference/sagemaker/list-model-cards.html)
+ [list-model-card-versions](https://docs.aws.amazon.com/cli/latest/reference/sagemaker/list-model-card-versions.html)
+ [update-model-card](https://docs.aws.amazon.com/cli/latest/reference/sagemaker/update-model-card.html)
+ [create-model-card-export-job](https://docs.aws.amazon.com/cli/latest/reference/sagemaker/create-model-card-export-job.html)
+ [describe-model-card-export-job](https://docs.aws.amazon.com/cli/latest/reference/sagemaker/describe-model-card-export-job.html)
+ [list-model-card-export-jobs](https://docs.aws.amazon.com/cli/latest/reference/sagemaker/list-model-card-export-jobs.html)
+ [delete-model-card](https://docs.aws.amazon.com/cli/latest/reference/sagemaker/delete-model-card.html)
# Model card FAQs
Refer to the following FAQ items for answers to commonly asked questions about Amazon SageMaker Model Card.
## Q. What is model risk?
A: You can use models for a variety of business applications ranging from predicting cyber attacks and approving loan applications to detecting the category of an email. Each of these applications assumes a different level of risk. For example, incorrectly detecting a cyber attack has much greater business impact than incorrectly categorizing an email. Given these varied risk profiles of a model, you can use model cards to provide a risk rating of `low`, `medium`, or `high` for a model. If you don’t know the risk of your model, you can set the status to `unknown`. Customers are responsible for assigning the risk profile for each model. Based on the risk rating, organizations may have different rules in place for deploying those models to production. For more information, see [Risk ratings](model-cards.md#model-cards-risk-rating).
## Q. What is the intended use of a model?
The intended use of a model describes how you should use the model in your production applications. This goes beyond technical requirements like the type of instance to which you should deploy a model and instead refers to the types of applications to create with the model, the scenarios in which you can expect a reasonable performance from the model, or the type of data to use with the model. We recommend providing this information in the model card for better model governance. You can define a kind of model specification in the intended use field and ensure that model developers and consumers follow this specification while training and deploying their models. For more information, see [Intended uses of a model](model-cards.md#model-cards-intended-uses).
## Q. Does SageMaker AI auto-populate information in my model card?
When creating a model card using either the SageMaker Python SDK or the AWS console, SageMaker AI automatically populates information about your trained model. This includes comprehensive training details and all model information that's available through the `describe-model` API call. If you work in Amazon SageMaker Studio, you can auto-populate your model cards by calling the [DescribeModelPackage](https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_DescribeModelPackage.html)API.
## Q. Can I customize a model card?
Amazon SageMaker Model Cards have a defined structure to them that cannot be modified. This structure gives you guidance on what information should be captured in a model card. While you cannot change the structure of the model card, there is some flexibility introduced through custom properties in the **Additional information** section of the model card.
## Q. Can I edit a model card once it is created?
Model cards have versions associated with them. A given model version is immutable across all attributes other than the model card status. If you make any other changes to the model card, such as evaluation metrics, description, or intended uses, SageMaker AI creates a new version of the model card to reflect the updated information. This is to ensure that a model card, once created, cannot be tampered with.
Model cards are automatically updated whenever you make changes to your model package versions in the Model Registry.
## Q. Can I create model cards for models that were not trained using SageMaker AI?
A: Yes. You can create model cards for models not trained in SageMaker AI, but no information is automatically populated in the card. You must supply all the information needed in the model card for non-SageMaker AI models.
## Q. Can I export or share model cards?
A: Yes. You can export each version of a model card to a PDF, downloaded, and share it.
## Q. Do I need to register my model in the Model Registry to use model cards?
A: Model cards are accessible through both SageMaker AI Console and Model Registry. When using Model Registry, you will automatically receive a model card for each version of your model.
## Q. What is the difference between model cards and Model Registry?
A: Model cards are intended to provide organizations with a mechanism to document as much detail about their model as they like by following SageMaker AI’s prescriptive guidance along with providing their own custom information. You can introduce model cards at the very start of the ML process and use them to define the business problem that the model should solve and any considerations to think about while using the model. After a model is trained, you can populate the model card associated with that model with information about the model and how it was trained. Model cards are associated with models and are immutable once associated with a model. This ensures that the model card is the single source of truth for all the information related to a model, including how it was trained and how it should be used.
The Model Registry is a catalog that stores metadata about your models. Each entry in the model registry corresponds to a unique model version. That model version contains information about the model such as where the model artifacts are stored in Amazon S3, what container is needed to deploy the model, and custom metadata that should be attached to the model. Each model package version has a model card associated with it.
## Q. What is the relationship between model cards and Model Registry?
A: Model cards are integrated into the Model Registry object. Each version of a model package in the Model Registry is linked to its corresponding model card. You can access the model card schema for each version by using the [ModelPackageModelCard](https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_ModelPackageModelCard.html) API.
## Q. Are model card versions related to model versions in the Model Registry?
A: Yes, there is a one-to-one relationship between model cards and models in the Model Registry. Each model version stored in the Model Registry has exactly one corresponding model card associated with it.
## Q. Are model cards integrated with SageMaker Model Monitor?
A: No. You can upload the performance metrics computed by SageMaker Model Monitor to the model card by uploading a metrics file to Amazon S3 and linking that to the card, but there is no native integration between Model Monitor and model cards. Model dashboards are integrated with Model Monitor. For more information on model dashboards, see [Amazon SageMaker Model Dashboard](https://docs.aws.amazon.com/sagemaker/latest/dg/model_dashboard.html).
# Controlled access to assets with Amazon SageMaker Assets
Use Amazon SageMaker Assets to provide controlled and regulated access to *assets*, models or data tables, belonging to your organization. Within SageMaker Assets, users from different AWS accounts can create and share assets related to specific business problems without additional administrator overhead. Instead of having permissions being statically tied to their identity, users can provide permissions to assets that they’re using for their active workflows.
Assets are ML assets or data assets. ML assets are metadata that point to Amazon SageMaker Feature Store feature groups or SageMaker Model Registry Model Groups. Data assets are metadata that point to Amazon Redshift tables or AWS Glue tables.
For example, the asset for a model group contains the model group name and the Amazon Resource Name (ARN) for the model package group. The asset points to the underlying collection of models. The asset itself can be shared between users.
Users can create assets for their own projects. They can make them visible to users who aren't members of those projects. The users who aren't project members can search through the assets and read their metadata. They can use the metadata to determine whether they want to access to the underlying source of data.
To understand the SageMaker Assets workflow better, imagine that you have two groups of users in your organization, Group A and Group B. The users in Group A are looking to predict home prices. They’re looking to collaborate with the users in Group B who are in a different AWS account. They have housing data stored in AWS Glue tables. They also have different models saved as model packages within a model group. With SageMaker Assets, the users in Group A can share their AWS Glue tables and model packages with the users in Group B in a few clicks. Without administrator intervention, the users in Group A provided precisely scoped permissions to the users in Group B.
Users can create assets and publish them to make them visible throughout the organization. Other users can request access to those assets.
**Topics**
+ [Set up SageMaker Assets (administrator guide)](sm-assets-set-up.md)
+ [Work with assets (user guide)](sm-assets-user-guide.md)
# Set up SageMaker Assets (administrator guide)
**Important**
SageMaker Assets is only available in Amazon SageMaker Studio. If you're using Amazon SageMaker Studio Classic, you must migrate to Studio. For more information about Studio and Studio Classic, see [Machine learning environments offered by Amazon SageMaker AI](machine-learning-environments.md). For information about migrating, see [Migration from Amazon SageMaker Studio Classic](studio-updated-migrate.md).
As business needs change, your users need to collaborate effectively to solve business problems as they arise. To solve them, users must share data and models with each other.
SageMaker Assets integrates Amazon SageMaker Studio with Amazon DataZone, a data management service. SageMaker Assets is a platform that helps your users share models and data with each other. You can use the following information to set up the integration between SageMaker Assets and Amazon DataZone.
You create an Amazon DataZone domain for your business line or organization. The *domain* is the core feature of Amazon DataZone. All of your users' data and models exist within the domain.
Within the Amazon DataZone domain, a subset of your users work on specific *projects*. A project typically corresponds to a particular business problem. Within the project, members can create datasets and models. By default, project members only have access to the data and models within the project. They can provide access to their data and models to other users within the organization.
Within the project, you create environments. For SageMaker Assets specifically, an environment is a collection of configured resources used to launch Amazon SageMaker Studio. For more information about the terminology used in Amazon DataZone, see [Terminology and concepts](https://docs.aws.amazon.com/datazone/latest/userguide/datazone-concepts.html).
**Important**
Depending on the set up you choose, Amazon SageMaker Studio uses one of the following:
An Amazon SageMaker AI domain that Amazon DataZone creates as part of your SageMaker AI environment.
Your existing Amazon SageMaker AI domain that you migrate to Amazon DataZone
You can access Studio from the Amazon SageMaker AI domain, but we recommend accessing it from the project you've created. For information about accessing Studio, see [Work with assets (user guide)](sm-assets-user-guide.md).
## Set up Amazon DataZone with a new SageMaker AI domain
Use the steps in the following list and the documentation it references to set up Amazon DataZone with an Amazon SageMaker AI domain that it creates.
1. Create an Amazon DataZone domain that corresponds to your users' organization or business line. For information about creating an Amazon DataZone domain, see [Create domains](https://docs.aws.amazon.com/datazone/latest/userguide/create-domain.html).
1. Enable the SageMaker AI blueprint within Amazon DataZone. For information about enabling the SageMaker AI blueprint, see [Enable built-in blueprints in the AWS account that owns the Amazon DataZone domain](https://docs.aws.amazon.com/datazone/latest/userguide/working-with-blueprints.html#enable-default-blueprints).
1. Create a project within the domain that corresponds to the business problem that users in your domain are solving. For information about creating a project, see [Create a new project](https://docs.aws.amazon.com/datazone/latest/userguide/create-new-project.html).
1. Create an environment profile that you can use as a template to create SageMaker AI environments for your users. For information about creating an environment profile, see [Create an environment profile](https://docs.aws.amazon.com/datazone/latest/userguide/create-environment-profile.html).
1. Create a SageMaker AI environment. Within the project, your users use the SageMaker AI environment to launch Amazon SageMaker Studio. Within Studio, they can create assets and use SageMaker Assets to share them. For information about creating an environment, see [Create a new environment](https://docs.aws.amazon.com/datazone/latest/userguide/create-environment-profile.html).
1. Add SageMaker AI as one of the trusted services within Amazon DataZone. To add SageMaker AI as one of the services, see [Add SageMaker AI as a trusted service in the AWS account that owns the Amazon DataZone domain](https://docs.aws.amazon.com/datazone/latest/userguide/working-with-blueprints.html#add-sagemaker-as-trusted-service).
## Set up Amazon DataZone with your SageMaker AI domain
Use the steps in the following list and the documentation it references to set up Amazon DataZone with an existing Amazon SageMaker AI domain.
1. Create an Amazon DataZone domain that corresponds to your users' organization or business line. For information about creating an Amazon DataZone domain, see [Create domains](https://docs.aws.amazon.com/datazone/latest/userguide/create-domain.html).
1. Enable the SageMaker AI blueprint within Amazon DataZone. For information about enabling a custom blueprint, see [Amazon DataZone custom AWS service blueprints](https://docs.aws.amazon.com/datazone/latest/userguide/working-with-custom-blueprint.html).
1. Create a project within the domain that corresponds to the business problem that users in your domain are solving. For information about creating a project, see [Create a new project](https://docs.aws.amazon.com/datazone/latest/userguide/create-new-project.html).
1. Enable SageMaker AI as one of the trusted services within Amazon DataZone. To enable SageMaker AI as one of the services, see [Add Amazon SageMaker AI as a trusted service in the AWS account that owns the Amazon DataZone domain ](https://docs.aws.amazon.com/datazone/latest/userguide/working-with-blueprints.html#add-sagemaker-as-trusted-service).
1. Create Amazon DataZone users within the SageMaker AI domain.
1. Onboard existing users to the Amazon DataZone domain.
**Note**
If your SageMaker AI users are SSO and your Amazon DataZone domain is SSO, you can automatically map the users from the Amazon SageMaker AI domain to the Amazon DataZone domain.
To onboard existing SageMaker AI users, run the [Amazon DataZone Import SageMaker AI Domain](https://github.com/aws/amazon-sagemaker-examples/tree/default/%20%20%20ml_ops/sm-datazone_import) script in your environment. You must pass the name of your AWS Region and the AWS account ID of your Amazon SageMaker AI domain as arguments. The following is an example AWS CLI command that runs the script.
```
python example-script AWS Region 111122223333
```
The script does the following:
1. Asks you for your Amazon SageMaker AI domain ID.
1. Asks you for your Amazon DataZone domain ID.
1. Asks you for your Amazon DataZone project.
1. Prompts you to specify the users that you're importing.
1. Adds tags to your users and the Amazon SageMaker AI domain.
1. Map your Amazon DataZone users to your SageMaker AI user profiles. For each SageMaker AI user profile, the script will prompt you for a Amazon DataZone user ID. You can modify the script for your own use case.
1. Attaches a federation role to the environment, so that Amazon DataZone can access your Amazon SageMaker AI domain domain and migrate it.
The script goes through each user in the Amazon SageMaker AI domain and prompts you to specify the corresponding user in the Amazon DataZone domain. It automatically adds tags for the user in the Amazon DataZone domain to the users in the corresponding SageMaker AI domain. It also updates the custom environment blueprint with the mapping between users in each domain.
**Note**
The SageMaker AI environment uses the latest version of the SageMaker Distribution Image. SageMaker AI Distribution Images have popular libraries packages for machine learning. For more information, see [SageMaker Studio image support policy](sagemaker-distribution.md).
After you've created the environment, you can create AWS Glue and Amazon Redshift tables and databases. For more information, see [Query data in Athena or Amazon Redshift](https://docs.aws.amazon.com/datazone/latest/userguide/query-athena-with-deep-link-in-project.html).
## Viewing and modifying your users' permissions
After you create a SageMaker AI environment, you can change your users' permissions to suit the needs of your organization. The SageMaker AI blueprint specifies permissions for all of your users. They can perform actions with all of the SageMaker AI services, but the permissions are scoped down to resources created within the Amazon DataZone domain.
**Important**
The environment that you create uses an IAM role that has limited permissions and a permissions boundary. To change your users' permissions, you can modify or replace the permissions boundary. For example, you can change the permissions boundary if your users need access to a resource such as an Amazon S3 bucket that has been created within the environment.
You can view the permissions in the ARN of the IAM role used to create the SageMaker AI domain.
Use the following procedure to view or edit the permissions of the IAM role of your users.
**To view or edit the permissions of your users**
1. Open the [ Amazon SageMaker AI console](https://console.aws.amazon.com/sagemaker).
1. Choose **Domains**.
1. Choose the name of the domain that has the same name as your Amazon DataZone domain.
1. Choose **Domain settings**.
1. Under **Execution role**, copy the ARN of the execution role.
1. Open the [ IAM console](https://console.aws.amazon.com/IAM).
1. Choose **Roles**.
1. Paste the ARN and delete everything except the role name after the last forward slash.
1. Choose the role to view the permissions.
1. Under **Permissions**, modify the policies to suit the needs of your organization.
1. (Optional) Select **Permissions boundary**, and choose **Set permissions boundary**.
1. Select a policy to set as the permissions boundary.
# Work with assets (user guide)
Use SageMaker Assets to seamlessly collaborate on machine learning projects with other individuals in your organization. With SageMaker Assets, you and your collaborators create and share models and data tables with each other. Within SageMaker Assets, these models and data tables are known as *assets*.
SageMaker Assets is a feature within Amazon SageMaker Studio. You or your administrator create a Studio environment within an Amazon DataZone project. For more information about setting up Amazon DataZone, see [Set up SageMaker Assets (administrator guide)](sm-assets-set-up.md).
Assets are ML assets or data assets. ML assets are metadata that point to the following:
+ Feature Store feature groups
+ SageMaker AI model groups
The underlying model groups and feature groups are the sources of data. If you update a feature group or model group, the asset for the model group or feature group gets updated within the day.
Data assets are metadata that point to the following:
+ Amazon Redshift tables
+ AWS Glue tables
For data assets, the data source is the mechanism that pulls metadata from the AWS Glue tables and Amazon Redshift tables into the asset. For example, a data source pulls the metadata from an AWS Glue table into the asset for that table.
You can make an asset visible to everyone in your organization by publishing it. Individuals can review the metadata in the asset and request access. If you provide access, they get access to the underlying machine learning source of data or table.
Your administrator has likely given you access to the feature groups, model groups, and tables. If they haven't, see the information in [Set up SageMaker Assets (administrator guide)](sm-assets-set-up.md) to help you get started.
The following sections provide reference information for feature groups and model groups.
## Feature groups
Amazon SageMaker Feature Store provides a centralized location to help you store and manage your features. It's a highly performant repository that you can use for feature engineering.
Within Feature Store, features are stored in a feature group. A feature group is a collection of features related to a project that you're working on. For example, if you're working on a project related to predicting housing prices, a feature group might include features such as location or number of bedrooms.
For more information about how you can use feature groups to streamline the process of feature engineering, see [Create, store, and share features with Feature Store](feature-store.md).
## Model groups
You can use SageMaker AI model groups within SageMaker Model Registry to organize and manage different versions of your models. You can compare the different versions of the models to see which one performs best for your use case. For more information about SageMaker Model Registry, see [Model Registration Deployment with Model Registry](model-registry.md).
The following is background information on Amazon Redshift and AWS Glue.
Amazon Redshift is a large scale data warehousing service that provides fast query performance on large datasets. For more information about Amazon Redshift, see [Amazon Redshift Serverless](https://docs.aws.amazon.com/redshift/latest/gsg/new-user-serverless.html).
AWS Glue is an extract, transform, load (ETL) service that you can use to simplify the process of data preparation. For more information about AWS Glue, see [What is AWS Glue?](https://docs.aws.amazon.com/glue/latest/dg/what-is-glue.html)
You can use the SQL editor to connect AWS Glue and Amazon Redshift databases and run queries. You can share any tables that you create in the editor within SageMaker Assets. For more information, see [Data preparation with SQL in Studio](sagemaker-sql-extension.md).
**Topics**
+ [Terminology and Concepts](#sm-assets-terminology)
+ [Step 1: Access SageMaker Assets](#sm-assets-access)
+ [Step 2: Share assets and manage access to them](#sm-assets-share)
+ [Step 3: Manage access requests](#sm-assets-manage-requests)
+ [Step 4: Find assets and request access to them](#sm-assets-request-access)
+ [Step 5: Use a shared asset in your machine learning workflows](#sm-assets-consume)
## Terminology and Concepts
Before you get started with using SageMaker Assets, it's helpful to familiarize yourself with the following terminology and concepts:
+ Asset – The metadata that points to the models or data tables that you're sharing. You either request access to an asset that someone else owns or share your asset with others. You and your teammates access the asset and the underlying data table or model associated with it.
+ Subscribed assets – To request access to an asset, you submit a subscription request. If your request is approved, the asset appears under your subscribed assets.
+ Owned assets – The assets that you've shared with your teammates.
+ Asset catalog – The assets that you've shared across your organization.
## Step 1: Access SageMaker Assets
Access SageMaker Assets to view your assets and share them with others. Use the following information to help you get started with using it.
You access SageMaker Assets from a *project* within an Amazon DataZone domain. A project is a collaboration between you and your team members. Within the project, you and the other members of your project have access to the assets that you and your other team members create within the inventory catalog. You can publish the assets to the published catalog to make them visible to other individuals in your organization.
Those individuals can request access to your asset. If you provide them with access, they can get access to the updated source of data. For example, if an individual subscribes to an AWS Glue table that you update, they can access the updated AWS Glue table in real time.
Use the following procedure to access SageMaker Assets.
**To access SageMaker Assets**
1. Open the [ Amazon DataZone](https://console.aws.amazon.com/datazone) console.
1. Choose **View domains**.
1. Next to the domain containing your project, choose **Open data portal**.
1. Under **Analytics Tools**, choose **SageMaker AI Studio**.
1. Choose **Open Amazon SageMaker AI**.
1. Choose **Assets**.
The assets that have been shared with you are under **Subscribed assets**. The assets that you and your project members create are under **Owned assets**. The assets that you and the other members of your organization have published are in the **Assets catalog**.
## Step 2: Share assets and manage access to them
After you create machine learning models, feature groups, or data tables, you can make them visible to the individuals collaborating with you on your project or your organization more broadly. You can respond to requests for access to the asset. If you approve an individual's request, they can modify the asset's underlying source of data.
When you're sharing an asset, you have two options:
+ Publish to asset catalog – Make the asset visible to everyone in your organization
+ Publish to inventory – Make the asset visible to everyone working on your project
If you've published your asset to the asset catalog, individuals in your organization can find it in the assets catalog. They can view your asset's metadata and decide if they want to request access to them. If you approve their request, they get access to the underlying source of data.
If you publish to inventory, you and the other members of your project can access the asset without any additional action.
Assets published to the inventory only appear under **Owned assets**. Assets published to the catalog appear under **Owned assets** and **Assets catalog**.
When you publish a data table, you must create a data source that pulls the metadata from the underlying AWS Glue table or Amazon Redshift table into the asset. Use the following procedures to publish a AWS Glue or Amazon Redshift table.
------
#### [ Publish an AWS Glue table ]
To publish an asset for an AWS Glue table, you create a data source for it and publish it. A data source is the mechanism that pulls the metadata from the AWS Glue table into the asset.
Use the following procedure to publish an AWS Glue table.
**To publish a AWS Glue table**
1. Navigate to the **SageMaker Assets** landing page.
1. Select **Owned assets**.
1. Choose **View data sources**.
1. Choose **Create data source**.
1. For **Name**, specify a name for the data source.
1. For **Description**, provide a description.
1. For **Type**, select **AWS Glue**.
1. For **Data selection**, select the database containing the AWS Glue table.
1. For **Table selection criteria**, specify the name of the table.
**Note**
Even though you can specify more than one table, we strongly suggest providing only one table name.
1. Choose **Next**.
1.
+ For **Publish asset to the catalog**, select **Yes** to publish to the asset catalog.
+ For **Publish asset to the catalog**, select **No** to publish to the asset catalog.
1. Choose **Next**.
1. Under **Asset details**, choose **Run on a schedule** or **Run on demand** to determine how the metadata from the AWS Glue table is pulled into the asset.
1. (Optional) If you choose **Run on a schedule**, specify the schedule that pulls the metadata into the asset.
1. Choose **Next**.
1. Choose **Create**.
1. (Optional) If you haven't created a schedule, choose **Run** to bring the metadata from the AWS Glue table into the asset.
------
#### [ Publish an Amazon Redshift table ]
To publish an asset for an Amazon Redshift table, you create a data source for it and publish it. A data source is the mechanism that pulls the metadata from the Amazon Redshift table into the asset.
Use the following procedure to publish an Amazon Redshift table.
**To publish an Amazon Redshift table**
1. Navigate to the **SageMaker Assets** landing page.
1. Select **Owned assets**.
1. Choose **View data sources**.
1. Choose **Create data source**.
1. For **Name**, specify a name for the data source.
1. For **Description**, provide a description.
1. For **Type**, select **Amazon Redshift**.
1.
+ Select **Redshift cluster**.
1. For **Redshift cluster**, specify the name of the Amazon Redshift cluster containing the database for the table.
1. For **Secret**, specify the name of the AWS Secrets Manager secret containing the credentials for the cluster.
+ Select **Redshift serverless**.
1. For **Redshift workgroup**, specify the name of the Amazon Redshift workgroup containing the database for the table.
1. For **Secret**, specify the name of the AWS Secrets Manager secret containing the credentials for the workgroup.
1. For **Publish source selection**, select the database containing the Amazon Redshift table.
1. For **Table selection criteria**, specify the name of the table.
**Note**
Even though you can specify more than one table, we strongly suggest providing only one table name.
1. Choose **Next**.
1.
+ For **Publish asset to the catalog**, select **Yes** to publish to the asset catalog.
+ For **Publish asset to the catalog**, select **No** to publish to the asset catalog.
1. Choose **Next**.
1. Under **Asset details**, choose **Run on a schedule** or **Run on demand** to determine how the metadata from the Amazon Redshift table is pulled into the asset.
1. (Optional) If you choose **Run on a schedule**, specify the schedule that pulls the metadata into the asset.
1. Choose **Next**.
1. Choose **Create**.
1. (Optional) If you haven't created a schedule, choose **Run** to bring the metadata from the Amazon Redshift table into the asset.
------
Use the following procedures to publish an asset for a feature group or model package group.
------
#### [ Publish a feature group ]
Use the following procedure to navigate to a feature group that you've created and publish it to your owned assets or asset catalog.
**To publish the feature group to your owned assets or asset catalog**
1. Within Studio, select **Data** on the left hand navigation.
1. Select the feature group that you're publishing.
1. Choose the ![\[\]](http://docs.aws.amazon.com/sagemaker/latest/dg/images/sm-assets-publish-icon.png) icon.
1.
+ Select **Publish to asset catalog** to publish to the asset catalog.
+ Select **Publish to inventory** to publish to the owned assets of your group.
------
#### [ Publish a model group ]
Use the following procedure to navigate to a model group that you've created and publish it to your owned assets or asset catalog.
**To publish the model group to your owned assets or asset catalog**
1. Within Studio, select **Models** on the left hand navigation.
1. Select the model group that you're publishing.
1. Choose the ![\[\]](http://docs.aws.amazon.com/sagemaker/latest/dg/images/sm-assets-publish-icon.png) icon.
1.
+ Select **Publish to asset catalog** to publish to the asset catalog.
+ Select **Publish to inventory** to publish to the owned assets of your group.
------
Use the following procedure to publish an asset from your owned assets to the asset catalog.
**To publish an asset from the SageMaker Assets page**
1. Within Studio, navigate to **Assets**.
1. Select **Owned assets**.
1. Specify the name of your asset in the search bar.
1. Choose the asset.
1. Choose **Publish**.
You can use the following SageMaker Python SDK code to publish a feature group or model package group. The code assumes that you've already created the feature group or model package group.
```
from sagemaker.asset import AssetManager
publisher = AssetPublisher()
publisher.publish_to_catalog(name-of-your-feature-group-or-model-package)
```
## Step 3: Manage access requests
After you've published an asset, users outside of your project might want to access it. You can provide, reject, or revoke access requests. You can also delete assets to only make the underlying source of data only available to yourself.
Use the following procedure to respond to subscription requests.
**To approve subscription requests**
1. Navigate to the **SageMaker Assets** page.
1. Choose **Manage asset assets**.
1. Select **Incoming subscription requests**.
1.
+ (Optional) Choose **Approve** and provide reason.
+ (Optional) Choose **Reject**.
You can revoke access to an asset that you've previously approved. If you choose to revoke access, users lose access to both the asset and the underlying asset. source. Use the following procedure to revoke access.
**To revoke access**
1. Navigate to the **SageMaker Assets** page.
1. Choose **Manage asset assets**.
1. Select **Incoming subscription requests**.
1. Select the **Approved** tab.
1. Choose **Revoke** next to the asset.
You can also unpublish assets, making them only show up as owned assets. The assets won't be visible in the resouce catalog, but the individuals whose subscription requests you've approved can still access them.
**To unpublish an asset**
1. Navigate to the **SageMaker Assets** page.
1. Under **Owned assets**, select the asset that you're unpublishing.
1. Choose **Unpublish**.
You can also delete assets from the same page where you unpublish them. Deleting an asset doesn't delete the source of data. Asset deletion only makes the asset invisible to the other members of your project or organization.
## Step 4: Find assets and request access to them
You can request access to assets that other users have published to the resource catalog. If they approve the subscription request, you get access to the underlying source of data.
At the top of the SageMaker Assets page, you can specify a search query to find assets that other users in your organization have published. You can also select an asset type to view all the published assets of that type. For example, you can select **Glue Table** to view all of the published AWS Glue tables.
You can also view the asset type directly under the name of the asset. The following are the available names for the asset types:
+ Redshift table
+ Glue table
+ Models
+ Feature group
**Note**
Feature groups in the following stores have the type of **Glue table**:
Offline
Offline and online
**To make a subscription request**
1. Navigate to the **SageMaker Assets** page.
1.
+ In the search bar, specify the name of the asset and choose **Search**.
+ For **Types**, select the asset type and find an asset that you're accessing within the resource catalog.
1. Choose the asset.
1. Choose **Subscribe**.
1. Provide a reason for the request.
1. Choose **Submit**.
Your subscription request appears under **Outgoing subscription requests** under **Manage asset requests**. If the publisher of the asset approves your request, it appears under **Subscribed assets**. You can now use the Amazon Redshift, AWS Glue table, or ML source of data in your machine learning workflows.
## Step 5: Use a shared asset in your machine learning workflows
If your subscription request to an asset is approved, you can use it in your machine learning workflows.
The feature groups to which you've been given access appear in your list of feature groups in Studio.
The model groups to which you've been given access appear in your list of model groups in Studio. You can open your model group in model registry from SageMaker Assets. Use the following procedure to open the model group within model registry. **Subscribed assets**.
**To open a model group from SageMaker Assets**
1. Select the model group.
1. Choose **Open in Model Registry**.
You can access AWS Glue or Amazon Redshift tables in Data Wrangler within SageMaker Canvas. SageMaker Canvas is an application that lets you perform exploratory data analysis (EDA) and train models without code. For more information about SageMaker Canvas, see [Amazon SageMaker Canvas](canvas.md).
You can also bring the data from your AWS Glue or Amazon Redshift tables into your Jupyter notebooks by using the SQL extension. You can convert your data into pandas dataframes for your machine learning workflows. For more information, see [Data preparation with SQL in Studio](sagemaker-sql-extension.md).
# Amazon SageMaker Model Dashboard
Amazon SageMaker Model Dashboard is a centralized portal, accessible from the SageMaker AI console, where you can view, search, and explore all of the models in your account. You can track which models are deployed for inference and if they are used in batch transform jobs or hosted on endpoints. If you set up monitors with Amazon SageMaker Model Monitor, you can also track the performance of your models as they make real-time predictions on live data. You can use the dashboard to find models that violate thresholds you set for data quality, model quality, bias and explainability. The dashboard’s comprehensive presentation of all your monitor results helps you quickly identify models that don’t have these metrics configured.
The Model Dashboard aggregates model-related information from several SageMaker AI features. In addition to the services provided in Model Monitor, you can view model cards, visualize workflow lineage, and track your endpoint performance. You no longer have to sort through logs, query in notebooks, or access other AWS services to collect the data you need. With a cohesive user experience and integration into existing services, SageMaker AI’s Model Dashboard provides an out-of-the-box model governance solution to help you ensure quality coverage across all your models.
**Prerequisites**
To use the Model Dashboard, you should have one or more models in your account. You can train models using Amazon SageMaker AI or import models you've trained elsewhere. To create a model in SageMaker AI, you can use the `CreateModel` API. For more information, see [CreateModel](https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_CreateModel.html). You can also use SageMaker AI-provided ML environments, such as Amazon SageMaker Studio Classic, which provides project templates that set up model training and deployment for you. For information about how to get started with Studio Classic, see [Amazon SageMaker Studio Classic](https://docs.aws.amazon.com/sagemaker/latest/dg/studio.htm).
While this is not a mandatory prerequisite, customers gain the most value out of the dashboard if they set up model monitoring jobs using SageMaker Model Monitor for models deployed to endpoints. For prerequisites and instructions on how to use SageMaker Model Monitor, see [Data and model quality monitoring with Amazon SageMaker Model Monitor](model-monitor.md).
## Model Dashboard elements
The Model Dashboard view extracts high-level details from each model to provide a comprehensive summary of every model in your account. If your model is deployed for inference, the dashboard helps you track the performance of your model and endpoint in real time.
Important details to highlight in this page include:
+ **Risk rating**: A user-specified parameter from the model card with a **low**, **medium**, or **high** value. The model card’s risk rating is a categorical measure of the business impact of the model’s predictions. Models are used for a variety of business applications, each of which assumes a different level of risk. For example, incorrectly detecting a cyber attack has much greater business impact than incorrectly categorizing an email. If you don’t know the model risk, you can set it to **unknown**. For information about Amazon SageMaker Model Cards see [Model Cards](https://docs.aws.amazon.com/sagemaker/latest/dg/model-cards.html).
+ Model Monitor alerts: Model Monitor alerts are a primary focus of the Model Dashboard, and reviewing the existing documentation on the various monitors provided by SageMaker AI is a helpful way to get started. For an in-depth explanation of the SageMaker Model Monitor feature and sample notebooks, see [Data and model quality monitoring with Amazon SageMaker Model Monitor](model-monitor.md).
The Model Dashboard displays Model Monitor status values by the following monitor types:
+ *Data Quality*: Compares live data to training data. If they diverge, your model's inferences may no longer be accurate. For additional details about the Data Quality monitor, see [Data quality](model-monitor-data-quality.md).
+ *Model Quality*: Compares the predictions that the model makes with the actual Ground Truth labels that the model attempts to predict. For additional details about the Model Quality monitor, see [Model quality](model-monitor-model-quality.md).
+ *Bias Drift*: Compares the distribution of live data to training data, which can also cause inaccurate predictions. For additional details about the Bias Drift monitor, see [Bias drift for models in production](clarify-model-monitor-bias-drift.md).
+ *Feature Attribution Drift*: Also known as explainability drift. Compares the relative rankings of your features in training data versus live data, which could also be a result of bias drift. For additional details about the Feature Attribution Drift monitor, see [Feature attribution drift for models in production](clarify-model-monitor-feature-attribution-drift.md).
Each Model Monitor status is one of the following values:
+ **None**: No monitor is scheduled
+ **Inactive**: A monitor was scheduled, but it was deactivated
+ **OK**: A monitor is scheduled and is active, and has not encountered the necessary number of violations in recent model monitor executions to raise an alert
+ Time and date: An active monitor raised an alert at the specified time and date
+ **Endpoint**: The endpoints which host your model for real-time inference. Within the Model Dashboard, you can select the endpoint column to view performance metrics such as CPU, GPU, disk, and memory utilization of your endpoints in real time to help you track the performance of your compute instances.
+ **Batch transform job**: The most recent batch transform job that ran using this model. This column helps you determine if a model is actively used for batch inference.
+ Model details: Each entry in the dashboard links to a model details page where you can dive deeper into an individual model. You can access the model’s lineage graph, which visualizes the workflow from data preparation to deployment, and metadata for each step. You can also create and view the model card, review alert details and history, assess the performance of your real-time endpoints, and access other infrastructure-related details.
# Model Monitor schedules and alerts
Using the Python SDK, you can create a model monitor for data quality, model quality, bias drift, or feature attribution drift. For more information about using SageMaker Model Monitor, see [Data and model quality monitoring with Amazon SageMaker Model Monitor](model-monitor.md). The Model Dashboard populates information from all the monitors you create on all your models in your account. You can track the status of each monitor, which indicates whether your monitor is running as expected or failed due to an internal error. You can also activate or deactivate any monitor in the model details page itself. For instructions about how to view scheduled monitors for a model, see [View scheduled monitors](model-dashboard-schedule-view.md). For instructions about how to activate or deactivate model monitors, see [Activate or deactivate a model monitor](model-dashboard-schedule-activate.md).
A properly-configured and actively-running model monitor might raise alerts, in which case the monitoring executions produce violation reports. For details about how alerts work and how to view alert results, history, and links to job reports for debug, see [View and edit alerts](model-dashboard-alerts.md).
# View scheduled monitors
Use SageMaker Model Monitor to continuously monitor your machine learning models for data drift, model quality, bias, and other issues that might impact model performance. After you've set up monitoring schedules, you can view the details of these scheduled monitors through the SageMaker AI console. The following procedure outlines the steps to access and review the scheduled monitors for a particular model, including their current status:
**To view a model’s scheduled monitors**
1. Open the [SageMaker AI console](https://console.aws.amazon.com/sagemaker/).
1. Choose **Governance** in the left panel.
1. Choose **Model Dashboard**.
1. In the **Models** section of the Model Dashboard, select the model name of the scheduled monitors you want to view.
1. View the scheduled monitors in the **Monitor schedule** section. You can review the status for each monitor in the **Status schedule** column, which is one of the following values:
+ **Failed**: The monitoring schedule failed due to a problem with the configuration or settings (such as incorrect user permissions).
+ **Pending**: The monitor is in the process of becoming scheduled.
+ **Stopped**: The schedule is stopped by the user.
+ **Scheduled**: The schedule is created and runs at the frequency you specified.
# Activate or deactivate a model monitor
Use the following procedure to activate or deactivate a model monitor.
**To activate or deactivate a model monitor, complete the following steps:**
1. Open the [SageMaker AI console](https://console.aws.amazon.com/sagemaker/).
1. Choose **Governance** in the left panel.
1. Choose **Model Dashboard**.
1. In the **Models** section of the Model Dashboard, select the model name of the alert you want to modify.
1. Choose the radio box next to the monitor schedule of the alert you want to modify.
1. (optional) Choose **Deactivate monitor schedule** if you want to deactivate your monitor schedule.
1. (optional) Choose **Activate monitor schedule** if you want to activate your monitor schedule.
# View and edit alerts
The Model Dashboard displays alerts you configured in Amazon CloudWatch. You can modify the alert criteria within the dashboard itself. The alert criteria depend upon two parameters:
+ **Datapoints to alert**: Within the evaluation period, how many execution failures raise an alert.
+ **Evaluation period**: The number of most recent monitoring executions to consider when evaluating alert status.
The following image shows an example scenario of a series of Model Monitor executions in which we set a hypothetical **Evaluation period** of 3 and a **Datapoints to alert** value of 2. After every monitoring execution, the number of failures are counted within the **Evaluation period** of 3. If the number of failures meets or exceeds the **Datapoints to alert** value 2, the monitor raises an alert and remains in alert status until the number of failures within the **Evaluation period** becomes less than 2 in subsequent iterations. In the image, the evaluation windows are red when the monitor raises an alert or remains in alert status, and green otherwise.
Note that even if the evaluation window size has not reached the **Evaluation period** of 3, as shown in the first 2 rows of the image, the monitor still raises an alert if the number of failures meets or exceeds the **Datapoints to alert** value of 2.
![\[A sequence of seven example monitoring executions.\]](http://docs.aws.amazon.com/sagemaker/latest/dg/images/model_monitor/model-dashboard-alerts-window.png)
Within the monitor details page, you can view your alert history, edit existing alert criteria, and view job reports to help you debug alert failures. For instructions about how to view alert history or job reports for failed monitoring executions, see [View alert history or job reports](model-dashboard-alerts-view.md). For instructions about how to edit alert criteria, see [Edit alert criteria](model-dashboard-alerts-edit.md).
# View alert history or job reports
**To view alert history or job reports of failed executions, complete the following steps:**
1. Open the [SageMaker AI console](https://console.aws.amazon.com/sagemaker/).
1. Choose **Governance** in the left panel.
1. Choose **Model Dashboard**.
1. In the **Models** section of the Model Dashboard, select the model name of the alert history you want to view.
1. In the **Schedule name** column, select the monitor name of the alert history you want to view.
1. To view alert history, select the **Alert history** tab.
1. (optional) To view job reports of monitoring executions, complete the following steps:
1. In the **Alert history** tab, choose **View executions** for the alert you want to investigate.
1. In the **Execution history** table, choose **View report** of the monitoring execution you want to investigate.
**The report displays the following information:**
+ **Feature**: The user-defined ML feature monitored
+ **Constraint**: The specific check within the monitor
+ **Violation details**: Information about why the constraint was violated
# Edit alert criteria
**To edit an alert in the Model Dashboard, complete the following steps:**
1. Open the [SageMaker AI console](https://console.aws.amazon.com/sagemaker/).
1. Choose **Governance** in the left panel.
1. Choose **Model Dashboard**.
1. In the **Models** section of the Model Dashboard, select the model name of the alert you want to modify.
1. Choose the radio box next to the monitor schedule of the alert you want to modify.
1. Choose **Edit Alert** in the **Monitor schedule** section.
1. (optional) Change **Datapoints to alert** if you want to change the number of failures within the **Evaluation period** that initiate an alert.
1. (optional) Change **Evaluation period** if you want to change the number of most recent monitoring executions to consider when evaluating alert status.
# View a model lineage graph
When you train a model, Amazon SageMaker AI creates a visualization of your entire ML workflow from data preparation to deployment. This visualization is called a model lineage graph. The following page describes how to view a model lineage graph in the SageMaker AI console.
Model lineage graphs use entities to represent individual steps in your workflow. For example, a basic model lineage graph might have an entity representing your training set, which is associated with an entity representing your training job, which is associated with another entity representing your model. In addition, the graph stores information about each step in your workflow. With this information, you can recreate any step in the workflow or track model and dataset lineage. For example, SageMaker AI Lineage stores the S3 URI of your input data sources with each job so you can perform further analysis of the data sources for compliance verification.
While the model lineage graph can help you view the steps in individual workflows, there are many other capabilities that you can leverage using the AWS SDK. For example, with the AWS SDK you can create or query your entities. For more information about the full set of features in SageMaker AI Lineage and example notebooks, see [Amazon SageMaker ML Lineage Tracking](lineage-tracking.md).
# Introduction to entities
Amazon SageMaker AI automatically creates tracking entities for SageMaker AI jobs, models, model packages, and endpoints if the data is available. For a basic workflow, suppose you train a model using a dataset. SageMaker AI automatically generates a lineage graph with three entities:
+ **Dataset** : A type of artifact, which is an entity representing a URI addressable object or data. An artifact is generally either an input or an output to a trial component or action.
+ **TrainingJob**: A type of trial component, which is an entity representing processing, training, and transform jobs.
+ **Model**: Another type of artifact. Like the **Dataset** artifact, a **Model** is a URI addressable object. In this case, it is an output of the **TrainingJob** trial component.
Your model lineage graph expands quickly if you add additional steps to your workflow, such as data preprocessing or postprocessing, if you deploy your model to an endpoint, or if you include your model in a model package, among many other possibilities. For the complete list of SageMaker AI entities, see [Amazon SageMaker ML Lineage Tracking](lineage-tracking.md).
## Entity properties
Each node in the graph displays the entity type, but you can choose the vertical ellipsis to the right of the entity type to see specific details related to your workflow. In our previous barebones lineage graph, you can choose the vertical ellipsis next to **DataSet** to see specific values for the following properties (common to all artifact entities):
+ **Name**: The name of your dataset.
+ **Source URI**: The Amazon S3 location of your dataset.
For the `TrainingJob` entity, you can see the specific values for the following properties (common to all `TrialComponent` entities):
+ **Name**: The name of the training job.
+ **Job ARN**: The Amazon Resource Name (ARN) of your training job.
For the **Model** entity, you see the same properties as listed for **DataSet** since they are both artifact entities. For a list of the entities and their associated properties, see [Lineage Tracking Entities](lineage-tracking-entities.md).
## Entity queries
Amazon SageMaker AI automatically generates graphs of lineage entities as you use them. However if you are running many iterations of an experiment and don't want to view every lineage graph, the AWS SDK can help you perform queries across all your workflows. For example, you can query your lineage entities for all the processing jobs that use an endpoint. Or, you can see all the downstream trails that use an artifact. For a list of all the queries you can perform, see [Querying Lineage Entities](querying-lineage-entities.md).
## View a model’s lineage graph
**To view the lineage graph for a model, complete the following steps:**
1. Open the [SageMaker AI console](https://console.aws.amazon.com/sagemaker/).
1. Choose **Governance** in the left panel.
1. Choose **Model Dashboard**.
1. In the **Models** section of the Model Dashboard, select the model name of the lineage graph you want to view.
1. Choose **View lineage** in the **Model Overview** section.
# View Endpoint Status
If you want to use your trained model to perform inference on live data, you deploy your model to a real-time endpoint. To ensure appropriate latency of your predictions, you want to make sure the instances that host your model are running efficiently. Model Dashboard’s endpoint monitoring feature displays real-time information about your endpoint configuration and helps you track endpoint performance with metrics.
**Monitor settings**
The Model Dashboard links to existing SageMaker AI endpoint details pages which display real-time graphs of metrics you can select in Amazon CloudWatch. Within your dashboard, you can track these metrics as your endpoint is handling real-time inference requests. Some metrics you can select are the following:
+ `CpuUtilization`: The sum of each individual CPU core's utilization, with each ranging from 0%–100%.
+ `MemoryUtilization`: The percentage of memory used by the containers on an instance, ranging from 0%–100%.
+ `DiskUtilization`: The percentage of disk space used by the containers on an instance, ranging from 0%–100%.
For the complete list of metrics you can view in real time, see [Amazon SageMaker AI metrics in Amazon CloudWatch](monitoring-cloudwatch.md).
**Runtime settings**
Amazon SageMaker AI supports automatic scaling (auto scaling) for your hosted models. Auto scaling dynamically adjusts the number of instances provisioned for a model in response to changes in your workload. When the workload increases, auto scaling brings more instances online. When the workload decreases, auto scaling removes unnecessary instances so that you don't pay for provisioned instances that you aren't using. You can customize the following runtime settings in the Model Dashboard:
+ *Update weights*: Change the amount of workload assigned to each instance with numerical weighting. For more information about instance weighting during auto scaling, see [Configure instance weighting for Amazon EC2 Auto Scaling](https://docs.aws.amazon.com/autoscaling/ec2/userguide/ec2-auto-scaling-mixed-instances-groups-instance-weighting.html).
+ *Update instance count*: Change the number of total instances that can service your workload when it increases.
For more information about endpoint runtime settings, see [CreateEndpointConfig](https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_CreateEndpointConfig.html).
**Endpoint configuration settings**
Endpoint configuration settings display the settings you specified when you created the endpoint. These settings inform SageMaker AI which resources to provision for your endpoint. Some settings included are the following:
+ *Data capture*: You can choose to capture information about your endpoint's inputs and outputs. For example, you may want to sample incoming traffic to see if the results correlate to training data. You can customize your sampling frequency, the format of the stored data, and Amazon S3 location of stored data. For more information about setting up your data capture configuration, see [Data capture](model-monitor-data-capture.md).
+ *Production variants*: See the previous discussion in *Runtime settings*.
+ *Async invocation configuration*: If your endpoint is asynchronous, this section includes the maximum number of concurrent requests sent by the SageMaker AI client to the model container, the Amazon S3 location of your success and failure notifications, and the output location of your endpoint outputs. For more information about asynchronous outputs, see [Asynchronous endpoint operations](async-inference-create-invoke-update-delete.md).
+ *Encryption key*: You can enter your encryption key if you want to encrypt your outputs.
For more information about endpoint configuration settings, see [CreateEndpointConfig](https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_CreateEndpointConfig.html).
## View status and configuration for an endpoint
**To view the status and configuration for a model’s endpoint, complete the following steps:**
1. Open the [SageMaker AI console](https://console.aws.amazon.com/sagemaker/).
1. Choose **Governance** in the left panel.
1. Choose **Model Dashboard**.
1. In the **Models** section of the Model Dashboard, select the model name of the endpoint you want to view.
1. Select the endpoint name in the **Endpoints** section.
# Model Dashboard FAQ
Refer to the following FAQ topics for answers to commonly asked questions about Amazon SageMaker Model Dashboard.
## Q. What is Model Dashboard?
Amazon SageMaker Model Dashboard is a centralized repository of all models created in your account. The models are generally the outputs of SageMaker training jobs, but you can also import models trained elsewhere and host them on SageMaker AI. Model Dashboard provides a single interface for IT administrators, model risk managers, and business leaders to track all deployed models and aggregates data from multiple AWS services to provide indicators about how your models are performing. You can view details about model endpoints, batch transform jobs, and monitoring jobs for additional insights into model performance. The dashboard’s visual display helps you quickly identify which models have missing or inactive monitors so you can ensure all models are periodically checked for data drift, model drift, bias drift, and feature attribution drift. Lastly, the dashboard’s ready access to model details helps you dive deep so you can access logs, infrastructure-related information, and resources to help you debug monitoring failures.
## Q. What are the prerequisites to use Model Dashboard?
You should have one or more models created in SageMaker AI, either trained on SageMaker AI or externally trained. While this is not a mandatory prerequisite, you gain the most value from the dashboard if you set up model monitoring jobs via Amazon SageMaker Model Monitor for models deployed to endpoints.
## Q. Who should use Model Dashboard?
Model risk managers, ML practitioners, data scientists and business leaders can get a comprehensive overview of models using the Model Dashboard. The dashboard aggregates and displays data from Amazon SageMaker Model Cards, Endpoints and Model Monitor services to display valuable information such as model metadata from the model card and model registry, endpoints where the models are deployed, and insights from model monitoring.
## Q. How do I use Model Dashboard?
Model Dashboard is available out of the box with Amazon SageMaker AI and does not require any prior configuration. However, if you have set up model monitoring jobs using SageMaker Model Monitor and Clarify, you use Amazon CloudWatch to configure alerts that raise a flag in the dashboard when model performance deviates from an acceptable range. You can create and add new model cards to the dashboard, and view all the monitoring results associated with endpoints. Model Dashboard currently does not support cross-account models.
## Q. What is Amazon SageMaker Model Monitor?
With Amazon SageMaker Model Monitor, you can select the data you want to monitor and analyze without writing any code. SageMaker Model Monitor lets you select data, such as prediction output, from a menu of options and captures metadata such as timestamp, model name, and endpoint so you can analyze model predictions. You can specify the sampling rate of data capture as a percentage of overall traffic in the case of high volume real-time predictions. This data is stored in your own Amazon S3 bucket. You can also encrypt this data, configure fine-grained security, define data retention policies, and implement access control mechanisms for secure access.
## Q. What types of model monitors does SageMaker AI support?
SageMaker Model Monitor provides the following types of [model monitors](https://docs.aws.amazon.com/sagemaker/latest/dg/model-monitor.html):
+ *Data Quality*: Monitor drift in data quality.
+ *Model Quality*: Monitor drift in model quality metrics, such as accuracy.
+ *Bias Drift for Models in Production*: Monitor bias in your model's predictions by comparing the distribution of training and live data.
+ *Feature Attribution Drift for Models in Production*: Monitor drift in feature attribution by comparing the relative rankings of features in training and live data.
## Q. What inference methods does SageMaker Model Monitor support?
Model Monitor currently supports endpoints that host a single model for real-time inference and does not support monitoring of [multi-model endpoints](https://docs.aws.amazon.com/sagemaker/latest/dg/multi-model-endpoints.html).
## Q. How can I get started with SageMaker Model Monitor?
You can use the following resources to get started with model monitoring:
+ [Data quality monitor example notebook](https://github.com/aws/amazon-sagemaker-examples/blob/main/sagemaker_model_monitor/introduction/SageMaker-ModelMonitoring.ipynb)
+ [Model quality monitor example notebook](https://github.com/aws/amazon-sagemaker-examples/blob/main/sagemaker_model_monitor/introduction/SageMaker-ModelMonitoring.ipynb)
+ [Bias drift monitor example notebook](https://github.com/aws/amazon-sagemaker-examples/blob/main/sagemaker_model_monitor/fairness_and_explainability/SageMaker-Model-Monitor-Fairness-and-Explainability.ipynb)
+ [Feature attribution drift monitor example notebook](https://github.com/aws/amazon-sagemaker-examples/blob/main/sagemaker_model_monitor/fairness_and_explainability/SageMaker-Model-Monitor-Fairness-and-Explainability.ipynb)
For more examples of model monitoring, see the GitHub repository [amazon-sagemaker-examples](https://github.com/aws/amazon-sagemaker-examples/tree/main/sagemaker_model_monitor).
## Q. How does Model Monitor work?
Amazon SageMaker Model Monitor automatically monitors machine learning models in production, using rules to detect drift in your model. Model Monitor notifies you when quality issues arise through alerts. To learn more, see [How Amazon SageMaker Model Monitor works](model-monitor.md#model-monitor-how-it-works).
## Q. When and how do you bring your own container (BYOC) for Model Monitor?
Model Monitor computes model metrics and statistics on tabular data only. For use cases other than tabular datasets, such as images or text, you can bring your own containers (BYOC) to monitor your data and models. For example, you can use BYOC to monitor an image classification model that takes images as input and outputs a label. To learn more about container contracts, see [Support for Your Own Containers With Amazon SageMaker Model Monitor](model-monitor-byoc-containers.md).
## Q. Where can I find examples of BYOC for Model Monitor?
You can find helpful BYOC examples in the following links:
+ [Data and model quality monitoring with Amazon SageMaker Model Monitor](model-monitor.md)
+ [GitHub example repository](https://github.com/aws/amazon-sagemaker-examples/tree/master/sagemaker_model_monitor)
+ [Support for Your Own Containers With Amazon SageMaker Model Monitor](model-monitor-byoc-containers.md)
+ [Detecting data drift in NLP using BYOC Model Monitor](https://aws.amazon.com/blogs/machine-learning/detect-nlp-data-drift-using-custom-amazon-sagemaker-model-monitor)
+ [ Detecting and analyzing incorrect predictions in CV ](https://aws.amazon.com/blogs//machine-learning/detecting-and-analyzing-incorrect-model-predictions-with-amazon-sagemaker-model-monitor-and-debugger)
## Q. How do I integrate Model Monitor with Pipelines?
For details about how to integrate Model Monitor and Pipelines, see [ Amazon Pipelines now integrates with SageMaker Model Monitor and SageMaker Clarify ](https://aws.amazon.com/about-aws/whats-new/2021/12/amazon-sagemaker-pipelines-integrates-sagemaker-model-monitor-sagemaker-clarify/).
For an example, see the GitHub sample notebook [Pipelines integration with Model Monitor and Clarify](https://github.com/aws/amazon-sagemaker-examples/blob/main/sagemaker-pipelines/tabular/model-monitor-clarify-pipelines/sagemaker-pipeline-model-monitor-clarify-steps.ipynb).
## Q. Are there any performance concerns using `DataCapture`?
When turned on, data capture occurs asynchronously on the SageMaker AI endpoints. To prevent impact to inference requests, `DataCapture` stops capturing requests at high levels of disk usage. It is recommended you keep your disk utilization below 75% to ensure `DataCapture` continues capturing requests.