# Amazon SageMaker Model Cards Model Cards **Important** Amazon SageMaker Model Card is integrated with SageMaker Model Registry. If you're registering a model within Model Registry, you can use the integration to add auditing information. For more information, see [Update the Details of a Model Version](model-registry-details.md). Use Amazon SageMaker Model Cards to document critical details about your machine learning (ML) models in a single place for streamlined governance and reporting. Model cards can help you to capture key information about your models throughout their lifecycle and implement responsible AI practices. Catalog details such as the intended use and risk rating of a model, training details and metrics, evaluation results and observations, and additional call-outs such as considerations, recommendations, and custom information. By creating model cards, you can do the following: + Provide guidance on how a model should be used. + Support audit activities with detailed descriptions of model training and performance. + Communicate how a model is intended to support business goals. Model cards provide prescriptive guidance on what information to document and include fields for custom information. After creating a model card, you can export it to a PDF or download it to share with relevant stakeholders. Any edits other than an approval status update made to a model card result in additional model card versions in order to have an immutable record of model changes. **Topics** + [ ## Prerequisites ](#model-cards-prerequisites) + [ ## Intended uses of a model ](#model-cards-intended-uses) + [ ## Risk ratings ](#model-cards-risk-rating) + [ ## Model card JSON schema ](#model-cards-json-schema) + [ # Create a model card ](model-cards-create.md) + [ # Model cards actions ](model-cards-manage.md) + [ # Set up cross-account support for Amazon SageMaker Model Cards ](model-cards-xaccount.md) + [ # Low-level SageMaker APIs for model cards ](model-cards-apis.md) + [ # Model card FAQs ](model-cards-faqs.md) ## Prerequisites To get started with Amazon SageMaker Model Cards, you must have permission to create, edit, view, and export model cards. ## Intended uses of a model Specifying the intended uses of a model helps ensure that model developers and users have the information they need to train or deploy the model responsibly. The intended uses of a model should describe the scenarios in which the model is appropriate to use as well as the scenarios in which the model is not recommended to use. We recommend including: + The general purpose of the model + Use cases for which the model was intended + Use cases for which the model was not intended + Assumptions made when developing the model The intended uses of a model go beyond technical details and describe how a model should be used in production, the scenarios in which is appropriate to use a model, and additional considerations such as the type of data to use with the model or any assumptions made during development. ## Risk ratings Developers create ML models for use cases with varying levels of risk. For example, a model that approves loan applications might be a higher risk model than one that detects the category of an email. Given the varied risk profiles of a model, model cards provide a field for you to categorize a model’s risk rating. This risk rating can be `unknown`, `low`, `medium`, or `high`. Use these risk rating fields to label unknown, low, medium, or high-risk models and help your organization comply with any existing rules about putting certain models into production. ## Model card JSON schema Evaluation details for a model card must be provided in JSON format. If you have existing JSON format evaluation reports generated by [SageMaker Clarify](https://docs.aws.amazon.com/sagemaker/latest/dg/clarify-processing-job-run.html) or [SageMaker AI Model Monitor](https://docs.aws.amazon.com/sagemaker/latest/dg/model-monitor-model-quality.html), upload them to Amazon S3 and provide an S3 URI to automatically parse evaluation metrics. For more information and sample reports, see the [example metrics](https://github.com/aws/amazon-sagemaker-examples/tree/main/sagemaker_model_governance/example_metrics) folder in the *Amazon SageMaker Model Governance - Model Cards* example notebook. When creating a model card using the SageMaker Python SDK, model content must be in the model card JSON schema and provided as a string. Provide model content similar to the following example. ### Model card JSON schema sample file ``` { "$schema": "http://json-schema.org/draft-07/schema#", "$id": "http://json-schema.org/draft-07/schema#", "title": "SageMakerModelCardSchema", "description": "Internal model card schema for SageMakerRepositoryService without model_package_details", "version": "0.1.0", "type": "object", "additionalProperties": false, "properties": { "model_overview": { "description": "Overview about the model", "type": "object", "additionalProperties": false, "properties": { "model_description": { "description": "description of model", "type": "string", "maxLength": 1024 }, "model_creator": { "description": "Creator of model", "type": "string", "maxLength": 1024 }, "model_artifact": { "description": "Location of the model artifact", "type": "array", "maxContains": 15, "items": { "type": "string", "maxLength": 1024 } }, "algorithm_type": { "description": "Algorithm used to solve the problem", "type": "string", "maxLength": 1024 }, "problem_type": { "description": "Problem being solved with the model", "type": "string" }, "model_owner": { "description": "Owner of model", "type": "string", "maxLength": 1024 } } }, "intended_uses": { "description": "Intended usage of model", "type": "object", "additionalProperties": false, "properties": { "purpose_of_model": { "description": "Why the model was developed?", "type": "string", "maxLength": 2048 }, "intended_uses": { "description": "intended use cases", "type": "string", "maxLength": 2048 }, "factors_affecting_model_efficiency": { "type": "string", "maxLength": 2048 }, "risk_rating": { "description": "Risk rating for model card", "$ref": "#/definitions/risk_rating" }, "explanations_for_risk_rating": { "type": "string", "maxLength": 2048 } } }, "business_details": { "description": "Business details of model", "type": "object", "additionalProperties": false, "properties": { "business_problem": { "description": "What business problem does the model solve?", "type": "string", "maxLength": 2048 }, "business_stakeholders": { "description": "Business stakeholders", "type": "string", "maxLength": 2048 }, "line_of_business": { "type": "string", "maxLength": 2048 } } }, "training_details": { "description": "Overview about the training", "type": "object", "additionalProperties": false, "properties": { "objective_function": { "description": "the objective function the model will optimize for", "function": { "$ref": "#/definitions/objective_function" }, "notes": { "type": "string", "maxLength": 1024 } }, "training_observations": { "type": "string", "maxLength": 1024 }, "training_job_details": { "type": "object", "additionalProperties": false, "properties": { "training_arn": { "description": "SageMaker Training job arn", "type": "string", "maxLength": 1024 }, "training_datasets": { "description": "Location of the model datasets", "type": "array", "maxContains": 15, "items": { "type": "string", "maxLength": 1024 } }, "training_environment": { "type": "object", "additionalProperties": false, "properties": { "container_image": { "description": "SageMaker training image uri", "type": "array", "maxContains": 15, "items": { "type": "string", "maxLength": 1024 } } } }, "training_metrics": { "type": "array", "items": { "maxItems": 50, "$ref": "#/definitions/training_metric" } }, "user_provided_training_metrics": { "type": "array", "items": { "maxItems": 50, "$ref": "#/definitions/training_metric" } }, "hyper_parameters": { "type": "array", "items": { "maxItems": 100, "$ref": "#/definitions/training_hyper_parameter" } }, "user_provided_hyper_parameters": { "type": "array", "items": { "maxItems": 100, "$ref": "#/definitions/training_hyper_parameter" } } } } } }, "evaluation_details": { "type": "array", "default": [], "items": { "type": "object", "required": [ "name" ], "additionalProperties": false, "properties": { "name": { "type": "string", "pattern": ".{1,63}" }, "evaluation_observation": { "type": "string", "maxLength": 2096 }, "evaluation_job_arn": { "type": "string", "maxLength": 256 }, "datasets": { "type": "array", "items": { "type": "string", "maxLength": 1024 }, "maxItems": 10 }, "metadata": { "description": "additional attributes associated with the evaluation results", "type": "object", "additionalProperties": { "type": "string", "maxLength": 1024 } }, "metric_groups": { "type": "array", "default": [], "items": { "type": "object", "required": [ "name", "metric_data" ], "properties": { "name": { "type": "string", "pattern": ".{1,63}" }, "metric_data": { "type": "array", "items": { "anyOf": [ { "$ref": "#/definitions/simple_metric" }, { "$ref": "#/definitions/linear_graph_metric" }, { "$ref": "#/definitions/bar_chart_metric" }, { "$ref": "#/definitions/matrix_metric" } ] } } } } } } } }, "additional_information": { "additionalProperties": false, "type": "object", "properties": { "ethical_considerations": { "description": "Any ethical considerations that the author wants to provide", "type": "string", "maxLength": 2048 }, "caveats_and_recommendations": { "description": "Caveats and recommendations for people who might use this model in their applications.", "type": "string", "maxLength": 2048 }, "custom_details": { "type": "object", "additionalProperties": { "$ref": "#/definitions/custom_property" } } } } }, "definitions": { "source_algorithms": { "type": "array", "minContains": 1, "maxContains": 1, "items": { "type": "object", "additionalProperties": false, "required": [ "algorithm_name" ], "properties": { "algorithm_name": { "description": "The name of an algorithm that was used to create the model package. The algorithm must be either an algorithm resource in your SageMaker account or an algorithm in AWS Marketplace that you are subscribed to.", "type": "string", "maxLength": 170 }, "model_data_url": { "description": "The Amazon S3 path where the model artifacts, which result from model training, are stored.", "type": "string", "maxLength": 1024 } } } }, "inference_specification": { "type": "object", "additionalProperties": false, "required": [ "containers" ], "properties": { "containers": { "description": "Contains inference related information which were used to create model package.", "type": "array", "minContains": 1, "maxContains": 15, "items": { "type": "object", "additionalProperties": false, "required": [ "image" ], "properties": { "model_data_url": { "description": "The Amazon S3 path where the model artifacts, which result from model training, are stored.", "type": "string", "maxLength": 1024 }, "image": { "description": "Inference environment path. The Amazon EC2 Container Registry (Amazon ECR) path where inference code is stored.", "type": "string", "maxLength": 255 }, "nearest_model_name": { "description": "The name of a pre-trained machine learning benchmarked by Amazon SageMaker Inference Recommender model that matches your model.", "type": "string" } } } } } }, "risk_rating": { "description": "Risk rating of model", "type": "string", "enum": [ "High", "Medium", "Low", "Unknown" ] }, "custom_property": { "description": "Additional property in section", "type": "string", "maxLength": 1024 }, "objective_function": { "description": "objective function that training job is optimized for", "additionalProperties": false, "properties": { "function": { "type": "string", "enum": [ "Maximize", "Minimize" ] }, "facet": { "type": "string", "maxLength": 63 }, "condition": { "type": "string", "maxLength": 63 } } }, "training_metric": { "description": "training metric data", "type": "object", "required": [ "name", "value" ], "additionalProperties": false, "properties": { "name": { "type": "string", "pattern": ".{1,255}" }, "notes": { "type": "string", "maxLength": 1024 }, "value": { "type": "number" } } }, "training_hyper_parameter": { "description": "training hyper parameter", "type": "object", "required": [ "name" ], "additionalProperties": false, "properties": { "name": { "type": "string", "pattern": ".{1,255}" }, "value": { "type": "string", "pattern": ".{0,255}" } } }, "linear_graph_metric": { "type": "object", "required": [ "name", "type", "value" ], "additionalProperties": false, "properties": { "name": { "type": "string", "pattern": ".{1,255}" }, "notes": { "type": "string", "maxLength": 1024 }, "type": { "type": "string", "enum": [ "linear_graph" ] }, "value": { "anyOf": [ { "type": "array", "items": { "type": "array", "items": { "type": "number" }, "minItems": 2, "maxItems": 2 }, "minItems": 1 } ] }, "x_axis_name": { "$ref": "#/definitions/axis_name_string" }, "y_axis_name": { "$ref": "#/definitions/axis_name_string" } } }, "bar_chart_metric": { "type": "object", "required": [ "name", "type", "value" ], "additionalProperties": false, "properties": { "name": { "type": "string", "pattern": ".{1,255}" }, "notes": { "type": "string", "maxLength": 1024 }, "type": { "type": "string", "enum": [ "bar_chart" ] }, "value": { "anyOf": [ { "type": "array", "items": { "type": "number" }, "minItems": 1 } ] }, "x_axis_name": { "$ref": "#/definitions/axis_name_array" }, "y_axis_name": { "$ref": "#/definitions/axis_name_string" } } }, "matrix_metric": { "type": "object", "required": [ "name", "type", "value" ], "additionalProperties": false, "properties": { "name": { "type": "string", "pattern": ".{1,255}" }, "notes": { "type": "string", "maxLength": 1024 }, "type": { "type": "string", "enum": [ "matrix" ] }, "value": { "anyOf": [ { "type": "array", "items": { "type": "array", "items": { "type": "number" }, "minItems": 1, "maxItems": 20 }, "minItems": 1, "maxItems": 20 } ] }, "x_axis_name": { "$ref": "#/definitions/axis_name_array" }, "y_axis_name": { "$ref": "#/definitions/axis_name_array" } } }, "simple_metric": { "description": "metric data", "type": "object", "required": [ "name", "type", "value" ], "additionalProperties": false, "properties": { "name": { "type": "string", "pattern": ".{1,255}" }, "notes": { "type": "string", "maxLength": 1024 }, "type": { "type": "string", "enum": [ "number", "string", "boolean" ] }, "value": { "anyOf": [ { "type": "number" }, { "type": "string", "maxLength": 63 }, { "type": "boolean" } ] }, "x_axis_name": { "$ref": "#/definitions/axis_name_string" }, "y_axis_name": { "$ref": "#/definitions/axis_name_string" } } }, "axis_name_array": { "type": "array", "items": { "type": "string", "maxLength": 63 } }, "axis_name_string": { "type": "string", "maxLength": 63 } } } ``` # Create a model card **Important** Custom IAM policies that allow Amazon SageMaker Studio or Amazon SageMaker Studio Classic to create Amazon SageMaker resources must also grant permissions to add tags to those resources. The permission to add tags to resources is required because Studio and Studio Classic automatically tag any resources they create. If an IAM policy allows Studio and Studio Classic to create resources but does not allow tagging, "AccessDenied" errors can occur when trying to create resources. For more information, see [Provide permissions for tagging SageMaker AI resources](security_iam_id-based-policy-examples.md#grant-tagging-permissions). [AWS managed policies for Amazon SageMaker AI](security-iam-awsmanpol.md) that give permissions to create SageMaker resources already include permissions to add tags while creating those resources. You can create an Amazon SageMaker Model Card using either the SageMaker AI console or the SageMaker Python SDK. You can also use the API operations directly. For more information about the API operations, see [Low-level SageMaker APIs for model cards](model-cards-apis.md). ## Create a model card using the SageMaker AI console Go to the Amazon SageMaker AI console. In the navigation pane, under **Governance**, choose **Model cards**. On the upper right-hand corner, choose **Create model card**. Go through the four steps in the **Create model card** prompt to document details about your model. ### Step 1: Enter model details and intended use If your model is an AWS resource, specify the exact model name in this field to auto-populate model details. To browse existing model names, see **Models** in the Amazon SageMaker AI console. Each unique model name can have only one associated model card. If your model is not an AWS resource, provide a unique name for your model. To add a model as an AWS resource, see [Create a model](https://docs.aws.amazon.com/sagemaker/latest/dg/realtime-endpoints-deployment.html#realtime-endpoints-deployment-create-model) in the *Amazon SageMaker AI Developer Guide*. Alternatively, you can add your model as a model package using [SageMaker AI Marketplace](https://docs.aws.amazon.com/sagemaker/latest/dg/sagemaker-marketplace.html) or [SageMaker AI Model Registry](https://docs.aws.amazon.com/sagemaker/latest/dg/model-registry.html). For more information on intended uses, see [Intended uses of a model](model-cards.md#model-cards-intended-uses). For more information on risk ratings, see [Risk ratings](model-cards.md#model-cards-risk-rating). #### Step 2: Enter training details Add any training details, training observations, datasets, hyperparameters, and details about the model's objective function to the model card. The objective function in a model card can be any function that is optimized during training. This can include, but is not limited to, cost functions, loss functions, or objective metrics. In this section, document the objective function that is most critical to training your model. We recommend that you catalog the following attributes of your objective function: + Optimization direction + Metric + Description For example, you might minimize (optimization direction) cross entropy loss (metric) for a binary classification problem (description) or maximize likelihood for logistic regression. Additionally, you can provide notes about why you chose this objective function over others. #### Step 3: Enter evaluation details If you have existing evaluation reports generated by SageMaker Clarify or Model Monitor, either provide an S3 URI for those reports or upload them manually to add them to the model card. For more information on SageMaker Clarify, see [Run SageMaker Clarify Processing Jobs for Bias Analysis and Explainability](https://docs.aws.amazon.com/sagemaker/latest/dg/clarify-processing-job-run.html). For more information on monitoring drift in model quality metrics using Model Monitor, see [Monitor model quality](https://docs.aws.amazon.com/sagemaker/latest/dg/model-monitor-model-quality.html). To add your own evaluation report, choose **Generic model card evaluation**. All model card evaluation reports must be in the [Model card JSON schema](model-cards.md#model-cards-json-schema). #### Step 4: Enter additional details Add custom model card detail fields for any additional information that you want to address on your model card. For example, you might include the custom field *Line of business* with a value of *Personal finance*. #### Save model card After reviewing the information in your model card, choose **Save** in the lower right-hand corner to save your model card. ## Create a model card using the SageMaker Python SDK Before creating a model card, you must first define the content of your model card. When using the SageMaker Python SDK, model content consists of a model overview, training details, intended uses, evaluation details, and additional information. You can create model cards for: + Models that are hosted within SageMaker AI + Model packages (models) within the SageMaker Model Registry + Models that are hosted or registered outside of SageMaker AI You can also create model cards without associating any models to them. We recommend adding the models that you've trained to the SageMaker Model Registry. The model registry helps you catalog models and track model versions. When you create a model card, the information about the model from the model registry automatically populates the model card. You can edit the model card or add information to it after you create it. For information about using the model registry, see [Model Registration Deployment with Model Registry](model-registry.md). For information about creating a model card from a model registry, see [Create a model card for your model in the SageMaker Model Registry](#model-cards-sdk-create-model-registry). **Note** To use model cards with the SageMaker Python SDK, you first need to establish a SageMaker AI Session. For more information, see [Session](https://sagemaker.readthedocs.io/en/stable/api/utility/session.html) in the SageMaker Python SDK API reference. To create a model card for models that aren't in the SageMaker Model Registry, see [Create a model that isn't in the model registry](#model-card-sdk-create-model-non-model-registry). ### Create a model that isn't in the model registry Use the information in the following sections to create a model card for a model that you haven't added to the model registry. #### Step 1: Define model overview Define an overview of your model. ``` model_overview = ModelOverview.from_model_name( model_name=model_name, sagemaker_session=sagemaker_session, model_description="A-description-of-your-model", problem_type="Problem-type", # For example, "Binary Classification" algorithm_type="Algorithm-type", # For example, "Logistic Regression" model_creator="Name-of-model-creator", model_owner="Name-of-model-owner", ) ``` If your model is an AWS resource, then overview information such as the model ARN, inference container URI, and the S3 location of model artifacts is automatically retrievable. Print the associated AWS metadata with the following commands: ``` print(model_overview.model_id) print(model_overview.inference_environment.container_image) print(model_overview.model_artifact) ``` #### Step 2: Define training details To define your model's training details, you must first define its objective function. ``` objective_function = ObjectiveFunction( function=Function( function=ObjectiveFunctionEnum.MINIMIZE, facet=FacetEnum.LOSS, ), notes="An-explanation-about-objective-function", ) ``` Next, you can define your training details using your existing model overview, session, and objective function. Add any training observations here. ``` training_details = TrainingDetails.from_model_overview( model_overview=model_overview, sagemaker_session=sagemaker_session, objective_function=objective_function, training_observations="Model-training-observations", ) ``` Once again, if your model is an AWS resource, certain training details are autopopulated. Print the training job ARN, training container URI, and training metrics with the following commands: ``` print(training_details.training_job_details.training_arn) print(training_details.training_job_details.training_environment.container_image) print([{"name": i.name, "value": i.value} for i in training_details.training_job_details.training_metrics]) ``` #### Define evaluation details To define your model's evaluation details, you must first define one or more metric groups to describe metrics used for any evaluation jobs. ``` my_metric_group = MetricGroup( name="binary classification metrics", metric_data=[Metric(name="accuracy", type=MetricTypeEnum.NUMBER, value=0.5)] ) ``` Next, you can define your evaluation details using evaluation metrics and datasets for each evaluation job. Add any evaluation observations here and give your evaluation job a unique name. ``` evaluation_details = [ EvaluationJob( name="Example-evaluation-job", evaluation_observation="Evaluation-observations", datasets=["s3://path/to/evaluation/data"], metric_groups=[my_metric_group], ) ] ``` If you have existing evaluation reports generated by [SageMaker AI Clarify](https://docs.aws.amazon.com/sagemaker/latest/dg/clarify-processing-job-run.html) or [SageMaker AI Model Monitor](https://docs.aws.amazon.com/sagemaker/latest/dg/model-monitor-model-quality.html), upload them to Amazon S3 and provide an S3 URI to automatically parse evaluation metrics. To add your own generic model card evaluation report, provide a report in the [evaluation results JSON format](https://docs.aws.amazon.com/sagemaker/latest/dg/model-cards-json-schema.html). ``` report_type = "clarify_bias.json" example_evaluation_job.add_metric_group_from_json( f"example_metrics/{report_type}", EvaluationMetricTypeEnum.CLARIFY_BIAS ) ``` #### Step 3: Define intended uses Define the intended uses of the model, including the general purpose of the model and the use cases for which it was intended. It is also recommended to include any factors that potentially a this model's efficacy in a particular use case and your organization's risk rating of the model. For more information, see [Intended uses of a model](model-cards.md#model-cards-intended-uses)and [Risk ratings](model-cards.md#model-cards-risk-rating). ``` intended_uses = IntendedUses( purpose_of_model="Purpose-of-the-model", intended_uses="The-intended-uses-of-this-model", factors_affecting_model_efficiency="Any-factors-effecting-model-efficacy", risk_rating=RiskRatingEnum.LOW, explanations_for_risk_rating="Explanation-for-low-risk-rating", ) ``` #### Define additional information Lastly, you can add additional custom information to your model card. You can document any ethical considerations, caveats, and recommendations about the model. You can also add any custom details that you would like in the form of key-value pairs. ``` additional_information = AdditionalInformation( ethical_considerations="Any-ethical-considerations", caveats_and_recommendations="Any-caveats-and-recommendations", custom_details={"custom details1": "details-value"}, ) ``` #### Step 4: Create model card Name your model card, define a model card, and then use that definition to create a model card using the SageMaker Python SDK. ``` model_card_name = "my-model-card" my_card = ModelCard( name=model_card_name, status=ModelCardStatusEnum.DRAFT, model_overview=model_overview, training_details=training_details, intended_uses=intended_uses, evaluation_details=evaluation_details, additional_information=additional_information, sagemaker_session=sagemaker_session, ) my_card.create() ``` ### Create a model card for your model in the SageMaker Model Registry Before you begin creating a model card, make sure that you've created a model package group and a model package. For more information about using model registry, see [Model Registration Deployment with Model Registry](model-registry.md). **Important** You must have permissions to use the operations in SageMaker Model Registry. We recommend using `AmazonSageMakerModelRegistryFullAccess` AWS managed policy. For more information about the managed policy, see [AWS Managed Policies for Model Registry](security-iam-awsmanpol-model-registry.md). Use the SageMaker Python SDK to create a model card for a model package within the SageMaker Model Registry. A model package is a model that you've trained. When you create a model card, Amazon SageMaker Model Cards automatically imports the data from the model package into the model card. When you create a model card for a model package, Amazon SageMaker Model Card uses the [DescribeModelPackage](https://docs.aws.amazon.com/APIReference/API_DescribeModelPackage.html#API_DescribeModelPackage_ResponseSyntax) operation to add the data from the model package to the model card. The following are examples of the fields that can be imported from a model package into a model card: + [ModelDataUrl](https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_ModelPackageContainerDefinition.html#sagemaker-Type-ModelPackageContainerDefinition-ModelDataUrl) + [ModelPackageDescription](https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_DescribeModelPackage.html#sagemaker-DescribeModelPackage-response-ModelPackageDescription) + [ModelPackageGroupName](https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_DescribeModelPackage.html#sagemaker-DescribeModelPackage-response-ModelPackageGroupName) + [ModelPackageStatus](https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_DescribeModelPackage.html#sagemaker-DescribeModelPackage-response-ModelPackageStatus) + [ModelPackageVersion](https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_DescribeModelPackage.html#sagemaker-DescribeModelPackage-response-ModelPackageVersion) Use the following code to define the model package and create a model card from it: ``` mp_details = ModelPackage.from_model_package_arn( model_package_arn="example_model_package_arn", sagemaker_session=sagemaker_session, ) model_card_name = "example-model-card" my_card = ModelCard( name=model_card_name, status=ModelCardStatusEnum.status, model_package_details=mp_details, sagemaker_session=sagemaker_session, ) my_card.create() ``` For `status`, you're specifying the approval status of the model card. If you don't specify a status, SageMaker Model Cards uses the default value of `DRAFT`. If you don't specify a SageMaker AI session, SageMaker Model Cards uses the default SageMaker AI session. You must specify a name for the model and the Amazon Resource Name (ARN) of the model package. For information about getting the Amazon Resource Name (ARN) for the model package, see [View and Update the Details of a Model Version (Boto3)](model-registry-details.md#model-registry-details-api). The model card that you've created from the model package might have information that is either missing or inaccurate. You can add information to the model card or edit it. For more information about managing your model cards, see [Model cards actions](model-cards-manage.md). SageMaker Model Registry supports versioning of your model packages. You can version your model package and create a model card for each version. The information from model cards of preceding versions carry over to model cards created from subsequent versions. For example, you could have version 1, version 2, and version 3 of a model package. Suppose you've already created a model card for version 1, but you haven't created one for version 2. If you create a model card for version 3, Amazon SageMaker Model Cards automatically carries over the information from the model card for version 1 to the model card for version 3. **Note** You can also create model cards for model packages that don't use versioning. However, most machine learning workflows involve multiple versions of the same model, so we recommend doing the following: Creating a version for each model package Creating a model card for each version of the model package # Model cards actions After you've created a model card, you can manage them. Managing model cards include the following actions: + Editing a model card + Deleting a model card + Exporting a model card to a PDF You can manage using either the Amazon SageMaker AI console or the SageMaker Python SDK. For more information about using the Python SDK see [Amazon SageMaker Model Cards](https://sagemaker.readthedocs.io/en/stable/api/governance/model_card.html) in the SageMaker Python SDK API reference. For example notebook using the SageMaker Python SDK, see the *[Amazon SageMaker Model Governance - Model Card](https://github.com/aws/amazon-sagemaker-examples/tree/main/sagemaker_model_governance/model_card.ipynb)* example notebook. **Topics** + [ # Edit a model card ](model-cards-console-edit.md) + [ # Export a model card ](model-cards-console-export.md) + [ # Delete a model card ](model-cards-console-delete.md) # Edit a model card To edit a model card, navigate to the model card of your choice by selecting its name in the Amazon SageMaker Model Card console and choose **Edit**. After you save a model card, you cannot edit the name of that model card. After you save a model card version, you cannot update that version of the model card. Any edits that you need to make are saved as a subsequent version in order to have an immutable record of model changes. To view different versions of the model card, choose **Actions**, **Select version**, and then choose the version that you want to view. You can edit a model card using the `model_card.update()` method. Updating a model card creates a new model card version in order to have an immutable record of model changes. You cannot update the name of a model card. ``` my_card.model_overview.model_description = "updated-model-decription" my_card.update() ``` # Export a model card Follow these steps to export a model card. 1. Go to the Amazon SageMaker Model Card console. 1. Choose the name of the model card you want to export. 1. In the model card overview, choose **Actions** and then **Export PDF**. 1. Enter an S3 URI or browse available S3 buckets for your model card PDF. 1. If your model card exports successfully, you can either choose **Download PDF** in the resulting banner or download your PDF directly from Amazon S3. You can export a model card in the SageMaker Python SDK by specifying an S3 output path and export your model card PDF to it with the following commands: ``` s3_output_path = f"s3://{bucket}/{prefix}/export" pdf_s3_url = my_card.export_pdf(s3_output_path=s3_output_path).delete() ``` # Delete a model card Follow these steps to permanently delete one or more model cards. 1. Go to the Amazon SageMaker Model Cards console. 1. Check the box to the left of the name of the card(s) you want to delete. 1. Choose **Delete** in the upper right-hand corner. 1. Confirm your request to permanently delete one or more cards.. You can also delete a model card when viewing the model card overview in the console by choosing **Actions** and then **Delete model card**. Within the SageMaker Python SDK, you can permanently delete a model card with the following command: ``` my_card.delete() ``` # Set up cross-account support for Amazon SageMaker Model Cards Set up cross-account support Use cross-account support in Amazon SageMaker Model Cards to share model cards between AWS accounts. The account where the model cards are created is the *model card account*. Users in the model card account share them with the *shared accounts*. The users in a shared account can update the model cards or create PDFs of them. Users in the model card account share their model cards through AWS Resource Access Manager (AWS RAM). AWS RAM helps you share resources across AWS accounts. For an introduction to AWS RAM, see [What is AWS Resource Access Manager?](https://docs.aws.amazon.com/ram/latest/userguide/what-is.html) The following is the process to share model cards: 1. A user in the model card account sets up the cross-account model card sharing using AWS Resource Access Manager. 1. If the model cards are encrypted with AWS KMS keys, the user setting up model sharing must also provide users in the shared account with AWS KMS permissions. 1. A user in the shared account accepts the invite to the resource share. 1. A user in the shared account provides the other users with permissions to access the model cards. If you're a user in the model card account, see the following sections: + [Set up cross-account model card sharing](#model-cards-xaccount-set-up) + [Set up AWS KMS permissions for the shared account](#model-cards-xaccount-kms) + [Get responses to your resource share invitation](#model-cards-xaccount-set-up-responses) If you're a user in the shared account, see [Set up IAM user permissions in the shared account](#model-cards-xaccount-shared-account-permissions) about setting up permissions for yourself and the other users in the account. ## Set up cross-account model card sharing Set up cross-account model card sharing Use AWS Resource Access Manager (AWS RAM) to grant users in your AWS account access to view or update model cards created in a different AWS account. To set up model card sharing, you must create a resource share. A resource share specifies: + The resources being shared + Who or what has access to the resources + Managed permissions for the resources For more information about resource shares, see [Terms and concepts for AWS RAM](https://docs.aws.amazon.com/ram/latest/userguide/getting-started-terms-and-concepts.html). We recommend taking the time to understand AWS RAM conceptually before you go through the process of creating a resource share. **Important** You must have permissions to create a resource share. For more information about permissions, see [How AWS RAM works with IAM](https://docs.aws.amazon.com/ram/latest/userguide/security-iam-policies.html). For procedures to create a resource share and additional information about them, see [Create a resource share](https://docs.aws.amazon.com/ram/latest/userguide/getting-started-sharing.html#getting-started-sharing-create). When you go through the procedure of creating a resource share, you specify `sagemaker:ModelCard` as the resource type. You must also specify the Amazon Resource Number (ARN) of the AWS RAM resource-based policy. You can specify either the default policy or the policy that has additional permissions to create a PDF of the model card. With the default `AWSRAMPermissionSageMakerModelCards` resource-based policy, the users in the shared account have permissions to do the following operations: + [DescribeModelCard](https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_DescribeModelCard.html) + [ListModelCardVersions](https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_ListModelCardVersions.html) + [UpdateModelCard](https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_UpdateModelCard.html) With the `AWSRAMPermissionSageMakerModelCardsAllowExport` resource-based policy, the users in the shared account have permissions to do all of the preceding actions. They also have permissions to create a model card export job and describe it through the following operations: + [CreateModelCardExportJob](https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_CreateModelCardExportJob.html) + [DescribeModelCardExportJob](https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_DescribeModelCardExportJob.html) The users in the shared account can create an export job to generate a PDF of a model card. They can also describe an export job that has been created to find the PDF's Amazon S3 URI. Model cards and export jobs are resources. The model card account owns the export jobs created by a user in the shared account. For example, a user in account A shares model card X with shared account B. A user in account B creates export job Y for model card X that stores the output in an Amazon S3 location that the user in account B specifies. Even though account B created export job Y, it belongs to account A. Each AWS account has resource quotas. For information about quotas related to model cards, see [Amazon SageMaker AI endpoints and quotas](https://docs.aws.amazon.com/general/latest/gr/sagemaker.html#limits_sagemaker). ### Set up AWS KMS permissions for the shared account If the model cards that you're sharing have been encrypted with AWS Key Management Service keys, you also need to share the access to the keys with the shared account. Otherwise, the users in the shared account can't view, update, or export the model cards. For an overview of AWS KMS, see [AWS Key Management Service](https://docs.aws.amazon.com/kms/latest/developerguide/overview.html). To provide AWS KMS permissions to users in the shared account, update your key policy with the following statement: ``` { "Effect": "Allow", "Principal": { "AWS": [ "arn:aws:iam::shared-account-id::role/example-IAM-role" ] }, "Action": [ "kms:GenerateDataKey", "kms:Decrypt", ] "Resource": "arn:aws:kms:AWS-Region-of-model-card-account:model-card-account-id:key/AWS KMS-key-id" "Condition": { "Bool": {"kms:GrantIsForAWSResource": true }, "StringEquals": { "kms:ViaService": [ "sagemaker.AWS-Region.amazonaws.com", "s3.AWS-Region.amazonaws.com" ], }, "StringLike": { "kms:EncryptionContext:aws:sagemaker:model-card-arn": "arn:aws:sagemaker:AWS-Region:model-card-account-id:model-card/model-card-name" } } } ``` The preceding statement provides users in the shared account with `kms:Decrypt` and `kms:GenerateDataKey` permissions. With `kms:Decrypt`, users can decrypt the model cards. With `kms:GenerateDataKey`, users can encrypt the model cards that they update or the PDFs that they create. ### Get responses to your resource share invitation After you've created a resource share, the shared accounts that you've specified in the resource share receive an invitation to join it. They must accept the invite to access the resources. For information about accepting a resource share invite, see [Using shared AWS resources ](https://docs.aws.amazon.com/ram/latest/userguide/getting-started-shared.html)in the *AWS Resource Access Manager User Guide*. ### Set up IAM user permissions in the shared account The following information assumes that you've accepted the resource share invitation from the model card account. For more information about accepting a resource share invitation, see [Using shared AWS resources ](https://docs.aws.amazon.com/ram/latest/userguide/getting-started-shared.html). You and the other users in your account use an IAM role to access the model cards shared from the model card account. Use the following template to change the policy of the IAM role. You can modify the template for your own use case. ------ #### [ JSON ] **** ``` { "Version":"2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "sagemaker:DescribeModelCard", "sagemaker:UpdateModelCard", "sagemaker:CreateModelCardExportJob", "sagemaker:ListModelCardVersions", "sagemaker:DescribeModelCardExportJob" ], "Resource": [ "arn:aws:sagemaker:us-east-1:111122223333:model-card/example-model-card-name-0", "arn:aws:sagemaker:us-east-1:111122223333:model-card/example-model-card-name-1/*" ] }, { "Effect": "Allow", "Action": "s3:PutObject", "Resource": "arn:aws:s3:::amzn-s3-demo-bucket-storing-the-pdf-of-the-model-card/model-card-name/*" } ] } ``` ------ To access model cards encrypted using AWS KMS, you must provide users in your account with the following AWS KMS permissions. ``` { "Effect": "Allow", "Action": [ "kms:GenerateDataKey", "kms:Decrypt", ], "Resource": "arn:aws:kms:AWS-Region:AWS-account-id-where-the-model-card-is-created:key/AWS Key Management Service-key-id" } ``` # Low-level SageMaker APIs for model cards Model card APIs You can create an Amazon SageMaker Model Card directly through the SageMaker API or the AWS Command Line Interface (AWS CLI). **Note** When creating a model card with the low-level APIs, the content must be in the model card JSON schema and provided as a string. For more information, see [Model card JSON schema](model-cards.md#model-cards-json-schema). ## SageMaker API Use the following SageMaker API commands to work with Amazon SageMaker Model Cards: + [CreateModelCard](https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_CreateModelCard.html) + [DescribeModelCard](https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_DescribeModelCard.html) + [ListModelCards](https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_ListModelCards.html) + [ListModelCardVersions](https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_ListModelCardVersions.html) + [UpdateModelCard](https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_UpdateModelCard.html) + [CreateModelCardExportJob](https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_CreateModelCardExportJob.html) + [DescribeModelCardExportJob](https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_DescribeModelCardExportJob.html) + [ListModelCardExportJobs](https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_ListModelCardExportJobs.html) + [DeleteModelCard](https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_DeleteModelCard.html) ## AWS CLI Use the following AWS CLI commands to work with Amazon SageMaker Model Cards: + [create-model-card](https://docs.aws.amazon.com/cli/latest/reference/sagemaker/create-model-card.html) + [describe-model-card](https://docs.aws.amazon.com/cli/latest/reference/sagemaker/describe-model-card.html) + [list-model-cards](https://docs.aws.amazon.com/cli/latest/reference/sagemaker/list-model-cards.html) + [list-model-card-versions](https://docs.aws.amazon.com/cli/latest/reference/sagemaker/list-model-card-versions.html) + [update-model-card](https://docs.aws.amazon.com/cli/latest/reference/sagemaker/update-model-card.html) + [create-model-card-export-job](https://docs.aws.amazon.com/cli/latest/reference/sagemaker/create-model-card-export-job.html) + [describe-model-card-export-job](https://docs.aws.amazon.com/cli/latest/reference/sagemaker/describe-model-card-export-job.html) + [list-model-card-export-jobs](https://docs.aws.amazon.com/cli/latest/reference/sagemaker/list-model-card-export-jobs.html) + [delete-model-card](https://docs.aws.amazon.com/cli/latest/reference/sagemaker/delete-model-card.html) # Model card FAQs Refer to the following FAQ items for answers to commonly asked questions about Amazon SageMaker Model Card. ## Q. What is model risk? A: You can use models for a variety of business applications ranging from predicting cyber attacks and approving loan applications to detecting the category of an email. Each of these applications assumes a different level of risk. For example, incorrectly detecting a cyber attack has much greater business impact than incorrectly categorizing an email. Given these varied risk profiles of a model, you can use model cards to provide a risk rating of `low`, `medium`, or `high` for a model. If you don’t know the risk of your model, you can set the status to `unknown`. Customers are responsible for assigning the risk profile for each model. Based on the risk rating, organizations may have different rules in place for deploying those models to production. For more information, see [Risk ratings](model-cards.md#model-cards-risk-rating). ## Q. What is the intended use of a model? The intended use of a model describes how you should use the model in your production applications. This goes beyond technical requirements like the type of instance to which you should deploy a model and instead refers to the types of applications to create with the model, the scenarios in which you can expect a reasonable performance from the model, or the type of data to use with the model. We recommend providing this information in the model card for better model governance. You can define a kind of model specification in the intended use field and ensure that model developers and consumers follow this specification while training and deploying their models. For more information, see [Intended uses of a model](model-cards.md#model-cards-intended-uses). ## Q. Does SageMaker AI auto-populate information in my model card? When creating a model card using either the SageMaker Python SDK or the AWS console, SageMaker AI automatically populates information about your trained model. This includes comprehensive training details and all model information that's available through the `describe-model` API call. If you work in Amazon SageMaker Studio, you can auto-populate your model cards by calling the [DescribeModelPackage](https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_DescribeModelPackage.html)API. ## Q. Can I customize a model card? Amazon SageMaker Model Cards have a defined structure to them that cannot be modified. This structure gives you guidance on what information should be captured in a model card. While you cannot change the structure of the model card, there is some flexibility introduced through custom properties in the **Additional information** section of the model card. ## Q. Can I edit a model card once it is created? Model cards have versions associated with them. A given model version is immutable across all attributes other than the model card status. If you make any other changes to the model card, such as evaluation metrics, description, or intended uses, SageMaker AI creates a new version of the model card to reflect the updated information. This is to ensure that a model card, once created, cannot be tampered with. Model cards are automatically updated whenever you make changes to your model package versions in the Model Registry. ## Q. Can I create model cards for models that were not trained using SageMaker AI? A: Yes. You can create model cards for models not trained in SageMaker AI, but no information is automatically populated in the card. You must supply all the information needed in the model card for non-SageMaker AI models. ## Q. Can I export or share model cards? A: Yes. You can export each version of a model card to a PDF, downloaded, and share it. ## Q. Do I need to register my model in the Model Registry to use model cards? A: Model cards are accessible through both SageMaker AI Console and Model Registry. When using Model Registry, you will automatically receive a model card for each version of your model. ## Q. What is the difference between model cards and Model Registry? A: Model cards are intended to provide organizations with a mechanism to document as much detail about their model as they like by following SageMaker AI’s prescriptive guidance along with providing their own custom information. You can introduce model cards at the very start of the ML process and use them to define the business problem that the model should solve and any considerations to think about while using the model. After a model is trained, you can populate the model card associated with that model with information about the model and how it was trained. Model cards are associated with models and are immutable once associated with a model. This ensures that the model card is the single source of truth for all the information related to a model, including how it was trained and how it should be used. The Model Registry is a catalog that stores metadata about your models. Each entry in the model registry corresponds to a unique model version. That model version contains information about the model such as where the model artifacts are stored in Amazon S3, what container is needed to deploy the model, and custom metadata that should be attached to the model. Each model package version has a model card associated with it. ## Q. What is the relationship between model cards and Model Registry? A: Model cards are integrated into the Model Registry object. Each version of a model package in the Model Registry is linked to its corresponding model card. You can access the model card schema for each version by using the [ModelPackageModelCard](https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_ModelPackageModelCard.html) API. ## Q. Are model card versions related to model versions in the Model Registry? A: Yes, there is a one-to-one relationship between model cards and models in the Model Registry. Each model version stored in the Model Registry has exactly one corresponding model card associated with it. ## Q. Are model cards integrated with SageMaker Model Monitor? A: No. You can upload the performance metrics computed by SageMaker Model Monitor to the model card by uploading a metrics file to Amazon S3 and linking that to the card, but there is no native integration between Model Monitor and model cards. Model dashboards are integrated with Model Monitor. For more information on model dashboards, see [Amazon SageMaker Model Dashboard](https://docs.aws.amazon.com/sagemaker/latest/dg/model_dashboard.html).