# Amazon SageMaker Partner AI Apps overview Partner AI Apps With Amazon SageMaker Partner AI Apps, users get access to generative AI and machine learning (ML) development applications built, published, and distributed by industry-leading application providers. Partner AI Apps are certified to run on SageMaker AI. With Partner AI Apps, users can accelerate and improve how they build solutions based on foundation models (FM) and classic ML models without compromising the security of their sensitive data. The data stays completely within their trusted security configuration and is never shared with a third party.  ## How it works Partner AI Apps are full application stacks that include an Amazon Elastic Kubernetes Service cluster and an array of accompanying services that can include Application Load Balancer, Amazon Relational Database Service, Amazon Simple Storage Service buckets, Amazon Simple Queue Service queues, and Redis caches. These service applications can be shared across all users in a SageMaker AI domain and are provisioned by an admin. After provisioning the application by purchasing a subscription through the AWS Marketplace, the admin can give users in the SageMaker AI domain permissions to access the Partner AI App directly from Amazon SageMaker Studio, Amazon SageMaker Unified Studio (preview), or using a pre-signed URL. For information about launching an application from Studio, see [Launch Amazon SageMaker Studio](studio-updated-launch.md). Partner AI Apps offers the following benefits for administrators and users.  + Administrators use the SageMaker AI console to browse, discover, select, and provision the Partner AI Apps for use by their data science and ML teams. After the Partner AI Apps are deployed, SageMaker AI runs them on service-managed AWS accounts. This significantly reduces the operational overhead associated with building and operating these applications, and contributes to the security and privacy of customer data. + Data scientists and ML developers can access Partner AI Apps from within their ML development environment in Amazon SageMaker Studio or Amazon SageMaker Unified Studio (preview). They can use the Partner AI Apps to analyze their data, experiments, and models created on SageMaker AI. This minimizes context switching and helps accelerate building foundation models and bringing new generative AI capabilities to market. ## Integration with AWS services Partner AI Apps uses the existing AWS Identity and Access Management (IAM) configuration for authorization and authentication. As a result, users don’t need to provide separate credentials to access each Partner AI App from Amazon SageMaker Studio. For more information about authorization and authentication with Partner AI Apps, see [Set up Partner AI Apps](partner-app-onboard.md). Partner AI Apps also integrates with Amazon CloudWatch to provide operational monitoring and management. Customers can also browse Partner AI Apps, and get details about them, such as features, customer experience, and pricing, from the AWS Management Console. For information about Amazon CloudWatch, see [How Amazon CloudWatch works](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/cloudwatch_architecture.html). Partner AI applications such as Deepchecks support integration with Amazon Bedrock to enable LLM-based evaluation features such as "LLM as a judge" evaluations and automated annotation capabilities. When Amazon Bedrock integration is enabled, the Partner AI App uses your customer-managed Amazon Bedrock account to access foundation models, ensuring that your data remains within your trusted security configuration. For more information about configuring Amazon Bedrock integration, see [Configure Amazon Bedrock integration](partner-app-onboard.md#partner-app-onboard-admin-bedrock). ## Supported types Partner AI Apps support the following types: + Comet + Deepchecks + Fiddler + Lakera Guard When the admin launches a Partner AI App, they must select the configuration of the instance cluster that the Partner AI App is launched with. This configuration is known as the Partner AI App's tier. A Partner AI App's tier can be one of the following values: + `small` + `medium` + `large` The following sections give information about each of the Partner AI App types, and details about the Partner AI App's tier values. ### Comet overview Comet provides an end-to-end model evaluation platform for AI developers, with LLM evaluations, experiment tracking, and production monitoring. We recommend the following Partner AI App tiers based on the workload: + `small` – Recommended for up to 5 users and 20 running jobs. + `medium` – Recommended for up to 50 users and 100 running jobs. + `large` – Recommended for up to 500 users and more than 100 running jobs. **Note** SageMaker AI does not support viewing the Comet UI as part of the output of a Jupyter notebook. ### Deepchecks overview AI application developers and stakeholders can use Deepchecks to continuously validate LLM-based applications including characteristics, performance metrics, and potential pitfalls throughout the entire lifecycle from pre-deployment and internal experimentation to production. We recommend the following Partner AI App tiers based on the speed desired for the workload: + `small` – Processes 200 tokens per second. + `medium` – Processes 500 tokens per second. + `large` – Processes 1300 tokens per second. ### Fiddler overview The Fiddler AI Observability Platform facilitates validating, monitoring, and analyzing ML models in production, including tabular, deep learning, computer vision, and natural language processing models. We recommend the following Partner AI App tiers based on the speed desired for the workload: + `small` – Processing 10MM events across 5 models, 100 features, and 20 iterations takes about 53 minutes. + `medium` – Processing 10MM events across 5 models, 100 features, and 20 iterations takes about 23 minutes. + `large` – Processing 10MM events across 5 models, 100 features, and 100 iterations takes about 27 minutes. ### Lakera Guard overview Lakera Guard is a low-latency AI application firewall to secure generative AI applications from gen AI-specific threats. We recommend the following Partner AI App tiers based on the workload: + `small` – Recommended for up to 20 Robotic Process Automations (RPAs). + `medium` – Recommended for up to 100 RPAs. + `large` – Recommended for up to 200 RPAs. # Set up Partner AI Apps The following topics describe the permissions needed to start using Amazon SageMaker Partner AI Apps. The permissions required are split into two parts, depending on the user permissions level: + **Administrative permissions** – Permissions for administrators setting up data scientist and machine learning (ML) developer environments. + AWS Marketplace + Partner AI Apps management + AWS License Manager + **User permissions** – Permissions for data scientists and machine learning developers. + User authorization + Identity propagation + SDK access ## Prerequisites Admins can complete the following prerequisites to set up Partner AI Apps. + (Optional) Onboard to a SageMaker AI domain. Partner AI Apps can be accessed directly from a SageMaker AI domain. For more information, see [Amazon SageMaker AI domain overview](gs-studio-onboard.md). + If using Partner AI Apps in a SageMaker AI domain in VPC-only mode, admins must create an endpoint with the following format to connect to the Partner AI Apps. For more information about using Studio in VPC-only mode, see [Connect Amazon SageMaker Studio in a VPC to External Resources](studio-updated-and-internet-access.md). ``` aws.sagemaker.region.partner-app ``` + (Optional) If admins are interacting with the domain using the AWS CLI, they must also complete the following prerequisites. 1. Update the AWS CLI by following the steps in [Installing the current AWS CLI Version](https://docs.aws.amazon.com/cli/latest/userguide/install-cliv1.html#install-tool-bundled).  1. From the local machine, run `aws configure` and provide AWS credentials. For information about AWS credentials, see [Understanding and getting your AWS credentials](https://docs.aws.amazon.com/general/latest/gr/aws-sec-cred-types.html). ## Administrative permissions The administrator must add the following permissions to enable Partner AI Apps in SageMaker AI. + Permission to complete AWS Marketplace subscription for Partner AI Apps + Set up Partner AI App execution role ### AWS Marketplace subscription for Partner AI Apps Admins must complete the following steps to add permissions for AWS Marketplace. For information about using AWS Marketplace, see [Getting started as a buyer using AWS Marketplace](https://docs.aws.amazon.com/marketplace/latest/buyerguide/buyer-getting-started.html). 1. Grant permissions for AWS Marketplace. Partner AI Apps administrators require these permissions to purchase subscriptions to Partner AI Apps from AWS Marketplace. To get access to AWS Marketplace, admins must attach the `AWSMarketplaceManageSubscriptions` managed policy to the IAM role that they're using to access the SageMaker AI console and purchase the app. For details about the `AWSMarketplaceManageSubscriptions` managed policy, see [AWS managed policies for AWS Marketplace buyers](https://docs.aws.amazon.com/marketplace/latest/buyerguide/buyer-security-iam-awsmanpol.html#security-iam-awsmanpol-awsmarketplacemanagesubscriptions). For information about attaching managed policies, see [Adding and removing IAM identity permissions](https://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies_manage-attach-detach.html). 1. Grant permissions for SageMaker AI to run operations on the admins behalf using other AWS services. Admins must grant SageMaker AI permissions to use these services and the resources that they act upon. The following policy definition demonstrates how to grant the required Partner AI Apps permissions. These permissions are needed in addition to the existing permissions for the admin role. For more information, see [How to use SageMaker AI execution roles](sagemaker-roles.md). ------ #### [ JSON ] **** ``` { "Version":"2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "sagemaker:CreatePartnerApp", "sagemaker:DeletePartnerApp", "sagemaker:UpdatePartnerApp", "sagemaker:DescribePartnerApp", "sagemaker:ListPartnerApps", "sagemaker:CreatePartnerAppPresignedUrl", "sagemaker:CreatePartnerApp", "sagemaker:AddTags", "sagemaker:ListTags", "sagemaker:DeleteTags" ], "Resource": "*" }, { "Effect": "Allow", "Action": [ "iam:PassRole" ], "Resource": "arn:aws:iam::*:role/*", "Condition": { "StringEquals": { "iam:PassedToService": "sagemaker.amazonaws.com" } } } ] } ``` ------ ### Set up Partner AI App execution role 1. Partner AI Apps require an execution role to interact with resources in the AWS account. Admins can create this execution role using the AWS CLI. The Partner AI App uses this role to complete actions related to Partner AI App functionality. ``` aws iam create-role --role-name PartnerAiAppExecutionRole --assume-role-policy-document '{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": { "Service": [ "sagemaker.amazonaws.com" ] }, "Action": "sts:AssumeRole" } ] }' ``` 1. Create the AWS License Manager service-linked role by following the steps in [Create a service-linked role for License Manager](https://docs.aws.amazon.com/license-manager/latest/userguide/license-manager-role-core.html#create-slr-core).  1. Grant permissions for the Partner AI App to access License Manager using the AWS CLI. These permissions are required to access the licenses for Partner AI App. This allows the Partner AI App to verify access to the Partner AI App license. ``` aws iam put-role-policy --role-name PartnerAiAppExecutionRole --policy-name LicenseManagerPolicy --policy-document '{ "Version": "2012-10-17", "Statement": { "Effect": "Allow", "Action": [ "license-manager:CheckoutLicense", "license-manager:CheckInLicense", "license-manager:ExtendLicenseConsumption", "license-manager:GetLicense", "license-manager:GetLicenseUsage" ], "Resource": "*" } }' ``` 1. If the Partner AI App requires access to an Amazon S3 bucket, then add Amazon S3 permissions to the execution role. For more information, see [Required permissions for Amazon S3 API operations](https://docs.aws.amazon.com/AmazonS3/latest/userguide/using-with-s3-policy-actions.html). ### Configure Amazon Bedrock integration Partner AI applications such as Deepchecks support integration with Amazon Bedrock to enable LLM-based evaluation features. When configuring a Partner AI App with Amazon Bedrock support, administrators can specify which foundation models and inference profiles are available for use within the application. If you need to increase the quota limit for your Amazon Bedrock models, see [Request an increase for Amazon Bedrock quotas](https://docs.aws.amazon.com/bedrock/latest/userguide/quotas-increase.html). 1. Ensure the Partner AI App execution role has the required Amazon Bedrock permissions. Add the following permissions to enable Amazon Bedrock model access: ``` aws iam put-role-policy --role-name PartnerAiAppExecutionRole --policy-name BedrockInferencePolicy --policy-document '{ "Version": "2012-10-17", "Statement": { "Effect": "Allow", "Action": [ "bedrock:InvokeModel", "bedrock:GetFoundationModel", "bedrock:GetInferenceProfile" ], "Resource": "*" } }' ``` 1. Identify the Amazon Bedrock models that your organization wants to make available to the Partner AI App. You can view available models in your region using the Amazon Bedrock console. For information about model availability across regions, see [Model support by AWS Region](https://docs.aws.amazon.com/bedrock/latest/userguide/models-regions.html). 1. (Optional) Create customer-managed inference profiles for cost tracking and model management. Inference profiles allow you to track Amazon Bedrock usage specifically for the Partner AI App and can enable cross-region inference when models are not available in your current region. For more information, see [ Using inference profiles in Amazon Bedrock](https://docs.aws.amazon.com/bedrock/latest/userguide/inference-profiles.html). 1. When creating or updating the Partner AI App, specify the allowed models and inference profiles using the `CreatePartnerApp` or `UpdatePartnerApp` API. The Partner AI App will only be able to access the models and inference profiles that you explicitly configure. **Important** Amazon Bedrock usage through Partner AI Apps is billed directly to your AWS account using your existing Amazon Bedrock pricing. The Partner AI App infrastructure costs are separate from Amazon Bedrock model inference costs. #### Deepchecks Amazon Bedrock integration Deepchecks supports Amazon Bedrock integration for LLM-based evaluation capabilities, including: + *LLM as a judge evaluations* - Use foundation models to automatically evaluate model outputs for quality, relevance, and other criteria + *Automated annotation* - Generate labels and annotations for datasets using foundation models + *Content analysis* - Analyze text data for bias, toxicity, and other quality metrics using LLM capabilities For detailed information about Deepchecks Amazon Bedrock features and configuration, see the Deepchecks documentation within the application. ## User permissions After admins have completed the administrative permissions settings, they must make sure that users have the permissions needed to access the Partner AI Apps. 1. Grant permissions for SageMaker AI to run operations on your behalf using other AWS services. Admins must grant SageMaker AI permissions to use these services and the resources that they act upon. Admins grant SageMaker AI these permissions using an IAM execution role. For more information about IAM roles, see [IAM roles](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles.html). The following policy definition demonstrates how to grant the required Partner AI Apps permissions. This policy can be added to the execution role of the user profile.  For more information, see [How to use SageMaker AI execution roles](sagemaker-roles.md). ------ #### [ JSON ] **** ``` { "Version":"2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "sagemaker:DescribePartnerApp", "sagemaker:ListPartnerApps", "sagemaker:CreatePartnerAppPresignedUrl" ], "Resource": "arn:aws:sagemaker:*:*:partner-app/app-*" } ] } ``` ------ 1. (Optional) If launching Partner AI Apps from Studio, add the `sts:TagSession` trust policy to the role used to launch Studio or the Partner AI Apps directly as follows. This makes sure that the identity can be propagated properly. ``` { "Effect": "Allow", "Principal": { "Service": "sagemaker.amazonaws.com" }, "Action": [ "sts:AssumeRole", "sts:TagSession" ] } ``` 1. (Optional) If using the SDK of a Partner AI App to access functionality in SageMaker AI, add the following `CallPartnerAppApi` permission to the role used to run the SDK code. If running the SDK code from Studio, add the permission to the Studio execution role. If running the code from anywhere other than Studio, add the permission to the IAM role used with the notebook. This gives the user access the Partner AI App functionality from the Partner AI App’s SDK. ------ #### [ JSON ] **** ``` { "Version":"2012-10-17", "Statement": [ { "Sid": "Statement1", "Effect": "Allow", "Action": [ "sagemaker:CallPartnerAppApi" ], "Resource": [ "arn:aws:sagemaker:us-east-1:111122223333:partner-app/app" ] } ] } ``` ------ ### Manage user authorization and authentication To provide access to Partner AI Apps to members of their team, admins must make sure that the identity of their users is propagated to the Partner AI Apps. This propagation makes sure users can properly access the Partner AI Apps' UI and perform authorized Partner AI App actions. Partner AI Apps support the following identity sources: + AWS IAM Identity Center + External identity providers (IdPs)  + IAM Session-based identity The following sections gives information about the identity sources that Partner AI Apps support, as well as important details related to that identity source. #### IAM Identity Center If a user is authenticated into Studio using IAM Identity Center and launches an application from Studio, the IAM Identity Center `UserName` is automatically propagated as the user identity for a Partner AI App. This is not the case if the user launches the Partner AI App directly using the `CreatePartnerAppPresignedUrl` API. #### External identity providers (IdPs) If using SAML for AWS account federation, admins have two options to carry over the IdP identity as the user identity for a Partner AI App. For information about setting up AWS account federation, see [How to Configure SAML 2.0 for AWS account Federation](https://saml-doc.okta.com/SAML_Docs/How-to-Configure-SAML-2.0-for-Amazon-Web-Service).  + **Principal Tag** – Admins can configure the IdP-specific IAM Identity Center application to pass identity information from the landing session using the AWS session `PrincipalTag` with the following `Name` attribute. When using SAML, the landing role session uses an IAM role. To use the `PrincipalTag`, admins must add the `sts:TagSession` permission to this landing role, as well as the Studio execution role. For more information about `PrincipalTag`, see [Configure SAML assertions for the authentication response](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_providers_create_saml_assertions.html#saml_role-session-tags). ``` https://aws.amazon.com/SAML/Attributes/PrincipalTag:SageMakerPartnerAppUser ``` + **Landing session name** – Admins can propagate the landing session name as the identity for the Partner AI App. To do this, they must set the `EnableIamSessionBasedIdentity` opt-in flag for each Partner AI App. For more information, see [`EnableIamSessionBasedIdentity`](#partner-app-onboard-user-iam-session). #### IAM session-based identity **Important** We do not recommend using this method for production accounts. For production accounts, use an identity provider for increased security. SageMaker AI supports the following options for identity propagation when using an IAM session-based identity. All of the options, except using a session tag with AWS STS, require setting the `EnableIamSessionBasedIdentity` opt-in flag for each application. For more information, see [`EnableIamSessionBasedIdentity`](#partner-app-onboard-user-iam-session). When propagating identities, SageMaker AI verifies whether an AWS STS Session tag is being used. If one is not used, then SageMaker AI propagates the IAM username or AWS STS session name. + **AWS STS Session tag** – Admins can set a `SageMakerPartnerAppUser` session tag for the launcher IAM session. When admins launch a Partner AI App using the SageMaker AI console or the AWS CLI, the `SageMakerPartnerAppUser` session tag is automatically passed as the user identity for the Partner AI App. The following example shows how to set the `SageMakerPartnerAppUser` session tag using the AWS CLI. The value of the key is added as a principal tag. ``` aws sts assume-role \ --role-arn arn:aws:iam::account:role/iam-role-used-to-launch-partner-ai-app \ --role-session-name session_name \ --tags Key=SageMakerPartnerAppUser,Value=user-name ``` When giving users access to a Partner AI App using `CreatePartnerAppPresignedUrl`, we recommend verifying the value for the `SageMakerPartnerAppUser` key. This helps to prevent unintended access to Partner AI App resources. The following trust policy verifies that the session tag exactly matches the associated IAM user. Admins can use any principal tag for this purpose. It should be configured on the role that is launching Studio or the Partner AI App. ------ #### [ JSON ] **** ``` { "Version":"2012-10-17", "Statement": [ { "Sid": "RoleTrustPolicyRequireUsernameForSessionName", "Effect": "Allow", "Action": [ "sts:AssumeRole", "sts:TagSession" ], "Principal": { "AWS": "arn:aws:iam::111122223333:root" }, "Condition": { "StringLike": { "aws:RequestTag/SageMakerPartnerAppUser": "prefix${aws:username}" } } } ] } ``` ------ + **Authenticated IAM user** – The username of the user is automatically propagated as the Partner AI App user. + **AWS STS session name** – If no `SageMakerPartnerAppUser` session tag is configured when using AWS STS, SageMaker AI returns an error when users launch a Partner AI App. To avoid this error, admins must set the `EnableIamSessionBasedIdentity` opt-in flag for each Partner AI App. For more information, see [`EnableIamSessionBasedIdentity`](#partner-app-onboard-user-iam-session). When the `EnableIamSessionBasedIdentity` opt-in flag is enabled, use the [IAM role trust policy](https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_iam-condition-keys.html#ck_rolesessionname) to make sure that the IAM session name is or contains the IAM username. This makes sure that users don't gain access by impersonating other users. The following trust policy verifies that the session name exactly matches the associated IAM user. Admins can use any principal tag for this purpose. It should be configured on the role that is launching Studio or the Partner AI App. ------ #### [ JSON ] **** ``` { "Version":"2012-10-17", "Statement": [ { "Sid": "RoleTrustPolicyRequireUsernameForSessionName", "Effect": "Allow", "Action": "sts:AssumeRole", "Principal": { "AWS": "arn:aws:iam::111122223333:root" }, "Condition": { "StringEquals": { "sts:RoleSessionName": "${aws:username}" } } } ] } ``` ------ Admins must also add the `sts:TagSession` trust policy to the role that is launching Studio or the Partner AI App. This makes sure that the identity can be propagated properly. ``` { "Effect": "Allow", "Principal": { "Service": "sagemaker.amazonaws.com" }, "Action": [ "sts:AssumeRole", "sts:TagSession" ] } ``` After setting the credentials, admins can give their users access to Studio or the Partner AI App from the AWS CLI using either the `CreatePresignedDomainUrl` or `CreatePartnerAppPresignedUrl` API calls, respectively. Users can also then launch Studio from the SageMaker AI console, and launch Partner AI Apps from Studio. ### `EnableIamSessionBasedIdentity` `EnableIamSessionBasedIdentity` is an opt-in flag. When the `EnableIamSessionBasedIdentity` flag is set, SageMaker AI passes IAM session information as the Partner AI App user identity. For more information about AWS STS sessions, see [Use temporary credentials with AWS resources](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_temp_use-resources.html). ### Access control To control access to Partner AI Apps, use an IAM policy attached to the user profile’s execution role. To launch a Partner AI App directly from Studio or using the AWS CLI, the user profile’s execution role must have a policy that gives permissions for the `CreatePartnerAppPresignedUrl` API. Remove this permission from the user profile’s execution role to make sure they can't launch Partner AI Apps. ### Root admin users The Comet and Fiddler Partner AI Apps require at least one root admin user. Root admin users have permissions to add both normal and admin users and manage resources. The usernames provided as root admin users must be consistent with the usernames from the identity source. While root admin users are persisted in SageMaker AI, normal admin users are not and exist only within the Partner AI App until the Partner AI App is terminated. Admins can update root admin users using the `UpdatePartnerApp` API call. When root admin users are updated, the updated list of root admin users is passed to the Partner AI App. The Partner AI App makes sure that all usernames in the list are granted root admin privileges. If a root admin user is removed from the list, the user still retains normal admin permissions until either: + The user is removed from the application. + Another admin user revokes admin permissions for the user. **Note** Fiddler doesn't support updating admin users. Only Comet supports updates to root admin users.  To delete a root admin user, you must first update the list of root admin users using the `UpdatePartnerApp` API. Then, remove or revoke the admin permissions through the Partner AI App's UI. If you remove a root admin user from the Partner AI App's UI without updating the list of root admin users with the `UpdatePartnerApp` API, the change is temporary. When SageMaker AI sends the next Partner AI App update request, SageMaker AI sends the root admin list that still includes the user to the Partner AI App. This overrides the deletion completed from the Partner AI App UI. # Partner AI App provisioning After admins have set up the required permissions, they can explore and provision Amazon SageMaker Partner AI Apps for users in the domain. Admins can view all of the available Partner AI Apps, as well as the Partner AI Apps that they have provisioned from the [Amazon SageMaker AI console](https://console.aws.amazon.com/sagemaker/). From the **Partner AI Apps** page, admins can view details about the pricing model for each Partner AI App and make them available to users. Admins can make them available by navigating to the AWS Marketplace to subscribe to that Partner AI App. Admins can provision new apps from the Partner AI Apps page. They can also view the Partner AI Apps that they have already provisioned from the **My Apps** tab. **Note** Applications that admins provision can be accessed by all users that admins give proper permissions to in an AWS account. Partner AI Apps are not restricted to a specific domain or user. ## Status When admins view a Partner AI App that they have provisioned, they can also see the status of their application with one of the following values. + **Deployed** – The application is ready for use. Admins can update the application configuration and delete the application. + **Error** – There was an issue with the application deployment. Admins can troubleshoot and configure the application again to deploy it. + **Not deployed** – The application has been subscribed to, but not deployed. Admins can configure the application to deploy it. ## Options When admins configure an application, they can decide the following options: + **App name** – A unique name for the application. + **App maintenance schedule** – Partner AI Apps undergo maintenance on a weekly basis. With this option, admins choose both the day of the week and the time that this maintenance happens. + **STS identity propagation** – Use this option to pass the AWS Security Token Service (AWS STS) launcher IAM session name as the Partner AI App user identity. For more information, see [Set up Partner AI Apps](partner-app-onboard.md). + **Admin management** – Some Partner AI Apps support adding up to five admins that have full rights to manage the Partner AI App functionality. This only applies to Comet and Fiddler. For more information, see [Set up Partner AI Apps](partner-app-onboard.md). + **Execution role** – The role that the Partner AI App uses to access resources and perform actions. For more information, see [Set up Partner AI Apps](partner-app-onboard.md). + **App version** – The version of the Partner AI App that admins want to use.  + **Tier selection** – The infrastructure deployment tier for the Partner AI App. The tier size impacts the speed and capabilities of the application. For more information, see [Set up Partner AI Apps](partner-app-onboard.md). + **Lakera S3 bucket policy** – This is only required by the Lakera-guard app to access an Amazon S3 bucket. # Set up the Amazon SageMaker Partner AI Apps SDKs The following topic outlines the process needed to install and use the application-specific SDKs with Amazon SageMaker Partner AI Apps. To install and use SDKs for applications, you must specify environment variables specific to Partner AI Apps, so the application’s SDK can pick up environment variables and trigger authorization. The following sections give information about the steps needed to complete this for each of the supported application types. ## Comet Comet offers two products: + Opik is an source LLM evaluation framework. + Comet’s ML platform can be used to track, compare, explain, and optimize models across the complete ML lifecycle. Comet supports the use of two different SDKs based on the product that you are interacting with. Complete the following procedure to install and use the Comet or Opik SDKs. For more information about the Comet SDK, see [Quickstart](https://www.comet.com/docs/v2/guides/quickstart/). For more information about the Opik SDK, see [Open source LLM evaluation framework](https://github.com/comet-ml/opik). 1. Launch the environment that you are using the Comet or Opik SDKs with Partner AI Apps in. For information about launching a JupyterLab application, see [Create a space](studio-updated-jl-user-guide-create-space.md). For information about launching a Code Editor, based on Code-OSS, Visual Studio Code - Open Source application, see [Launch a Code Editor application in Studio](code-editor-use-studio.md). 1. Launch a Jupyter notebook or Code Editor space. 1. From the development environment, install the compatible Comet, Opik, and SageMaker Python SDK versions. To be compatible: + The SageMaker Python SDK version must be at least `2.237.0`. + The Comet SDK version must be the latest version. + The Opik SDK version must match the version used by your Opik application. Verify the Opik version used in the Opik web application UI. The exception to this is that the Opik SDK version must be at least `1.2.0` when the Opik application version is `1.1.5`. **Note** SageMaker JupyterLab comes with SageMaker Python SDK installed. However, you may need to upgrade the SageMaker Python SDK if the version is lower than `2.237.0`. ``` %pip install sagemaker>=2.237.0 comet_ml ##or %pip install sagemaker>=2.237.0 opik= ``` 1. Set the following environment variables for the application resource ARN. These environment variables are used to communicate with the Comet and Opik SDKs. To retrieve these values, navigate to the details page for the application in Amazon SageMaker Studio. ``` os.environ['AWS_PARTNER_APP_AUTH'] = 'true' os.environ['AWS_PARTNER_APP_ARN'] = '' ``` 1. For the Comet application, the SDK URL is automatically included as part of the API key set in the following step. You may instead set the `COMET_URL_OVERRIDE` environment variable to manually override the SDK URL. ``` os.environ['COMET_URL_OVERRIDE'] = '' ``` 1. For the Opik application, the SDK URL is automatically included as part of the API key set in the following step. You may instead set the `OPIK_URL_OVERRIDE` environment variable to manually override the SDK URL. To get the Opik workspace name, see the Opik application and navigate to the user's workspace. ``` os.environ['OPIK_URL_OVERRIDE'] = '' os.environ['OPIK_WORKSPACE'] = '' ``` 1. Set the environment variable that identifies the API key for Comet or Opik. This is used to verify the connection from SageMaker to the application when the Comet and Opik SDKs are used. This API key is application-specific and is not managed by SageMaker. To get this key, you must log into the application and retrieve the API key. The Opik API key is the same as the Comet API key. ``` os.environ['COMET_API_KEY'] = '' os.environ["OPIK_API_KEY"] = os.environ["COMET_API_KEY"] ``` ## Fiddler Complete the following procedure to install and use the Fiddler Python Client. For information about the Fiddler Python Client, see [About Client 3.x](https://docs.fiddler.ai/python-client-3-x/about-client-3x). 1. Launch the notebook environment that you are using the Fiddler Python Client with Partner AI Apps in. For information about launching a JupyterLab application, see [Create a space](studio-updated-jl-user-guide-create-space.md). For information about launching a Code Editor, based on Code-OSS, Visual Studio Code - Open Source application, see [Launch a Code Editor application in Studio](code-editor-use-studio.md). 1. Launch a Jupyter notebook or Code Editor space. 1. From the development environnment, install the Fiddler Python Client and SageMaker Python SDK versions. To be compatible: + The SageMaker Python SDK version must be at least `2.237.0`. + The Fiddler Python Client version must be compatible with the version of Fiddler used in the application. After verifying the Fiddler version from the UI, see the Fiddler [Compatibility Matrix](https://docs.fiddler.ai/history/compatibility-matrix) for the compatible Fiddler Python Client version. **Note** SageMaker JupyterLab comes with SageMaker Python SDK installed. However, you may need to upgrade the SageMaker Python SDK if the version is lower than `2.237.0`. ``` %pip install sagemaker>=2.237.0 fiddler-client= ``` 1. Set the following environment variables for the application resource ARN and the SDK URL. These environment variables are used to communicate with the Fiddler Python Client. To retrieve these values, navigate to the details page for the Fiddler application in Amazon SageMaker Studio.   ``` os.environ['AWS_PARTNER_APP_AUTH'] = 'true' os.environ['AWS_PARTNER_APP_ARN'] = '' os.environ['AWS_PARTNER_APP_URL'] = '' ``` 1. Set the environment variable that identifies the API key for the Fiddler application. This is used to verify the connection from SageMaker to the Fiddler application when the Fiddler Python Client is used. This API key is application-specific and is not managed by SageMaker. To get this key, you must log into the Fiddler application and retrieve the API key. ``` os.environ['FIDDLER_KEY'] = '' ``` ## Deepchecks Complete the following procedure to install and use Deepchecks Python SDK. 1. Launch the notebook environment that you are using the Deepchecks Python SDK with Partner AI Apps in. For information about launching a JupyterLab application, see [Create a space](studio-updated-jl-user-guide-create-space.md). For information about launching a Code Editor, based on Code-OSS, Visual Studio Code - Open Source application, see [Launch a Code Editor application in Studio](code-editor-use-studio.md). 1. Launch a Jupyter notebook or Code Editor space. 1. From the development environment, install the compatible Deepchecks Python SDK and SageMaker Python SDK versions.  Partner AI Apps is running version `0.21.15` of Deepchecks. To be compatible: + The SageMaker Python SDK version must be at least `2.237.0`. + The Deepchecks Python SDK must use the minor version `0.21`. **Note** SageMaker JupyterLab comes with SageMaker Python SDK installed. However, you may need to upgrade the SageMaker Python SDK if the version is lower than `2.237.0`. ``` %pip install sagemaker>=2.237.0 deepchecks-llm-client>=0.21,<0.22 ``` 1. Set the following environment variables for the application resource ARN and the SDK URL. These environment variables are used to communicate with the Deepchecks Python SDK. To retrieve these values, navigate to the details page for the application in Amazon SageMaker Studio.   ``` os.environ['AWS_PARTNER_APP_AUTH'] = 'true' os.environ['AWS_PARTNER_APP_ARN'] = '' os.environ['AWS_PARTNER_APP_URL'] = '' ``` 1. Set the environment variable that identifies the API key for the Deepchecks application. This is used to verify the connection from SageMaker to the Deepchecks application when the Deepchecks Python SDK is used. This API key is application-specific and is not managed by SageMaker. To get this key, see [Setup: Python SDK Installation & API Key Retrieval](https://llmdocs.deepchecks.com/docs/setup-sdk-installation-api-key#generate-an-api-key-via-the-ui). ``` os.environ['DEEPCHECKS_API_KEY'] = '' ``` ## Lakera Lakera does not offer an SDK. Instead, you can interact with the Lakera Guard API through HTTP requests to the available endpoints in any programming language. For more information, see [Lakera Guard API](https://platform.lakera.ai/docs/api). To use the SageMaker Python SDK with Lakera, complete the following steps: 1. Launch the environment that you are using Partner AI Apps in. For information about launching a JupyterLab application, see [Create a space](studio-updated-jl-user-guide-create-space.md). For information about launching a Code Editor, based on Code-OSS, Visual Studio Code - Open Source application, see [Launch a Code Editor application in Studio](code-editor-use-studio.md). 1. Launch a Jupyter notebook or Code Editor space. 1. From the development environment, install the compatible SageMaker Python SDK version. The SageMaker Python SDK version must be at least `2.237.0` **Note** SageMaker JupyterLab comes with SageMaker Python SDK installed. However, you may need to upgrade the SageMaker Python SDK if the version is lower than `2.237.0`. ``` %pip install sagemaker>=2.237.0 ``` 1. Set the following environment variables for the application resource ARN and the SDK URL. To retrieve these values, navigate to the details page for the application in Amazon SageMaker Studio. ``` os.environ['AWS_PARTNER_APP_ARN'] = '' os.environ['AWS_PARTNER_APP_URL'] = '' ``` # Partner AI Apps in Studio After the admin has added the required permissions and authorized users, users can view the Amazon SageMaker Partner AI App in Amazon SageMaker Studio. From Studio, users can launch apps that have been approved for use by their administrator. ## Browsing and selecting To browse the available Partner AI Apps, users must navigate to Studio. For information about launching Studio, see [Launch Amazon SageMaker Studio](studio-updated-launch.md). After users have launched Studio, they can view all of the available Partner AI Apps by selecting the **Partner AI Apps** section in the left navigation. The **Partner AI Apps** page lists all of the Partner AI Apps, and gives information about whether the Partner AI Apps have been deployed by the admin. If the desired Partner AI Apps haven't been deployed, users can reach out to the admin to request that they deploy the Partner AI Apps for use in the SageMaker AI domain. If the application has been deployed, users can open the Partner AI App UI to start using it or view details of the Partner AI App. When users view the details of the application, they see the value of the following. + ARN – This is the resource ARN of the Partner AI App. + SDK URL – This is the URL of the Partner AI App that the Partner AI App SDK uses to support app-specific tasks such as logging model experiment tracking data from a JupyterLab notebook in Studio. Users can use these values to write code that uses the Partner AI App SDK for app-specific tasks. Each Partner AI App’s details page includes a sample notebook. To get started, users can launch the sample notebook in a JupyterLab space in the Studio environment. # Use AWS KMS Permissions for Amazon SageMaker Partner AI Apps Use AWS KMS Permissions You can protect your data at rest using encryption for Amazon SageMaker Partner AI Apps. By default, it uses server-side encryption with a SageMaker owned key. SageMaker also supports an option for server-side encryption with a customer managed KMS key. ## Server-side encryption with SageMaker managed keys (Default) Partner AI Apps encrypt all your data at rest using an AWS managed key by default. ## Server-side encryption with customer managed KMS keys (Optional) Partner AI Apps support the use of a symmetric customer managed key that you create, own, and manage to replace the existing AWS owned encryption. Because you have full control of this layer of encryption, you can perform such tasks as: + Establishing and maintaining key policies + Establishing and maintaining IAM policies and grants + Enabling and disabling key policies + Rotating key cryptographic material + Adding tags + Creating key aliases + Scheduling keys for deletion For more information, see [Customer managed keys](https://docs.aws.amazon.com/kms/latest/developerguide/concepts.html#customer-cmk) in the *AWS Key Management Service Developer Guide*. ## How Partner AI Apps use grants in AWS KMS Partner AI Apps require a grant to use your customer managed key. When you create an application encrypted with a customer managed key, Partner AI Apps creates a grant on your behalf by sending a CreateGrant request to AWS KMS. Grants in AWS KMS are used to give Partner AI Apps access to a KMS key in a customer account. You can revoke access to the grant, or remove the service's access to the customer managed key at any time. If you do, Partner AI App won't be able to access any of the data encrypted by the customer managed key, which affects operations that are dependent on that data. The application will not operate properly and will become irrecoverable. ## Create a customer managed key You can create a symmetric customer managed key by using the AWS Management Console or the AWS KMS APIs. **To create a symmetric customer managed key** Follow the steps for [Creating symmetric encryption KMS keys](https://docs.aws.amazon.com/kms/latest/developerguide/create-keys.html#create-symmetric-cmk) in the *AWS Key Management Service Developer Guide*. **Key policy** Key policies control access to your customer managed key. Every customer managed key must have exactly one key policy, which contains statements that determine who can use the key and how they can use it. When you create your customer managed key, you can specify a key policy. For more information, see [Determining access to AWS KMS keys](https://docs.aws.amazon.com/kms/latest/developerguide/determining-access.html) in the *AWS Key Management Service Developer Guide*. To use your customer managed key with your Partner AI App resources, the following API operations must be permitted in the key policy. The principal for these operations depends on whether the role is used to create or use the application. + Creating the application: + `[kms:CreateGrant](https://docs.aws.amazon.com/kms/latest/APIReference/API_CreateGrant.html)` + [https://docs.aws.amazon.com/kms/latest/APIReference/API_DescribeKey.html](https://docs.aws.amazon.com/kms/latest/APIReference/API_DescribeKey.html) + Using the application: + [https://docs.aws.amazon.com/kms/latest/APIReference/API_Decrypt.html](https://docs.aws.amazon.com/kms/latest/APIReference/API_Decrypt.html) + [https://docs.aws.amazon.com/kms/latest/APIReference/API_GenerateDataKey.html](https://docs.aws.amazon.com/kms/latest/APIReference/API_GenerateDataKey.html) The following are policy statement examples you can add for Partner AI Apps based on whether the persona is an administrator or user. For more information about specifying permissions in a policy, see [AWS KMS permissions](https://docs.aws.amazon.com/kms/latest/developerguide/kms-api-permissions-reference.html) in the *AWS Key Management Service Developer Guide*. For more information about troubleshooting, see [Troubleshooting key access](https://docs.aws.amazon.com/kms/latest/developerguide/policy-evaluation.html) in the *AWS Key Management Service Developer Guide*. **Administrator** The following policy statement is used for the administrator who is creating Partner AI Apps. ------ #### [ JSON ] **** ``` { "Version":"2012-10-17", "Id": "example-key-policy", "Statement": [ { "Sid": "Allow use of the key", "Effect": "Allow", "Principal": { "AWS": "arn:aws:iam::111122223333:role/" }, "Action": [ "kms:CreateGrant", "kms:DescribeKey" ], "Resource": "*", "Condition": { "StringEquals": { "kms:ViaService": "sagemaker.us-east-1.amazonaws.com" } } } ] } ``` ------ **User** The following policy statement is for the user of the Partner AI Apps. ------ #### [ JSON ] **** ``` { "Version":"2012-10-17", "Id":"example-key-policy", "Statement":[ { "Effect":"Allow", "Principal":{ "AWS":"arn:aws:iam::111122223333:role/user-role" }, "Action":[ "kms:Decrypt", "kms:GenerateDataKey" ], "Resource":"*", "Condition":{ "StringEquals":{ "kms:ViaService":"sagemaker.us-east-1.amazonaws.com" } } } ] } ``` ------ # Setting up cross-account sharing for Amazon SageMaker AI partner AI apps Cross-account sharing Amazon SageMaker AI integrates with AWS Resource Access Manager (AWS RAM) to enable resource sharing. AWS RAM is a service that enables you to share some Amazon SageMaker AI resources with other AWS accounts or through AWS Organizations. With AWS RAM, you share resources that you own by creating a *resource share*. A resource share specifies the resources to share, and the consumers with whom to share them. Consumers can be specific AWS accounts inside or outside of its organization in AWS Organizations. For more information about AWS RAM, see the *[AWS RAM User Guide](https://docs.aws.amazon.com/ram/latest/userguide/)*. This topic explains how to share resources that you own, and how to use resources that are shared with you. **Topics** + [ ## Prerequisites for sharing an Amazon SageMaker Partner AI App ](#partner-app-resource-sharing-ram-prereqs) + [ ## Sharing an Amazon SageMaker Partner AI App ](#partner-app-resource-sharing-share) + [ ## Accepting resource share invitations ](#partner-app-resource-sharing-responses) + [ ## Identifying a shared Amazon SageMaker Partner AI App ](#sharing-identify) + [ ## Responsibilities and permissions for shared Amazon SageMaker Partner AI Apps ](#sharing-perms) ## Prerequisites for sharing an Amazon SageMaker Partner AI App + To share an Amazon SageMaker Partner AI App, you must own it in your AWS account. This means that the resource must be allocated or provisioned in your account. You cannot share an Amazon SageMaker Partner AI App that has been shared with you. + To share an Amazon SageMaker Partner AI App with your organization or an organizational unit in AWS Organizations, you must enable sharing with AWS Organizations. For more information, see [ Enable Sharing with AWS Organizations](https://docs.aws.amazon.com/ram/latest/userguide/getting-started-sharing.html#getting-started-sharing-orgs) in the *AWS RAM User Guide*. ## Sharing an Amazon SageMaker Partner AI App To share an Amazon SageMaker Partner AI App, you must add it to a resource share. A resource share is an AWS RAM resource that lets you share your resources across AWS accounts. A resource share specifies the resources to share, and the consumers with whom they are shared. When you share an Amazon SageMaker Partner AI App using the [Amazon SageMaker AI console](https://console.aws.amazon.com/sagemaker), you add it to an existing resource share. To add the Amazon SageMaker Partner AI App to a new resource share, you must first create the resource share by using the [AWS RAM console](https://console.aws.amazon.com/ram). You can share an Amazon SageMaker Partner AI App that you own using the Amazon SageMaker AI console, AWS RAM console, or the AWS CLI. **To share an Amazon SageMaker Partner AI App that you own using the Amazon SageMaker AI console** 1. Sign in to the AWS Management Console and open the AWS RAM console at [https://console.aws.amazon.com/ram/home](https://console.aws.amazon.com/ram/home). 1. In the main pane, choose **Create a resource share**. 1. Enter a name for the resource share that you want to create. 1. In the **Resources** section, for **Resource type** select **SageMaker AI Partner Apps**. The partner apps that you can share appear in the table. 1. Select the partner apps that you want to share. 1. Optionally specify tags, and then choose **Next**. 1. Specify the AWS accounts with which you want to share your partner apps. 1. Review your resource share configuration and choose **Create resource share**. It might take the service a few minutes to finish creating the resource share. **To share an Amazon SageMaker Partner AI App that you own using the AWS RAM console** See [Creating a Resource Share](https://docs.aws.amazon.com/ram/latest/userguide/working-with-sharing.html#working-with-sharing-create) in the *AWS RAM User Guide*. **To share an Amazon SageMaker Partner AI App that you own using the AWS CLI** Use the [create-resource-share](https://docs.aws.amazon.com/cli/latest/reference/ram/create-resource-share.html) command. ## Accepting resource share invitations When a resource owner sets up a resource share, each consumer AWS account receives an invitation to join the resource share. The consumer AWS accounts must accept the invitation to gain access to any shared resources. For more information on accepting a resource share invitation through AWS RAM, see [Using shared AWS resources ](https://docs.aws.amazon.com/ram/latest/userguide/getting-started-shared.html)in the *AWS Resource Access Manager User Guide*. ## Identifying a shared Amazon SageMaker Partner AI App Owners and consumers can identify shared Amazon SageMaker Partner AI Apps using the Amazon SageMaker AI console and AWS CLI. **To identify a shared Amazon SageMaker Partner AI App by using the Amazon SageMaker AI console** See [Partner AI Apps in Studio](partner-apps-studio.md). **To identify a shared Amazon SageMaker Partner AI App by using the AWS CLI** Use the [list-partner-apps](https://docs.aws.amazon.com/cli/latest/reference/sagemaker/list-partner-apps.html) command. The command returns the Amazon SageMaker Partner AI Apps that you own and Amazon SageMaker Partner AI Apps that are shared with you. `OwnerId` shows the AWS account ID of the Amazon SageMaker Partner AI App owner. ## Responsibilities and permissions for shared Amazon SageMaker Partner AI Apps The account with which an Amazon SageMaker Partner AI App is shared needs to have the following AWS Identity and Access Management policy. ------ #### [ JSON ] **** ``` { "Version":"2012-10-17", "Statement" : [ { "Sid" : "AmazonSageMakerPartnerListAppsPermission", "Effect" : "Allow", "Action" : "sagemaker:ListPartnerApps", "Resource" : "*" }, { "Sid" : "AmazonSageMakerPartnerAppsPermission", "Effect" : "Allow", "Action" : [ "sagemaker:CreatePartnerAppPresignedUrl", "sagemaker:DescribePartnerApp", "sagemaker:CallPartnerAppApi" ], "Condition" : { "StringEquals" : { "aws:ResourceAccount" : [ "App-owner AWS account-1", "App-owner AWS account-2"] } }, "Resource" : "arn:aws:sagemaker:*:*:partner-app/*" } ] } ``` ------