Multi-container endpoints - Amazon SageMaker AI

Multi-container endpoints

SageMaker AI multi-container endpoints enable customers to deploy multiple containers, that use different models or frameworks, on a single SageMaker AI endpoint. The containers can be run in a sequence as an inference pipeline, or each container can be accessed individually by using direct invocation to improve endpoint utilization and optimize costs.

For information about invoking the containers in a multi-container endpoint in sequence, see Inference pipelines in Amazon SageMaker AI.

For information about invoking a specific container in a multi-container endpoint, see Invoke a multi-container endpoint with direct invocation

Topics

The following policy allows invoke_endpoint requests only when the value of the TargetContainerHostname field matches one of the specified regular expressions.

JSON
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Action": [
                "sagemaker:InvokeEndpoint"
            ],
            "Effect": "Allow",
            "Resource": "arn:aws:sagemaker:us-east-1:account-id:endpoint/endpoint_name",
            "Condition": {
                "StringLike": {
                    "sagemaker:TargetContainerHostname": [
                        "customIps*",
                        "common*"
                    ]
                }
            }
        }
    ]
}

The following policy denies invoke_endpoint requests when the value of the TargetContainerHostname field matches one of the specified regular expressions in the Deny statement.

JSON
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Action": [
                "sagemaker:InvokeEndpoint"
            ],
            "Effect": "Allow",
            "Resource": "arn:aws:sagemaker:us-east-1:account-id:endpoint/endpoint_name",
            "Condition": {
                "StringLike": {
                    "sagemaker:TargetContainerHostname": [
                        "*"
                    ]
                }
            }
        },
        {
            "Action": [
                "sagemaker:InvokeEndpoint"
            ],
            "Effect": "Deny",
            "Resource": "arn:aws:sagemaker:us-east-1:account-id:endpoint/endpoint_name",
            "Condition": {
                "StringLike": {
                    "sagemaker:TargetContainerHostname": [
                        "special*"
                    ]
                }
            }
        }
    ]
}

For information about SageMaker AI condition keys, see Condition Keys for SageMaker AI in the AWS Identity and Access Management User Guide.