AWS CDK Reference

Appearance

v2.274.0

Experimental

@aws-cdk/aws-sagemaker-alpha is an experimental (alpha) package: its API may change in any release, without a major version bump.

InvocationsScalingProps Struct ​

Properties for enabling SageMaker Endpoint utilization tracking.

Stability
Experimental
Source
lib/scalable-instance-count.ts

Example ​

ts
import * as sagemaker from '@aws-cdk/aws-sagemaker-alpha';

declare const model: sagemaker.Model;

const variantName = 'my-variant';
const endpointConfig = new sagemaker.EndpointConfig(this, 'EndpointConfig', {
  instanceProductionVariants: [
    {
      model: model,
      variantName: variantName,
    },
  ]
});

const endpoint = new sagemaker.Endpoint(this, 'Endpoint', { endpointConfig });
const productionVariant = endpoint.findInstanceProductionVariant(variantName);
const instanceCount = productionVariant.autoScaleInstanceCount({
  maxCapacity: 3
});
instanceCount.scaleOnInvocations('LimitRPS', {
  maxRequestsPerSecond: 30,
});

Properties ​

disableScaleInbooleandisable_scale_inbooldisableScaleInBooleanDisableScaleInboolDisableScaleIn*bool= false#
Indicates whether scale in by the target tracking policy is disabled.

If the value is true, scale in is disabled and the target tracking policy won't remove capacity from the scalable resource. Otherwise, scale in is enabled and the target tracking policy can remove capacity from the scalable resource.

maxRequestsPerSecondnumbermax_requests_per_secondint | floatmaxRequestsPerSecondNumberMaxRequestsPerSeconddoubleMaxRequestsPerSecond*float64required#
Max RPS per instance used for calculating the target SageMaker variant invocation per instance.

More documentation available here: https://docs.aws.amazon.com/sagemaker/latest/dg/endpoint-scaling-loadtest.html

policyNamestringpolicy_namestrpolicyNameStringPolicyNamestringPolicyName*string= Automatically generated name.#
A name for the scaling policy.
safetyFactornumbersafety_factorint | floatsafetyFactorNumberSafetyFactordoubleSafetyFactor*float64= 0.5#
Safety factor for calculating the target SageMaker variant invocation per instance.

More documentation available here: https://docs.aws.amazon.com/sagemaker/latest/dg/endpoint-scaling-loadtest.html

scaleInCooldownDurationscale_in_cooldownDurationscaleInCooldownDurationScaleInCooldownDurationScaleInCooldownDuration= Duration.seconds(300) for the fo…#
Period after a scale in activity completes before another scale in activity can start.
default

Default: Duration.seconds(300) for the following scalable targets: ECS services, Spot Fleet requests, EMR clusters, AppStream 2.0 fleets, Aurora DB clusters, Amazon SageMaker endpoint variants, Custom resources. For all other scalable targets, the default value is Duration.seconds(0): DynamoDB tables, DynamoDB global secondary indexes, Amazon Comprehend document classification endpoints, Lambda provisioned concurrency

scaleOutCooldownDurationscale_out_cooldownDurationscaleOutCooldownDurationScaleOutCooldownDurationScaleOutCooldownDuration= Duration.seconds(300) for the fo…#
Period after a scale out activity completes before another scale out activity can start.
default

Default: Duration.seconds(300) for the following scalable targets: ECS services, Spot Fleet requests, EMR clusters, AppStream 2.0 fleets, Aurora DB clusters, Amazon SageMaker endpoint variants, Custom resources. For all other scalable targets, the default value is Duration.seconds(0): DynamoDB tables, DynamoDB global secondary indexes, Amazon Comprehend document classification endpoints, Lambda provisioned concurrency