/AWS1/CL_SGMMETRICSCONFIG

The configuration for Utilization metrics.

CONSTRUCTOR

IMPORTING

Optional arguments:

iv_enableenhancedmetrics TYPE /AWS1/SGMENABLEENHANCEDMETRICS /AWS1/SGMENABLEENHANCEDMETRICS

Specifies whether to enable enhanced metrics for the endpoint. Enhanced metrics provide utilization and invocation data at instance and container granularity. Container granularity is supported for Inference Components. The default is False.

iv_enbldetailedobservability TYPE /AWS1/SGMENBETAILEDOBSERVABI00 /AWS1/SGMENBETAILEDOBSERVABI00

Indicates whether detailed observability is enabled for the endpoint. When set to True, the following metrics are published at the configured frequency:

• Container-level inference metrics scraped from the container's Prometheus endpoint (such as request latency, error counts, and throughput). Available metrics vary by framework.

• Per-GPU metrics (utilization, memory, and temperature) attributed to individual inference components.

• Per-instance host metrics (CPU, memory, and disk utilization).

• Inference component placement metrics (copy count per Availability Zone).

For first-party and Deep Learning Containers (DLC), the Prometheus endpoint path is determined automatically. For Bring-Your-Own-Container (BYOC) cases, you can optionally set ContainerMetricsConfig to specify a custom endpoint path. If not specified, the default path /metrics on port 8080 is used.

When set to False, these additional metrics are not published. Standard invocation and utilization metrics controlled by EnableEnhancedMetrics are unaffected.

The default value for new endpoint configurations is True. For existing endpoint configurations created before this feature, the value is False unless explicitly set.

iv_metricpubfrequencyinsecs TYPE /AWS1/SGMMETRICPUBFREQUENCYI00 /AWS1/SGMMETRICPUBFREQUENCYI00

The interval, in seconds, at which metrics are published to Amazon CloudWatch. Defaults to 60. Valid values: 10, 30, 60, 120, 180, 240, 300.

When EnableEnhancedMetrics is set to False, this interval applies to utilization metrics only. Invocation metrics continue to be published at the default 60-second interval. When EnableEnhancedMetrics is set to True, this interval applies to both utilization and invocation metrics.

When EnableDetailedObservability is set to True, this interval applies to per-GPU metrics, per-instance host metrics, container metrics, and fleet-level inference component lifecycle and placement metrics.

---

Queryable Attributes

EnableEnhancedMetrics

Specifies whether to enable enhanced metrics for the endpoint. Enhanced metrics provide utilization and invocation data at instance and container granularity. Container granularity is supported for Inference Components. The default is False.

Accessible with the following methods

Method	Description
GET_ENABLEENHANCEDMETRICS()	Getter for ENABLEENHANCEDMETRICS, with configurable default
ASK_ENABLEENHANCEDMETRICS()	Getter for ENABLEENHANCEDMETRICS w/ exceptions if field has
HAS_ENABLEENHANCEDMETRICS()	Determine if ENABLEENHANCEDMETRICS has a value

EnableDetailedObservability

Indicates whether detailed observability is enabled for the endpoint. When set to True, the following metrics are published at the configured frequency:

• Container-level inference metrics scraped from the container's Prometheus endpoint (such as request latency, error counts, and throughput). Available metrics vary by framework.

• Per-GPU metrics (utilization, memory, and temperature) attributed to individual inference components.

• Per-instance host metrics (CPU, memory, and disk utilization).

• Inference component placement metrics (copy count per Availability Zone).

For first-party and Deep Learning Containers (DLC), the Prometheus endpoint path is determined automatically. For Bring-Your-Own-Container (BYOC) cases, you can optionally set ContainerMetricsConfig to specify a custom endpoint path. If not specified, the default path /metrics on port 8080 is used.

When set to False, these additional metrics are not published. Standard invocation and utilization metrics controlled by EnableEnhancedMetrics are unaffected.

The default value for new endpoint configurations is True. For existing endpoint configurations created before this feature, the value is False unless explicitly set.

Accessible with the following methods

Method	Description
GET_ENBLDETEDOBSERVABILITY()	Getter for ENABLEDETAILEDOBSERVABILITY, with configurable de
ASK_ENBLDETEDOBSERVABILITY()	Getter for ENABLEDETAILEDOBSERVABILITY w/ exceptions if fiel
HAS_ENBLDETEDOBSERVABILITY()	Determine if ENABLEDETAILEDOBSERVABILITY has a value

MetricPublishFrequencyInSeconds

The interval, in seconds, at which metrics are published to Amazon CloudWatch. Defaults to 60. Valid values: 10, 30, 60, 120, 180, 240, 300.

When EnableEnhancedMetrics is set to False, this interval applies to utilization metrics only. Invocation metrics continue to be published at the default 60-second interval. When EnableEnhancedMetrics is set to True, this interval applies to both utilization and invocation metrics.

When EnableDetailedObservability is set to True, this interval applies to per-GPU metrics, per-instance host metrics, container metrics, and fleet-level inference component lifecycle and placement metrics.

Accessible with the following methods

Method	Description
GET_METRICPUBFREQUENCYINSECS()	Getter for METRICPUBFREQUENCYINSECONDS, with configurable de
ASK_METRICPUBFREQUENCYINSECS()	Getter for METRICPUBFREQUENCYINSECONDS w/ exceptions if fiel
HAS_METRICPUBFREQUENCYINSECS()	Determine if METRICPUBFREQUENCYINSECONDS has a value