Monitoring the performance of Amazon Bedrock
You can monitor all parts of your Amazon Bedrock application using Amazon CloudWatch, which collects raw data and processes it into readable, near real-time metrics. You can graph the metrics using the CloudWatch console. You can also set alarms that watch for certain thresholds, and send notifications or take actions when values exceed those thresholds.
For more information, see What is Amazon CloudWatch in the Amazon CloudWatch User Guide.
Amazon Bedrock provides comprehensive monitoring capabilities across different components of your application:
-
Monitor model invocation using CloudWatch Logs and Amazon S3 - Track and analyze model invocations using CloudWatch Logs and Amazon S3.
-
Monitor knowledge bases using CloudWatch Logs - Monitor knowledge base operations and performance.
-
Monitor Amazon Bedrock Guardrails using CloudWatch metrics - Track guardrail evaluations and policy enforcement.
-
Monitor Amazon Bedrock Agents using CloudWatch Metrics - Monitor agent invocations and performance metrics.
-
Amazon Bedrock runtime metrics - View key runtime metrics including invocations, latency, errors, and token counts.
-
Monitor Amazon Bedrock job state changes using Amazon EventBridge - Track job state changes and automate responses to events.
-
Monitor Amazon Bedrock API calls using CloudTrail - Audit API calls and track user activity.
Topics
Amazon Bedrock runtime metrics
The following table describes runtime metrics provided by Amazon Bedrock.
| Metric name | Unit | Description |
|---|---|---|
| Invocations | SampleCount | Number of successful requests to the Converse, ConverseStream, InvokeModel, and InvokeModelWithResponseStream API operations. |
|
InvocationLatency |
MilliSeconds |
Latency of the invocations.
|
|
InvocationClientErrors |
SampleCount |
Number of invocations that result in client-side errors. |
|
InvocationServerErrors |
SampleCount |
Number of invocations that result in AWS server-side errors. |
|
InvocationThrottles |
SampleCount |
Number of invocations that the system throttled. Throttled requests and other invocation errors don't count as either Invocations or Errors. The number of throttles you see will depend on your retry settings in the SDK. For more information, see Retry behavior in the AWS SDKs and Tools Reference Guide. |
|
InputTokenCount |
SampleCount |
Number of tokens in the input. |
| LegacyModelInvocations | SampleCount | Number of invocations using Legacy models |
|
OutputTokenCount |
SampleCount |
Number of tokens in the output. |
|
OutputImageCount |
SampleCount |
Number of images in the output (only applicable for image generation models). |
There are also metrics for Amazon Bedrock Guardrails and Amazon Bedrock Agents.
CloudWatch metrics for Amazon Bedrock
For each delivery success or failure attempt, the following Amazon CloudWatch metrics are emitted
under the namespace AWS/Bedrock, and Across all model IDs
dimension:
-
ModelInvocationLogsCloudWatchDeliverySuccess -
ModelInvocationLogsCloudWatchDeliveryFailure -
ModelInvocationLogsS3DeliverySuccess -
ModelInvocationLogsS3DeliveryFailure -
ModelInvocationLargeDataS3DeliverySuccess -
ModelInvocationLargeDataS3DeliveryFailure
To retrieve metrics for your Amazon Bedrock operations, you specify the following information:
-
The metric dimension. A dimension is a set of name-value pairs that you use to identify a metric. Amazon Bedrock supports the following dimensions:
-
ModelId– all metrics -
ModelId + ImageSize + BucketedStepSize– OutputImageCount
-
-
The metric name, such as
InvocationClientErrors.
You can get metrics for Amazon Bedrock with the AWS Management Console, the AWS CLI, or the CloudWatch API. You can use the CloudWatch API through one of the AWS Software Development Kits (SDKs) or the CloudWatch API tools.
To view Amazon Bedrock metrics in the CloudWatch console, go to the metrics section in the navigation pane and select the all metrics option, then search for the model ID.
You must have the appropriate CloudWatch permissions to monitor Amazon Bedrock with CloudWatch For more information, see Authentication and Access Control for Amazon CloudWatch in the Amazon CloudWatch User Guide.
