/AWS1/CL_BDKCUSTOMMETRICDEFN¶
The definition of a custom metric for use in an Amazon Bedrock evaluation job. A custom metric definition includes a metric name, prompt (instructions) and optionally, a rating scale. Your prompt must include a task description and input variables. The required input variables are different for model-as-a-judge and RAG evaluations.
For more information about how to define a custom metric in Amazon Bedrock, see Create a prompt for a custom metrics (LLM-as-a-judge model evaluations) and Create a prompt for a custom metrics (RAG evaluations).
CONSTRUCTOR¶
IMPORTING¶
Required arguments:¶
iv_name TYPE /AWS1/BDKMETRICNAME /AWS1/BDKMETRICNAME¶
The name for a custom metric. Names must be unique in your Amazon Web Services region.
iv_instructions TYPE /AWS1/BDKCUSTMETRICINSTRUCTI00 /AWS1/BDKCUSTMETRICINSTRUCTI00¶
The prompt for a custom metric that instructs the evaluator model how to rate the model or RAG source under evaluation.
Optional arguments:¶
it_ratingscale TYPE /AWS1/CL_BDKRATINGSCALEITEM=>TT_RATINGSCALE TT_RATINGSCALE¶
Defines the rating scale to be used for a custom metric. We recommend that you always define a ratings scale when creating a custom metric. If you don't define a scale, Amazon Bedrock won't be able to visually display the results of the evaluation in the console or calculate average values of numerical scores. For more information on specifying a rating scale, see Specifying an output schema (rating scale).
Queryable Attributes¶
name¶
The name for a custom metric. Names must be unique in your Amazon Web Services region.
Accessible with the following methods¶
| Method | Description |
|---|---|
GET_NAME() |
Getter for NAME, with configurable default |
ASK_NAME() |
Getter for NAME w/ exceptions if field has no value |
HAS_NAME() |
Determine if NAME has a value |
instructions¶
The prompt for a custom metric that instructs the evaluator model how to rate the model or RAG source under evaluation.
Accessible with the following methods¶
| Method | Description |
|---|---|
GET_INSTRUCTIONS() |
Getter for INSTRUCTIONS, with configurable default |
ASK_INSTRUCTIONS() |
Getter for INSTRUCTIONS w/ exceptions if field has no value |
HAS_INSTRUCTIONS() |
Determine if INSTRUCTIONS has a value |
ratingScale¶
Defines the rating scale to be used for a custom metric. We recommend that you always define a ratings scale when creating a custom metric. If you don't define a scale, Amazon Bedrock won't be able to visually display the results of the evaluation in the console or calculate average values of numerical scores. For more information on specifying a rating scale, see Specifying an output schema (rating scale).
Accessible with the following methods¶
| Method | Description |
|---|---|
GET_RATINGSCALE() |
Getter for RATINGSCALE, with configurable default |
ASK_RATINGSCALE() |
Getter for RATINGSCALE w/ exceptions if field has no value |
HAS_RATINGSCALE() |
Determine if RATINGSCALE has a value |