EvaluationLevel

class aws_cdk.aws_bedrock_agentcore_alpha.EvaluationLevel(value)

Bases: object

(experimental) The level at which a custom evaluator assesses agent performance.

Determines what granularity of data the evaluator operates on.

Stability:: experimental
ExampleMetadata:: infused

Example:

# Create a custom LLM-as-a-Judge evaluator
evaluator = agentcore.Evaluator(self, "MyEvaluator",
    evaluator_name="my_custom_evaluator",
    level=agentcore.EvaluationLevel.SESSION,
    evaluator_config=agentcore.EvaluatorConfig.llm_as_aJudge(
        instructions="Evaluate whether the agent response is helpful and accurate.",
        model_id="us.anthropic.claude-sonnet-4-6",
        rating_scale=agentcore.EvaluatorRatingScale.categorical([label="Good", definition="The response is helpful and accurate.", label="Bad", definition="The response is not helpful or contains errors."
        ])
    )
)

# Use the custom evaluator in an online evaluation configuration
agentcore.OnlineEvaluationConfig(self, "MyEvaluation",
    online_evaluation_config_name="my_evaluation",
    evaluators=[
        agentcore.EvaluatorReference.builtin(agentcore.BuiltinEvaluator.HELPFULNESS),
        agentcore.EvaluatorReference.custom(evaluator)
    ],
    data_source=agentcore.DataSourceConfig.from_cloud_watch_logs(
        log_group_names=["/aws/bedrock-agentcore/my-agent"],
        service_names=["my-agent.default"]
    )
)

Parameters:

value (str) –

The evaluation level string.

Stability:

experimental

Attributes

SESSION = <aws_cdk.aws_bedrock_agentcore_alpha.EvaluationLevel object>

TOOL_CALL = <aws_cdk.aws_bedrock_agentcore_alpha.EvaluationLevel object>

TRACE = <aws_cdk.aws_bedrock_agentcore_alpha.EvaluationLevel object>

value

(experimental) The string value of the evaluation level.

Stability:: experimental