CREATEEVALUATOR()`¶

About CreateEvaluator¶

Creates a custom evaluator for agent quality assessment. Custom evaluators use LLM-as-a-Judge configurations with user-defined prompts, rating scales, and model settings to evaluate agent performance at tool call, trace, or session levels.

Method Signature¶

METHODS /AWS1/IF_BDO~CREATEEVALUATOR
  IMPORTING
    !IV_CLIENTTOKEN TYPE /AWS1/BDOCLIENTTOKEN OPTIONAL
    !IV_EVALUATORNAME TYPE /AWS1/BDOCUSTOMEVALUATORNAME OPTIONAL
    !IV_DESCRIPTION TYPE /AWS1/BDOEVALUATORDESCRIPTION OPTIONAL
    !IO_EVALUATORCONFIG TYPE REF TO /AWS1/CL_BDOEVALUATORCONFIG OPTIONAL
    !IV_LEVEL TYPE /AWS1/BDOEVALUATORLEVEL OPTIONAL
  RETURNING
    VALUE(OO_OUTPUT) TYPE REF TO /aws1/cl_bdocreateevaluatorrsp
  RAISING
    /AWS1/CX_BDOACCESSDENIEDEX
    /AWS1/CX_BDOCONFLICTEXCEPTION
    /AWS1/CX_BDOINTERNALSERVEREX
    /AWS1/CX_BDOSERVICEQUOTAEXCDEX
    /AWS1/CX_BDOTHROTTLINGEX
    /AWS1/CX_BDOVALIDATIONEX
    /AWS1/CX_BDOCLIENTEXC
    /AWS1/CX_BDOSERVEREXC
    /AWS1/CX_RT_TECHNICAL_GENERIC
    /AWS1/CX_RT_SERVICE_GENERIC.

IMPORTING¶

Required arguments:¶

`iv_evaluatorname` `TYPE /AWS1/BDOCUSTOMEVALUATORNAME` `/AWS1/BDOCUSTOMEVALUATORNAME`¶

The name of the evaluator. Must be unique within your account.

`io_evaluatorconfig` `TYPE REF TO /AWS1/CL_BDOEVALUATORCONFIG` `/AWS1/CL_BDOEVALUATORCONFIG`¶

The configuration for the evaluator, including LLM-as-a-Judge settings with instructions, rating scale, and model configuration.

`iv_level` `TYPE /AWS1/BDOEVALUATORLEVEL` `/AWS1/BDOEVALUATORLEVEL`¶

The evaluation level that determines the scope of evaluation. Valid values are TOOL_CALL for individual tool invocations, TRACE for single request-response interactions, or SESSION for entire conversation sessions.

Optional arguments:¶

`iv_clienttoken` `TYPE /AWS1/BDOCLIENTTOKEN` `/AWS1/BDOCLIENTTOKEN`¶

A unique, case-sensitive identifier to ensure that the API request completes no more than one time. If you don't specify this field, a value is randomly generated for you. If this token matches a previous request, the service ignores the request, but doesn't return an error. For more information, see Ensuring idempotency.

`iv_description` `TYPE /AWS1/BDOEVALUATORDESCRIPTION` `/AWS1/BDOEVALUATORDESCRIPTION`¶

The description of the evaluator that explains its purpose and evaluation criteria.

RETURNING¶

`oo_output` `TYPE REF TO /aws1/cl_bdocreateevaluatorrsp` `/AWS1/CL_BDOCREATEEVALUATORRSP`¶

Domain /AWS1/RT_ACCOUNT_ID

Primitive Type NUMC

Examples¶

Syntax Example¶

This is an example of the syntax for calling the method. It includes every possible argument and initializes every possible value. The data provided is not necessarily semantically accurate (for example the value "string" may be provided for something that is intended to be an instance ID, or in some cases two arguments may be mutually exclusive). The syntax shows the ABAP syntax for creating the various data structures.

DATA(lo_result) = lo_client->createevaluator(
  io_evaluatorconfig = new /aws1/cl_bdoevaluatorconfig(
    io_llmasajudge = new /aws1/cl_bdollmasajudgeevalu00(
      io_modelconfig = new /aws1/cl_bdoevaluatormodelcfg(
        io_bdrkevaluatormodelconfig = new /aws1/cl_bdobdrevaluatormdel00(
          io_addlmodelrequestfields = /AWS1/CL_RT_DOCUMENT=>FROM_JSON_STR( |\{"foo":"this is a JSON object..."\}| )
          io_inferenceconfig = new /aws1/cl_bdoinferenceconf(
            it_stopsequences = VALUE /aws1/cl_bdononemptystrlist_w=>tt_nonemptystringlist(
              ( new /aws1/cl_bdononemptystrlist_w( |string| ) )
            )
            iv_maxtokens = 123
            iv_temperature = '0.1'
            iv_topp = '0.1'
          )
          iv_modelid = |string|
        )
      )
      io_ratingscale = new /aws1/cl_bdoratingscale(
        it_categorical = VALUE /aws1/cl_bdocatgclscaledefn=>tt_categoricalscaledefinitions(
          (
            new /aws1/cl_bdocatgclscaledefn(
              iv_definition = |string|
              iv_label = |string|
            )
          )
        )
        it_numerical = VALUE /aws1/cl_bdonumericalscaledefn=>tt_numericalscaledefinitions(
          (
            new /aws1/cl_bdonumericalscaledefn(
              iv_definition = |string|
              iv_label = |string|
              iv_value = '0.1'
            )
          )
        )
      )
      iv_instructions = |string|
    )
  )
  iv_clienttoken = |string|
  iv_description = |string|
  iv_evaluatorname = |string|
  iv_level = |string|
).

This is an example of reading all possible response values

lo_result = lo_result.
IF lo_result IS NOT INITIAL.
  lv_customevaluatorarn = lo_result->get_evaluatorarn( ).
  lv_evaluatorid = lo_result->get_evaluatorid( ).
  lv_timestamp = lo_result->get_createdat( ).
  lv_evaluatorstatus = lo_result->get_status( ).
ENDIF.

/AWS1/IF_BDO=>CREATEEVALUATOR()¶