Request Syntax URI Request Parameters Request Body Response Syntax Response Elements Errors See Also

StartBatchEvaluation

Starts a batch evaluation job that evaluates agent performance across multiple sessions. Batch evaluations pull agent traces from CloudWatch Logs or an existing online evaluation configuration and run specified evaluators and insights against them.

Request Syntax


POST /evaluations/batch-evaluate HTTP/1.1
Content-type: application/json

{
   "batchEvaluationName": "string",
   "clientToken": "string",
   "dataSourceConfig": { ... },
   "description": "string",
   "evaluationMetadata": { ... },
   "evaluators": [ 
      { 
         "evaluatorId": "string"
      }
   ]
}

URI Request Parameters

The request does not use any URI parameters.

Request Body

The request accepts the following data in JSON format.

batchEvaluationName

The name of the batch evaluation. Must be unique within your account.

Type: String

Pattern: [a-zA-Z][a-zA-Z0-9_]{0,47}

Required: Yes

clientToken

A unique, case-sensitive identifier to ensure that the API request completes no more than one time. If this token matches a previous request, the service ignores the request, but does not return an error.

Type: String

Length Constraints: Minimum length of 33. Maximum length of 256.

Pattern: [a-zA-Z0-9](-*[a-zA-Z0-9]){0,256}

Required: No

dataSourceConfig

The data source configuration that specifies where to pull agent session traces from for evaluation.

Type: DataSourceConfig object

Note: This object is a Union. Only one member of this object can be specified or returned.

Required: Yes

description

The description of the batch evaluation.

Type: String

Length Constraints: Minimum length of 0. Maximum length of 200.

Required: No

evaluationMetadata

Optional metadata for the evaluation, including session-specific ground truth data and test scenario identifiers.

Type: EvaluationMetadata object

Note: This object is a Union. Only one member of this object can be specified or returned.

Required: No

evaluators

The list of evaluators to apply during the batch evaluation. Can include both built-in evaluators and custom evaluators. Maximum of 10 evaluators.

Type: Array of Evaluator objects

Array Members: Minimum number of 0 items. Maximum number of 10 items.

Required: No

Response Syntax


HTTP/1.1 202
Content-type: application/json

{
   "batchEvaluationArn": "string",
   "batchEvaluationId": "string",
   "batchEvaluationName": "string",
   "createdAt": "string",
   "description": "string",
   "evaluators": [ 
      { 
         "evaluatorId": "string"
      }
   ],
   "outputConfig": { ... },
   "status": "string"
}

Response Elements

If the action is successful, the service sends back an HTTP 202 response.

The following data is returned in JSON format by the service.

batchEvaluationArn

The Amazon Resource Name (ARN) of the created batch evaluation.

Type: String

batchEvaluationId

The unique identifier of the created batch evaluation.

Type: String

Pattern: [a-zA-Z][a-zA-Z0-9-_]{0,99}-[a-zA-Z0-9]{10}

batchEvaluationName

The name of the batch evaluation.

Type: String

Pattern: [a-zA-Z][a-zA-Z0-9_]{0,47}

createdAt

The timestamp when the batch evaluation was created.

Type: Timestamp

description

The description of the batch evaluation.

Type: String

Length Constraints: Minimum length of 0. Maximum length of 200.

evaluators

The list of evaluators applied during the batch evaluation.

Type: Array of Evaluator objects

outputConfig

The output configuration specifying where evaluation results are written.

Type: OutputConfig object

Note: This object is a Union. Only one member of this object can be specified or returned.

status

The status of the batch evaluation.

Type: String

Errors

For information about the errors that are common to all actions, see Common Error Types.

AccessDeniedException

The exception that occurs when you do not have sufficient permissions to perform an action. Verify that your IAM policy includes the necessary permissions for the operation you are trying to perform.

HTTP Status Code: 403

ConflictException

The exception that occurs when the request conflicts with the current state of the resource. This can happen when trying to modify a resource that is currently being modified by another request, or when trying to create a resource that already exists.

HTTP Status Code: 409

InternalServerException

The exception that occurs when the service encounters an unexpected internal error. This is a temporary condition that will resolve itself with retries. We recommend implementing exponential backoff retry logic in your application.

HTTP Status Code: 500

ServiceQuotaExceededException

The exception that occurs when the request would cause a service quota to be exceeded. Review your service quotas and either reduce your request rate or request a quota increase.

HTTP Status Code: 402

ThrottlingException

The exception that occurs when the request was denied due to request throttling. This happens when you exceed the allowed request rate for an operation. Reduce the frequency of requests or implement exponential backoff retry logic in your application.

HTTP Status Code: 429

UnauthorizedException

This exception is thrown when the JWT bearer token is invalid or not found for OAuth bearer token based access

HTTP Status Code: 401

ValidationException

The exception that occurs when the input fails to satisfy the constraints specified by the service. Check the error message for details about which input parameter is invalid and correct your request.

HTTP Status Code: 400