Request Syntax Request Parameters Response Syntax Response Elements Errors Examples See Also

CreateEvaluation

Creates a new Evaluation of an MLModel. An MLModel is evaluated on a set of observations associated to a DataSource. Like a DataSource for an MLModel, the DataSource for an Evaluation contains values for the Target Variable. The Evaluation compares the predicted result for each observation to the actual outcome and provides a summary so that you know how effective the MLModel functions on the test data. Evaluation generates a relevant performance metric, such as BinaryAUC, RegressionRMSE or MulticlassAvgFScore based on the corresponding MLModelType: BINARY, REGRESSION or MULTICLASS.

CreateEvaluation is an asynchronous operation. In response to CreateEvaluation, Amazon Machine Learning (Amazon ML) immediately returns and sets the evaluation status to PENDING. After the Evaluation is created and ready for use, Amazon ML sets the status to COMPLETED.

You can use the GetEvaluation operation to check progress of the evaluation during the creation operation.

Request Syntax


{
   "EvaluationDataSourceId": "string",
   "EvaluationId": "string",
   "EvaluationName": "string",
   "MLModelId": "string"
}

Request Parameters

For information about the parameters that are common to all actions, see Common Parameters.

The request accepts the following data in JSON format.

EvaluationDataSourceId

The ID of the DataSource for the evaluation. The schema of the DataSource must match the schema used to create the MLModel.

Type: String

Length Constraints: Minimum length of 1. Maximum length of 64.

Pattern: [a-zA-Z0-9_.-]+

Required: Yes

EvaluationId

A user-supplied ID that uniquely identifies the Evaluation.

Type: String

Length Constraints: Minimum length of 1. Maximum length of 64.

Pattern: [a-zA-Z0-9_.-]+

Required: Yes

EvaluationName

A user-supplied name or description of the Evaluation.

Type: String

Length Constraints: Maximum length of 1024.

Pattern: .*\S.*|^$

Required: No

MLModelId

The ID of the MLModel to evaluate.

The schema used in creating the MLModel must match the schema of the DataSource used in the Evaluation.

Type: String

Length Constraints: Minimum length of 1. Maximum length of 64.

Pattern: [a-zA-Z0-9_.-]+

Required: Yes

Response Syntax


{
   "EvaluationId": "string"
}

Response Elements

If the action is successful, the service sends back an HTTP 200 response.

The following data is returned in JSON format by the service.

EvaluationId

The user-supplied ID that uniquely identifies the Evaluation. This value should be identical to the value of the EvaluationId in the request.

Type: String

Length Constraints: Minimum length of 1. Maximum length of 64.

Pattern: [a-zA-Z0-9_.-]+

Errors

For information about the errors that are common to all actions, see Common Errors.

IdempotentParameterMismatchException

A second request to use or change an object was not allowed. This can result from retrying a request using a parameter that was not present in the original request.

HTTP Status Code: 400

InternalServerException

An error on the server occurred when trying to process a request.

HTTP Status Code: 500

InvalidInputException

An error on the client occurred. Typically, the cause is an invalid input value.

HTTP Status Code: 400

Examples

The following is a sample request and response of the CreateEvaluation operation:

This example illustrates one usage of CreateEvaluation.

Sample Request


POST / HTTP/1.1
Host: machinelearning.<region>.<domain>
x-amz-Date: <Date>
Authorization: AWS4-HMAC-SHA256 Credential=<Credential>, SignedHeaders=contenttype;date;host;user-agent;x-amz-date;x-amz-target;x-amzn-requestid,Signature=<Signature>
User-Agent: <UserAgentString>
Content-Type: application/x-amz-json-1.1
Content-Length: <PayloadSizeBytes>
Connection: Keep-Alive
X-Amz-Target: AmazonML_20141212.CreateEvaluation
{
  "EvaluationId": "CreateEvaluation-pr-2014-09-12-15-14-04-924", 
  "EvaluationName": "EXAMPLE", 
  "MLModelId": "EXAMPLE-pr-2014-09-12-15-14-04-924", 
  "EvaluationDataSourceId": "EXAMPLE-ev-ds-2014-09-12-15-14-04-411", 
}

Sample Response


HTTP/1.1 200 OK
x-amzn-RequestId: <RequestId>
Content-Type: application/x-amz-json-1.1
Content-Length: <PayloadSizeBytes>
Date: <Date>
{"EvaluationId":"CreateEvaluation-pr-2014-09-12-15-14-04-924"}