Request Syntax URI Request Parameters Request Body Response Syntax Response Elements Errors See Also

CreateAutomatedReasoningPolicyTestCase

Creates a test for an Automated Reasoning policy. Tests validate that your policy works as expected by providing sample inputs and expected outcomes. Use tests to verify policy behavior before deploying to production.

Request Syntax


POST /automated-reasoning-policies/policyArn/test-cases HTTP/1.1
Content-type: application/json

{
   "clientRequestToken": "string",
   "confidenceThreshold": number,
   "expectedAggregatedFindingsResult": "string",
   "guardContent": "string",
   "queryContent": "string"
}

URI Request Parameters

The request uses the following URI parameters.

policyArn

The Amazon Resource Name (ARN) of the Automated Reasoning policy for which to create the test.

Length Constraints: Minimum length of 1. Maximum length of 2048.

Pattern: arn:aws(-[^:]+)?:bedrock:[a-z0-9-]{1,20}:[0-9]{12}:automated-reasoning-policy/[a-z0-9]{12}(:([1-9][0-9]{0,11}))?

Required: Yes

Request Body

The request accepts the following data in JSON format.

clientRequestToken

A unique, case-sensitive identifier to ensure that the operation completes no more than one time. If this token matches a previous request, Amazon Bedrock ignores the request, but does not return an error.

Type: String

Length Constraints: Minimum length of 1. Maximum length of 256.

Pattern: [a-zA-Z0-9](-*[a-zA-Z0-9])*

Required: No

confidenceThreshold

The minimum confidence level for logic validation. Content that meets the threshold is considered a high-confidence finding that can be validated.

Type: Double

Valid Range: Minimum value of 0. Maximum value of 1.

Required: No

expectedAggregatedFindingsResult

The expected result of the Automated Reasoning check. Valid values include: , TOO_COMPLEX, and NO_TRANSLATIONS.

VALID - The claims are true. The claims are implied by the premises and the Automated Reasoning policy. Given the Automated Reasoning policy and premises, it is not possible for these claims to be false. In other words, there are no alternative answers that are true that contradict the claims.
INVALID - The claims are false. The claims are not implied by the premises and Automated Reasoning policy. Furthermore, there exists different claims that are consistent with the premises and Automated Reasoning policy.
SATISFIABLE - The claims can be true or false. It depends on what assumptions are made for the claim to be implied from the premises and Automated Reasoning policy rules. In this situation, different assumptions can make input claims false and alternative claims true.
IMPOSSIBLE - Automated Reasoning can’t make a statement about the claims. This can happen if the premises are logically incorrect, or if there is a conflict within the Automated Reasoning policy itself.
TRANSLATION_AMBIGUOUS - Detected an ambiguity in the translation meant it would be unsound to continue with validity checking. Additional context or follow-up questions might be needed to get translation to succeed.
TOO_COMPLEX - The input contains too much information for Automated Reasoning to process within its latency limits.
NO_TRANSLATIONS - Identifies that some or all of the input prompt wasn't translated into logic. This can happen if the input isn't relevant to the Automated Reasoning policy, or if the policy doesn't have variables to model relevant input. If Automated Reasoning can't translate anything, you get a single NO_TRANSLATIONS finding. You might also see a NO_TRANSLATIONS (along with other findings) if some part of the validation isn't translated.

Type: String

Required: Yes

guardContent

The output content that's validated by the Automated Reasoning policy. This represents the foundation model response that will be checked for accuracy.

Type: String

Length Constraints: Minimum length of 0. Maximum length of 1024.

Required: Yes

queryContent

The input query or prompt that generated the content. This provides context for the validation.

Type: String

Length Constraints: Minimum length of 0. Maximum length of 1024.

Required: No

Response Syntax


HTTP/1.1 200
Content-type: application/json

{
   "policyArn": "string",
   "testCaseId": "string"
}

Response Elements

If the action is successful, the service sends back an HTTP 200 response.

The following data is returned in JSON format by the service.

policyArn

The Amazon Resource Name (ARN) of the policy for which the test was created.

Type: String

Length Constraints: Minimum length of 1. Maximum length of 2048.

Pattern: arn:aws(-[^:]+)?:bedrock:[a-z0-9-]{1,20}:[0-9]{12}:automated-reasoning-policy/[a-z0-9]{12}(:([1-9][0-9]{0,11}))?

testCaseId

The unique identifier of the created test.

Type: String

Length Constraints: Minimum length of 0. Maximum length of 12.

Pattern: [0-9A-Z]{12}

Errors

For information about the errors that are common to all actions, see Common Errors.

AccessDeniedException

The request is denied because of missing access permissions.

HTTP Status Code: 403

ConflictException

Error occurred because of a conflict while performing an operation.

HTTP Status Code: 400

InternalServerException

An internal server error occurred. Retry your request.

HTTP Status Code: 500

ResourceNotFoundException

The specified resource Amazon Resource Name (ARN) was not found. Check the Amazon Resource Name (ARN) and try your request again.

HTTP Status Code: 404

ServiceQuotaExceededException

The number of requests exceeds the service quota. Resubmit your request later.

HTTP Status Code: 400

ThrottlingException

The number of requests exceeds the limit. Resubmit your request later.

HTTP Status Code: 429

ValidationException

Input validation failed. Check your request parameters and retry the request.

HTTP Status Code: 400