SampleTrainingSessionWithResponseStream
Streams inference responses from the model associated with the specified training session. Use this operation for real-time token-by-token generation. The service passes the request body to the model without modification.
Request Syntax
POST /training-sessions/trainingSessionName/sample-with-response-stream HTTP/1.1
Content-Type: contentType
Accept: accept
X-Amzn-SageMaker-Model-Package: modelPackageArn
bodyURI Request Parameters
The request uses the following URI parameters.
- accept
-
The desired MIME type of the inference body in the response.
- contentType
-
The MIME type of the input data in the request body.
- modelPackageArn
-
The ARN of the model package whose checkpoint to invoke.
- trainingSessionName
-
The name of the training session whose model to invoke.
Length Constraints: Minimum length of 1. Maximum length of 64.
Pattern:
[a-zA-Z0-9-]+Required: Yes
Request Body
The request accepts the following binary data.
- body
-
The raw inference request body.
Response Syntax
HTTP/1.1 200
Content-Type: contentType
bodyResponse Elements
If the action is successful, the service sends back an HTTP 200 response.
The response returns the following HTTP headers.
- contentType
-
The MIME type of the streaming inference result.
The response returns the following as the HTTP body.
- body
-
The streaming inference response from the model, delivered as a series of chunks.
Errors
For information about the errors that are common to all actions, see Common Error Types.
- ConflictException
-
The training session is in a state that cannot accept new operations. The session must be in
RunningorBlockedstate to accept requests.HTTP Status Code: 409
- InternalServerException
-
An internal service error occurred. Retry the request using exponential backoff.
HTTP Status Code: 500
- ResourceNotFoundException
-
The specified training session or operation does not exist, or the operation does not belong to the specified training session.
HTTP Status Code: 404
- ThrottlingException
-
The request was throttled. Retry the request after a brief wait using exponential backoff.
HTTP Status Code: 429
- ValidationException
-
The request is not valid. Check the request syntax and parameters. Examples include invalid sequence numbers, malformed client tokens, missing required fields, or invalid operation configuration.
HTTP Status Code: 400
See Also
For more information about using this API in one of the language-specific AWS SDKs, see the following:
