Class: Aws::SageMakerGatewayRuntime::Client
- Inherits:
-
Seahorse::Client::Base
- Object
- Seahorse::Client::Base
- Aws::SageMakerGatewayRuntime::Client
- Includes:
- ClientStubs
- Defined in:
- gems/aws-sdk-sagemakergatewayruntime/lib/aws-sdk-sagemakergatewayruntime/client.rb
Overview
An API client for SageMakerGatewayRuntime. To construct a client, you need to configure a :region and :credentials.
client = Aws::SageMakerGatewayRuntime::Client.new(
region: region_name,
credentials: credentials,
# ...
)
For details on configuring region and credentials see the developer guide.
See #initialize for a full list of supported configuration options.
Instance Attribute Summary
Attributes inherited from Seahorse::Client::Base
API Operations collapse
-
#invoke_global_inference_gateway(params = {}) ⇒ Types::InvokeGlobalInferenceGatewayOutput
Buffered (non-streaming) inference.
-
#invoke_global_inference_gateway_with_response_stream(params = {}) ⇒ Types::InvokeGlobalInferenceGatewayWithResponseStreamOutput
Streaming inference.
Instance Method Summary collapse
-
#initialize(options) ⇒ Client
constructor
A new instance of Client.
Methods included from ClientStubs
#api_requests, #stub_data, #stub_responses
Methods inherited from Seahorse::Client::Base
add_plugin, api, clear_plugins, define, new, #operation_names, plugins, remove_plugin, set_api, set_plugins
Methods included from Seahorse::Client::HandlerBuilder
#handle, #handle_request, #handle_response
Constructor Details
#initialize(options) ⇒ Client
Returns a new instance of Client.
490 491 492 |
# File 'gems/aws-sdk-sagemakergatewayruntime/lib/aws-sdk-sagemakergatewayruntime/client.rb', line 490 def initialize(*args) super end |
Instance Method Details
#invoke_global_inference_gateway(params = {}) ⇒ Types::InvokeGlobalInferenceGatewayOutput
Buffered (non-streaming) inference. Returns the full OpenAI completion body.
Errors carry their origin in the Message prefix. Every typed
error's text is in the Message member (JSON key Message), and
starts with one of these prefixes:
Global Inference Gateway:-- the gateway rejected or failed the request itself: validation, authorization, throttling, routing (the gateway or model was not found, or no capacity is available), a gateway limit or time limit, or failing to reach the inference gateway (Global Inference Gateway: could not reach the inference gateway: <reason>, orGlobal Inference Gateway: timed out waiting for the inference gateway). This includes the 400 for an invalid request and the 413 for a request body that is too large.inference gateway:-- the inference gateway serving the model returned an error status. The message gives that status code and up to 256 bytes of its error message, for exampleinference gateway: 503 <message>. A BadGatewayException can instead readinference gateway: response ended unexpectedly, with no status, when the inference gateway's connection failed while it was sending the response.inference gateway (<component>):-- the same, when the inference gateway reports which of its components produced the error, for exampleinference gateway (model server): 502 <message>. The component is one of:config-- the inference gateway's configuration and proxy layer.scheduler-- the inference gateway's request scheduler.model selector-- the inference gateway's model selector.model server-- the model server.
Every error status the inference gateway returns is a typed error, chosen by that status:
- 400 and any other 4xx not listed here -- a 400 validation error.
- 401 and 403 -- AccessDeniedException (403).
- 404 -- ResourceNotFoundException.
- 408 -- GatewayTimeoutException (504).
- 429 -- ThrottlingException, with the
Retry-Afterheader. - 502, and 500, 503 or any other 5xx not listed here -- BadGatewayException (502). A 500 or 503 from the inference gateway comes after the request was dispatched, so it is not modeled as retryable, and the gateway may already have tried another inference gateway. Standard SDK retry modes may still retry a 5xx status; set max attempts to 1 if a repeated inference is unacceptable.
- 504 -- GatewayTimeoutException.
603 604 605 606 |
# File 'gems/aws-sdk-sagemakergatewayruntime/lib/aws-sdk-sagemakergatewayruntime/client.rb', line 603 def invoke_global_inference_gateway(params = {}, = {}) req = build_request(:invoke_global_inference_gateway, params) req.send_request() end |
#invoke_global_inference_gateway_with_response_stream(params = {}) ⇒ Types::InvokeGlobalInferenceGatewayWithResponseStreamOutput
Streaming inference. Returns an AWS event stream of PayloadPart events that carry the model server's server-sent event (SSE) bytes, so SDKs decode the stream into events for you.
The errors listed for this operation, other than ModelStreamError and InternalStreamFailure, are returned before the response starts (before the 200 status), so they end the request, not the stream. None of them can occur after the 200. ModelStreamError and InternalStreamFailure appear only as events in the stream, after the 200. Once the 200 is sent, the stream carries the response as PayloadPart events. If reading the inference gateway's response fails, the stream carries one InternalStreamFailure event and then ends. ModelStreamError is reserved and is not currently sent.
Errors carry their origin in the Message prefix. Every typed
error's text is in the Message member (JSON key Message), and
starts with one of these prefixes:
Global Inference Gateway:-- the gateway rejected or failed the request itself: validation, authorization, throttling, routing (the gateway or model was not found, or no capacity is available), a gateway limit or time limit, or failing to reach the inference gateway (Global Inference Gateway: could not reach the inference gateway: <reason>, orGlobal Inference Gateway: timed out waiting for the inference gateway). This includes the 400 for an invalid request and the 413 for a request body that is too large.inference gateway:-- the inference gateway serving the model returned an error status. The message gives that status code and up to 256 bytes of its error message, for exampleinference gateway: 503 <message>.inference gateway (<component>):-- the same, when the inference gateway reports which of its components produced the error, for exampleinference gateway (model server): 502 <message>. The component is one of:config-- the inference gateway's configuration and proxy layer.scheduler-- the inference gateway's request scheduler.model selector-- the inference gateway's model selector.model server-- the model server.
An InternalStreamFailure event's Message uses the same prefixes,
with the same text the buffered operation uses for the same failure.
Its inference gateway: message carries no status: inference
gateway: response ended unexpectedly when the inference gateway's
connection failed while it was sending the response.
Every error status the inference gateway returns before the 200 is a typed error, chosen by that status as for InvokeGlobalInferenceGateway:
- 400 and any other 4xx not listed here -- a 400 validation error.
- 401 and 403 -- AccessDeniedException (403).
- 404 -- ResourceNotFoundException.
- 408 -- GatewayTimeoutException (504).
- 429 -- ThrottlingException, with the
Retry-Afterheader. - 502, and 500, 503 or any other 5xx not listed here -- BadGatewayException (502). A 500 or 503 from the inference gateway comes after the request was dispatched, so it is not modeled as retryable, and the gateway may already have tried another inference gateway. Standard SDK retry modes may still retry a 5xx status; set max attempts to 1 if a repeated inference is unacceptable.
- 504 -- GatewayTimeoutException.
840 841 842 843 844 845 846 847 848 849 850 851 852 853 854 855 856 857 858 859 860 861 |
# File 'gems/aws-sdk-sagemakergatewayruntime/lib/aws-sdk-sagemakergatewayruntime/client.rb', line 840 def invoke_global_inference_gateway_with_response_stream(params = {}, = {}, &block) params = params.dup event_stream_handler = case handler = params.delete(:event_stream_handler) when EventStreams::ResponseStream then handler when Proc then EventStreams::ResponseStream.new.tap(&handler) when nil then EventStreams::ResponseStream.new else msg = "expected :event_stream_handler to be a block or "\ "instance of Aws::SageMakerGatewayRuntime::EventStreams::ResponseStream"\ ", got `#{handler.inspect}` instead" raise ArgumentError, msg end yield(event_stream_handler) if block_given? req = build_request(:invoke_global_inference_gateway_with_response_stream, params) req.context[:event_stream_handler] = event_stream_handler req.handlers.add(Aws::Binary::DecodeHandler, priority: 95) req.send_request(, &block) end |