[ aws ]
Amazon SageMaker Inference Router Runtime is the runtime API for Amazon SageMaker AI inference routers. An inference router sends each request that you make to one of the destinations of a routing target. A destination is a SageMaker AI endpoint or inference component, in your account or in another account, in the router’s Amazon Web Services Region or in another Region in the same Amazon Web Services partition. The router chooses a healthy destination with capacity, and prefers destinations in its own Region.
Use InvokeInferenceRouter to get an inference in real time. Use InvokeInferenceRouterAsync to submit a request whose input and response are stored in Amazon S3.
The inference router runtime endpoint supports HTTP/1.1 only. HTTP/2 isn’t supported. If you use the Amazon Web Services SDKs or the Amazon Web Services CLI, you don’t need to change anything.