INVOKEENDPOINTWITHBIDISTREAM()`¶

About InvokeEndpointWithBidirectionalStream¶

Invokes a model endpoint with bidirectional streaming capabilities. This operation establishes a persistent connection that allows you to send multiple requests and receive streaming responses from the model in real-time.

Bidirectional streaming is useful for interactive applications such as chatbots, real-time translation, or any scenario where you need to maintain a conversation-like interaction with the model. The connection remains open, allowing you to send additional input and receive responses without establishing a new connection for each request.

For an overview of Amazon SageMaker AI, see How It Works.

Amazon SageMaker AI strips all POST headers except those supported by the API. Amazon SageMaker AI might add additional headers. You should not rely on the behavior of headers outside those enumerated in the request syntax.

Calls to InvokeEndpointWithBidirectionalStream are authenticated by using Amazon Web Services Signature Version 4. For information, see Authenticating Requests (Amazon Web Services Signature Version 4) in the Amazon S3 API Reference.

The bidirectional stream maintains the connection until either the client closes it or the model indicates completion. Each request and response in the stream is sent as an event with optional headers for data type and completion state.

Endpoints are scoped to an individual account, and are not public. The URL does not contain the account ID, but Amazon SageMaker AI determines the account ID from the authentication token that is supplied by the caller.

Method Signature¶

METHODS /AWS1/IF_SGH~INVOKEENDPOINTWITHBIDISTREAM
  IMPORTING
    !IV_ENDPOINTNAME TYPE /AWS1/SGHSTRING OPTIONAL
    !IO_BODY TYPE REF TO /AWS1/IF_SGHREQSTREAMEVENT_IS OPTIONAL
    !IV_TARGETVARIANT TYPE /AWS1/SGHSTRING OPTIONAL
    !IV_MODELINVOCATIONPATH TYPE /AWS1/SGHSTRING OPTIONAL
    !IV_MODELQUERYSTRING TYPE /AWS1/SGHSTRING OPTIONAL
  RETURNING
    VALUE(OO_OUTPUT) TYPE REF TO /aws1/cl_sghinvendptwbistrmout
  RAISING
    /AWS1/CX_SGHINPUTVLDTNERROR
    /AWS1/CX_SGHINTSERVERERROR
    /AWS1/CX_SGHINTSTREAMFAILURE
    /AWS1/CX_SGHMODELERROR
    /AWS1/CX_SGHMODELSTREAMERROR
    /AWS1/CX_SGHSVCUNAVAILERROR
    /AWS1/CX_SGHCLIENTEXC
    /AWS1/CX_SGHSERVEREXC
    /AWS1/CX_RT_TECHNICAL_GENERIC
    /AWS1/CX_RT_SERVICE_GENERIC.

IMPORTING¶

Required arguments:¶

`iv_endpointname` `TYPE /AWS1/SGHSTRING` `/AWS1/SGHSTRING`¶

The name of the endpoint to invoke.

`io_body` `TYPE REF TO /AWS1/IF_SGHREQSTREAMEVENT_IS` `/AWS1/IF_SGHREQSTREAMEVENT_IS`¶

The request payload stream.

Optional arguments:¶

`iv_targetvariant` `TYPE /AWS1/SGHSTRING` `/AWS1/SGHSTRING`¶

Target variant for the request.

`iv_modelinvocationpath` `TYPE /AWS1/SGHSTRING` `/AWS1/SGHSTRING`¶

Model invocation path.

`iv_modelquerystring` `TYPE /AWS1/SGHSTRING` `/AWS1/SGHSTRING`¶

Model query string.

RETURNING¶

`oo_output` `TYPE REF TO /aws1/cl_sghinvendptwbistrmout` `/AWS1/CL_SGHINVENDPTWBISTRMOUT`¶

Domain /AWS1/RT_ACCOUNT_ID

Primitive Type NUMC

Examples¶

Syntax Example¶

This is an example of the syntax for calling the method. It includes every possible argument and initializes every possible value. The data provided is not necessarily semantically accurate (for example the value "string" may be provided for something that is intended to be an instance ID, or in some cases two arguments may be mutually exclusive). The syntax shows the ABAP syntax for creating the various data structures.

DATA(lo_result) = lo_client->invokeendpointwithbidistream(
  io_body = lo_stream
  iv_endpointname = |string|
  iv_modelinvocationpath = |string|
  iv_modelquerystring = |string|
  iv_targetvariant = |string|
).

This is an example of reading all possible response values

lo_result = lo_result.
IF lo_result IS NOT INITIAL.
  TRY.
    WHILE lo_result->get_body( )->/aws1/if_rt_stream_reader~data_available( ) = ABAP_TRUE.
      lo_event = lo_result->get_body( )->READ( ).
      IF lo_event->get_payloadpart( ) IS NOT INITIAL.
        " process this kind of event
      ENDIF.
    ENDWHILE.
  CATCH /aws1/cx_sghintstreamfailure.
    " handle error in stream
  CATCH /aws1/cx_sghmodelstreamerror.
    " handle error in stream
  ENDTRY.
  lv_string = lo_result->get_invokedproductionvariant( ).
ENDIF.

/AWS1/IF_SGH=>INVOKEENDPOINTWITHBIDISTREAM()¶