Interface InvokeGlobalInferenceGatewayRequest.Builder

  • Method Details

    • gatewayId

      The ID of the Global Inference Gateway to invoke. IAM policies refer to the gateway by its ARN, arn:<partition>:sagemaker:<region>:<account>:global-inference-gateway/<id> .

      Parameters:
      gatewayId - The ID of the Global Inference Gateway to invoke. IAM policies refer to the gateway by its ARN, arn:<partition>:sagemaker:<region>:<account>:global-inference-gateway/<id> .
      Returns:
      Returns a reference to this object so that method calls can be chained together.
    • accept

      The MIME type you want for the response. It is passed to the model server.

      Parameters:
      accept - The MIME type you want for the response. It is passed to the model server.
      Returns:
      Returns a reference to this object so that method calls can be chained together.
    • contentType

      The MIME type of the request body. Pass application/json (parameters such as charset=utf-8 are allowed): the body is one OpenAI JSON object. The Global Inference Gateway rejects other content types with a 415 error.

      Parameters:
      contentType - The MIME type of the request body. Pass application/json (parameters such as charset=utf-8 are allowed): the body is one OpenAI JSON object. The Global Inference Gateway rejects other content types with a 415 error.
      Returns:
      Returns a reference to this object so that method calls can be chained together.
    • customerIdentityToken

      InvokeGlobalInferenceGatewayRequest.Builder customerIdentityToken(String customerIdentityToken)

      Per-caller customer identity token (a JWT), forwarded unchanged to the inference gateway, which validates it. The gateway does not use it to authorize the request. Include it in the SigV4 signed headers so it cannot be altered in transit. Redacted from logs, metrics and traces.

      Parameters:
      customerIdentityToken - Per-caller customer identity token (a JWT), forwarded unchanged to the inference gateway, which validates it. The gateway does not use it to authorize the request. Include it in the SigV4 signed headers so it cannot be altered in transit. Redacted from logs, metrics and traces.
      Returns:
      Returns a reference to this object so that method calls can be chained together.
    • body

      The inference request body: one OpenAI JSON object, at most 10 MiB. Its content is passed to the model server unchanged, except that the request is always sent with "stream": false: this operation returns one JSON document, so a "stream": true in the body is overridden (use InvokeGlobalInferenceGatewayWithResponseStream to stream). A body that is not one JSON object, or that repeats the top-level model key with different values, is rejected with a 400 validation error. A body larger than 10 MiB is rejected with PayloadTooLargeException (413). The gateway reads only model and stream, and does not store the body.

      Parameters:
      body - The inference request body: one OpenAI JSON object, at most 10 MiB. Its content is passed to the model server unchanged, except that the request is always sent with "stream": false: this operation returns one JSON document, so a "stream": true in the body is overridden (use InvokeGlobalInferenceGatewayWithResponseStream to stream). A body that is not one JSON object, or that repeats the top-level model key with different values, is rejected with a 400 validation error. A body larger than 10 MiB is rejected with PayloadTooLargeException (413). The gateway reads only model and stream, and does not store the body.
      Returns:
      Returns a reference to this object so that method calls can be chained together.
    • overrideConfiguration

      Description copied from interface: AwsRequest.Builder
      Add an optional request override configuration.
      Specified by:
      overrideConfiguration in interface AwsRequest.Builder
      Parameters:
      overrideConfiguration - The override configuration.
      Returns:
      This object for method chaining.
    • overrideConfiguration

      Description copied from interface: AwsRequest.Builder
      Add an optional request override configuration.
      Specified by:
      overrideConfiguration in interface AwsRequest.Builder
      Parameters:
      builderConsumer - A Consumer to which an empty AwsRequestOverrideConfiguration.Builder will be given.
      Returns:
      This object for method chaining.