Submit prompts and generate responses using the API - Amazon Bedrock

Submit prompts and generate responses using the API

Amazon Bedrock offers the followingAPI operations for carrying out model inference:

  • InvokeModel – Submit a prompt and generate a response. The request body is model-specific. To generate streaming responses, use InvokeModelWithResponseStream.

  • Converse – Submit a prompt and generate responses with a structure unified across all models. Model-specific request fields can be specified in the additionalModelRequestFields field. You can also include system prompts and previous conversation for context. To generate streaming responses, use ConverseStream.

  • StartAsyncInvoke – Submit a prompt and generate a response asynchronously that can be retrieved later. Used to generate videos.

  • InvokeModelWithBidirectionalStream –

  • OpenAI Chat completions API – Use the OpenAI Chat Completions API with models supported by Amazon Bedrock to generate a response.

For model inference, you need to determine the following parameters:

  • Model ID – The ID or Amazon Resource Name (ARN) of the model or inference profile to use in the modelId field for inference. The following table describes how to find IDs for different types of resources:

    Model type Description Find ID in console Find ID in API Relevant documentation
    Base model A foundation model from a provider. Choose Base models from the left navigation pane, search for a model, and look for the Model ID. Send a GetFoundationModel or ListFoundationModels request and find the modelId in the response. See a list of IDs at Supported foundation models in Amazon Bedrock.
    Inference profile Increases throughput by allowing invocation of a model in multiple regions. Choose Cross-Region inference from the left navigation pane and look for an Inference profile ID. Send a GetInferenceProfile or ListInferenceProfiles request and find the inferenceProfileId in the response. See a list of IDs at Supported Regions and models for inference profiles.
    Prompt A prompt that was constructed using Prompt management. Choose Prompt management from the left navigation pane, select a prompt in the Prompts section, and look for the Prompt ARN. Send a GetPrompt or ListPrompts request and find the promptArn in the response. Learn about creating a prompt in Prompt management at Construct and store reusable prompts with Prompt management in Amazon Bedrock.
    Provisioned Throughput Provides a higher level of throughput for a model at a fixed cost. Choose Provisioned Throughput from the left navigation pane, select a Provisioned Throughput, and look for the ARN. Send a GetProvisionedModelThroughput or ListProvisionedModelThroughputs request and find the provisionedModelArn in the response. Learn how to purchase a Provisioned Throughput for a model at Increase model invocation capacity with Provisioned Throughput in Amazon Bedrock.
    Custom model A model whose parameters are shifted from a foundation model based on training data. After purchasing Provisioned Throughput for a custom model, follow the steps to find the ID for the Provisioned Throughput. After purchasing Provisioned Throughput for a custom model, follow the steps to find the ID for the Provisioned Throughput. Learn how to customize a model at Customize your model to improve its performance for your use case. After customization, you must purchase Provisioned Throughput for it and use the ID of the Provisioned Throughput.

  • Request body – Contains the inference parameters for a model and other configurations. Each base model has its own inference parameters. The inference parameters for a custom or provisioned model depends on the base model from which it was created. For more information, see Inference request parameters and response fields for foundation models.

Select a topic to learn how to use the model invocation APIs.

Topics