InferenceConfiguration
The configuration parameters that control how the foundation model behaves during evaluation, including response generation settings.
Contents
- maxTokens
-
The maximum number of tokens to generate in the model response during evaluation.
Type: Integer
Valid Range: Minimum value of 1.
Required: No
- stopSequences
-
The list of sequences that will cause the model to stop generating tokens when encountered.
Type: Array of strings
Array Members: Minimum number of 0 items. Maximum number of 2500 items.
Length Constraints: Minimum length of 1.
Required: No
- temperature
-
The temperature value that controls randomness in the model's responses. Lower values produce more deterministic outputs.
Type: Float
Valid Range: Minimum value of 0. Maximum value of 1.
Required: No
- topP
-
The top-p sampling parameter that controls the diversity of the model's responses by limiting the cumulative probability of token choices.
Type: Float
Valid Range: Minimum value of 0. Maximum value of 1.
Required: No
See Also
For more information about using this API in one of the language-specific AWS SDKs, see the following:
