/AWS1/CL_BDZTEXTINFERENCECFG¶
Configuration settings for text generation using a language model via the RetrieveAndGenerate operation. Includes parameters like temperature, top-p, maximum token count, and stop sequences.
The valid range of maxTokens depends on the accepted values for your chosen model's inference parameters. To see the inference parameters for your model, see Inference parameters for foundation models.
CONSTRUCTOR¶
IMPORTING¶
Optional arguments:¶
iv_temperature TYPE /AWS1/RT_FLOAT_AS_STRING /AWS1/RT_FLOAT_AS_STRING¶
Controls the random-ness of text generated by the language model, influencing how much the model sticks to the most predictable next words versus exploring more surprising options. A lower temperature value (e.g. 0.2 or 0.3) makes model outputs more deterministic or predictable, while a higher temperature (e.g. 0.8 or 0.9) makes the outputs more creative or unpredictable.
iv_topp TYPE /AWS1/RT_FLOAT_AS_STRING /AWS1/RT_FLOAT_AS_STRING¶
A probability distribution threshold which controls what the model considers for the set of possible next tokens. The model will only consider the top p% of the probability distribution when generating the next token.
iv_maxtokens TYPE /AWS1/BDZMAXTOKENS /AWS1/BDZMAXTOKENS¶
The maximum number of tokens to generate in the output text. Do not use the minimum of 0 or the maximum of 65536. The limit values described here are arbitary values, for actual values consult the limits defined by your specific model.
it_stopsequences TYPE /AWS1/CL_BDZRAGSTOPSEQUENCES_W=>TT_RAGSTOPSEQUENCES TT_RAGSTOPSEQUENCES¶
A list of sequences of characters that, if generated, will cause the model to stop generating further tokens. Do not use a minimum length of 1 or a maximum length of 1000. The limit values described here are arbitary values, for actual values consult the limits defined by your specific model.
Queryable Attributes¶
temperature¶
Controls the random-ness of text generated by the language model, influencing how much the model sticks to the most predictable next words versus exploring more surprising options. A lower temperature value (e.g. 0.2 or 0.3) makes model outputs more deterministic or predictable, while a higher temperature (e.g. 0.8 or 0.9) makes the outputs more creative or unpredictable.
Accessible with the following methods¶
| Method | Description |
|---|---|
GET_TEMPERATURE() |
Getter for TEMPERATURE, with configurable default |
ASK_TEMPERATURE() |
Getter for TEMPERATURE w/ exceptions if field has no value |
STR_TEMPERATURE() |
String format for TEMPERATURE, with configurable default |
HAS_TEMPERATURE() |
Determine if TEMPERATURE has a value |
topP¶
A probability distribution threshold which controls what the model considers for the set of possible next tokens. The model will only consider the top p% of the probability distribution when generating the next token.
Accessible with the following methods¶
| Method | Description |
|---|---|
GET_TOPP() |
Getter for TOPP, with configurable default |
ASK_TOPP() |
Getter for TOPP w/ exceptions if field has no value |
STR_TOPP() |
String format for TOPP, with configurable default |
HAS_TOPP() |
Determine if TOPP has a value |
maxTokens¶
The maximum number of tokens to generate in the output text. Do not use the minimum of 0 or the maximum of 65536. The limit values described here are arbitary values, for actual values consult the limits defined by your specific model.
Accessible with the following methods¶
| Method | Description |
|---|---|
GET_MAXTOKENS() |
Getter for MAXTOKENS, with configurable default |
ASK_MAXTOKENS() |
Getter for MAXTOKENS w/ exceptions if field has no value |
HAS_MAXTOKENS() |
Determine if MAXTOKENS has a value |
stopSequences¶
A list of sequences of characters that, if generated, will cause the model to stop generating further tokens. Do not use a minimum length of 1 or a maximum length of 1000. The limit values described here are arbitary values, for actual values consult the limits defined by your specific model.
Accessible with the following methods¶
| Method | Description |
|---|---|
GET_STOPSEQUENCES() |
Getter for STOPSEQUENCES, with configurable default |
ASK_STOPSEQUENCES() |
Getter for STOPSEQUENCES w/ exceptions if field has no value |
HAS_STOPSEQUENCES() |
Determine if STOPSEQUENCES has a value |