PromptInferenceConfigurationProps

class aws_cdk.aws_bedrock_alpha.PromptInferenceConfigurationProps(*, max_tokens=None, stop_sequences=None, temperature=None, top_p=None)

Bases: object

(experimental) Properties for creating a prompt inference configuration.

Parameters:

  • max_tokens (Union[int, float, None]) – (experimental) The maximum number of tokens to return in the response. Default: - No limit specified

  • stop_sequences (Optional[Sequence[str]]) – (experimental) A list of strings that define sequences after which the model will stop generating. Default: - No stop sequences

  • temperature (Union[int, float, None]) – (experimental) Controls the randomness of the response. Higher values make output more random, lower values more deterministic. Valid range is 0.0 to 1.0. Default: - Model default temperature

  • top_p (Union[int, float, None]) – (experimental) The percentage of most-likely candidates that the model considers for the next token. Valid range is 0.0 to 1.0. Default: - Model default topP

Stability:

experimental

ExampleMetadata:

fixture=default infused

Example:

cmk = kms.Key(self, "cmk")
claude_model = bedrock.BedrockFoundationModel.ANTHROPIC_CLAUDE_SONNET_V1_0

variant1 = bedrock.PromptVariant.text(
    variant_name="variant1",
    model=claude_model,
    prompt_variables=["topic"],
    prompt_text="This is my first text prompt. Please summarize our conversation on: {{topic}}.",
    inference_configuration=bedrock.PromptInferenceConfiguration.text(
        temperature=1,
        top_p=0.999,
        max_tokens=2000
    )
)

prompt1 = bedrock.Prompt(self, "prompt1",
    prompt_name="prompt1",
    description="my first prompt",
    default_variant=variant1,
    variants=[variant1],
    kms_key=cmk
)

Attributes

max_tokens

(experimental) The maximum number of tokens to return in the response.

Default:

  • No limit specified

Stability:

experimental

stop_sequences

(experimental) A list of strings that define sequences after which the model will stop generating.

Default:

  • No stop sequences

Stability:

experimental

temperature

(experimental) Controls the randomness of the response.

Higher values make output more random, lower values more deterministic. Valid range is 0.0 to 1.0.

Default:

  • Model default temperature

Stability:

experimental

top_p

(experimental) The percentage of most-likely candidates that the model considers for the next token.

Valid range is 0.0 to 1.0.

Default:

  • Model default topP

Stability:

experimental