Advanced prompting techniques

Amazon Nova 2 models offer an optional reasoning mode that enhances the model's approach to complex problem-solving by allowing it to work through problems systematically before responding. Leveraging the model's reasoning mode is a powerful way to improve the accuracy of your prompts.

When to use it: Reasoning mode is recommended for complex tasks such as use cases with:

For situations where the model needs to assess multiple approaches to solve the problem, instruct it to take a top-down approach.

Example:

{{User query}}. Start with the big picture and break it down into progressively smaller, more detailed subproblems or steps.

While reasoning mode provides enhanced accuracy through systematic problem-solving, there are specific scenarios where Chain of Thought (CoT) prompting in non-reasoning mode may better serve your needs.

When to use it:

Guiding the model's CoT direction:

{{User query}} Please follow these steps:

1. {{Step 1}}
2. {{Step 2}}
...

Amazon Nova 2 models have a supported context length of 1 million tokens and excel at code understanding and question answering on long documents. Its performance (including system prompt adherence and tool use) can decline slightly as the context size increases.

How to use it:

Example Template:

// Provide your long inputs at the top of your prompt
BEGIN INPUT DOCUMENTS

DOCUMENT 1 START
{{Your document}}
DOCUMENT 1 END

END INPUT DOCUMENTS

// Then specify your query and instructions
BEGIN QUESTION
{{User query}}
END QUESTION

BEGIN INSTRUCTIONS
{{Instructions}}
END INSTRUCTIONS

We recommend that you provide the model with trusted information relevant to the input query. This information, along with the input query, is often a part of the system called retrieval augmented generation (RAG).

System:
In this session, the model has access to search results and a user's question, your job is to answer the user's question using only information from the search results. 

Model Instructions:
- DO NOT USE INFORMATION THAT IS NOT IN SEARCH RESULTS!

User: {Query} 
Resource: Search Results: {Reference texts}

Ground using citation markers

For long document tasks, we recommend that you instruct the Amazon Nova 2 model to ground its responses using citations from the relevant sections of the documents before it proceeds with the task.

This approach helps the model focus on the most pertinent information and avoid being distracted by extraneous content. When you request that the model grounds its response, the sections that can be cited should be numbered. For example, Passage %[1]%, Passage %[2]% and so on.

You are an AI financial assistant. Your task is to find patterns and insights from multi-year financial documents 

Passage %[1]%
{{Your document}}

Passage %[2]%
{{Your document}}

## Task:
Analyze my LLC's reports across multiple years to identify significant performance trends, segment growth patterns and strategic shifts.

## Context information:
- You have access to my LLC's annual financial reports (10-K) for multiple fiscal years in PDF format
- These reports contain comprehensive financial data including income statements, balance sheets, cash flow statements and management discussions
- The analysis should focus on year-over-year comparisons to identify meaningful trends
- I operate two business segments, one in Massachusetts and one in New York

Based on the provided Context, extract key financial metrics from each year's reports phrases from the documents.
Place citations as inline markers (e.g., %[1]%, %[2]%, etc.) directly within the relevant parts of the response 
text. Do not include a separate citation section after the response.
## Response Schema:
%% (Extracted Financial Metrics)
%% (Extracted Financial Metrics)
%% (Extracted Financial Metrics)
...

## Task
Analyze my LLC's financial reports across multiple years to identify significant performance trends, segment growth patterns and strategic shifts.

{{extracted financial metrics}}

## Model Instructions:
- Organize data chronologically to identify meaningful trends
- DO compare segment performance across the five-year period
- DO identify significant strategic shifts or investments mentioned in management discussions
- DO NOT make speculative predictions beyond what is supported by the data
- ALWAYS note any changes in accounting practices or reporting methodologies that might affect year-over-year comparisons

## Response style and format requirements:
- Respond in markdown
- Structure the analysis with clear headings and subheadings
- Present key financial metrics in tabular format showing all five years side-by-side
- Include percentage changes year-over-year for all major metrics
- Create a section dedicated to visualizing the most significant trends (with descriptions of what would be shown in charts)
- Limit the executive summary to 250 words maximum
- Format segment analysis as separate sections with consistent metrics across all segments
- MUST include a Key Insights bullet-pointed list at the end of each major section

Use Nova web grounding

Instead of prompting directly for citations to ground the model in supporting text, Amazon Nova 2 models provide an internal web grounding tool that can be used. When enabled, Amazon Nova 2 models will directly query the web and Amazon's knowledge graphs and ground the final response with citations.

To learn more on how to leverage Amazon Nova Web Grounding, you can reference the Amazon Nova Web Grounding user guide.

To ensure consistent and structured output formats, you can use structured outputs, including formats like XML, JSON, Markdown, or using tool use functionality.

For example, if the downstream parser expects specific naming conventions for keys in a JSON object, you should specify the response schema at the end of the prompt. Additionally, if you prefer responses to be in JSON format without any preamble text, instruct the model accordingly. That is, explicitly state PLEASE GENERATE ONLY THE JSON OUTPUT. DO NOT PROVIDE ANY PREAMBLE. to ensure clean output.

Common formatting schemas

The following are examples of common formatting schemas.

JSON:

JSON_format = """Write your response following the JSON format below:
```json
{
  "key1": "value1",
  "key2": "value2",
  "key3": [{
    "key3_1": "value_3_1 written in YYYY/MM/DD format",
    "key3_2": "value_3_2 day of the week written in full form",
    ...
  }]
}
```
"""

XML:

ML_format = """Write your response following the XML format below:

<output>
    <task>"task1"</task>
    <subtask>
        <task1_result> ( task 1 result )</task1_result>
        <task2_result> ( task 2 result )</task2_result>
        <task3_result> ( task 3 result )</task3_result>
    </subtask>
    <task>"task2"</task>
    <subtask>
        <task1_result> ( task 1 result )</task1_result>
        <task2_result> ( task 2 result )</task2_result>
        <task3_result> ( task 3 result )</task3_result>
    </subtask>
</output>

"""

Markdown:

markdown_schema = """Write your response following the markdown format below:
## Introduction
( 2-3 line intro)

## Design Guidance 
(Bulleted list of design guidance)

## Step by Step Instructions on Execution
( Bulleted list of instructions with each with bold title.)

## Conclusion
( conclusion )
"""

Prefill assistant content

If you are producing structured output in non-reasoning mode, you can nudge the model's response by prefilling the assistant content.

Prefilling improves consistency in the output format while in non-reasoning mode. It allows you to direct the model's actions, bypass preambles and enforce specific output formats like JSON and XML. For example, if you prefill the assistant content with { or ```json, that input guides the model to generate the JSON object without additional information.

import boto3
import json

# Create a Bedrock Runtime client.
client = boto3.client(
    "bedrock-runtime",
    region_name="us-east-1"
)

request_body = {
    "system": [
        {"text": "You write JSON objects based on the given instructions"}
    ],
    "messages": [
        {
            "role": "user",
            "content": [{"text": "Provide details about the best selling full-frame cameras in past three years. Answer in JSON format with keys like name, brand, price and a summary."}]
        },
        {
            "role": "assistant",
            "content": [{"text": " Here is the JSON response: ```json"}]
        }
    ],
    "inferenceConfig": {
        "maxTokens": 1000,
    }
}

# Invoke the model and extract the response body.
response = client.invoke_model(
    modelId="amazon.nova-2-lite-v1:0",
    body=json.dumps(request_body)
)

model_response = json.loads(response["body"].read())

Specify a tool to use for complex schemas

Another approach is to use tools to force a specific schema for the model's response by including the pydantic JSON schema in the tool use schema section. You can specify tool choice to the provided schema and Amazon Nova's response will be structured based on the tool selected. To learn more about how to leverage tool use see Tool Use.

Examples

Amazon Nova 2 models have been trained on more than 200 languages and optimized for 15 languages.

Prompt for accurate translations

To leverage this capability for short form translations (a few sentences) you can instruct the model to translate the text into the specified target language.

Translate the following text into {target language}. Please output only the translated text with no prefix or introduction: {text}

Translate the following sentence from {source_language} to {target language}: {text}

{text} How do you say this sentence in {target_language}

Enforce consistent writing conventions

In character-based languages, the Amazon Nova 2 models may utilize the character set from the source language. You can use the following prompt to enforce a consistent output.

When translating, ensure to use the correct orthography / script / writing convention of the target language, not the source language's characters

Agentic systems

Set the right inference parameters

Tool calling requires a very specific structured output from the model and is improved by using the following inference parameters:

Consider latency requirements

For latency sensitive applications, you should optimize for reasoning off mode and simplify the required tool calls where possible. Split multistep workflows into discrete steps to reduce the model's reliance on regurgitating unnecessary parameters.

Use intentional wording for tool calling instructions

Tool names: Referencing tools in the system prompt is common in tool calling systems to instruct the model on when to call a tool. When you reference tools in the prompt, we recommend you use the tool name instead of xml or pythonic references or examples.

Use the 'run_shell_command' tool for running shell commands

Call run_shell_command() to run shell commands

Leverage thinking Commands

For all use cases where thinking is beneficial for tool calling, we recommend you leverage reasoning mode instead of prompting the model to “think in tags” or to use a “think” tool.

Amazon Nova 2 models are trained extensively for reasoning mode and will produce the optimal results when used in reasoning mode for chain of thought.

Tool call ordering

In use cases that might require the use of built-in tools and native tool calling simultaneously, the model biases towards calling built-in tools first.

Don't instruct the model to act differently in the prompt. Instead, design your workflow to take this into account.

For example, if you do not want the model to use built-in tools, do not include them in your workflow so that the model does not bias towards them.

Create quality tool schemas

Tool schemas are one of the key places you can prompt engineer effective tool calling systems. However, it's important to consider what gets captured in the tool schema itself, how each schema element is described semantically and how the system prompt references tools and schema elements within system instructions.

Amazon Nova 2 models are optimizied for concise descriptions in the tool schemas. Keep it brief.

Tool schema versus system prompt guidelines:

Include in the tool schema:

Include in the system prompt:

You are a software engineering issue root cause analysis agent. You are tasked with reviewing a customer issue and examining the repository to identify a plan to resolve the issue.
      # Core Mandates
- **DO NOT** update the original issue that was posted by the user. You only add *additional* comments to the reported issue if necessary

# Primary Workflows
1. **Understand:** Analyze the user's request and explore the codebase thoroughly using **get_file_contents** to grasp file structures and conventions.
2. **Plan:** Create a coherent, evidence-based plan for resolving the task and share it with the user following the format below

# Tool Usage 
- **Read the Issue:** Always start by using the **read_issue** tool to get the details about the requested issue
- **File Paths:** Always end the file path with "/" if you are searching a directory using the **get_file_contents** tools
- **Parallelism:** Execute multiple independent tool calls in parallel when feasible

# Error Handling and Troubleshooting
- **File Exploration:** If you get an error that a file doesn't exist, try searching at the directory level first to validate the file path

# Output Formatting
Return your plan in markdown in the following format

## Issue
<Your root cause analysis of the issue>

## Resolution Plan
<your step by step plan of how to solve the issue>

Create sub-agents

Consider creating specialized sub-agents instead of a single agent with many tools when you encounter:

Use tools for multimodal inputs

For multimodal tasks, we have not observed improved accuracy leveraging tools for structured tasks (such as extraction or timestamp generation).

Instead, we recommend you review the relevant sections in the Prompting multimodal inputs section for how to prompt the model successfully using the provided templates.

Next steps