本文為英文版的機器翻譯版本，如內容有任何歧義或不一致之處，概以英文版為準。

# 為擷取並產生 RAG 評估任務建立提示資料集
<a name="knowledge-base-evaluation-prompt-retrieve-generate"></a>

擷取並產生評估任務需要使用 JSON 行格式的提示資料集。您的資料集最多可有 1000 個提示

## 為擷取並產生評估任務準備資料集，其中 Amazon Bedrock 會調用您的知識庫
<a name="knowledge-base-evaluation-prompt-retrieve-generate-invoke"></a>

若要建立 Amazon Bedrock 調用您知識庫的僅擷取評估任務，您的提示資料集必須包含下列鍵值對：
+ `referenceResponses` – 此父金鑰用於指定您預期 [https://docs.aws.amazon.com/bedrock/latest/APIReference/API_agent-runtime_RetrieveAndGenerate.html](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_agent-runtime_RetrieveAndGenerate.html) 會傳回的基本事實回應。在 `text` 索引鍵中指定基本事實。如果您在評估任務中選擇**內容涵蓋範圍**指標，則需要 `referenceResponses`。
+ `prompt` – 此父金鑰用於指定您想要讓模型在評估任務執行時回應的提示 (使用者查詢)。

以下是包含 6 個輸入並使用 JSON 行格式的自訂資料集範例。

```
{"conversationTurns":[{"prompt":{"content":[{"text":"Provide the prompt you want to use during inference"}]},"referenceResponses":[{"content":[{"text":"Specify a ground-truth response"}]}]}]}
{"conversationTurns":[{"prompt":{"content":[{"text":"Provide the prompt you want to use during inference"}]},"referenceResponses":[{"content":[{"text":"Specify a ground-truth response"}]}]}]}
{"conversationTurns":[{"prompt":{"content":[{"text":"Provide the prompt you want to use during inference"}]},"referenceResponses":[{"content":[{"text":"Specify a ground-truth response"}]}]}]}
{"conversationTurns":[{"prompt":{"content":[{"text":"Provide the prompt you want to use during inference"}]},"referenceResponses":[{"content":[{"text":"Specify a ground-truth response"}]}]}]}
{"conversationTurns":[{"prompt":{"content":[{"text":"Provide the prompt you want to use during inference"}]},"referenceResponses":[{"content":[{"text":"Specify a ground-truth response"}]}]}]}
{"conversationTurns":[{"prompt":{"content":[{"text":"Provide the prompt you want to use during inference"}]},"referenceResponses":[{"content":[{"text":"Specify a ground-truth response"}]}]}]}
```

為了清楚起見，下列提示已展開。在實際提示資料集中，每行 (提示) 必須是有效的 JSON 物件。

```
{
    "conversationTurns": [
        {
            "prompt": {
                "content": [
                    {
                        "text": "What is the recommended service interval for your product?"
                    }
                ]
            },
            "referenceResponses": [
                {
                    "content": [
                        {
                            "text": "The recommended service interval for our product is two years."
                        }
                    ]
                }
            ]
        }
    ]
}
```

## 使用您自己的推論回應資料準備資料集以用於擷取並產生評估任務
<a name="knowledge-base-evaluation-prompt-retrieve-generate-byoir"></a>

若要在提供自己的推論回應資料的地方建立擷取並產生評估任務，您的提示資料集是對話回合的清單，並在每次回合時包含以下內容。每個任務只能評估一個 RAG 來源。
+ `prompt` – 您提供給模型以產生結果的提示。
+ `referenceResponses` – 此父金鑰用於指定 LLM 擷取了擷取結果和輸入查詢後，預期其中最終輸出的基本事實回應。
+ `referenceContexts` (選用) – 此選用父金鑰用於指定您預期從 RAG 來源擷取的基本事實段落。只有在您想要在自己的自訂評估指標中使用它時，才需要包含此金鑰。Amazon Bedrock 提供的內建指標不會使用此屬性。
+ `output` – 來自 RAG 來源的輸出，包含下列項目：
  + `text` – RAG 系統中 LLM 的最終輸出。
  + `retrievedPassages` – 此父金鑰用於指定 RAG 來源擷取的內容。

您的 `output` 資料也必須包含字串 `knowledgeBaseIdentifier`，以定義您用來產生推論回應的 RAG 來源。您也可以包含可識別您使用之 LLM 的選用 `modelIdentifier` 字串。對於 `retrievalResults` 和 `retrievedReferences`，您可以提供選用的名稱和中繼資料。

以下是包含 6 個輸入並使用 JSON 行格式的自訂資料集範例。

```
{"conversationTurns":[{"prompt":{"content":[{"text":"Provide the prompt you used to generate the response"}]},"referenceResponses":[{"content":[{"text":"A ground truth for the final response generated by the LLM"}]}],"referenceContexts":[{"content":[{"text":"A ground truth for a received passage"}]}],"output":{"text":"The output of the LLM","modelIdentifier":"(Optional) a string identifying your model","knowledgeBaseIdentifier":"A string identifying your RAG source","retrievedPassages":{"retrievalResults":[{"name":"(Optional) a name for your retrieval","content":{"text":"The retrieved content"},"metadata":{"(Optional) a key for your metadata":"(Optional) a value for your metadata"}}]}}}]}
{"conversationTurns":[{"prompt":{"content":[{"text":"Provide the prompt you used to generate the response"}]},"referenceResponses":[{"content":[{"text":"A ground truth for the final response generated by the LLM"}]}],"referenceContexts":[{"content":[{"text":"A ground truth for a received passage"}]}],"output":{"text":"The output of the LLM","modelIdentifier":"(Optional) a string identifying your model","knowledgeBaseIdentifier":"A string identifying your RAG source","retrievedPassages":{"retrievalResults":[{"name":"(Optional) a name for your retrieval","content":{"text":"The retrieved content"},"metadata":{"(Optional) a key for your metadata":"(Optional) a value for your metadata"}}]}}}]}
{"conversationTurns":[{"prompt":{"content":[{"text":"Provide the prompt you used to generate the response"}]},"referenceResponses":[{"content":[{"text":"A ground truth for the final response generated by the LLM"}]}],"referenceContexts":[{"content":[{"text":"A ground truth for a received passage"}]}],"output":{"text":"The output of the LLM","modelIdentifier":"(Optional) a string identifying your model","knowledgeBaseIdentifier":"A string identifying your RAG source","retrievedPassages":{"retrievalResults":[{"name":"(Optional) a name for your retrieval","content":{"text":"The retrieved content"},"metadata":{"(Optional) a key for your metadata":"(Optional) a value for your metadata"}}]}}}]}
{"conversationTurns":[{"prompt":{"content":[{"text":"Provide the prompt you used to generate the response"}]},"referenceResponses":[{"content":[{"text":"A ground truth for the final response generated by the LLM"}]}],"referenceContexts":[{"content":[{"text":"A ground truth for a received passage"}]}],"output":{"text":"The output of the LLM","modelIdentifier":"(Optional) a string identifying your model","knowledgeBaseIdentifier":"A string identifying your RAG source","retrievedPassages":{"retrievalResults":[{"name":"(Optional) a name for your retrieval","content":{"text":"The retrieved content"},"metadata":{"(Optional) a key for your metadata":"(Optional) a value for your metadata"}}]}}}]}
{"conversationTurns":[{"prompt":{"content":[{"text":"Provide the prompt you used to generate the response"}]},"referenceResponses":[{"content":[{"text":"A ground truth for the final response generated by the LLM"}]}],"referenceContexts":[{"content":[{"text":"A ground truth for a received passage"}]}],"output":{"text":"The output of the LLM","modelIdentifier":"(Optional) a string identifying your model","knowledgeBaseIdentifier":"A string identifying your RAG source","retrievedPassages":{"retrievalResults":[{"name":"(Optional) a name for your retrieval","content":{"text":"The retrieved content"},"metadata":{"(Optional) a key for your metadata":"(Optional) a value for your metadata"}}]}}}]}
{"conversationTurns":[{"prompt":{"content":[{"text":"Provide the prompt you used to generate the response"}]},"referenceResponses":[{"content":[{"text":"A ground truth for the final response generated by the LLM"}]}],"referenceContexts":[{"content":[{"text":"A ground truth for a received passage"}]}],"output":{"text":"The output of the LLM","modelIdentifier":"(Optional) a string identifying your model","knowledgeBaseIdentifier":"A string identifying your RAG source","retrievedPassages":{"retrievalResults":[{"name":"(Optional) a name for your retrieval","content":{"text":"The retrieved content"},"metadata":{"(Optional) a key for your metadata":"(Optional) a value for your metadata"}}]}}}]}
```

以下顯示為了清楚起見而展開的提示資料集格式。在實際提示資料集中，每行 (提示) 必須是有效的 JSON 物件。

```
{
    "conversationTurns": [
        {
            "prompt": {
                "content": [
                    {
                        "text": "Provide the prompt you used to generate the responses"
                    }
                ]
            },
            "referenceResponses": [
                {
                    "content": [
                        {
                            "text": "A ground truth for the final response generated by the LLM"
                        }
                    ]
                }
            ],
            "referenceContexts": [
                {
                    "content": [
                        {
                            "text": "A ground truth for a received passage"
                        }
                    ]
                }
            ],
            "output": {
                "text": "The output of the LLM",
                "modelIdentifier": "(Optional) a string identifying your model",
                "knowledgeBaseIdentifier": "A string identifying your RAG source",
                "retrievedPassages": {
                    "retrievalResults": [
                        {
                            "name": "(Optional) a name for your retrieval",
                            "content": {
                                "text": "The retrieved content"
                            },
                            "metadata": {
                                "(Optional) a key for your metadata": "(Optional) a value for your metadata"
                            }
                        }
                    ]
                }
            }
        }
    ]
}
```