本文為英文版的機器翻譯版本，如內容有任何歧義或不一致之處，概以英文版為準。

微調 Nova 2.0

先決條件

開始訓練任務之前，請注意以下事項：

什麼是 SFT？

受監督的微調 (SFT) 會使用標記的輸入輸出對來訓練語言模型。此模型會從包含提示和回應的示範範例中學習，改善其功能以符合特定任務、指示或所需行為。

資料準備

概觀

Nova 2.0 SFT 資料使用與 Nova 1.0 相同的 Converse API 格式，並新增選用的推理內容欄位。如需完整的格式規格，請參閱：

支援的功能

目前限制

支援的媒體格式

資料格式範例

Text-only (Nova 1.0 compatible)

{  
  "schemaVersion": "bedrock-conversation-2024",  
  "system": [  
    {  
      "text": "You are a digital assistant with a friendly personality"  
    }  
  ],  
  "messages": [  
    {  
      "role": "user",  
      "content": [  
        {  
          "text": "What country is right next to Australia?"  
        }  
      ]  
    },  
    {  
      "role": "assistant",  
      "content": [  
        {  
          "text": "The closest country is New Zealand"  
        }  
      ]  
    }  
  ]  
}

Text with reasoning (Nova 2.0)

{  
  "schemaVersion": "bedrock-conversation-2024",  
  "system": [  
    {  
      "text": "You are a digital assistant with a friendly personality"  
    }  
  ],  
  "messages": [  
    {  
      "role": "user",  
      "content": [  
        {  
          "text": "What country is right next to Australia?"  
        }  
      ]  
    },  
    {  
      "role": "assistant",  
      "content": [  
        {  
          "reasoningContent": {  
            "reasoningText": {  
              "text": "I need to use my world knowledge of geography to answer this question"  
            }  
          }  
        },  
        {  
          "text": "The closest country to Australia is New Zealand, located to the southeast across the Tasman Sea."  
        }  
      ]  
    }  
  ]  
}

Image + text input

{  
  "schemaVersion": "bedrock-conversation-2024",  
  "system": [  
    {  
      "text": "You are a helpful assistant."  
    }  
  ],  
  "messages": [  
    {  
      "role": "user",  
      "content": [  
        {  
          "image": {  
            "format": "jpeg",  
            "source": {  
              "s3Location": {  
                "uri": "s3://your-bucket/your-path/your-image.jpg",  
                "bucketOwner": "your-aws-account-id"  
              }  
            }  
          }  
        },  
        {  
          "text": "Which country is highlighted in the image?"  
        }  
      ]  
    },  
    {  
      "role": "assistant",  
      "content": [  
        {  
          "reasoningContent": {  
            "reasoningText": {  
              "text": "I will determine the highlighted country by examining its location on the map and using my geographical knowledge"  
            }  
          }  
        },  
        {  
          "text": "The highlighted country is New Zealand"  
        }  
      ]  
    }  
  ]  
}

Video + text input

{  
  "schemaVersion": "bedrock-conversation-2024",  
  "system": [  
    {  
      "text": "You are a helpful assistant."  
    }  
  ],  
  "messages": [  
    {  
      "role": "user",  
      "content": [  
        {  
          "video": {  
            "format": "mp4",  
            "source": {  
              "s3Location": {  
                "uri": "s3://your-bucket/your-path/your-video.mp4",  
                "bucketOwner": "your-aws-account-id"  
              }  
            }  
          }  
        },  
        {  
          "text": "What is shown in this video?"  
        }  
      ]  
    },  
    {  
      "role": "assistant",  
      "content": [  
        {  
          "reasoningContent": {  
            "reasoningText": {  
              "text": "I will analyze the video content to identify key elements"  
            }  
          }  
        },  
        {  
          "text": "The video shows a map with New Zealand highlighted"  
        }  
      ]  
    }  
  ]  
}

工具呼叫

Nova 2.0 SFT 支援工具呼叫模式的訓練模型，讓您的模型了解呼叫外部工具或函數的時間和方式。

工具呼叫的資料格式

工具呼叫訓練資料包含定義可用工具的toolConfig區段，以及示範工具使用模式的對話輪換。

範例輸入

{  
  "schemaVersion": "bedrock-conversation-2024",  
  "system": [  
    {  
      "text": "You are an expert in composing function calls."  
    }  
  ],  
  "toolConfig": {  
    "tools": [  
      {  
        "toolSpec": {  
          "name": "getItemCost",  
          "description": "Retrieve the cost of an item from the catalog",  
          "inputSchema": {  
            "json": {  
              "type": "object",  
              "properties": {  
                "item_name": {  
                  "type": "string",  
                  "description": "The name of the item to retrieve cost for"  
                },  
                "item_id": {  
                  "type": "string",  
                  "description": "The ASIN of item to retrieve cost for"  
                }  
              },  
              "required": [  
                "item_id"  
              ]  
            }  
          }  
        }  
      },  
      {  
        "toolSpec": {  
          "name": "getItemAvailability",  
          "description": "Retrieve whether an item is available in a given location",  
          "inputSchema": {  
            "json": {  
              "type": "object",  
              "properties": {  
                "zipcode": {  
                  "type": "string",  
                  "description": "The zipcode of the location to check in"  
                },  
                "quantity": {  
                  "type": "integer",  
                  "description": "The number of items to check availability for"  
                },  
                "item_id": {  
                  "type": "string",  
                  "description": "The ASIN of item to check availability for"  
                }  
              },  
              "required": [  
                "item_id", "zipcode"  
              ]  
            }  
          }  
        }  
      }  
    ]  
  },  
  "messages": [  
    {  
      "role": "user",  
      "content": [  
        {  
          "text": "I need to check whether there are twenty pieces of the following item available. Here is the item ASIN on Amazon: id-123. Please check for the zipcode 94086"  
        }  
      ]  
    },  
    {  
      "role": "assistant",  
      "content": [  
        {  
          "reasoningContent": {  
            "reasoningText": {  
              "text": "The user wants to check how many pieces of the item with ASIN id-123 are available in the zipcode 94086"  
            }  
          }  
        },  
        {  
          "toolUse": {  
            "toolUseId": "getItemAvailability_0",  
            "name": "getItemAvailability",  
            "input": {  
              "zipcode": "94086",  
              "quantity": 20,  
              "item_id": "id-123"  
            }  
          }  
        }  
      ]  
    },  
    {  
      "role": "user",  
      "content": [  
        {  
          "toolResult": {  
            "toolUseId": "getItemAvailability_0",  
            "content": [  
              {  
                "text": "[{\"name\": \"getItemAvailability\", \"results\": {\"availability\": true}}]"  
              }  
            ]  
          }  
        }  
      ]  
    },  
    {  
      "role": "assistant",  
      "content": [  
        {  
          "text": "Yes, there are twenty pieces of item id-123 available at 94086. Would you like to place an order or know the total cost?"  
        }  
      ]  
    }  
  ]  
}

工具呼叫需求

建立工具呼叫訓練資料時，請遵循下列要求：

重要說明

理解文件

Nova 2.0 SFT 支援訓練以文件為基礎的任務，讓您的模型了解如何分析和回應有關 PDF 文件的問題。

文件理解的資料格式

文件了解訓練資料包含使用者內容區塊中的文件參考，模型學習會擷取文件內容並解釋原因。

範例輸入

{  
  "schemaVersion": "bedrock-conversation-2024",  
  "messages": [  
    {  
      "role": "user",  
      "content": [  
        {  
          "text": "What are the ways in which a customer can experience issues during checkout on Amazon?"  
        },  
        {  
          "document": {  
            "format": "pdf",  
            "source": {  
              "s3Location": {  
                "uri": "s3://my-bucket-name/path/to/documents/customer_service_debugging.pdf",  
                "bucketOwner": "123456789012"  
              }  
            }  
          }  
        }  
      ]  
    },  
    {  
      "role": "assistant",  
      "content": [  
        {  
          "text": "Customers can experience issues with 1. Data entry, 2. Payment methods, 3. Connectivity while placing the order. Which one would you like to dive into?"  
        }  
      ],  
      "reasoning_content": [  
        {  
          "text": "I need to find the relevant section in the document to answer the question.",  
          "type": "text"  
        }  
      ]  
    }  
  ]  
}

文件理解限制

文件理解的最佳實務

影片理解

Nova 2.0 SFT 支援視訊型任務的訓練，讓您的模型了解如何分析和回應有關視訊內容的問題。

影片理解的資料格式

影片了解訓練資料包含使用者內容區塊中的影片參考，模型學習會擷取影片內容的資訊和原因。

範例輸入

{  
  "schemaVersion": "bedrock-conversation-2024",  
  "messages": [  
    {  
      "role": "user",  
      "content": [  
        {  
          "text": "What are the ways in which a customer can experience issues during checkout on Amazon?"  
        },  
        {  
          "video": {  
            "format": "mp4",  
            "source": {  
              "s3Location": {  
                "uri": "s3://my-bucket-name/path/to/videos/customer_service_debugging.mp4",  
                "bucketOwner": "123456789012"  
              }  
            }  
          }  
        }  
      ]  
    },  
    {  
      "role": "assistant",  
      "content": [  
        {  
          "text": "Customers can experience issues with 1. Data entry, 2. Payment methods, 3. Connectivity while placing the order. Which one would you like to dive into?"  
        }  
      ],  
      "reasoning_content": [  
        {  
          "text": "I need to find the relevant section in the video to answer the question.",  
          "type": "text"  
        }  
      ]  
    }  
  ]  
}

影片理解限制

支援的影片格式

影片理解的最佳實務

原因與非原因模式

了解推理內容

推理內容 （也稱為chain-of-thought) 會在產生最終答案之前擷取模型的中繼思考步驟。assistant 反之，請使用 reasoningContent 欄位來包含這些推理追蹤。

如何計算損失

您可以在多迴轉對話中reasoningContent跨多個助理迴轉包含 。

格式化準則

何時啟用推理模式

在下列情況下，在您的訓練組態reasoning_enabled: true中設定 ：

reasoning_enabled = true 允許使用 在非合理資料集上訓練 Nova。不過，這樣做可能會導致模型失去其推理功能，因為 Nova 主要學習在不套用推理的情況下產生資料中呈現的回應。如果您想要在非合理資料集上訓練 Nova，但仍預期推論期間的推理，您可以在訓練期間停用推理 (reasoning_enabled = false)，但啟用它以進行推理。雖然此方法允許在推論時間使用推理，但與推論相比，它不保證改善效能，無需推理。一般而言，在使用推理資料集時啟用訓練推理和推論，並在使用非合理的資料集時停用兩者推理。

設定reasoning_enabled: false時機：

產生推理資料

如果您的資料集缺少推理追蹤，您可以使用像 Nova Premier 這樣的推理能力模型來建立它們。將輸入輸出對提供給模型，並擷取其推理程序，以建置推理擴增資料集。

使用推理權杖進行訓練

啟用推理模式的訓練時，模型會學習將內部推理與最終答案分開。訓練程序：

有效的推理內容

高品質推理內容應包括：

這有助於模型開發「在回答之前思考」的能力。

資料集準備準則

大小和品質

多樣性

包含下列各種範例：

輸出格式

在助理回應中明確指定所需的輸出格式：

多回合對話

對於多迴轉資料集，請記住：

品質檢查清單

上傳資料

資料集應上傳至 SageMaker 訓練任務可存取的儲存貯體。如需設定正確許可的詳細資訊，請參閱先決條件。

開始訓練任務

選取超參數並更新配方

Nova 2.0 的設定大致上與 Nova 1.0 的設定相同。輸入資料上傳到 S3 後，請使用微調資料夾下的 SageMaker Hyperpod 配方。對於 Nova 2.0，以下是您可以根據使用案例更新的一些關鍵超參數。以下是 Nova 2.0 SFT PEFT 配方的範例。

請使用 SageMaker PySDK 的 v2.254.1，以嚴格相容於 Nova 訓練。將 SDK 升級到 v3.0 版本會導致重大變更。即將推出對 SageMaker PySDK v3 的支援。

範例輸入

!pip install sagemaker==2.254.1

run:  
  name: {peft_recipe_job_name}  
  model_type: amazon.nova-2-lite-v1:0:256k  
  model_name_or_path: {peft_model_name_or_path}  
  data_s3_path: {train_dataset_s3_path} # SageMaker Hyperpod (SMHP) only and not compatible with SageMaker Training jobs. Note replace my-bucket-name with your real bucket name for SMHP job  
  replicas: 4                      # Number of compute instances for training, allowed values are 4, 8, 16, 32  
  output_s3_path: ""               # Output artifact path (Hyperpod job-specific; not compatible with standard SageMaker Training jobs). Note replace my-bucket-name with your real bucket name for SMHP job  
  
training_config:  
  max_steps: 10                   # Maximum training steps. Minimal is 4.  
  save_steps: 10                      # How many training steps the checkpoint will be saved. Should be less than or equal to max_steps  
  save_top_k: 1                    # Keep top K best checkpoints. Note supported only for SageMaker HyperPod jobs. Minimal is 1.  
  max_length: 32768                # Sequence length (options: 8192, 16384, 32768 [default], 65536)  
  global_batch_size: 32            # Global batch size (options: 32, 64, 128)  
  reasoning_enabled: true          # If data has reasoningContent, set to true; otherwise False  
  
  lr_scheduler:  
    warmup_steps: 15               # Learning rate warmup steps. Recommend 15% of max_steps  
    min_lr: 1e-6                   # Minimum learning rate, must be between 0.0 and 1.0  
  
  optim_config:                    # Optimizer settings  
    lr: 1e-5                       # Learning rate, must be between 0.0 and 1.0  
    weight_decay: 0.0              # L2 regularization strength, must be between 0.0 and 1.0  
    adam_beta1: 0.9                # Exponential decay rate for first-moment estimates, must be between 0.0 and 1.0  
    adam_beta2: 0.95               # Exponential decay rate for second-moment estimates, must be between 0.0 and 1.0  
  
  peft:                            # Parameter-efficient fine-tuning (LoRA)  
    peft_scheme: "lora"            # Enable LoRA for PEFT  
    lora_tuning:  
      alpha: 64                    # Scaling factor for LoRA weights ( options: 32, 64, 96, 128, 160, 192),  
      lora_plus_lr_ratio: 64.0

配方也包含與 Nova 1.0 大致相同的超參數。值得注意的超參數為：

開始訓練任務

from sagemaker.pytorch import PyTorch  
  
# define OutputDataConfig path  
if default_prefix:  
    output_path = f"s3://{bucket_name}/{default_prefix}/{sm_training_job_name}"  
else:  
    output_path = f"s3://{bucket_name}/{sm_training_job_name}"  
  
recipe_overrides = {  
    "run": {  
        "replicas": instance_count,  # Required  
        "output_s3_path": output_path  
    },  
}  
  
estimator = PyTorch(  
    output_path=output_path,  
    base_job_name=sm_training_job_name,  
    role=role,  
    disable_profiler=True,  
    debugger_hook_config=False,  
    instance_count=instance_count,  
    instance_type=instance_type,  
    training_recipe=training_recipe,  
    recipe_overrides=recipe_overrides,  
    max_run=432000,  
    sagemaker_session=sagemaker_session,  
    image_uri=image_uri,  
    tags=[  
        {'Key': 'model_name_or_path', 'Value': model_name_or_path},  
    ]  
)  
  
print(f"\nsm_training_job_name:\n{sm_training_job_name}\n")  
print(f"output_path:\n{output_path}")

from sagemaker.inputs import TrainingInput  
  
train_input = TrainingInput(  
    s3_data=train_dataset_s3_path,  
    distribution="FullyReplicated",  
    s3_data_type="Converse",  
)  
  
estimator.fit(inputs={"validation": val_input}, wait=False)

若要開始任務：

筆記本會處理任務提交並提供狀態追蹤。