本文属于机器翻译版本。若本译文内容与英语原文存在差异，则一律以英文原文为准。

微调 Nova 2.0

先决条件

在开始训练作业之前，请注意具备以下内容：

什么是 SFT？

监督微调 (SFT) 使用带标签的输入输出对训练语言模型。该模型从由提示和响应组成的演示示例中学习，完善其功能以适应特定的任务、指令或所需行为。

数据准备

概述

Nova 2.0 SFT 数据使用与 Nova 1.0 相同的 Converse API 格式，但增加了可选的推理内容字段。有关完整的格式规范，请参阅：

支持的功能

目前的局限性

支持的媒体格式

数据格式示例

Text-only (Nova 1.0 compatible)

{  
  "schemaVersion": "bedrock-conversation-2024",  
  "system": [  
    {  
      "text": "You are a digital assistant with a friendly personality"  
    }  
  ],  
  "messages": [  
    {  
      "role": "user",  
      "content": [  
        {  
          "text": "What country is right next to Australia?"  
        }  
      ]  
    },  
    {  
      "role": "assistant",  
      "content": [  
        {  
          "text": "The closest country is New Zealand"  
        }  
      ]  
    }  
  ]  
}

Text with reasoning (Nova 2.0)

{  
  "schemaVersion": "bedrock-conversation-2024",  
  "system": [  
    {  
      "text": "You are a digital assistant with a friendly personality"  
    }  
  ],  
  "messages": [  
    {  
      "role": "user",  
      "content": [  
        {  
          "text": "What country is right next to Australia?"  
        }  
      ]  
    },  
    {  
      "role": "assistant",  
      "content": [  
        {  
          "reasoningContent": {  
            "reasoningText": {  
              "text": "I need to use my world knowledge of geography to answer this question"  
            }  
          }  
        },  
        {  
          "text": "The closest country to Australia is New Zealand, located to the southeast across the Tasman Sea."  
        }  
      ]  
    }  
  ]  
}

Image + text input

{  
  "schemaVersion": "bedrock-conversation-2024",  
  "system": [  
    {  
      "text": "You are a helpful assistant."  
    }  
  ],  
  "messages": [  
    {  
      "role": "user",  
      "content": [  
        {  
          "image": {  
            "format": "jpeg",  
            "source": {  
              "s3Location": {  
                "uri": "s3://your-bucket/your-path/your-image.jpg",  
                "bucketOwner": "your-aws-account-id"  
              }  
            }  
          }  
        },  
        {  
          "text": "Which country is highlighted in the image?"  
        }  
      ]  
    },  
    {  
      "role": "assistant",  
      "content": [  
        {  
          "reasoningContent": {  
            "reasoningText": {  
              "text": "I will determine the highlighted country by examining its location on the map and using my geographical knowledge"  
            }  
          }  
        },  
        {  
          "text": "The highlighted country is New Zealand"  
        }  
      ]  
    }  
  ]  
}

Video + text input

{  
  "schemaVersion": "bedrock-conversation-2024",  
  "system": [  
    {  
      "text": "You are a helpful assistant."  
    }  
  ],  
  "messages": [  
    {  
      "role": "user",  
      "content": [  
        {  
          "video": {  
            "format": "mp4",  
            "source": {  
              "s3Location": {  
                "uri": "s3://your-bucket/your-path/your-video.mp4",  
                "bucketOwner": "your-aws-account-id"  
              }  
            }  
          }  
        },  
        {  
          "text": "What is shown in this video?"  
        }  
      ]  
    },  
    {  
      "role": "assistant",  
      "content": [  
        {  
          "reasoningContent": {  
            "reasoningText": {  
              "text": "I will analyze the video content to identify key elements"  
            }  
          }  
        },  
        {  
          "text": "The video shows a map with New Zealand highlighted"  
        }  
      ]  
    }  
  ]  
}

工具调用

Nova 2.0 SFT 支持根据工具调用模式训练模型，使您的模型能够学习何时以及如何调用外部工具或函数。

调用工具的数据格式

工具调用训练数据包括定义可用工具的toolConfig部分，以及演示工具使用模式的对话回合。

样本输入

{  
  "schemaVersion": "bedrock-conversation-2024",  
  "system": [  
    {  
      "text": "You are an expert in composing function calls."  
    }  
  ],  
  "toolConfig": {  
    "tools": [  
      {  
        "toolSpec": {  
          "name": "getItemCost",  
          "description": "Retrieve the cost of an item from the catalog",  
          "inputSchema": {  
            "json": {  
              "type": "object",  
              "properties": {  
                "item_name": {  
                  "type": "string",  
                  "description": "The name of the item to retrieve cost for"  
                },  
                "item_id": {  
                  "type": "string",  
                  "description": "The ASIN of item to retrieve cost for"  
                }  
              },  
              "required": [  
                "item_id"  
              ]  
            }  
          }  
        }  
      },  
      {  
        "toolSpec": {  
          "name": "getItemAvailability",  
          "description": "Retrieve whether an item is available in a given location",  
          "inputSchema": {  
            "json": {  
              "type": "object",  
              "properties": {  
                "zipcode": {  
                  "type": "string",  
                  "description": "The zipcode of the location to check in"  
                },  
                "quantity": {  
                  "type": "integer",  
                  "description": "The number of items to check availability for"  
                },  
                "item_id": {  
                  "type": "string",  
                  "description": "The ASIN of item to check availability for"  
                }  
              },  
              "required": [  
                "item_id", "zipcode"  
              ]  
            }  
          }  
        }  
      }  
    ]  
  },  
  "messages": [  
    {  
      "role": "user",  
      "content": [  
        {  
          "text": "I need to check whether there are twenty pieces of the following item available. Here is the item ASIN on Amazon: id-123. Please check for the zipcode 94086"  
        }  
      ]  
    },  
    {  
      "role": "assistant",  
      "content": [  
        {  
          "reasoningContent": {  
            "reasoningText": {  
              "text": "The user wants to check how many pieces of the item with ASIN id-123 are available in the zipcode 94086"  
            }  
          }  
        },  
        {  
          "toolUse": {  
            "toolUseId": "getItemAvailability_0",  
            "name": "getItemAvailability",  
            "input": {  
              "zipcode": "94086",  
              "quantity": 20,  
              "item_id": "id-123"  
            }  
          }  
        }  
      ]  
    },  
    {  
      "role": "user",  
      "content": [  
        {  
          "toolResult": {  
            "toolUseId": "getItemAvailability_0",  
            "content": [  
              {  
                "text": "[{\"name\": \"getItemAvailability\", \"results\": {\"availability\": true}}]"  
              }  
            ]  
          }  
        }  
      ]  
    },  
    {  
      "role": "assistant",  
      "content": [  
        {  
          "text": "Yes, there are twenty pieces of item id-123 available at 94086. Would you like to place an order or know the total cost?"  
        }  
      ]  
    }  
  ]  
}

工具调用要求

创建调用训练数据的工具时，请遵循以下要求：

重要提示

文档理解

Nova 2.0 SFT 支持基于文档的任务训练，使您的模型能够学习如何分析和回答有关 PDF 文档的问题。

用于理解文档的数据格式

文档理解训练数据包括用户内容块中的文档引用，模型学习提取和推理而不是文档内容。

样本输入

{  
  "schemaVersion": "bedrock-conversation-2024",  
  "messages": [  
    {  
      "role": "user",  
      "content": [  
        {  
          "text": "What are the ways in which a customer can experience issues during checkout on Amazon?"  
        },  
        {  
          "document": {  
            "format": "pdf",  
            "source": {  
              "s3Location": {  
                "uri": "s3://my-bucket-name/path/to/documents/customer_service_debugging.pdf",  
                "bucketOwner": "123456789012"  
              }  
            }  
          }  
        }  
      ]  
    },  
    {  
      "role": "assistant",  
      "content": [  
        {  
          "text": "Customers can experience issues with 1. Data entry, 2. Payment methods, 3. Connectivity while placing the order. Which one would you like to dive into?"  
        }  
      ],  
      "reasoning_content": [  
        {  
          "text": "I need to find the relevant section in the document to answer the question.",  
          "type": "text"  
        }  
      ]  
    }  
  ]  
}

文档理解局限性

理解文档的最佳实践

视频理解

Nova 2.0 SFT 支持基于视频的任务训练，使您的模型能够学习如何分析和回答有关视频内容的问题。

用于视频理解的数据格式

视频理解训练数据包括用户内容块中的视频参考，模型学习提取视频内容的信息和推理。

样本输入

{  
  "schemaVersion": "bedrock-conversation-2024",  
  "messages": [  
    {  
      "role": "user",  
      "content": [  
        {  
          "text": "What are the ways in which a customer can experience issues during checkout on Amazon?"  
        },  
        {  
          "video": {  
            "format": "mp4",  
            "source": {  
              "s3Location": {  
                "uri": "s3://my-bucket-name/path/to/videos/customer_service_debugging.mp4",  
                "bucketOwner": "123456789012"  
              }  
            }  
          }  
        }  
      ]  
    },  
    {  
      "role": "assistant",  
      "content": [  
        {  
          "text": "Customers can experience issues with 1. Data entry, 2. Payment methods, 3. Connectivity while placing the order. Which one would you like to dive into?"  
        }  
      ],  
      "reasoning_content": [  
        {  
          "text": "I need to find the relevant section in the video to answer the question.",  
          "type": "text"  
        }  
      ]  
    }  
  ]  
}

视频理解限制

支持的视频格式

视频理解的最佳实践

推理模式与非推理模式

了解推理内容

推理内容（也称为 chain-of-thought）在生成最终答案之前捕捉模型的中间思维步骤。assistant反过来，使用该reasoningContent字段来包含这些推理痕迹。

损失是如何计算的

你可以在多回合对话中加入reasoningContent多个助手回合。

格式指南

何时启用推理模式

在以下reasoning_enabled: true情况下在训练配置中进行设置：

允许使用reasoning_enabled = true非推理数据集训练 Nova。但是，这样做可能会导致模型失去其推理能力，因为Nova主要是在不应用推理的情况下学习生成数据中呈现的响应。如果你想在非推理数据集上训练 Nova，但仍然期望在推理期间进行推理，则可以在训练期间禁用推理 (reasoning_enabled = false)，但将其启用以进行推理。虽然这种方法允许在推理时使用推理，但与不进行推理的推理相比，它并不能保证性能得到改善。通常，在使用推理数据集时，可以同时启用训练和推理的推理，而在使用非推理数据集时，可以同时为训练和推理启用推理。

设置reasoning_enabled: false时间：

生成推理数据

如果你的数据集缺少推理轨迹，你可以使用像 Nova Premier 这样的具有推理功能的模型来创建推理轨迹。向模型提供您的输入输出对，并捕获其推理过程以构建推理增强数据集。

使用推理代币进行训练

在启用推理模式的情况下训练时，模型会学会将内部推理与最终答案分开。培训过程：

有效的推理内容

高质量的推理内容应包括：

这有助于模型培养 “在回答之前先思考” 的能力。

数据集准备指南

尺寸和质量

多元化

包括以下各种示例：

输出格式

在助手响应中明确指定所需的输出格式：

多回合对话

对于多回合数据集，请记住：

质量清单

上传数据

数据集应上传到 SageMaker 训练作业可以访问的存储桶。有关设置正确权限的信息，请参阅先决条件。

开始训练作业

选择超参数并更新配方

Nova 2.0 的设置与 Nova 1.0 的设置基本相同。将输入数据上传到 S3 后，使用 Fine tuning 文件夹下的 SageMaker Hyperpod Recipes 中的配方。对于 Nova 2.0，以下是一些可以根据用例更新的关键超参数。以下是 Nova 2.0 SFT PEFT 配方的示例。

请使用 SageMaker pySDK 的 v2.254.1，以便与 Nova 训练完全兼容。将 SDK 升级到 v3.0 版本将导致重大更改。对 SageMaker pySDK v3 的支持即将推出。

示例输入

!pip install sagemaker==2.254.1

run:  
  name: {peft_recipe_job_name}  
  model_type: amazon.nova-2-lite-v1:0:256k  
  model_name_or_path: {peft_model_name_or_path}  
  data_s3_path: {train_dataset_s3_path} # SageMaker Hyperpod (SMHP) only and not compatible with SageMaker Training jobs. Note replace my-bucket-name with your real bucket name for SMHP job  
  replicas: 4                      # Number of compute instances for training, allowed values are 4, 8, 16, 32  
  output_s3_path: ""               # Output artifact path (Hyperpod job-specific; not compatible with standard SageMaker Training jobs). Note replace my-bucket-name with your real bucket name for SMHP job  
  
training_config:  
  max_steps: 10                   # Maximum training steps. Minimal is 4.  
  save_steps: 10                      # How many training steps the checkpoint will be saved. Should be less than or equal to max_steps  
  save_top_k: 1                    # Keep top K best checkpoints. Note supported only for SageMaker HyperPod jobs. Minimal is 1.  
  max_length: 32768                # Sequence length (options: 8192, 16384, 32768 [default], 65536)  
  global_batch_size: 32            # Global batch size (options: 32, 64, 128)  
  reasoning_enabled: true          # If data has reasoningContent, set to true; otherwise False  
  
  lr_scheduler:  
    warmup_steps: 15               # Learning rate warmup steps. Recommend 15% of max_steps  
    min_lr: 1e-6                   # Minimum learning rate, must be between 0.0 and 1.0  
  
  optim_config:                    # Optimizer settings  
    lr: 1e-5                       # Learning rate, must be between 0.0 and 1.0  
    weight_decay: 0.0              # L2 regularization strength, must be between 0.0 and 1.0  
    adam_beta1: 0.9                # Exponential decay rate for first-moment estimates, must be between 0.0 and 1.0  
    adam_beta2: 0.95               # Exponential decay rate for second-moment estimates, must be between 0.0 and 1.0  
  
  peft:                            # Parameter-efficient fine-tuning (LoRA)  
    peft_scheme: "lora"            # Enable LoRA for PEFT  
    lora_tuning:  
      alpha: 64                    # Scaling factor for LoRA weights ( options: 32, 64, 96, 128, 160, 192),  
      lora_plus_lr_ratio: 64.0

该配方还包含与 Nova 1.0 基本相同的超参数。值得注意的超参数有：

开始训练作业

from sagemaker.pytorch import PyTorch  
  
# define OutputDataConfig path  
if default_prefix:  
    output_path = f"s3://{bucket_name}/{default_prefix}/{sm_training_job_name}"  
else:  
    output_path = f"s3://{bucket_name}/{sm_training_job_name}"  
  
recipe_overrides = {  
    "run": {  
        "replicas": instance_count,  # Required  
        "output_s3_path": output_path  
    },  
}  
  
estimator = PyTorch(  
    output_path=output_path,  
    base_job_name=sm_training_job_name,  
    role=role,  
    disable_profiler=True,  
    debugger_hook_config=False,  
    instance_count=instance_count,  
    instance_type=instance_type,  
    training_recipe=training_recipe,  
    recipe_overrides=recipe_overrides,  
    max_run=432000,  
    sagemaker_session=sagemaker_session,  
    image_uri=image_uri,  
    tags=[  
        {'Key': 'model_name_or_path', 'Value': model_name_or_path},  
    ]  
)  
  
print(f"\nsm_training_job_name:\n{sm_training_job_name}\n")  
print(f"output_path:\n{output_path}")

from sagemaker.inputs import TrainingInput  
  
train_input = TrainingInput(  
    s3_data=train_dataset_s3_path,  
    distribution="FullyReplicated",  
    s3_data_type="Converse",  
)  
  
estimator.fit(inputs={"validation": val_input}, wait=False)

要开始这项工作，请执行以下操作：

笔记本处理作业提交并提供状态跟踪。