监督式微调

简介

监督式微调使用包含目标任务输入-输出对的数据集。换句话说，您可以提供提示（问题、指令等）示例以及正确或所需响应，并基于这些示例继续训练模型。调整模型权重以最大限度地减少监督损失，通常是其预测与目标响应词元之间的交叉熵。

何时使用 SFT？

当您拥有定义明确的任务且清楚预期输出时，SFT 最为适用。如果您能明确表述“给定 X 输入，正确/所需输出是 Y”，并且能收集到此类 X-Y 映射的示例，那么监督式微调是理想选择。SFT 表现出色的一些场景包括：

当您可以指定正确行为是什么样子时，SFT 是教授 LLM 新技能或行为最直接的方式。它利用模型现有的语言理解能力，并将其聚焦于任务上。如果差距在于知识而非行为，则不应使用 SFT；它无法让模型学习新的事实、术语或近期事件。在这种情况下，建议优先考虑对大规模领域内语料库进行持续预训练，或在推理时通过检索增强生成来引入外部知识。当您可以衡量质量但无法标记单个正确答案时，使用可验证的奖励或将 LLM 作为评判工具的强化微调可能比 SFT 更为可取。

根据任务复杂性以及未经微调的 Nova 模型性能，为每个任务规划数千到数万个演示样本，其中数据质量、一致性和多样性远比原始数量更为重要。

何时使用参数高效微调，何时使用全秩 SFT？

Nova 自定义配方使您能够执行参数高效微调（尤其是 LoRA）或全秩 SFT。如果您需要直接、具有成本效益的模型更新，或者数据量很少，建议选择参数高效微调方法，这样您只需训练小型适配器，而保留大部分主干网络不变（全秩 SFT 会更新所有模型参数）。

适用于 SFT 的数据混合

数据混合允许您将自定义训练数据集与 Nova 的专有训练数据相结合。此功能适用于 Nova 1.0 和 Nova 2.0 模型。

Nova 专有数据类型：Nova 支持文本和多模态 SFT 数据类型。它分为多个数据类别，每个类别都包含与该类别相关的多种任务。

Nova 专有数据类别：文本数据集包括多个类别，例如：自主决策、任务完成、以目标为导向的数据集（agents）、推理和非推理类的精确任务执行数据集（reasoning-instruction-following、instruction-following）、展示战略思维和逐步任务细分（planning）的序列、负责任的人工智能（rai）、长上下文、事实、数学、词干等等。同样，多模态数据集包括视频、屏幕截图、图表等。

数据混合功能允许您将自己的微调训练样本与用于微调 Nova 的 Nova 数据集样本进行混合。这可以防止在自定义训练上过拟合以及 Nova 能力的“灾难性遗忘”，或者在从新的预训练检查点开始训练时帮助您构建能力。

要混合 Nova 数据，只需在配方 YAML 文件的 training_config 部分下添加 data_mixing 数据块。文本和多模态数据混合块的内容不同，且 Nova 1.0 与 Nova 2.0 的数据混合块也有所差异。请参考相应的配方。

支持的模型

支持的模态

YAML 配置示例

适用于 Nova 1.0 文本混合的示例数据混合块

## Run config
run:
  name: "my-lora-run"             # A descriptive name for your training job
  model_type: "amazon.nova-lite-v1:0:300k"  # Model variant specification, do not change
  model_name_or_path: "nova-lite/prod"      # Base model path, do not change
  replicas: 4                     # Number of compute instances for training, allowed values are 4, 8, 16
  data_s3_path: ""                # Customer data path
  output_s3_path: ""              # Output artifact path, SageMaker HyperPod job-specific configuration - not compatible with standard SageMaker Training jobs

  ## MLFlow configs
  mlflow_tracking_uri: "" # Required for MLFlow
  mlflow_experiment_name: "my-lora-experiment" # Optional for MLFlow. Note: leave this field non-empty
  mlflow_run_name: "my-lora-run" # Optional for MLFlow. Note: leave this field non-empty
  
## Training specific configs
training_config:
  max_length: 32768               # Maximum context window size (tokens).
  global_batch_size: 64           # Global batch size, allowed values are 16, 32, 64

  trainer:
    max_epochs: 2                # Number of training epochs

  model:
    hidden_dropout: 0.0          # Dropout for hidden states, must be between 0.0 and 1.0
    attention_dropout: 0.0       # Dropout for attention weights, must be between 0.0 and 1.0
    ffn_dropout: 0.0             # Dropout for feed-forward networks, must be between 0.0 and 1.0

    optim:
      lr: 1e-5                 # Learning rate
      name: distributed_fused_adam  # Optimizer algorithm, do not change
      adam_w_mode: true        # Enable AdamW mode
      eps: 1e-06               # Epsilon for numerical stability
      weight_decay: 0.0        # L2 regularization strength, must be between 0.0 and 1.0
      betas:                   # Adam optimizer betas, must be between 0.0 and 1.0
        - 0.9
        - 0.999
      sched:
        warmup_steps: 10     # Learning rate warmup steps
        constant_steps: 0    # Steps at constant learning rate
        min_lr: 1e-6         # Minimum learning rate

    peft:
      peft_scheme: "lora"      # Enable LoRA for parameter-efficient fine-tuning
      lora_tuning:
        loraplus_lr_ratio: 8.0  # LoRA+ learning rate scaling factor, must be between 0.0 and 100.0
        alpha: 32            # Scaling factor for LoRA weights. Allowed values are 32, 64, 96, 128, 160 and 192
        adapter_dropout: 0.01  # Regularization for LoRA parameters. Must be between 0.0 and 1.0

data_mixing:
  dataset_catalog: sft_text       # Nova text dataset catalog
  sources:
    customer_data:
      percent: 50                 # Percent of overall mix to draw from customer data
    nova_data:                    # The remainder will be drawn from Nova data. The categories below must add to 100
      agents: 1                   # autonomous decision-making, task completion, goal-oriented behavior in AI systems
      chat: 51                    # Conversational exchanges demonstrating natural dialogue flow
      code: 8                     # Programming examples and solutions spanning multiple languages
      rai: 1                      # ethical AI principles, safety considerations, and responsible technology deployment
      instruction-following: 13   # precise task execution based on varying levels of user prompts and directives
      stem: 6                     # science, technology, engineering, and mathematics content
      planning: 2                 # sequences demonstrating strategic thinking and step-by-step task breakdown
      reasoning: 8                # logical problem-solving and analytical thinking demonstrations
      rag: 1                      # retrieval-augmented generation examples
      translation: 9              # language translation tasks

Nova 2.0 配置示例

run:
  name: my-lora-sft-run
  model_type: amazon.nova-2-lite-v1:0:256k
  model_name_or_path: nova-lite-2/prod
  data_s3_path: s3://my-bucket-name/train.jsonl
  replicas: 4
  output_s3_path: s3://my-bucket-name/outputs/
  mlflow_tracking_uri: ""
  mlflow_experiment_name: "my-lora-sft-experiment"
  mlflow_run_name: "my-lora-sft-run"
  
training_config:
  max_steps: 100
  save_steps: 10
  save_top_k: 5
  max_length: 32768
  global_batch_size: 32
  reasoning_enabled: true
  lr_scheduler:
    warmup_steps: 15
    min_lr: 1e-6
  optim_config:
    lr: 1e-5
    weight_decay: 0.0
    adam_beta1: 0.9
    adam_beta2: 0.95
  peft:
    peft_scheme: "lora"
    lora_tuning:
      alpha: 64
      lora_plus_lr_ratio: 64.0

Nova 2.0 文本数据混合

data_mixing:
  dataset_catalog: sft_1p5_text_chat
  sources:
    customer_data:
      percent: 50
    nova_data:
      agents: 1
      baseline: 10
      chat: 0.5
      code: 10
      factuality: 0.1
      identity: 1
      long-context: 1
      math: 2
      rai: 1
      instruction-following: 13
      stem: 0.5
      planning: 10
      reasoning-chat: 0.5
      reasoning-code: 0.5
      reasoning-factuality: 0.5
      reasoning-instruction-following: 45
      reasoning-math: 0.5
      reasoning-planning: 0.5
      reasoning-rag: 0.4
      reasoning-rai: 0.5
      reasoning-stem: 0.4
      rag: 1
      translation: 0.1

Nova 1.0 多模态数据混合

data_mixing:
  dataset_catalog: sft_mm
  sources:
    customer_data:
      percent: 50
    nova_data:
      agents: 11
      docs: 17
      perception: 10
      rag: 4
      rai: 3
      reasoning: 10
      stem: 17
      text: 10
      video: 18

Nova 2.0 多模态数据混合

data_mixing:
  dataset_catalog: sft_1p5_mm_chat
  sources:
    customer_data:
      percent: 50
    nova_data:
      charts: 1
      chat: 38
      code: 20
      docs: 3
      general: 2
      grounding: 1
      rag: 4
      screenshot: 4
      text: 8
      translation: 4
      video: 15

模型检查点

Nova 1.0 检查点

Nova 2.0 检查点

训练阶段：

训练方法

开始之前的先决条件

成功实现 SFT 的系统性方法

实验排序和数据混合

为 SFT 准备数据集

Nova 1.0：https://docs.aws.amazon.com/sagemaker/latest/dg/nova-fine-tune.html 中介绍了数据准备

Nova 2.0：使用与 Nova 1.0 相同的 Converse API 格式 https://docs.aws.amazon.com/bedrock/latest/userguide/conversation-inference-call.html。Nova 2.0 数据格式可以包含其他推理字段：https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_ReasoningContentBlock.html

推理内容记录了模型在生成最终答案之前的中间思考步骤。在 assistant 回复中，使用 reasoningContent 字段以包含推理轨迹。请为推理内容使用纯文本，除非任务特别要求，否则避免使用 <thinking> 和 </thinking> 之类的标记标签。

负责任的人工智能工具包和内容审核

内容审核设置：Nova Forge 客户可以为 Nova Lite 1.0 和 Pro 1.0 模型使用可自定义内容审核设置（CCMS）。CCMS 允许您调整内容审核控制，以符合特定的业务需求，同时保持必要的负责任的人工智能保护措施。要确定业务使用案例是否符合 CCMS 条件，请联系 Amazon Web Services 客户经理。

Nova Forge 提供负责任的人工智能工具包，其中包括训练数据、评估基准测试和运行时控制，以帮助您使模型符合 Nova 负责任的人工智能指南。

训练数据：数据混合中的“rai”类别包含强调负责任的人工智能原则、安全考虑和负责任技术部署的案例和场景。在持续预训练期间，使用这些数据来负责任地使模型对齐。

评估：提供了基准测试任务，用于测试模型检测和拒绝不当、有害或不正确内容的能力。使用这些评估来衡量基础模型性能与自定义模型性能之间的差异。

运行时控制：默认情况下，Nova 的运行时控制会在推理期间对模型响应进行审核。要针对特定业务案例修改这些控制，请联系 Amazon Web Services 客户经理，申请可自定义内容审核设置（CCMS）。

安全责任共担

安全是 Amazon Web Services 与客户的共同责任。更改基础模型或使用持续预训练来提高特定使用案例的性能，可能会影响新模型的安全性、公平性及其他特性。

我们采用稳健的适配方法，在尽可能保留基础模型原生的安全性、公平性及其他防护机制的同时，同时最大限度地减少未针对性地对模型进行自定义的任务的性能影响。

您有责任做到以下几点：

有关更多信息，请参阅《Amazon Web Services 负责任的人工智能使用指南》、《Amazon Web Services 负责任的人工智能政策》、《AWS 可接受使用政策》以及计划使用的服务的《AWS 服务条款》。

可自定义内容审核设置（CCMS）

CCMS 允许您根据业务需求调整相关控制，与此同时，系统会保留一些不可配置的基本控制，以确保负责任地使用人工智能。

这些设置支持通过以下三种配置实现内容生成：

内容审核共分为四大类别：

无论 CCMS 配置如何，Amazon Nova 都会强制执行不可配置的基本控制，以确保负责任地使用人工智能，例如防止对儿童造成伤害和保护隐私的控制。

CCMS 使用建议

使用 CCMS 时，建议使用持续预训练（CPT），从 RAI 对齐前检查点（PRE-TRAINING-Early、PRE-TRAINING-Mid 或 PRE-TRAINING-Final）开始，而不是从 GA/FINAL 检查点开始。这些检查点未经安全训练，也未引导至特定的 RAI 行为，因此您可以根据内容审核要求更有效地对其进行自定义。

提示：将 CCMS 与数据混合结合使用时，请考虑调整 nova_data 配置中的“rai”类别百分比，使其符合特定的内容审核要求。

可用性

CCMS 目前可供已获批准的客户在以下条件下使用：

要为 Forge 模型启用 CCMS，请联系 Amazon Web Services 客户经理。

评估方法

先决条件

开箱即用的基准测试：使用开箱即用的基准测试来验证通用任务的性能。有关更多详细信息，请单击此处：https://docs.aws.amazon.com/sagemaker/latest/dg/nova-hp-evaluate.html

自带数据

您也可以通过将数据格式化为以下格式，然后使用下述容器来提供自定义数据，如果需要，还可以获取推理结果以及用于校准的对数概率。

为每个任务创建具有以下结构的 jsonl：

{
  "metadata": "{key:4, category:'apple'}",
  "system": "arithmetic-patterns, please answer the following with no other words: ",
  "query": "What is the next number in this series? 1, 2, 4, 8, 16, ?",
  "response": "32"
}
      

评估作业在推理阶段生成的输出将具有以下结构：

{
  "prompt": "[{'role': 'system', 'content': 'arithmetic-patterns, please answer the following with no other words: '}, {'role': 'user', 'content': 'What is the next number in this series? 1, 2, 4, 8, 16, ?'}]",
  "inference": "['32']",
  "gold": "32",
  "metadata": "{key:4, category:'apple'}"
}
      

字段描述：

准备评估配置

启动评估作业的命令。使用 "--override-parameters" 修改配方中的任何条目。

hyperpod start-job -n kubeflow \
  --recipe evaluation/nova/nova_micro_p5_48xl_bring_your_own_dataset_eval \
  --override-parameters '{
    "instance_type": "p5.48xlarge",
    "container": "708977205387.dkr.ecr.us-east-1.amazonaws.com/nova-evaluation-repo:SM-HP-Eval-latest",
    "recipes.run.name": "<your-eval-job-name>",
    "recipes.run.model_name_or_path": "<checkpoint-s3-uri>",
    "recipes.run.output_s3_path": "s3://<your-bucket>/eval-results/",
    "recipes.run.data_s3_path": "s3://<your-bucket>/eval-data.jsonl"
  }'
      

最佳实践

准备训练作业配置

超参数

除数据混合外的全套超参数：

## Run config
run:
  name: my-lora-sft-run
  model_type: amazon.nova-2-lite-v1:0:256k
  model_name_or_path: nova-lite-2/prod
  data_s3_path: s3://my-bucket-name/train.jsonl  # SageMaker HyperPod (SMHP) only and not compatible with SageMaker Training jobs. Note replace my-bucket-name with your real bucket name for SMHP job
  replicas: 4                      # Number of compute instances for training, allowed values are 4, 8, 16, 32
  output_s3_path: s3://my-bucket-name/outputs/               # Output artifact path (Hyperpod job-specific; not compatible with standard SageMaker Training jobs). Note replace my-bucket-name with your real bucket name for SMHP job
  
  ## MLFlow configs
  mlflow_tracking_uri: "" # Required for MLFlow
  mlflow_experiment_name: "my-lora-sft-experiment" # Optional for MLFlow. Note: leave this field non-empty
  mlflow_run_name: "my-lora-sft-run" # Optional for MLFlow. Note: leave this field non-empty
  
training_config:
  max_steps: 100                   # Maximum training steps. Minimal is 4.
  save_steps: 10 # This parameter suggests after how many training steps the checkpoints will be saved. Should be less than or equal to max_steps(please override this value with a numerical value equal or less than max_steps value; min: 4)
  save_top_k: 5                    # Keep top K best checkpoints. Note supported only for SageMaker HyperPod jobs. Minimal is 1.
  max_length: 32768                # Sequence length (options: 8192, 16384, 32768 [default], 65536)
  global_batch_size: 32            # Golbal batch size (options: 32, 64, 128)
  reasoning_enabled: true          # If data has reasoningContent, set to true; otherwise False

  lr_scheduler:
    warmup_steps: 15               # Learning rate warmup steps. Recommend 15% of max_steps
    min_lr: 1e-6                   # Minimum learning rate, must be between 0.0 and 1.0

  optim_config:                    # Optimizer settings
    lr: 1e-5                       # Learning rate, must be between 0.0 and 1.0
    weight_decay: 0.0              # L2 regularization strength, must be between 0.0 and 1.0
    adam_beta1: 0.9                # Exponential decay rate for first-moment estimates, must be between 0.0 and 1.0
    adam_beta2: 0.95               # Exponential decay rate for second-moment estimates, must be between 0.0 and 1.0

  peft:                            # Parameter-efficient fine-tuning (LoRA)
    peft_scheme: "lora"            # Enable LoRA for PEFT
    lora_tuning:
      alpha: 64                    # Scaling factor for LoRA weights ( options: 32, 64, 96, 128, 160, 192),
      lora_plus_lr_ratio: 64.0     # LoRA+ learning rate scaling factor (0.0–100.0)

LoRa 最相关的参数是学习率、alpha（缩放参数）、用于训练的 epoch 数和预热步骤数；对于全秩微调，最相关的参数则主要是学习率、用于训练的 epoch 数和预热步骤数。配方中已预填充推荐的默认值。

设置数据混合块

将 data_mixing 部分添加到配方，并在数据集类别中设置相应的百分比分布。

下面介绍每个可用的 Nova 数据类别。

包含数据混合的 Nova 1.0 配置

data_mixing:
  dataset_catalog: sft_text       # Nova text dataset catalog
  sources:
    customer_data:
      percent: 50                 # Percent of overall mix to draw from customer data
    nova_data:                    # The remainder will be drawn from Nova data. The categories below must add to 100
      agents: 1                   # autonomous decision-making, task completion, goal-oriented behavior in AI systems
      chat: 51                    # Conversational exchanges demonstrating natural dialogue flow
      code: 8                     # Programming examples and solutions spanning multiple languages
      rai: 1                      # ethical AI principles, safety considerations, and responsible technology deployment
      instruction-following: 13   # precise task execution based on varying levels of user prompts and directives
      stem: 6                     # Technical content covering science, technology, engineering, and mathematics
      planning: 2                 # Sequences demonstrating strategic thinking and step-by-step task breakdown
      reasoning: 8                # Logical deduction, critical thinking, and analytical problem-solving scenarios
      rag: 1                      # combining retrieved external knowledge with generated responses
      translation: 9              # Multi-language content pairs showing accurate translation

这些类别的含义是什么？

多模态数据混合（Nova 1.0）

data_mixing:
  dataset_catalog: sft_mm        # Nova multi-modal dataset catalog
  sources:
    customer_data:
      percent: 50                 # Percent of overall mix to draw from customer data
    nova_data:                    # The remainder will be drawn from Nova data. The categories below must add to 100
      agents: 11                  # Combining visual and textual inputs
      docs: 17                    # Document-centric data combining text, images, layouts, and formatting
      perception: 10              # Visual-linguistic pairs t
      rag: 4                      # Combining retrieved external knowledge with generated responses
      rai: 3                      # Ethical AI principles, safety considerations, and responsible technology deployment
      reasoning: 10               # Logical analysis, problem-solving, and drawing conclusions
      stem: 17                    # Technical content pairing visual elements (diagrams, charts, equations) with text
      text: 10                    # A balanced pool of contextual text data create from the text-only SFT datasets
      video: 18                   # Video datasets

这些类别的含义是什么？

包含数据混合的 Nova 2.0 配置

data_mixing:
  dataset_catalog: sft_1p5_text_chat       # Nova text dataset catalog
  sources:
    customer_data:
      percent: 50                 # Percent of overall mix to draw from customer data
    nova_data:                    # The remainder will be drawn from Nova data. The categories below must add to 100
      agents: 1                   # autonomous decision-making, task completion, goal-oriented behavior in AI systems
      baseline: 10                 # [New in Nova 1.5]
      chat: 0.5                    # Conversational exchanges demonstrating natural dialogue flow
      code: 10                     # Programming examples and solutions spanning multiple languages
      factuality: 0.1               # [New in Nova 1.5]
      identity: 1                 # [New in Nova 1.5]
      long-context: 1             # [New in Nova 1.5]
      math: 2                     # [New in Nova 1.5]
      rai: 1                      # ethical AI principles, safety considerations, and responsible technology deployment
      instruction-following: 13   # precise task execution based on varying levels of user prompts and directives
      stem: 0.5                     # Technical content covering science, technology, engineering, and mathematics
      planning: 10                 # Sequences demonstrating strategic thinking and step-by-step task breakdown
      reasoning-chat: 0.5
      reasoning-code: 0.5
      reasoning-factuality: 0.5
      reasoning-instruction-following: 45
      reasoning-math: 0.5
      reasoning-planning: 0.5
      reasoning-rag: 0.4
      reasoning-rai: 0.5
      reasoning-stem: 0.4
      rag: 1                      # combining retrieved external knowledge with generated responses
      translation: 0.1

这些类别的含义是什么？

多模态数据混合（Nova 2.0）

data_mixing:
  dataset_catalog: sft_1p5_mm_chat       # Nova text dataset catalog
  sources:
    customer_data:
      percent: 50                 # Percent of overall mix to draw from customer data
    nova_data:                    # The remainder will be drawn from Nova data. The categories below must add to 100
      charts: 1
      chat: 38
      code: 20
      docs: 3
      general: 2
      grounding: 1
      rag: 4
      screenshot: 4
      text: 8
      translation: 4
      video: 15

注意：Nova 2.0 包含 Nova 1.0 中不支持的视频数据类别支持。

这些类别的含义是什么？

如何启动作业

如果您只需要获取启动首次 SFT 运行的基本细节，也可以参考 README。

容器信息：

完成所有设置后，从 sagemaker-hyperpod-cli 存储库的根目录开始，导航到默认的 nova sft 配方文件夹：

对于 Nova 1.0 SFT：

对于 Nova 2.0 SFT：

数据混合配置看起来类似，但在底部会有一个额外的数据混合部分，类似于这样

data_mixing:
  dataset_catalog: sft_text_lite
  sources:
    nova_data:   # percent inputs for Nova data must sum to 100%; use 0% if you want to exclude a data grouping
      agents: 20
      business-and-finance: 20
      scientific: 20
      code: 20
      factual-and-news: 20
      longform-text: 0
      health-and-medicine: 0
      humanities-and-education: 0
      legal: 0
      math: 0
      additional-languages: 0
      social-and-personal-interest: 0
      entertainment: 0
      reasoning: 0
      other: 0
      tables: 0
    customer_data: # percent input of customer data. 100 = use only customer data, 0 = use only the nova_data mix above
      percent: 25

这里有两个顶级数据类别：

提示：再次运行 pip install-e.，就可以提交作业了！

我们将在此覆盖几个参数以使用数据混合：

hyperpod start-job \
 -n kubeflow \
 --recipe fine-tuning/nova/forge/nova_2_0/nova_lite/SFT/nova_lite_2_0_p5x8_gpu_sft_with_datamix \
 --override-parameters '{
 "instance_type": "ml.p5.48xlarge",
 "recipes.run.name": "nova-sft-datamixing",
 "container": "708977205387.dkr.ecr.us-east-1.amazonaws.com/nova-fine-tune-repo:SM-HP-SFT-V2-Datamix",
 "recipes.run.data_s3_path": "s3://sft-data/sft_train_data.jsonl",
 "recipes.run.validation_data_s3_path": "s3://sft-data/sft_val_data.jsonl",
 "recipes.run.output_s3_path": "s3://sft-data/output/
 }'

输出应包含作业名称，如下所示：

⚡ MY Desktop ⚡ % hyperpod start-job \
 -n kubeflow \
 --recipe training/nova/forge/nova_2_0/nova_lite/sft/nova_lite_2_0_p5x8_gpu_pretrain_with_datamix \
 --override-parameters '{
 "instance_type": "ml.p5.48xlarge",
 "recipes.run.name": "nova-sft-datamixing",
 "container": "708977205387.dkr.ecr.us-east-1.amazonaws.com/nova-fine-tune-repo:SM-HP-SFT-V2-Datamix",
 "recipes.run.data_s3_path": "s3://sft-data/sft_train_data.jsonl",
 "recipes.run.validation_data_s3_path": "3://sft-data/sft_val_data.jsonl",
 "recipes.run.output_s3_path": "3://sft-data/output/
 }'

输出将如下所示：

Final command: python3 /local/home/my/Downloads/sagemaker-hyperpod-cli/src/hyperpod_cli/sagemaker_hyperpod_recipes/main.py recipes=training/nova/nova_micro_p5x8_gpu_pretrain cluster_type=k8s cluster=k8s base_results_dir=/local/home/niphaded/Downloads/sagemaker-hyperpod-cli/results cluster.pullPolicy="IfNotPresent" cluster.restartPolicy="OnFailure" cluster.namespace="kubeflow" instance_type="p5d.48xlarge" container="900867814919.dkr.ecr.us-east-1.amazonaws.com/nova-fine-tune-repo:sft-datamix-rig-final"
Prepared output directory at /local/home/my/Downloads/sagemaker-hyperpod-cli/results/my-sft-run-wzdyn/k8s_templates
Found credentials in shared credentials file: ~/.aws/credentials
Helm script created at /local/home/my/Downloads/sagemaker-hyperpod-cli/results/my-sft-run-wzdyn/niphaded-sft-run-wzdyn_launch.sh
Running Helm script: /local/home/my/Downloads/sagemaker-hyperpod-cli/results/my-sft-run-wzdyn/niphaded-sft-run-wzdyn_launch.sh

NAME: my-sft-run-wzdyn
LAST DEPLOYED: Tue Aug 26 16:21:06 2025
NAMESPACE: kubeflow
STATUS: deployed
REVISION: 1
TEST SUITE: None
Launcher successfully generated: /local/home/my/Downloads/sagemaker-hyperpod-cli/src/hyperpod_cli/sagemaker_hyperpod_recipes/launcher/nova/k8s_templates/SFT

{
 "Console URL": "https://us-east-1.console.aws.amazon.com/sagemaker/home?region=us-east-1#/cluster-management/hyperpod-eks-ga-0703"
}

您可以使用 hyperpod list-pods -n kubeflow --job-name my-sft-run-wzdyn 查看作业状态

hyperpod list-pods -n kubeflow --job-name my-sft-run-wzdyn 
{
 "pods": [
  {
   "PodName": "my-sft-run-wzdyn-master-0",
   "Namespace": "kubeflow",
   "Status": "Pending",
   "CreationTime": "2025-08-26 16:21:06+00:00"
  },
  {
   "PodName": "my-sft-run-wzdyn-worker-0",
   "Namespace": "kubeflow",
   "Status": "Pending",
   "CreationTime": "2025-08-26 16:21:06+00:00"
  }
 ]
}

或者直接使用 kubectl 命令查找它们。

例如，

kubectl get pods -o wide -w -n kubeflow | (head -n1 ; grep my-sft-run)

NAME                                                         READY   STATUS      RESTARTS   AGE     IP              NODE                           NOMINATED NODE   READINESS GATES
my-sft-run-5suc8-master-0                              0/1     Completed   0          3h23m   172.31.32.132   hyperpod-i-00b3d8a1bf25714e4   <none>           <none>
my-sft-run-5suc8-worker-0                              0/1     Completed   0          3h23m   172.31.44.196   hyperpod-i-0aa7ccfc2bd26b2a0   <none>           <none>
my-sft-run-5suc8-worker-1                              0/1     Completed   0          3h23m   172.31.46.84    hyperpod-i-026df6406a7b7e55c   <none>           <none>
my-sft-run-5suc8-worker-2                              0/1     Completed   0          3h23m   172.31.28.68    hyperpod-i-0802e850f903f28f1   <none>           <none>

专业提示：务必始终使用 -o wide 标志，因为作业运行的 EKS 节点将帮助您在 AWS UI 中更快地找到日志

如何监控作业

您可通过以下三种方式之一查看日志：

a) 使用 CloudWatch

日志位于 Amazon Web Services 账户（包含 HyperPod 集群）的 CloudWatch 下。如需在浏览器中查看，请导航到账户中的 CloudWatch 主页，然后搜索集群名称。例如，如果集群名为 my-hyperpod-rig，则日志组将具有以下前缀：

日志应该如下所示：

b) 使用 CloudWatch Insights

如果您手边有作业名称，并且不想完成上述所有步骤，则只需查询 /aws/sagemaker/Clusters/my-hyperpod-rig/{UUID} 下的所有日志，即可找到单个日志。

CPT

fields @timestamp, @message, @logStream, @log 
| filter @message like /(?i)Starting CPT Job/
| sort @timestamp desc 
| limit 100

要完成作业，请将“Starting SFT Job”替换为“SFT Job completed”

然后，您可以单击结果，并选择显示“Epoch 0”的结果，因为那将是主节点。

C) 使用 Amazon CLI

您可以选择使用 ... 来跟踪日志。在此之前，请使用 aws --version 检查 AWS CLI 版本。建议使用提供的工具脚本，以便在终端中实时跟踪日志

V1：

aws logs get-log-events \
 --log-group-name /aws/sagemaker/YourLogGroupName \
 --log-stream-name YourLogStream \
 --start-from-head | jq -r '.events[].message'

V2：

aws logs tail /aws/sagemaker/YourLogGroupName \
  --log-stream-name YourLogStream \
 --since 10m \
 --follow

d) 设置机器学习流程：

您可以通过 MLFlow 跟踪指标。

创建 MLFlow 应用程序

使用 Studio UI：如果您通过 Studio UI 创建训练作业，系统会自动创建默认 MLFlow 应用程序，并在“高级选项”下默认选择该应用程序。

使用 CLI：如果您使用 CLI，则必须创建一个 MLFlow 应用程序，并将其作为输入传递给训练作业 API 请求。

mlflow_app_name="<enter your MLflow app name>"  
role_arn="<enter your role ARN>"   
bucket_name="<enter your bucket name>"   
region="<enter your region>"  
  
mlflow_app_arn=$(aws sagemaker create-mlflow-app \  
  --name $mlflow_app_name \  
  --artifact-store-uri "s3://$bucket_name" \  
  --role-arn $role_arn \  
  --region $region)

访问 MLFlow 应用程序

使用 CLI：创建预签名 URL 以访问 MLFlow 应用程序 UI：

aws sagemaker create-presigned-mlflow-app-url \  
  --arn $mlflow_app_arn \  
  --region $region \  
  --output text

机器学习流程设置完成后，您可以在配方中传递 URI，或者在开始作业时使用覆盖。有关如何执行此操作的示例，请参阅 README。

SFT 后如何评估模型？

先决条件

开箱即用的基准测试：使用开箱即用的基准测试来验证通用任务的性能。有关更多详细信息，请单击此处。

自带数据：

您也可以通过将数据格式化为以下格式，然后使用下述容器来提供自定义数据，如果需要，还可以获取推理结果以及用于校准的对数概率。

为每个任务创建具有以下结构的 jsonl：

{
  "metadata": "{key:4, category:'apple'}",
  "system": "arithmetic-patterns, please answer the following with no other words: ",
  "query": "What is the next number in this series? 1, 2, 4, 8, 16, ?",
  "response": "32"
}

评估作业在推理阶段生成的输出将具有以下结构：

{
  "prompt": "[{'role': 'system', 'content': 'arithmetic-patterns, please answer the following with no other words: '}, {'role': 'user', 'content': 'What is the next number in this series? 1, 2, 4, 8, 16, ?'}]",
  "inference": "['32']",
  "gold": "32",
  "metadata": "{key:4, category:'apple'}"
}

字段描述：

准备评估配置

启动评估作业的命令。使用“--override-parameters”修改配方中的任何条目。

hyperpod start-job -n kubeflow \
  --recipe evaluation/nova/nova_micro_p5_48xl_bring_your_own_dataset_eval \
  --override-parameters '{
    "instance_type": "p5.48xlarge",
    "container": "708977205387.dkr.ecr.us-east-1.amazonaws.com/nova-evaluation-repo:SM-HP-Eval-latest",
    "recipes.run.name": "<your-eval-job-name>",
    "recipes.run.model_name_or_path": "<checkpoint-s3-uri>",
    "recipes.run.output_s3_path": "s3://<your-bucket>/eval-results/",
    "recipes.run.data_s3_path": "s3://<your-bucket>/eval-data.jsonl"
  }'

启动评估作业

不同配方及其对应映像的作业启动命令。

经验教训和提示