View a markdown version of this page

Formats de jeu de données pris en charge pour les tâches Bring-Your-Own-Dataset (BYOD) - Amazon SageMaker AI

Les traductions sont fournies par des outils de traduction automatique. En cas de conflit entre le contenu d'une traduction et celui de la version originale en anglais, la version anglaise prévaudra.

Formats de jeu de données pris en charge pour les tâches Bring-Your-Own-Dataset (BYOD)

Le score personnalisé et les types LLM-as-judge d'évaluation nécessitent un fichier JSONL de jeu de données personnalisé situé dans S3. AWS Vous devez fournir le fichier sous la forme d'un fichier de lignes JSON respectant l'un des formats pris en charge suivants. Les exemples de ce document sont développés pour plus de clarté.

Chaque format a ses propres nuances, mais tous nécessitent au minimum une invite de l'utilisateur.

Champs obligatoires
Champ Obligatoire
Invite de l'utilisateur Oui
Invite système Non
Vérité fondamentale Uniquement pour Custom Scorer
Catégorie Non

1. Format OpenAI

{
    "messages": [
        {
            "role": "system",    # System prompt (looks for system role)
            "content": "You are a helpful assistant."
        },
        {
            "role": "user",       # Query (looks for user role)
            "content": "Hello!"
        },
        {
            "role": "assistant",  # Ground truth (looks for assistant role)
            "content": "Hello to you!"
        }
    ]
}

2. SageMaker Évaluation

{
   "system":"You are an English major with top marks in class who likes to give minimal word responses: ",
   "query":"What is the symbol that ends the sentence as a question",
   "response":"?", # Ground truth
   "category": "Grammar"
}

3. HuggingFace Achèvement rapide

Les formats standard et conversationnel sont pris en charge.

# Standard

{
    "prompt" : "What is the symbol that ends the sentence as a question", # Query
    "completion" : "?" # Ground truth
}

# Conversational
{
    "prompt": [
        {
            "role": "user", # Query (looks for user role)
            "content": "What is the symbol that ends the sentence as a question"
        }
    ],
    "completion": [
        {
            "role": "assistant", # Ground truth (looks for assistant role)
            "content": "?"
        }
    ]
}

4. HuggingFace Préférence

Support du format standard (chaîne) et du format conversationnel (tableau de messages).

# Standard: {"prompt": "text", "chosen": "text", "rejected": "text"}
{
     "prompt" : "The sky is", # Query
     "chosen" : "blue", # Ground truth
     "rejected" : "green"
}

# Conversational:
{
    "prompt": [
        {
            "role": "user", # Query (looks for user role)
            "content": "What color is the sky?"
        }
    ],
    "chosen": [
        {
            "role": "assistant", # Ground truth (looks for assistant role)
            "content": "It is blue."
        }
    ],
    "rejected": [
        {
            "role": "assistant",
            "content": "It is green."
        }
    ]
}

5. Format Verl

Le format Verl (formats actuels et anciens) est pris en charge pour les cas d'utilisation de l'apprentissage par renforcement. Documents Verl pour référence : https://verl.readthedocs.io/en/latest/preparation/prepare_data.html

Les utilisateurs du format VERL ne fournissent généralement pas de réponse fondée sur la vérité. Si vous souhaitez en fournir un de toute façon, utilisez l'un des champs extra_info.answer ou reward_model.ground_truth ; extra_info a priorité.

SageMaker préserve les champs spécifiques à Verl suivants sous forme de métadonnées, le cas échéant :

  • id

  • data_source

  • ability

  • reward_model

  • extra_info

  • attributes

  • difficulty

# Newest VERL format where `prompt` is an array of messages.
{
  "data_source": "openai/gsm8k",
  "prompt": [
    {
      "content": "You are a helpful math tutor who explains solutions to questions step-by-step.",
      "role": "system"
    },
    {
      "content": "Natalia sold clips to 48 of her friends in April, and then she sold half as many clips in May. How many clips did Natalia sell altogether in April and May? Let's think step by step and output the final answer after \"####\".",
      "role": "user"
    }
  ],
  "ability": "math",
  "extra_info": {
    "answer": "Natalia sold 48/2 = <<48/2=24>>24 clips in May.\nNatalia sold 48+24 = <<48+24=72>>72 clips altogether in April and May.\n#### 72",
    "index": 0,
    "question": "Natalia sold clips to 48 of her friends in April, and then she sold half as many clips in May. How many clips did Natalia sell altogether in April and May?",
    "split": "train"
  },
  "reward_model": {
    "ground_truth": "72" # Ignored in favor of extra_info.answer
  }
}

# Legacy VERL format where `prompt` is a string. Also supported.
{
  "data_source": "openai/gsm8k",
  "prompt": "Natalia sold clips to 48 of her friends in April, and then she sold half as many clips in May. How many clips did Natalia sell altogether in April and May? Let's think step by step and output the final answer after \"####\".",
  "extra_info": {
    "answer": "Natalia sold 48/2 = <<48/2=24>>24 clips in May.\nNatalia sold 48+24 = <<48+24=72>>72 clips altogether in April and May.\n#### 72",
    "index": 0,
    "question": "Natalia sold clips to 48 of her friends in April, and then she sold half as many clips in May. How many clips did Natalia sell altogether in April and May?",
    "split": "train"
  }
}