Mempersiapkan data untuk fine-tuning multimodal

Berikut ini adalah pedoman dan persyaratan untuk menyiapkan data untuk menyempurnakan model Pemahaman:

Merancang contoh pelatihan yang efektif

Data pelatihan Anda harus menunjukkan perilaku yang Anda inginkan untuk ditunjukkan oleh model. SFT mengajarkan model bagaimana merespons, bukan apa yang harus diketahui. Jika Anda menemukan diri Anda membuat contoh pelatihan terutama untuk menyuntikkan pengetahuan faktual (misalnya, “Apa E-45 arti kode kesalahan?” dengan jawaban "E-45 menunjukkan batas waktu sensor”), pertimbangkan apakah RAG atau teknik cepat dapat mencapai hasil yang sama tanpa fine-tuning.

Saat mengonversi data sumber ke dalam format percakapan, ikuti prinsip-prinsip ini:

Pola transformasi data umum

Pola berikut menggambarkan bagaimana mengubah format data perusahaan umum menjadi contoh pelatihan SFT yang efektif.

Pola 1: Catatan terstruktur untuk klasifikasi atau pencarian

Jika data sumber Anda adalah tabel atau database catatan dengan bidang sepertiissue,cause, danresolution, bingkai setiap catatan sebagai pertanyaan pengguna dengan bidang input yang relevan dan respons asisten sebagai bidang keluaran target.

// train.jsonl
{
  "schemaVersion": "bedrock-conversation-2024",
  "system": [
    {
      "text": "You are a technical support assistant. Given an issue description, identify the most likely root cause and recommended resolution."
    }
  ],
  "messages": [
    {
      "role": "user",
      "content": [
        {
          "text": "A user reports: 'Application crashes on startup after the latest update.' What is the likely cause and resolution?"
        }
      ]
    },
    {
      "role": "assistant",
      "content": [
        {
          "text": "Cause: Incompatible configuration file from the previous version.\nResolution: Delete the local configuration cache and restart the application to regenerate default settings."
        }
      ]
    }
  ]
}

Pola 2: Data hierarkis atau taksonomi ke Tanya Jawab terstruktur

Ketika data sumber Anda adalah hierarki kode atau taksonomi seperti pohon kategori produk atau sistem klasifikasi, buat pertanyaan yang meminta model untuk menavigasi hierarki. Sertakan konteks klasifikasi dalam prompt sistem atau giliran pengguna.

// train.jsonl
{
  "schemaVersion": "bedrock-conversation-2024",
  "system": [
    {
      "text": "You are a product catalog assistant. Answer questions about product categories and classifications accurately."
    }
  ],
  "messages": [
    {
      "role": "user",
      "content": [
        {
          "text": "What top-level category does product code 'EL-4520' belong to, and what subcategories does it include?"
        }
      ]
    },
    {
      "role": "assistant",
      "content": [
        {
          "text": "Product code EL-4520 belongs to the Electronics category. It includes the subcategories: Audio Equipment, Portable Speakers, and Wireless Accessories."
        }
      ]
    }
  ]
}

Pola 3: Dokumen prosedural untuk dialog instruksional

Jika data sumber Anda terdiri dari prosedur operasi standar, buku runbook, atau panduan langkah demi langkah, ubah prosedur menjadi pertanyaan berorientasi tugas. Pengguna bertanya bagaimana melakukan tugas dan asisten memberikan langkah-langkah dalam format yang konsisten.

// train.jsonl
{
  "schemaVersion": "bedrock-conversation-2024",
  "system": [
    {
      "text": "You are an IT operations assistant. Provide step-by-step instructions based on approved runbooks and standard operating procedures."
    }
  ],
  "messages": [
    {
      "role": "user",
      "content": [
        {
          "text": "How do I reset a user's multi-factor authentication when they have lost access to their device?"
        }
      ]
    },
    {
      "role": "assistant",
      "content": [
        {
          "text": "Follow these steps to reset a user's multi-factor authentication:\n1. Verify the user's identity using the approved verification procedure.\n2. Disable the existing MFA configuration in the identity management console.\n3. Generate a new temporary access code and send it to the user's verified email.\n4. Instruct the user to log in with the temporary code and re-enroll their new device.\n5. Confirm the new MFA device is working by requesting a test authentication.\n6. Document the reset in the support ticket."
        }
      ]
    }
  ]
}

Contoh format dataset

Contoh format kumpulan data berikut memberikan panduan untuk Anda ikuti.

// train.jsonl
{
  "schemaVersion": "bedrock-conversation-2024",
  "system": [
    {
      "text": "You are a digital assistant with a friendly personality"
    }
  ],
  "messages": [
    {
      "role": "user",
      "content": [
        {
          "text": "What is the capital of Mars?"
        }
      ]
    },
    {
      "role": "assistant",
      "content": [
        {
          "text": "Mars does not have a capital. Perhaps it will one day."
        }
      ]
    }
  ]
}

// train.jsonl{
    "schemaVersion": "bedrock-conversation-2024",
    "system": [{
        "text": "You are a smart assistant that answers questions respectfully"
    }],
    "messages": [{
            "role": "user",
            "content": [{
                    "text": "What does the text in this image say?"
                },
                {
                    "image": {
                        "format": "png",
                        "source": {
                            "s3Location": {
                                "uri": "s3://your-bucket/your-path/your-image.png",
                                "bucketOwner": "your-aws-account-id"
                            }
                        }
                    }
                }
            ]
        },
        {
            "role": "assistant",
            "content": [{
                "text": "The text in the attached image says 'LOL'."
            }]
        }
    ]
}

{
    "schemaVersion": "bedrock-conversation-2024",
    "system": [{
        "text": "You are a helpful assistant designed to answer questions crisply and to the point"
    }],
    "messages": [{
            "role": "user",
            "content": [{
                    "text": "How many white items are visible in this video?"
                },
                {
                    "video": {
                        "format": "mp4",
                        "source": {
                            "s3Location": {
                                "uri": "s3://your-bucket/your-path/your-video.mp4",
                                "bucketOwner": "your-aws-account-id"
                            }
                        }
                    }
                }
            ]
        },
        {
            "role": "assistant",
            "content": [{
                "text": "There are at least eight visible items that are white"
            }]
        }
    ]
}

Kendala set data

Amazon Nova menerapkan batasan berikut pada kustomisasi model untuk Memahami model.