Terjemahan disediakan oleh mesin penerjemah. Jika konten terjemahan yang diberikan bertentangan dengan versi bahasa Inggris aslinya, utamakan versi bahasa Inggris. # Kustomisasi model bobot terbuka Bagian ini memandu Anda melalui proses untuk memulai dengan kustomisasi model bobot terbuka. **Topics** + [Prasyarat](model-customize-open-weight-prereq.md) + [Membuat aset untuk kustomisasi model di UI](model-customize-open-weight-create-assets-ui.md) + [Pengajuan pekerjaan kustomisasi model AI](model-customize-open-weight-job.md) + [Pengajuan pekerjaan evaluasi model](model-customize-open-weight-evaluation.md) + [Deployment model](model-customize-open-weight-deployment.md) + [Contoh kumpulan data dan evaluator](model-customize-open-weight-samples.md) # Prasyarat Sebelum menggunakan fungsi , pastikan untuk melengkapi prasyarat berikut: + Onboard ke domain SageMaker AI dengan akses Studio. Jika Anda tidak memiliki izin untuk menetapkan Studio sebagai pengalaman default untuk domain Anda, hubungi administrator Anda. Untuk informasi selengkapnya, lihat [ikhtisar domain Amazon SageMaker AI](https://docs.aws.amazon.com/sagemaker/latest/dg/gs-studio-onboard.html). + Perbarui AWS CLI dengan mengikuti langkah-langkah dalam [Menginstal AWS CLI Versi saat ini](https://docs.aws.amazon.com/cli/latest/userguide/install-cliv1.html#install-tool-bundled). + Dari mesin lokal Anda, jalankan `aws configure` dan berikan AWS kredensil Anda. Untuk informasi tentang AWS kredensil, lihat [Memahami dan mendapatkan kredensil Anda AWS](https://docs.aws.amazon.com/IAM/latest/UserGuide/security-creds.html). ## Izin IAM yang diperlukan SageMaker Kustomisasi model AI memerlukan penambahan izin yang sesuai untuk eksekusi domain SageMaker AI Anda. Untuk melakukannya, Anda dapat membuat kebijakan izin IAM sebaris dan melampirkannya ke peran IAM. Untuk informasi tentang menambahkan kebijakan, lihat [Menambahkan dan menghapus izin identitas IAM](https://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies_manage-attach-detach.html) di *Panduan Pengguna AWS Identity and Access Management*. ``` { "Version": "2012-10-17", "Statement": [ { "Sid": "AllowNonAdminStudioActions", "Effect": "Allow", "Action": [ "sagemaker:CreatePresignedDomainUrl", "sagemaker:DescribeDomain", "sagemaker:DescribeUserProfile", "sagemaker:DescribeSpace", "sagemaker:ListSpaces", "sagemaker:DescribeApp", "sagemaker:ListApps" ], "Resource": [ "arn:aws:sagemaker:*:*:domain/*", "arn:aws:sagemaker:*:*:user-profile/*", "arn:aws:sagemaker:*:*:app/*", "arn:aws:sagemaker:*:*:space/*" ] }, { "Sid": "LambdaListPermissions", "Effect": "Allow", "Action": [ "lambda:ListFunctions" ], "Resource": [ "*" ] }, { "Sid": "LambdaPermissionsForRewardFunction", "Effect": "Allow", "Action": [ "lambda:CreateFunction", "lambda:DeleteFunction", "lambda:InvokeFunction", "lambda:GetFunction" ], "Resource": [ "arn:aws:lambda:*:*:function:*SageMaker*", "arn:aws:lambda:*:*:function:*sagemaker*", "arn:aws:lambda:*:*:function:*Sagemaker*" ], "Condition": { "StringEquals": { "aws:ResourceAccount": "${aws:PrincipalAccount}" } } }, { "Sid": "LambdaLayerForAWSSDK", "Effect": "Allow", "Action": [ "lambda:GetLayerVersion" ], "Resource": [ "arn:aws:lambda:*:336392948345:layer:AWSSDK*" ] }, { "Sid": "SageMakerPublicHubPermissions", "Effect": "Allow", "Action": [ "sagemaker:ListHubContents" ], "Resource": [ "arn:aws:sagemaker:*:aws:hub/SageMakerPublicHub" ] }, { "Sid": "SageMakerHubPermissions", "Effect": "Allow", "Action": [ "sagemaker:ListHubs", "sagemaker:ListHubContents", "sagemaker:DescribeHubContent", "sagemaker:DeleteHubContent", "sagemaker:ListHubContentVersions", "sagemaker:Search" ], "Resource": [ "*" ], "Condition": { "StringEquals": { "aws:ResourceAccount": "${aws:PrincipalAccount}" } } }, { "Sid": "JumpStartAccess", "Effect": "Allow", "Action": [ "s3:GetObject", "s3:ListBucket" ], "Resource": [ "arn:aws:s3:::jumpstart*" ] }, { "Sid": "ListMLFlowOperations", "Effect": "Allow", "Action": [ "sagemaker:ListMlflowApps", "sagemaker:ListMlflowTrackingServers" ], "Resource": [ "*" ] }, { "Sid": "MLFlowAccess", "Effect": "Allow", "Action": [ "sagemaker:UpdateMlflowApp", "sagemaker:DescribeMlflowApp", "sagemaker:CreatePresignedMlflowAppUrl", "sagemaker:CallMlflowAppApi", "sagemaker-mlflow:*" ], "Resource": [ "arn:aws:sagemaker:*:*:mlflow-app/*" ], "Condition": { "StringEquals": { "aws:ResourceAccount": "${aws:PrincipalAccount}" } } }, { "Sid": "BYODataSetS3Access", "Effect": "Allow", "Action": [ "s3:ListBucket", "s3:GetObject", "s3:PutObject" ], "Resource": [ "arn:aws:s3:::*SageMaker*", "arn:aws:s3:::*Sagemaker*", "arn:aws:s3:::*sagemaker*" ] }, { "Sid": "AllowHubPermissions", "Effect": "Allow", "Action": [ "sagemaker:ImportHubContent" ], "Resource": [ "arn:aws:sagemaker:*:*:hub/*", "arn:aws:sagemaker:*:*:hub-content/*" ], "Condition": { "StringEquals": { "aws:ResourceAccount": "${aws:PrincipalAccount}" } } }, { "Sid": "PassRoleForSageMaker", "Effect": "Allow", "Action": [ "iam:PassRole" ], "Resource": [ "arn:aws:iam::*:role/service-role/AmazonSageMaker-ExecutionRole-*" ], "Condition": { "StringEquals": { "iam:PassedToService": "sagemaker.amazonaws.com", "aws:ResourceAccount": "${aws:PrincipalAccount}" } } }, { "Sid": "PassRoleForAWSLambda", "Effect": "Allow", "Action": [ "iam:PassRole" ], "Resource": [ "arn:aws:iam::*:role/service-role/AmazonSageMaker-ExecutionRole-*" ], "Condition": { "StringEquals": { "iam:PassedToService": "lambda.amazonaws.com", "aws:ResourceAccount": "${aws:PrincipalAccount}" } } }, { "Sid": "PassRoleForBedrock", "Effect": "Allow", "Action": [ "iam:PassRole" ], "Resource": [ "arn:aws:iam::*:role/service-role/AmazonSageMaker-ExecutionRole-*" ], "Condition": { "StringEquals": { "iam:PassedToService": "bedrock.amazonaws.com", "aws:ResourceAccount": "${aws:PrincipalAccount}" } } }, { "Sid": "TrainingJobRun", "Effect": "Allow", "Action": [ "sagemaker:CreateTrainingJob", "sagemaker:DescribeTrainingJob", "sagemaker:ListTrainingJobs" ], "Resource": [ "arn:aws:sagemaker:*:*:training-job/*" ], "Condition": { "StringEquals": { "aws:ResourceAccount": "${aws:PrincipalAccount}" } } }, { "Sid": "ModelPackageAccess", "Effect": "Allow", "Action": [ "sagemaker:CreateModelPackage", "sagemaker:DescribeModelPackage", "sagemaker:ListModelPackages", "sagemaker:CreateModelPackageGroup", "sagemaker:DescribeModelPackageGroup", "sagemaker:ListModelPackageGroups", "sagemaker:CreateModel" ], "Resource": [ "arn:aws:sagemaker:*:*:model-package-group/*", "arn:aws:sagemaker:*:*:model-package/*", "arn:aws:sagemaker:*:*:model/*" ], "Condition": { "StringEquals": { "aws:ResourceAccount": "${aws:PrincipalAccount}" } } }, { "Sid": "TagsPermission", "Effect": "Allow", "Action": [ "sagemaker:AddTags", "sagemaker:ListTags" ], "Resource": [ "arn:aws:sagemaker:*:*:model-package-group/*", "arn:aws:sagemaker:*:*:model-package/*", "arn:aws:sagemaker:*:*:hub/*", "arn:aws:sagemaker:*:*:hub-content/*", "arn:aws:sagemaker:*:*:training-job/*", "arn:aws:sagemaker:*:*:model/*", "arn:aws:sagemaker:*:*:endpoint/*", "arn:aws:sagemaker:*:*:endpoint-config/*", "arn:aws:sagemaker:*:*:pipeline/*", "arn:aws:sagemaker:*:*:inference-component/*", "arn:aws:sagemaker:*:*:action/*" ], "Condition": { "StringEquals": { "aws:ResourceAccount": "${aws:PrincipalAccount}" } } }, { "Sid": "LogAccess", "Effect": "Allow", "Action": [ "logs:DescribeLogGroups", "logs:DescribeLogStreams", "logs:GetLogEvents" ], "Resource": [ "arn:aws:logs:*:*:log-group*", "arn:aws:logs:*:*:log-group:/aws/sagemaker/TrainingJobs:log-stream:*" ], "Condition": { "StringEquals": { "aws:ResourceAccount": "${aws:PrincipalAccount}" } } }, { "Sid": "BedrockDeploy", "Effect": "Allow", "Action": [ "bedrock:CreateModelImportJob" ], "Resource": [ "arn:aws:bedrock:*:*:*" ], "Condition": { "StringEquals": { "aws:ResourceAccount": "${aws:PrincipalAccount}" } } }, { "Sid": "BedrockOperations", "Effect": "Allow", "Action": [ "bedrock:GetModelImportJob", "bedrock:GetImportedModel", "bedrock:ListProvisionedModelThroughputs", "bedrock:ListCustomModelDeployments", "bedrock:ListCustomModels", "bedrock:ListModelImportJobs", "bedrock:GetEvaluationJob", "bedrock:CreateEvaluationJob", "bedrock:InvokeModel" ], "Resource": [ "arn:aws:bedrock:*:*:evaluation-job/*", "arn:aws:bedrock:*:*:imported-model/*", "arn:aws:bedrock:*:*:model-import-job/*", "arn:aws:bedrock:*:*:foundation-model/*" ], "Condition": { "StringEquals": { "aws:ResourceAccount": "${aws:PrincipalAccount}" } } }, { "Sid": "BedrockFoundationModelOperations", "Effect": "Allow", "Action": [ "bedrock:GetFoundationModelAvailability", "bedrock:ListFoundationModels" ], "Resource": [ "*" ] }, { "Sid": "SageMakerPipelinesAndLineage", "Effect": "Allow", "Action": [ "sagemaker:ListActions", "sagemaker:ListArtifacts", "sagemaker:QueryLineage", "sagemaker:ListAssociations", "sagemaker:AddAssociation", "sagemaker:DescribeAction", "sagemaker:AddAssociation", "sagemaker:CreateAction", "sagemaker:CreateContext", "sagemaker:DescribeTrialComponent" ], "Resource": [ "arn:aws:sagemaker:*:*:artifact/*", "arn:aws:sagemaker:*:*:action/*", "arn:aws:sagemaker:*:*:context/*", "arn:aws:sagemaker:*:*:action/*", "arn:aws:sagemaker:*:*:model-package/*", "arn:aws:sagemaker:*:*:context/*", "arn:aws:sagemaker:*:*:pipeline/*", "arn:aws:sagemaker:*:*:experiment-trial-component/*" ], "Condition": { "StringEquals": { "aws:ResourceAccount": "${aws:PrincipalAccount}" } } }, { "Sid": "ListOperations", "Effect": "Allow", "Action": [ "sagemaker:ListInferenceComponents", "sagemaker:ListWorkforces" ], "Resource": [ "*" ], "Condition": { "StringEquals": { "aws:ResourceAccount": "${aws:PrincipalAccount}" } } }, { "Sid": "SageMakerInference", "Effect": "Allow", "Action": [ "sagemaker:DescribeInferenceComponent", "sagemaker:CreateEndpoint", "sagemaker:CreateEndpointConfig", "sagemaker:DescribeEndpoint", "sagemaker:DescribeEndpointConfig", "sagemaker:ListEndpoints" ], "Resource": [ "arn:aws:sagemaker:*:*:inference-component/*", "arn:aws:sagemaker:*:*:endpoint/*", "arn:aws:sagemaker:*:*:endpoint-config/*" ], "Condition": { "StringEquals": { "aws:ResourceAccount": "${aws:PrincipalAccount}" } } }, { "Sid": "SageMakerPipelines", "Effect": "Allow", "Action": [ "sagemaker:DescribePipelineExecution", "sagemaker:ListPipelineExecutions", "sagemaker:ListPipelineExecutionSteps", "sagemaker:CreatePipeline", "sagemaker:UpdatePipeline", "sagemaker:StartPipelineExecution" ], "Resource": [ "arn:aws:sagemaker:*:*:pipeline/*" ], "Condition": { "StringEquals": { "aws:ResourceAccount": "${aws:PrincipalAccount}" } } } ] } ``` Jika Anda telah melampirkan [AmazonSageMakerFullAccessPolicy](https://docs.aws.amazon.com/aws-managed-policy/latest/reference/AmazonSageMakerFullAccess.html)ke peran eksekusi, Anda dapat menambahkan kebijakan yang dikurangi ini: ``` { "Version": "2012-10-17", "Statement": [ { "Sid": "LambdaListPermissions", "Effect": "Allow", "Action": [ "lambda:ListFunctions" ], "Resource": [ "*" ] }, { "Sid": "LambdaPermissionsForRewardFunction", "Effect": "Allow", "Action": [ "lambda:CreateFunction", "lambda:DeleteFunction", "lambda:InvokeFunction", "lambda:GetFunction" ], "Resource": [ "arn:aws:lambda:*:*:function:*SageMaker*", "arn:aws:lambda:*:*:function:*sagemaker*", "arn:aws:lambda:*:*:function:*Sagemaker*" ], "Condition": { "StringEquals": { "aws:ResourceAccount": "${aws:PrincipalAccount}" } } }, { "Sid": "LambdaLayerForAWSSDK", "Effect": "Allow", "Action": [ "lambda:GetLayerVersion" ], "Resource": [ "arn:aws:lambda:*:336392948345:layer:AWSSDK*" ] }, { "Sid": "S3Access", "Effect": "Allow", "Action": [ "s3:GetObject", "s3:PutObject" ], "Resource": [ "arn:aws:s3:::*SageMaker*", "arn:aws:s3:::*Sagemaker*", "arn:aws:s3:::*sagemaker*", "arn:aws:s3:::jumpstart*" ] }, { "Sid": "PassRoleForSageMakerAndLambdaAndBedrock", "Effect": "Allow", "Action": [ "iam:PassRole" ], "Resource": [ "arn:aws:iam::*:role/service-role/AmazonSageMaker-ExecutionRole-*" ], "Condition": { "StringEquals": { "iam:PassedToService": [ "lambda.amazonaws.com", "bedrock.amazonaws.com" ], "aws:ResourceAccount": "${aws:PrincipalAccount}" } } }, { "Sid": "BedrockDeploy", "Effect": "Allow", "Action": [ "bedrock:CreateModelImportJob" ], "Resource": [ "*" ], "Condition": { "StringEquals": { "aws:ResourceAccount": "${aws:PrincipalAccount}" } } }, { "Sid": "BedrockOperations", "Effect": "Allow", "Action": [ "bedrock:GetModelImportJob", "bedrock:GetImportedModel", "bedrock:ListProvisionedModelThroughputs", "bedrock:ListCustomModelDeployments", "bedrock:ListCustomModels", "bedrock:ListModelImportJobs", "bedrock:GetEvaluationJob", "bedrock:CreateEvaluationJob", "bedrock:InvokeModel" ], "Resource": [ "arn:aws:bedrock:*:*:evaluation-job/*", "arn:aws:bedrock:*:*:imported-model/*", "arn:aws:bedrock:*:*:model-import-job/*", "arn:aws:bedrock:*:*:foundation-model/*" ], "Condition": { "StringEquals": { "aws:ResourceAccount": "${aws:PrincipalAccount}" } } }, { "Sid": "BedrockFoundationModelOperations", "Effect": "Allow", "Action": [ "bedrock:GetFoundationModelAvailability", "bedrock:ListFoundationModels" ], "Resource": [ "*" ] } ] } ``` Anda kemudian harus mengklik **Edit Kebijakan Kepercayaan** dan menggantinya dengan kebijakan berikut, lalu klik **Perbarui Kebijakan**. ``` { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": { "Service": "lambda.amazonaws.com" }, "Action": "sts:AssumeRole" }, { "Effect": "Allow", "Principal": { "Service": "sagemaker.amazonaws.com" }, "Action": "sts:AssumeRole" }, { "Effect": "Allow", "Principal": { "Service": "bedrock.amazonaws.com" }, "Action": "sts:AssumeRole" } ] } ``` # Membuat aset untuk kustomisasi model di UI Anda dapat membuat dan mengelola kumpulan data dan aset evaluator yang dapat Anda gunakan untuk penyesuaian model di UI. ## Aset Pilih **Aset** di panel sebelah kiri dan Amazon SageMaker Studio UI lalu pilih **Datasets**. ![\[Gambar yang berisi akses ke kustomisasi model.\]](http://docs.aws.amazon.com/id_id/sagemaker/latest/dg/images/screenshot-open-model-16.png) Pilih **Unggah Dataset** untuk menambahkan kumpulan data yang akan Anda gunakan dalam pekerjaan penyesuaian model Anda. Dengan memilih **format input data yang diperlukan**, Anda dapat mengakses referensi format kumpulan data yang akan digunakan. ![\[Gambar yang berisi akses ke kustomisasi model.\]](http://docs.aws.amazon.com/id_id/sagemaker/latest/dg/images/screenshot-open-model-15.png) ## Evaluator Anda juga dapat menambahkan **Fungsi Hadiah** dan **Prompt Hadiah** untuk pekerjaan kustomisasi Pembelajaran Penguatan Anda. ![\[Gambar yang berisi akses ke kustomisasi model.\]](http://docs.aws.amazon.com/id_id/sagemaker/latest/dg/images/screenshot-open-model-14.png) UI juga memberikan panduan tentang format yang diperlukan untuk fungsi hadiah atau prompt hadiah. ![\[Gambar yang berisi akses ke kustomisasi model.\]](http://docs.aws.amazon.com/id_id/sagemaker/latest/dg/images/screenshot-open-model-13.png) ## Aset untuk kustomisasi model menggunakan AWS SDK Anda juga dapat menggunakan SageMaker AI Python SDK untuk membuat aset. Lihat contoh potongan kode di bawah ini: ``` from sagemaker.ai_registry.air_constants import REWARD_FUNCTION, REWARD_PROMPT from sagemaker.ai_registry.dataset import DataSet, CustomizationTechnique from sagemaker.ai_registry.evaluator import Evaluator # Creating a dataset example dataset = DataSet.create( name="sdkv3-gen-ds2", source="s3://sample-test-bucket/datasets/training-data/jamjee-sft-ds1.jsonl", # or use local filepath as source. customization_technique=CustomizationTechnique.SFT ) # Refreshes status from hub dataset.refresh() pprint(dataset.__dict__) # Creating an evaluator. Method : Lambda evaluator = Evaluator.create( name = "sdk-new-rf11", source="arn:aws:lambda:us-west-2:<>:function:8", type=REWARD_FUNCTION ) # Creating an evaluator. Method : Bring your own code evaluator = Evaluator.create( name = "eval-lambda-test", source="/path_to_local/eval_lambda_1.py", type = REWARD_FUNCTION ) # Optional wait, by default we have wait = True during create call. evaluator.wait() evaluator.refresh() pprint(evaluator) ``` # Pengajuan pekerjaan kustomisasi model AI Kemampuan kustomisasi model SageMaker AI dapat diakses dari halaman Model Amazon SageMaker Studio di panel sebelah kiri. Anda juga dapat menemukan halaman Aset tempat Anda dapat membuat dan mengelola Dataset dan Evaluator kustomisasi model Anda. ![\[Gambar yang berisi akses ke kustomisasi model.\]](http://docs.aws.amazon.com/id_id/sagemaker/latest/dg/images/screenshot-open-model-12.png) Untuk memulai pengiriman pekerjaan penyesuaian model, pilih opsi Model untuk mengakses tab Model Dasar Jumpstart: ![\[Gambar yang berisi cara memilih model dasar.\]](http://docs.aws.amazon.com/id_id/sagemaker/latest/dg/images/screenshot-open-model-11.png) Anda dapat langsung mengklik Sesuaikan model di kartu model atau Anda dapat mencari model apa pun dari Meta yang Anda minati untuk disesuaikan. ![\[Gambar yang berisi kartu model dan cara memilih model untuk disesuaikan.\]](http://docs.aws.amazon.com/id_id/sagemaker/latest/dg/images/screenshot-open-model-10.png) Setelah mengklik kartu model, Anda dapat mengakses halaman detail model dan meluncurkan tugas penyesuaian dengan mengklik Sesuaikan model dan kemudian memilih Sesuaikan dengan UI untuk memulai mengonfigurasi pekerjaan RLVR Anda. ![\[Gambar yang berisi cara meluncurkan pekerjaan kustomisasi.\]](http://docs.aws.amazon.com/id_id/sagemaker/latest/dg/images/screenshot-open-model-9.png) Anda kemudian dapat memasukkan nama model kustom Anda, memilih teknik kustomisasi model yang akan digunakan dan mengkonfigurasi hyperparameters pekerjaan Anda: ![\[Gambar yang berisi pemilihan teknik kustomisasi model.\]](http://docs.aws.amazon.com/id_id/sagemaker/latest/dg/images/screenshot-open-model-8.png) ![\[Gambar yang berisi pemilihan teknik kustomisasi model.\]](http://docs.aws.amazon.com/id_id/sagemaker/latest/dg/images/screenshot-open-model-7.png) ## Pengajuan pekerjaan kustomisasi model AI menggunakan SDK Anda juga dapat menggunakan SageMaker AI Python SDK untuk mengirimkan pekerjaan penyesuaian model: ``` # Submit a DPO model customization job from sagemaker.modules.train.dpo_trainer import DPOTrainer from sagemaker.modules.train.common import TrainingType trainer = DPOTrainer( model=BASE_MODEL, training_type=TrainingType.LORA, model_package_group_name=MODEL_PACKAGE_GROUP_NAME, training_dataset=TRAINING_DATASET, s3_output_path=S3_OUTPUT_PATH, sagemaker_session=sagemaker_session, role=ROLE_ARN ) ``` ## Memantau pekerjaan kustomisasi Anda Segera setelah mengirimkan pekerjaan Anda, Anda akan diarahkan kembali ke halaman pekerjaan pelatihan penyesuaian model Anda. ![\[Gambar yang berisi pemilihan teknik kustomisasi model.\]](http://docs.aws.amazon.com/id_id/sagemaker/latest/dg/images/screenshot-open-model-6.png) Setelah pekerjaan selesai, Anda dapat pergi ke halaman detail model kustom Anda dengan mengklik Buka **Model Kustom** tombol di sudut kanan atas. ![\[Gambar yang berisi pemilihan teknik kustomisasi model.\]](http://docs.aws.amazon.com/id_id/sagemaker/latest/dg/images/screenshot-open-model-5.png) Di halaman detail model kustom Anda dapat bekerja lebih lanjut dengan model kustom Anda dengan: 1. Memeriksa informasi tentang kinerja, lokasi artefak yang dihasilkan, konfigurasi pelatihan hiper-parameter dan log pelatihan. 1. Luncurkan pekerjaan evaluasi dengan kumpulan data yang berbeda (Kustomisasi lanjutan). 1. Terapkan model menggunakan titik akhir Inferensi SageMaker AI atau Impor Model Kustom Amazon Bedrock. ![\[Gambar yang berisi pemilihan teknik kustomisasi model.\]](http://docs.aws.amazon.com/id_id/sagemaker/latest/dg/images/screenshot-open-model-4.png) # Pengajuan pekerjaan evaluasi model Bagian ini menjelaskan evaluasi model kustom berat terbuka. Ini membantu Anda memulai melalui proses pengajuan pekerjaan evaluasi. Sumber daya tambahan disediakan untuk kasus penggunaan pengajuan pekerjaan evaluasi yang lebih maju. **Topics** + [Memulai](model-customize-evaluation-getting-started.md) + [Jenis evaluasi dan Job Submission](model-customize-evaluation-types.md) + [Format Metrik Evaluasi](model-customize-evaluation-metrics-formats.md) + [Format Set Data yang Didukung untuk Tugas Bring-Your-Own-Dataset (BYOD)](model-customize-evaluation-dataset-formats.md) + [Evaluasi dengan Preset dan Custom Scorers](model-customize-evaluation-preset-custom-scorers.md) # Memulai ## Kirim Job Evaluasi Melalui SageMaker Studio ### Langkah 1: Arahkan ke Evaluasi Dari Kartu Model Anda Setelah Anda menyesuaikan model Anda, navigasikan ke halaman evaluasi dari kartu model Anda. Untuk informasi tentang pelatihan model kustom berat terbuka: [https://docs.aws.amazon.com/sagemaker/latest/dg/model- .html customize-open-weight-job](https://docs.aws.amazon.com/sagemaker/latest/dg/model-customize-open-weight-job.html) SageMaker memvisualisasikan model kustom Anda pada tab My Models: ![\[Halaman kartu model terdaftar\]](http://docs.aws.amazon.com/id_id/sagemaker/latest/dg/images/getting-started-registered-model-card.png) Pilih Lihat versi terbaru, lalu pilih Evaluasi: ![\[Halaman kustomisasi model\]](http://docs.aws.amazon.com/id_id/sagemaker/latest/dg/images/getting-started-evaluate-from-model-card.png) ### Langkah 2: Kirim Job Evaluasi Anda Pilih tombol Kirim dan kirimkan pekerjaan evaluasi Anda. Ini mengirimkan pekerjaan benchmark MMLU minimal. Untuk informasi tentang jenis pekerjaan evaluasi yang didukung, lihat[Jenis evaluasi dan Job Submission](model-customize-evaluation-types.md). ![\[Halaman pengajuan pekerjaan evaluasi\]](http://docs.aws.amazon.com/id_id/sagemaker/latest/dg/images/getting-started-benchmark-submission.png) ### Langkah 3: Lacak Kemajuan Pekerjaan Evaluasi Anda Progres pekerjaan evaluasi Anda dilacak di tab Langkah evaluasi: ![\[Kemajuan pekerjaan evaluasi Anda\]](http://docs.aws.amazon.com/id_id/sagemaker/latest/dg/images/getting-started-benchmark-tracking.png) ### Langkah 4: Lihat Hasil Job Evaluasi Anda Hasil pekerjaan evaluasi Anda divisualisasikan di tab Hasil evaluasi: ![\[Metrik pekerjaan evaluasi Anda\]](http://docs.aws.amazon.com/id_id/sagemaker/latest/dg/images/getting-started-benchmark-results.png) ### Langkah 5: Lihat Evaluasi Selesai Anda Pekerjaan evaluasi Anda yang telah selesai ditampilkan di Evaluasi kartu model Anda: ![\[Pekerjaan evaluasi Anda yang telah selesai\]](http://docs.aws.amazon.com/id_id/sagemaker/latest/dg/images/getting-started-benchmark-completed-model-card.png) ## Kirim Job Evaluasi Anda Melalui SageMaker Python SDK ### Langkah 1: Buat Anda BenchMarkEvaluator Lulus model terlatih terdaftar Anda, lokasi keluaran AWS S3, dan MLFlow sumber daya ARN `BenchMarkEvaluator` ke lalu inisialisasi. ``` from sagemaker.train.evaluate import BenchMarkEvaluator, Benchmark evaluator = BenchMarkEvaluator( benchmark=Benchmark.MMLU, model="arn:aws:sagemaker:::model-package//", s3_output_path="s3:////eval/", mlflow_resource_arn="arn:aws:sagemaker:::mlflow-tracking-server/", evaluate_base_model=False ) ``` ### Langkah 2: Kirim Job Evaluasi Anda Panggil `evaluate()` metode untuk mengirimkan pekerjaan evaluasi. ``` execution = evaluator.evaluate() ``` ### Langkah 3: Lacak Kemajuan Pekerjaan Evaluasi Anda Hubungi `wait()` metode eksekusi untuk mendapatkan pembaruan langsung dari kemajuan pekerjaan evaluasi. ``` execution.wait(target_status="Succeeded", poll=5, timeout=3600) ``` ### Langkah 4: Lihat Hasil Job Evaluasi Anda Hubungi `show_results()` metode untuk menampilkan hasil pekerjaan evaluasi Anda. ``` execution.show_results() ``` # Jenis evaluasi dan Job Submission ## Benchmarking dengan dataset standar Gunakan tipe Evaluasi Benchmark untuk mengevaluasi kualitas model Anda di seluruh kumpulan data benchmark standar termasuk kumpulan data populer seperti MMLU dan BBH. | Tolok Ukur | Dataset Kustom Didukung | Modalitas | Deskripsi | Metrik-metrik | Strategi | Subtugas tersedia | | --- | --- | --- | --- | --- | --- | --- | | mmlu | Tidak | Teks | Pemahaman Bahasa Multi-tugas - Menguji pengetahuan di 57 mata pelajaran. | ketepatan | zs\$1cot | Ya | | mmlu\$1pro | Tidak | Teks | MMLU - Subset Profesional - Berfokus pada domain profesional seperti hukum, kedokteran, akuntansi, dan teknik. | ketepatan | zs\$1cot | Tidak | | bbh | Tidak | Teks | Tugas Penalaran Lanjutan - Kumpulan masalah menantang yang menguji keterampilan kognitif dan pemecahan masalah tingkat tinggi. | ketepatan | fs\$1cot | Ya | | gpqa | Tidak | Teks | Penjawab Pertanyaan Fisika Umum — Menilai pemahaman konsep fisika dan kemampuan pemecahan masalah terkait. | ketepatan | zs\$1cot | Tidak | | matematika | Tidak | Teks | Pemecahan Masalah Matematika — Mengukur penalaran matematis di seluruh topik termasuk aljabar, kalkulus, dan masalah kata. | exact\$1match | zs\$1cot | Ya | | strong\$1tolak | Tidak | Teks | Quality-Control Task — Menguji kemampuan model untuk mendeteksi dan menolak konten yang tidak pantas, berbahaya, atau salah. | defleksi | zs | Ya | | ifeval | Tidak | Teks | Instruksi-Mengikuti Evaluasi - Mengukur seberapa akurat model mengikuti instruksi yang diberikan dan menyelesaikan tugas untuk spesifikasi. | ketepatan | zs | Tidak | Untuk informasi selengkapnya tentang format BYOD, lihat[Format Set Data yang Didukung untuk Tugas Bring-Your-Own-Dataset (BYOD)](model-customize-evaluation-dataset-formats.md). ### Subtugas yang Tersedia Berikut daftar subtugas yang tersedia untuk evaluasi model di beberapa domain termasuk MMLU (Massive Multitask Language Understanding), BBH (Big Bench Hard), dan MATH. StrongReject Subtugas ini memungkinkan Anda menilai kinerja model Anda pada kemampuan dan bidang pengetahuan tertentu. **Subtugas MMLU** ``` MMLU_SUBTASKS = [ "abstract_algebra", "anatomy", "astronomy", "business_ethics", "clinical_knowledge", "college_biology", "college_chemistry", "college_computer_science", "college_mathematics", "college_medicine", "college_physics", "computer_security", "conceptual_physics", "econometrics", "electrical_engineering", "elementary_mathematics", "formal_logic", "global_facts", "high_school_biology", "high_school_chemistry", "high_school_computer_science", "high_school_european_history", "high_school_geography", "high_school_government_and_politics", "high_school_macroeconomics", "high_school_mathematics", "high_school_microeconomics", "high_school_physics", "high_school_psychology", "high_school_statistics", "high_school_us_history", "high_school_world_history", "human_aging", "human_sexuality", "international_law", "jurisprudence", "logical_fallacies", "machine_learning", "management", "marketing", "medical_genetics", "miscellaneous", "moral_disputes", "moral_scenarios", "nutrition", "philosophy", "prehistory", "professional_accounting", "professional_law", "professional_medicine", "professional_psychology", "public_relations", "security_studies", "sociology", "us_foreign_policy", "virology", "world_religions" ] ``` **Subtugas BBH** ``` BBH_SUBTASKS = [ "boolean_expressions", "causal_judgement", "date_understanding", "disambiguation_qa", "dyck_languages", "formal_fallacies", "geometric_shapes", "hyperbaton", "logical_deduction_five_objects", "logical_deduction_seven_objects", "logical_deduction_three_objects", "movie_recommendation", "multistep_arithmetic_two", "navigate", "object_counting", "penguins_in_a_table", "reasoning_about_colored_objects", "ruin_names", "salient_translation_error_detection", "snarks", "sports_understanding", "temporal_sequences", "tracking_shuffled_objects_five_objects", "tracking_shuffled_objects_seven_objects", "tracking_shuffled_objects_three_objects", "web_of_lies", "word_sorting" ] ``` **Subtugas Matematika** ``` MATH_SUBTASKS = [ "algebra", "counting_and_probability", "geometry", "intermediate_algebra", "number_theory", "prealgebra", "precalculus" ] ``` **StrongReject Subtugas** ``` STRONG_REJECT_SUBTASKS = [ "gcg_transfer_harmbench", "gcg_transfer_universal_attacks", "combination_3", "combination_2", "few_shot_json", "dev_mode_v2", "dev_mode_with_rant", "wikipedia_with_title", "distractors", "wikipedia", "style_injection_json", "style_injection_short", "refusal_suppression", "prefix_injection", "distractors_negated", "poems", "base64", "base64_raw", " base64_input_only", "base64_output_only", "evil_confidant", "aim", "rot_13", "disemvowel", "auto_obfuscation", "auto_payload_splitting", "pair", "pap_authority_endorsement", "pap_evidence_based_persuasion", "pap_expert_endorsement", "pap_logical_appeal", "pap_misrepresentation" ] ``` ### Kirimkan pekerjaan benchmark Anda ------ #### [ SageMaker Studio ] ![\[Konfigurasi minimal untuk benchmarking melalui Studio SageMaker\]](http://docs.aws.amazon.com/id_id/sagemaker/latest/dg/images/benchmark-submission-sagemaker-studio.png) ------ #### [ SageMaker Python SDK ] ``` from sagemaker.train.evaluate import get_benchmarks from sagemaker.train.evaluate import BenchMarkEvaluator Benchmark = get_benchmarks() # Create evaluator with MMLU benchmark evaluator = BenchMarkEvaluator( benchmark=Benchmark.MMLU, model="arn:aws:sagemaker:::model-package//", s3_output_path="s3:////", evaluate_base_model=False ) execution = evaluator.evaluate() ``` Untuk informasi lebih lanjut tentang pengajuan pekerjaan evaluasi melalui SageMaker Python SDK, lihat: [https://sagemaker.readthedocs.io/en/stable/model\$1customization/evaluation.html](https://sagemaker.readthedocs.io/en/stable/model_customization/evaluation.html) ------ ## Evaluasi Model Bahasa Besar sebagai Hakim (LLMAJ) Gunakan evaluasi LLM-as-a-judge (LLMAJ) untuk memanfaatkan model perbatasan lain untuk menilai respons model target Anda. Anda dapat menggunakan model AWS Bedrock sebagai juri dengan memanggil `create_evaluation_job` API untuk meluncurkan pekerjaan evaluasi. Untuk informasi selengkapnya tentang model juri yang didukung, lihat: [https://docs.aws.amazon.com/bedrock/latest/userguide/models-supported.html](https://docs.aws.amazon.com/bedrock/latest/userguide/models-supported.html) Anda dapat menggunakan 2 format metrik yang berbeda untuk menentukan evaluasi: + **Metrik bawaan:** Manfaatkan metrik bawaan AWS Bedrock untuk menganalisis kualitas respons inferensi model Anda. Untuk informasi lebih lanjut, lihat: [https://docs.aws.amazon.com/bedrock/latest/userguide/model- evaluation-type-judge-prompt .html](https://docs.aws.amazon.com/bedrock/latest/userguide/model-evaluation-type-judge-prompt.html) + **Metrik khusus:** Tentukan metrik kustom Anda sendiri dalam format metrik kustom Evaluasi Batuan Dasar untuk menganalisis kualitas respons inferensi model Anda menggunakan instruksi Anda sendiri. Untuk informasi lebih lanjut, lihat: [https://docs.aws.amazon.com/bedrock/latest/userguide/model- evaluation-custom-metrics-prompt -formats.html](https://docs.aws.amazon.com/bedrock/latest/userguide/model-evaluation-custom-metrics-prompt-formats.html) ### Kirim pekerjaan LLMAJ metrik bawaan ------ #### [ SageMaker Studio ] ![\[Konfigurasi minimal untuk benchmarking LLMAJ melalui Studio SageMaker\]](http://docs.aws.amazon.com/id_id/sagemaker/latest/dg/images/llmaj-as-judge-submission-sagemaker-studio.png) ------ #### [ SageMaker Python SDK ] ``` from sagemaker.train.evaluate import LLMAsJudgeEvaluator evaluator = LLMAsJudgeEvaluator( model="arn:aws:sagemaker:::model-package//", evaluator_model="", dataset="s3:////.jsonl", builtin_metrics=["", ""], s3_output_path="s3:////", evaluate_base_model=False ) execution = evaluator.evaluate() ``` Untuk informasi lebih lanjut tentang pengajuan pekerjaan evaluasi melalui SageMaker Python SDK, lihat: [https://sagemaker.readthedocs.io/en/stable/model\$1customization/evaluation.html](https://sagemaker.readthedocs.io/en/stable/model_customization/evaluation.html) ------ ### Kirim pekerjaan LLMAJ metrik khusus Tentukan metrik kustom Anda: ``` { "customMetricDefinition": { "name": "PositiveSentiment", "instructions": ( "You are an expert evaluator. Your task is to assess if the sentiment of the response is positive. " "Rate the response based on whether it conveys positive sentiment, helpfulness, and constructive tone.\n\n" "Consider the following:\n" "- Does the response have a positive, encouraging tone?\n" "- Is the response helpful and constructive?\n" "- Does it avoid negative language or criticism?\n\n" "Rate on this scale:\n" "- Good: Response has positive sentiment\n" "- Poor: Response lacks positive sentiment\n\n" "Here is the actual task:\n" "Prompt: {{prompt}}\n" "Response: {{prediction}}" ), "ratingScale": [ {"definition": "Good", "value": {"floatValue": 1}}, {"definition": "Poor", "value": {"floatValue": 0}} ] } } ``` Untuk informasi lebih lanjut, lihat: [https://docs.aws.amazon.com/bedrock/latest/userguide/model- evaluation-custom-metrics-prompt -formats.html](https://docs.aws.amazon.com/bedrock/latest/userguide/model-evaluation-custom-metrics-prompt-formats.html) ------ #### [ SageMaker Studio ] ![\[Unggah metrik kustom melalui Metrik kustom > Tambahkan metrik khusus\]](http://docs.aws.amazon.com/id_id/sagemaker/latest/dg/images/custom-llmaj-metrics-submission-sagemaker-studio.png) ------ #### [ SageMaker Python SDK ] ``` evaluator = LLMAsJudgeEvaluator( model="arn:aws:sagemaker:::model-package//", evaluator_model="", dataset="s3:////.jsonl", custom_metrics=custom_metric_dict = { "customMetricDefinition": { "name": "PositiveSentiment", "instructions": ( "You are an expert evaluator. Your task is to assess if the sentiment of the response is positive. " "Rate the response based on whether it conveys positive sentiment, helpfulness, and constructive tone.\n\n" "Consider the following:\n" "- Does the response have a positive, encouraging tone?\n" "- Is the response helpful and constructive?\n" "- Does it avoid negative language or criticism?\n\n" "Rate on this scale:\n" "- Good: Response has positive sentiment\n" "- Poor: Response lacks positive sentiment\n\n" "Here is the actual task:\n" "Prompt: {{prompt}}\n" "Response: {{prediction}}" ), "ratingScale": [ {"definition": "Good", "value": {"floatValue": 1}}, {"definition": "Poor", "value": {"floatValue": 0}} ] } }, s3_output_path="s3:////", evaluate_base_model=False ) ``` ------ ## Pencetak Skor Kustom Tentukan fungsi pencetak gol kustom Anda sendiri untuk meluncurkan pekerjaan evaluasi. Sistem ini menyediakan dua pencetak gol bawaan: Prime math dan Prime code. Anda juga dapat membawa fungsi pencetak gol Anda sendiri. Anda dapat menyalin kode fungsi pencetak gol Anda secara langsung atau membawa definisi fungsi Lambda Anda sendiri menggunakan ARN terkait. Secara default, kedua jenis pencetak gol menghasilkan hasil evaluasi yang mencakup metrik standar seperti skor F1, ROUGE, dan BLEU. Untuk informasi lebih lanjut tentang pencetak gol bawaan dan kustom serta persyaratan/kontrak masing-masing, lihat. [Evaluasi dengan Preset dan Custom Scorers](model-customize-evaluation-preset-custom-scorers.md) ### Daftarkan dataset Anda Bawa kumpulan data Anda sendiri untuk pencetak gol khusus dengan mendaftarkannya sebagai Kumpulan Data Konten SageMaker Hub. ------ #### [ SageMaker Studio ] Di Studio, unggah kumpulan data Anda menggunakan halaman Datasets khusus.. ![\[Dataset evaluasi terdaftar di Studio SageMaker\]](http://docs.aws.amazon.com/id_id/sagemaker/latest/dg/images/dataset-registration-sagemaker-studio.png) ------ #### [ SageMaker Python SDK ] Di SageMaker Python SDK, unggah dataset Anda menggunakan halaman Datasets khusus.. ``` from sagemaker.ai_registry.dataset import DataSet dataset = DataSet.create( name="your-bring-your-own-dataset", source="s3:////.jsonl" ) dataset.refresh() ``` ------ ### Kirim pekerjaan pencetak gol bawaan ------ #### [ SageMaker Studio ] ![\[Pilih dari eksekusi Kode atau jawaban Matematika untuk penilaian kustom Built-in\]](http://docs.aws.amazon.com/id_id/sagemaker/latest/dg/images/builtin-scorer-submission-sagemaker-studio.png) ------ #### [ SageMaker Python SDK ] ``` from sagemaker.train.evaluate import CustomScorerEvaluator from sagemaker.train.evaluate import get_builtin_metrics BuiltInMetric = get_builtin_metrics() evaluator_builtin = CustomScorerEvaluator( evaluator=BuiltInMetric.PRIME_MATH, dataset="arn:aws:sagemaker:::hub-content//DataSet/your-bring-your-own-dataset/", model="arn:aws:sagemaker:::model-package//", s3_output_path="s3:////", evaluate_base_model=False ) execution = evaluator.evaluate() ``` Pilih dari `BuiltInMetric.PRIME_MATH` atau `BuiltInMetric.PRIME_CODE` untuk Skor Bawaan. ------ ### Kirim pekerjaan pencetak gol khusus Tentukan fungsi hadiah khusus. Untuk informasi selengkapnya, lihat [Pencetak Skor Kustom (Bawa Metrik Anda Sendiri)](model-customize-evaluation-preset-custom-scorers.md#model-customize-evaluation-custom-scorers-byom). **Daftarkan fungsi hadiah khusus** ------ #### [ SageMaker Studio ] ![\[Menavigasi ke SageMaker Studio > Aset > Evaluator > Buat evaluator > Buat fungsi hadiah\]](http://docs.aws.amazon.com/id_id/sagemaker/latest/dg/images/custom-scorer-submission-sagemaker-studio.png) ![\[Kirimkan pekerjaan evaluasi Pencetak Skor Khusus yang mereferensikan fungsi hadiah preset terdaftar di Custom Scorer > Metrik khusus\]](http://docs.aws.amazon.com/id_id/sagemaker/latest/dg/images/custom-scorer-benchmark-submission-sagemaker-studio.png) ------ #### [ SageMaker Python SDK ] ``` from sagemaker.ai_registry.evaluator import Evaluator from sagemaker.ai_registry.air_constants import REWARD_FUNCTION evaluator = Evaluator.create( name = "your-reward-function-name", source="/path_to_local/custom_lambda_function.py", type = REWARD_FUNCTION ) ``` ``` evaluator = CustomScorerEvaluator( evaluator=evaluator, dataset="s3:////.jsonl", model="arn:aws:sagemaker:::model-package//", s3_output_path="s3:////", evaluate_base_model=False ) execution = evaluator.evaluate() ``` ------ # Format Metrik Evaluasi Mengevaluasi kualitas model Anda di seluruh format metrik ini: + Ringkasan Evaluasi Model + MLFlow + TensorBoard ## Ringkasan Evaluasi Model Saat Anda mengirimkan pekerjaan evaluasi Anda, Anda menentukan lokasi keluaran AWS S3. SageMaker secara otomatis mengunggah ringkasan evaluasi file.json ke lokasi. Jalur S3 ringkasan benchmark adalah sebagai berikut: ``` s3:////output/output//eval_results/ ``` **Lewati lokasi AWS S3** ------ #### [ SageMaker Studio ] ![\[Masuk ke lokasi artefak keluaran (URI AWS S3)\]](http://docs.aws.amazon.com/id_id/sagemaker/latest/dg/images/s3-output-path-submission-sagemaker-studio.png) ------ #### [ SageMaker Python SDK ] ``` evaluator = BenchMarkEvaluator( benchmark=Benchmark.MMLU, model="arn:aws:sagemaker:::model-package//", s3_output_path="s3:////eval/", evaluate_base_model=False ) execution = evaluator.evaluate() ``` ------ Baca langsung sebagai `.json` dari lokasi AWS S3 atau divisualisasikan secara otomatis di UI: ``` { "results": { "custom|gen_qa_gen_qa|0": { "rouge1": 0.9152812653966208, "rouge1_stderr": 0.003536439199232507, "rouge2": 0.774569918517409, "rouge2_stderr": 0.006368825746765958, "rougeL": 0.9111255645823356, "rougeL_stderr": 0.003603841524881021, "em": 0.6562150055991042, "em_stderr": 0.007948251702846893, "qem": 0.7522396416573348, "qem_stderr": 0.007224355240883467, "f1": 0.8428757602152095, "f1_stderr": 0.005186300690881584, "f1_score_quasi": 0.9156170336744968, "f1_score_quasi_stderr": 0.003667700152375464, "bleu": 100.00000000000004, "bleu_stderr": 1.464411857851008 }, "all": { "rouge1": 0.9152812653966208, "rouge1_stderr": 0.003536439199232507, "rouge2": 0.774569918517409, "rouge2_stderr": 0.006368825746765958, "rougeL": 0.9111255645823356, "rougeL_stderr": 0.003603841524881021, "em": 0.6562150055991042, "em_stderr": 0.007948251702846893, "qem": 0.7522396416573348, "qem_stderr": 0.007224355240883467, "f1": 0.8428757602152095, "f1_stderr": 0.005186300690881584, "f1_score_quasi": 0.9156170336744968, "f1_score_quasi_stderr": 0.003667700152375464, "bleu": 100.00000000000004, "bleu_stderr": 1.464411857851008 } } } ``` ![\[Contoh metrik kinerja untuk benchmark gen-qa kustom yang divisualisasikan di Studio SageMaker\]](http://docs.aws.amazon.com/id_id/sagemaker/latest/dg/images/gen-qa-metrics-visualization-sagemaker-studio.png) ## MLFlow penebangan **Berikan ARN SageMaker MLFlow sumber daya Anda** SageMaker Studio menggunakan MLFlow aplikasi default yang disediakan di setiap domain Studio saat Anda menggunakan kemampuan penyesuaian model untuk pertama kalinya. SageMaker Studio menggunakan ARN terkait MLflow aplikasi default dalam pengiriman pekerjaan evaluasi. Anda juga dapat mengirimkan pekerjaan evaluasi Anda dan secara eksplisit memberikan MLFlow ARN Sumber Daya untuk mengalirkan metrik ke pelacakan server/app terkait tersebut untuk analisis waktu nyata. **SageMaker SDK Python** ``` evaluator = BenchMarkEvaluator( benchmark=Benchmark.MMLU, model="arn:aws:sagemaker:::model-package//", s3_output_path="s3:////eval/", mlflow_resource_arn="arn:aws:sagemaker:::mlflow-tracking-server/", evaluate_base_model=False ) execution = evaluator.evaluate() ``` Visualisasi metrik tingkat model dan tingkat sistem: ![\[Kesalahan dan akurasi tingkat model sampel untuk tugas pembandingan MMLU\]](http://docs.aws.amazon.com/id_id/sagemaker/latest/dg/images/model-metrics-mlflow.png) ![\[Contoh metrik bawaan untuk tugas pembandingan LLMAJ\]](http://docs.aws.amazon.com/id_id/sagemaker/latest/dg/images/llmaj-metrics-mlflow.png) ![\[Contoh metrik tingkat sistem untuk tugas pembandingan MMLU\]](http://docs.aws.amazon.com/id_id/sagemaker/latest/dg/images/system-metrics-mlflow.png) ## TensorBoard Kirimkan pekerjaan evaluasi Anda dengan lokasi keluaran AWS S3. SageMaker secara otomatis mengunggah TensorBoard file ke lokasi. SageMaker mengunggah TensorBoard file ke AWS S3 di lokasi berikut: ``` s3:////output/output//tensorboard_results/eval/ ``` **Lewati lokasi AWS S3 sebagai berikut** ------ #### [ SageMaker Studio ] ![\[Masuk ke lokasi artefak keluaran (URI AWS S3)\]](http://docs.aws.amazon.com/id_id/sagemaker/latest/dg/images/s3-output-path-submission-sagemaker-studio.png) ------ #### [ SageMaker Python SDK ] ``` evaluator = BenchMarkEvaluator( benchmark=Benchmark.MMLU, model="arn:aws:sagemaker:::model-package//", s3_output_path="s3:////eval/", evaluate_base_model=False ) execution = evaluator.evaluate() ``` ------ **Contoh metrik tingkat model** ![\[SageMaker TensorBoard menampilkan hasil dari pekerjaan benchmarking\]](http://docs.aws.amazon.com/id_id/sagemaker/latest/dg/images/metrics-in-tensorboard.png) # Format Set Data yang Didukung untuk Tugas Bring-Your-Own-Dataset (BYOD) Custom Scorer dan jenis LLM-as-judge evaluasi memerlukan file JSONL dataset kustom yang terletak di S3. AWS Anda harus menyediakan file sebagai file JSON Lines mengikuti salah satu format yang didukung berikut. Contoh dalam dokumen ini diperluas untuk kejelasan. Setiap format memiliki nuansa tersendiri tetapi setidaknya semua membutuhkan prompt pengguna. **Bidang yang Wajib Diisi** | Bidang | Diperlukan | | --- | --- | | Prompt Pengguna | Ya | | Prompt Sistem | Tidak | | Kebenaran dasar | Hanya untuk Custom Scorer | | Kategori | Tidak | **1. Format OpenAI** ``` { "messages": [ { "role": "system", # System prompt (looks for system role) "content": "You are a helpful assistant." }, { "role": "user", # Query (looks for user role) "content": "Hello!" }, { "role": "assistant", # Ground truth (looks for assistant role) "content": "Hello to you!" } ] } ``` **2. SageMaker Evaluasi** ``` { "system":"You are an English major with top marks in class who likes to give minimal word responses: ", "query":"What is the symbol that ends the sentence as a question", "response":"?", # Ground truth "category": "Grammar" } ``` **3. HuggingFace Penyelesaian yang cepat** Format Standar dan Percakapan didukung. ``` # Standard { "prompt" : "What is the symbol that ends the sentence as a question", # Query "completion" : "?" # Ground truth } # Conversational { "prompt": [ { "role": "user", # Query (looks for user role) "content": "What is the symbol that ends the sentence as a question" } ], "completion": [ { "role": "assistant", # Ground truth (looks for assistant role) "content": "?" } ] } ``` **4. HuggingFace Preferensi** Support untuk format standar (string) dan format percakapan (array pesan). ``` # Standard: {"prompt": "text", "chosen": "text", "rejected": "text"} { "prompt" : "The sky is", # Query "chosen" : "blue", # Ground truth "rejected" : "green" } # Conversational: { "prompt": [ { "role": "user", # Query (looks for user role) "content": "What color is the sky?" } ], "chosen": [ { "role": "assistant", # Ground truth (looks for assistant role) "content": "It is blue." } ], "rejected": [ { "role": "assistant", "content": "It is green." } ] } ``` **5. Format Verl** Format Verl (baik format saat ini dan lama) didukung untuk kasus penggunaan pembelajaran penguatan. Verl docs untuk referensi: [https://verl.readthedocs.io/en/latest/preparation/prepare\$1data.html](https://verl.readthedocs.io/en/latest/preparation/prepare_data.html) Pengguna format VERL biasanya tidak memberikan respons kebenaran dasar. Jika Anda ingin menyediakannya, gunakan salah satu bidang `extra_info.answer` atau`reward_model.ground_truth`; `extra_info` diutamakan. SageMaker mempertahankan bidang khusus Verl berikut sebagai metadata jika ada: + `id` + `data_source` + `ability` + `reward_model` + `extra_info` + `attributes` + `difficulty` ``` # Newest VERL format where `prompt` is an array of messages. { "data_source": "openai/gsm8k", "prompt": [ { "content": "You are a helpful math tutor who explains solutions to questions step-by-step.", "role": "system" }, { "content": "Natalia sold clips to 48 of her friends in April, and then she sold half as many clips in May. How many clips did Natalia sell altogether in April and May? Let's think step by step and output the final answer after \"####\".", "role": "user" } ], "ability": "math", "extra_info": { "answer": "Natalia sold 48/2 = <<48/2=24>>24 clips in May.\nNatalia sold 48+24 = <<48+24=72>>72 clips altogether in April and May.\n#### 72", "index": 0, "question": "Natalia sold clips to 48 of her friends in April, and then she sold half as many clips in May. How many clips did Natalia sell altogether in April and May?", "split": "train" }, "reward_model": { "ground_truth": "72" # Ignored in favor of extra_info.answer } } # Legacy VERL format where `prompt` is a string. Also supported. { "data_source": "openai/gsm8k", "prompt": "Natalia sold clips to 48 of her friends in April, and then she sold half as many clips in May. How many clips did Natalia sell altogether in April and May? Let's think step by step and output the final answer after \"####\".", "extra_info": { "answer": "Natalia sold 48/2 = <<48/2=24>>24 clips in May.\nNatalia sold 48+24 = <<48+24=72>>72 clips altogether in April and May.\n#### 72", "index": 0, "question": "Natalia sold clips to 48 of her friends in April, and then she sold half as many clips in May. How many clips did Natalia sell altogether in April and May?", "split": "train" } } ``` # Evaluasi dengan Preset dan Custom Scorers Saat menggunakan jenis evaluasi Pencetak Skor Kustom, SageMaker Evaluasi mendukung dua pencetak skor bawaan (juga disebut sebagai “fungsi hadiah”) Prime Math dan Prime Code yang diambil dari perpustakaan pelatihan [Volcengine/verl RL](https://github.com/volcengine/verl), atau pencetak gol kustom Anda sendiri yang diimplementasikan sebagai Fungsi Lambda. ## Pencetak Skor Bawaan **Matematika Utama** Pencetak gol matematika utama mengharapkan kumpulan data JSONL khusus dari entri yang berisi pertanyaan matematika sebagai prompt/query dan jawaban yang benar sebagai kebenaran dasar. Dataset dapat berupa salah satu format yang didukung yang disebutkan dalam[Format Set Data yang Didukung untuk Tugas Bring-Your-Own-Dataset (BYOD)](model-customize-evaluation-dataset-formats.md). Contoh entri kumpulan data (diperluas untuk kejelasan): ``` { "system":"You are a math expert: ", "query":"How many vertical asymptotes does the graph of $y=\\frac{2}{x^2+x-6}$ have?", "response":"2" # Ground truth aka correct answer } ``` **Kode Perdana** Pencetak kode utama mengharapkan kumpulan data JSONL khusus dari entri yang berisi masalah pengkodean dan kasus uji yang ditentukan di lapangan. `metadata` Struktur kasus uji dengan nama fungsi yang diharapkan untuk setiap entri, input sampel, dan output yang diharapkan. Contoh entri kumpulan data (diperluas untuk kejelasan): ``` { "system":"\\nWhen tackling complex reasoning tasks, you have access to the following actions. Use them as needed to progress through your thought process.\\n\\n[ASSESS]\\n\\n[ADVANCE]\\n\\n[VERIFY]\\n\\n[SIMPLIFY]\\n\\n[SYNTHESIZE]\\n\\n[PIVOT]\\n\\n[OUTPUT]\\n\\nYou should strictly follow the format below:\\n\\n[ACTION NAME]\\n\\n# Your action step 1\\n\\n# Your action step 2\\n\\n# Your action step 3\\n\\n...\\n\\nNext action: [NEXT ACTION NAME]\\n\\n", "query":"A number N is called a factorial number if it is the factorial of a positive integer. For example, the first few factorial numbers are 1, 2, 6, 24, 120,\\nGiven a number N, the task is to return the list/vector of the factorial numbers smaller than or equal to N.\\nExample 1:\\nInput: N = 3\\nOutput: 1 2\\nExplanation: The first factorial number is \\n1 which is less than equal to N. The second \\nnumber is 2 which is less than equal to N,\\nbut the third factorial number is 6 which \\nis greater than N. So we print only 1 and 2.\\nExample 2:\\nInput: N = 6\\nOutput: 1 2 6\\nExplanation: The first three factorial \\nnumbers are less than equal to N but \\nthe fourth factorial number 24 is \\ngreater than N. So we print only first \\nthree factorial numbers.\\nYour Task: \\nYou don't need to read input or print anything. Your task is to complete the function factorialNumbers() which takes an integer N as an input parameter and return the list/vector of the factorial numbers smaller than or equal to N.\\nExpected Time Complexity: O(K), Where K is the number of factorial numbers.\\nExpected Auxiliary Space: O(1)\\nConstraints:\\n1<=N<=10^{18}\\n\\nWrite Python code to solve the problem. Present the code in \\n```python\\nYour code\\n```\\nat the end.", "response": "", # Dummy string for ground truth. Provide a value if you want NLP metrics like ROUGE, BLEU, and F1. ### Define test cases in metadata field "metadata": { "fn_name": "factorialNumbers", "inputs": ["5"], "outputs": ["[1, 2]"] } } ``` ## Pencetak Skor Kustom (Bawa Metrik Anda Sendiri) Sesuaikan alur kerja evaluasi model Anda sepenuhnya dengan logika pasca-pemrosesan khusus yang memungkinkan Anda menghitung metrik khusus yang disesuaikan dengan kebutuhan Anda. Anda harus menerapkan pencetak gol khusus Anda sebagai fungsi AWS Lambda yang menerima respons model dan mengembalikan skor hadiah. ### Contoh Muatan Masukan Lambda AWS Lambda kustom Anda mengharapkan input dalam format OpenAI. Contoh: ``` { "id": "123", "messages": [ { "role": "user", "content": "Do you have a dedicated security team?" }, { "role": "assistant", "content": "As an AI developed by Amazon, I do not have a dedicated security team..." } ], "reference_answer": { "compliant": "No", "explanation": "As an AI developed by Company, I do not have a traditional security team..." } } ``` ### Contoh Muatan Keluaran Lambda Wadah SageMaker evaluasi mengharapkan respons Lambda Anda mengikuti format ini: ``` { "id": str, # Same id as input sample "aggregate_reward_score": float, # Overall score for the sample "metrics_list": [ # OPTIONAL: Component scores { "name": str, # Name of the component score "value": float, # Value of the component score "type": str # "Reward" or "Metric" } ] } ``` ### Definisi Lambda Kustom [Temukan contoh pencetak gol kustom yang diimplementasikan sepenuhnya dengan input sampel dan output yang diharapkan di: https://docs.aws.amazon.com/sagemaker/ latest/dg/nova - .html\$1 -example implementing-reward-functions nova-reward-llm-judge](https://docs.aws.amazon.com/sagemaker/latest/dg/nova-implementing-reward-functions.html#nova-reward-llm-judge-example) Gunakan kerangka berikut sebagai titik awal untuk fungsi Anda sendiri. ``` def lambda_handler(event, context): return lambda_grader(event) def lambda_grader(samples: list[dict]) -> list[dict]: """ Args: Samples: List of dictionaries in OpenAI format Example input: { "id": "123", "messages": [ { "role": "user", "content": "Do you have a dedicated security team?" }, { "role": "assistant", "content": "As an AI developed by Company, I do not have a dedicated security team..." } ], # This section is the same as your training dataset "reference_answer": { "compliant": "No", "explanation": "As an AI developed by Company, I do not have a traditional security team..." } } Returns: List of dictionaries with reward scores: { "id": str, # Same id as input sample "aggregate_reward_score": float, # Overall score for the sample "metrics_list": [ # OPTIONAL: Component scores { "name": str, # Name of the component score "value": float, # Value of the component score "type": str # "Reward" or "Metric" } ] } """ ``` ### Bidang input dan output **Bidang masukan** | Bidang | Deskripsi | Catatan tambahan | | --- | --- | --- | | id | Pengidentifikasi unik untuk sampel | Bergema kembali dalam output. Format string | | pesan | Riwayat obrolan yang dipesan dalam format OpenAI | Array objek pesan | | pesan [] .role | Pembicara pesan | Nilai umum: “pengguna”, “asisten”, “sistem” | | pesan [] .content | Isi teks pesan | Tali polos | | Metadata | Informasi bentuk bebas untuk membantu penilaian | Objek; bidang opsional yang dilewatkan dari data pelatihan | **Bidang keluaran** **Bidang Keluaran** | Bidang | Deskripsi | Catatan tambahan | | --- | --- | --- | | id | Pengidentifikasi yang sama dengan sampel input | Harus cocok dengan masukan | | aggregate\$1reward\$1score | Skor keseluruhan untuk sampel | Float (misalnya, 0,0—1,0 atau rentang yang ditentukan tugas) | | metrics\$1list | Skor komponen yang membentuk agregat | Array objek metrik | ### Izin yang Diperlukan Pastikan peran SageMaker eksekusi yang Anda gunakan untuk menjalankan evaluasi memiliki izin AWS Lambda. ``` { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "lambda:InvokeFunction" ], "Resource": "arn:aws:lambda:region:account-id:function:function-name" } ] } ``` Pastikan peran eksekusi Fungsi AWS Lambda Anda memiliki izin eksekusi Lambda dasar, serta izin tambahan yang mungkin Anda perlukan untuk panggilan hilir apa pun. AWS ``` { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "logs:CreateLogGroup", "logs:CreateLogStream", "logs:PutLogEvents" ], "Resource": "arn:aws:logs:*:*:*" } ] } ``` # Deployment model Dari halaman detail model kustom, Anda juga dapat menerapkan model kustom menggunakan titik akhir SageMaker AI Inference atau Amazon Bedrock. ![\[Gambar yang berisi pemilihan teknik kustomisasi model.\]](http://docs.aws.amazon.com/id_id/sagemaker/latest/dg/images/screenshot-open-model-1.png) # Contoh kumpulan data dan evaluator ## Penyetelan halus yang diawasi (SFT) + Nama: TAT-QA + Lisensi: CC-BY-4.0 + Tautan: [https://huggingface. co/datasets/next-tat/TAT-QA](https://huggingface.co/datasets/next-tat/TAT-QA) + Preprocessing - Pemformatan **Satu Sampel** ``` { "prompt": "Given a table and relevant text descriptions, answer the following question.\n\nTable:\n\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
20192018
$'000$'000
Revenue from external customers
Australia144,621129,431
New Zealand13,0368,912
Total157,657138,343
\n\nParagraphs:\n 4. SEGMENT INFORMATION\n\n During the 2019 and 2018 financial years, the Group operated wholly within one business segment being the operation and management of storage centres in Australia and New Zealand.\n\n The Managing Director is the Group\u2019s chief operating decision maker and monitors the operating results on a portfolio wide basis. Monthly management reports are evaluated based upon the overall performance of NSR consistent with the presentation within the consolidated financial statements. The Group\u2019s financing (including finance costs and finance income) are managed on a Group basis and not allocated to operating segments.\n\n The operating results presented in the statement of profit or loss represent the same segment information as reported in internal management information.\n\n The revenue information above excludes interest income and is based on the location of storage centres.\n\nQuestion: How many business segments were present in 2019 and 2018?\n\nAnswer:", "completion": "one", "data_idx": "2951" } ``` ## Optimalisasi Preferensi Langsung (DPO) + Nama: Ultrafeedback + Lisensi: MIT + Tautan: [https://huggingface. co/datasets/openbmb/UltraFeedback](https://huggingface.co/datasets/openbmb/UltraFeedback) + Langkah-langkah pra-pemrosesan: Memformat dan subsampling. **Satu Sampel** ``` { "source": "evol_instruct", "prompt": "Can you write a C++ program that prompts the user to enter the name of a country and checks if it borders the Mediterranean Sea? Here's some starter code to help you out:\n#include \n#include \nusing namespace std;\nint main() {\n string country;\n // prompt user for input\n cout << \"Enter the name of a country: \";\n cin >> country;\n // check if country borders the Mediterranean Sea\n // [C++ code]\n return 0;\n}", "chosen": "Here's a C++ program that prompts the user to enter the name of a country and checks if it borders the Mediterranean Sea:\n\n#include \n#include \n#include \n#include \n#include \n\nusing namespace std;\n\nint main() {\n // store countries and their bordering seas in a map\n map> countries;\n countries[\"Algeria\"] = {\"Mediterranean Sea\", \"North African Coast\"};\n countries[\"France\"] = {\"Mediterranean Sea\", \"English Channel\"};\n countries[\"Italy\"] = {\"Mediterranean Sea\", \"Adriatic Sea\"};\n countries[\"Malta\"] = {\"Mediterranean Sea\", \"North African Coast\"};\n countries[\"Spain\"] = {\"Mediterranean Sea\", \"Atlantic Ocean\"};\n countries[\"Tunisia\"] = {\"Mediterranean Sea\", \"North African Coast\"};\n\n string country;\n cout << \"Enter the name of a country: \";\n cin >> country;\n\n // check if country is in the map\n if (countries.find(country)!= countries.end()) {\n // check if country borders the Mediterranean Sea\n set borders = countries.at(country);\n if (borders.find(\"Mediterranean Sea\")!= borders.end()) {\n cout << country << \" borders the Mediterranean Sea.\" << endl;\n } else {\n cout << country << \" does not border the Mediterranean Sea.\" << endl;\n }\n } else {\n cout << country << \" is not a recognized country.\" << endl;\n }\n\n return 0;\n}\n\nIn this program, we first define a map called countries that stores each country's name as a key and a set of its surrounding seas as the value. We use a set to store each sea name so that we can easily check if a country borders a certain sea.\n\nWe then present the user with a prompt to enter the name of a country. We use a while loop to ensure that the user enters a valid country name. We check if the country is in the map, and if so, we retrieve its bordering seas from the map.\n\nFinally, we check if the Mediterranean Sea is among the set of borders. If it is, we inform the user that the country borders the Mediterranean Sea. If not, we inform the user that the country does not border the Mediterranean Sea.", "chosen-rating": 5.0, "chosen-model": "starchat", "rejected": "Sure, here is the program using the C++11 algorithm \"cds::algorithm::GreaterEqual\":\n#include \n#include \n#include \n#include \n#include \n\nusing namespace std;\n\nint main() {\n string country;\n cout << \"Enter the name of a country: \";\n cin >> country;\n std::vector vec;\n vec.push_back(country);\n size_t index = std::find_if(vec.begin(), vec.end(), [](const string& s) {\n return std::any_of(s.begin(), s.end(), [](const char& c) {\n return c == '}}';\n });\n if (index != vec.end()) {\n if ((*index)[0] == 'M') {\n cout << \"Country is bordered by the Mediterranean Sea.\" << endl;\n } else {\n cout << \"Country does not border the Mediterranean Sea.\" << endl;\n }\n } else {\n cout << \"Country is not found.\" << endl;\n }\n return 0;\n}", "rejected-rating": 1.25, "rejected-model": "pythia-12b" } ``` ## Pembelajaran Penguatan dari Umpan Balik AI (RLAIF) - Penilaian Berpasangan **Set Data Masukan** [Dataset sumber: https://github.com/WeOpenML/ PandaLM](https://github.com/WeOpenML/PandaLM) **Satu Sampel** ``` { "data_source": "WeOpenML/PandaLM", "prompt": [ { "role": "user", "content": "Below are two responses for a given task. The task is defined by the Instruction with an Input that provides further context. Evaluate the responses and generate a reference answer for the task.\n\n ### Instruction:\nCompare the given products.\n\n### Input:\niPhone 11 and Google Pixel 4\n\n ### Response 1:\nThe iPhone 11 has a larger screen size and a longer battery life than the Google Pixel 4.\n\n ### Response 2:\nThe iPhone 11 and Google Pixel 4 are both flagship smartphones released in 2018. The iPhone 11 has a 6.1-inch LCD display, while the Google Pixel 4 has a 5.7-inch OLED display. The iPhone 11 has an A13 Bionic chipset, while the Google Pixel 4 has a Qualcomm Snapdragon 845 chipset. The iPhone 11 has a dual-camera system, while the Google Pixel 4 has a single camera system. The iPhone 11 has a longer battery life, while the Google Pixel 4 has a faster processor.\n\n### Evaluation:\n" } ], "ability": "pairwise-judging", "reward_model": { "style": "llmj", "ground_truth": "2\n\n### Reason: Response 2 provides a more detailed and comprehensive comparison of the two products, including their specifications and features. Response 1 only mentions two aspects of the products and does not provide as much information.\n\n### Reference: The iPhone 11 and Google Pixel 4 are both flagship smartphones released in 2018. The iPhone 11 has a 6.1-inch LCD display, while the Google Pixel 4 has a 5.7-inch OLED display. The iPhone 11 has an A13 Bionic chipset, while the Google Pixel 4 has a Qualcomm Snapdragon 845 chipset. The iPhone 11 has a dual-camera system, while the Google Pixel 4 has a single camera system. The iPhone 11 has a longer battery life, while the Google Pixel 4 has a faster processor." }, "extra_info": { "split": "train", "index": 0, "raw_output_sequence": "2\n\n### Reason: Response 2 provides a more detailed and comprehensive comparison of the two products, including their specifications and features. Response 1 only mentions two aspects of the products and does not provide as much information.\n\n### Reference: The iPhone 11 and Google Pixel 4 are both flagship smartphones released in 2018. The iPhone 11 has a 6.1-inch LCD display, while the Google Pixel 4 has a 5.7-inch OLED display. The iPhone 11 has an A13 Bionic chipset, while the Google Pixel 4 has a Qualcomm Snapdragon 845 chipset. The iPhone 11 has a dual-camera system, while the Google Pixel 4 has a single camera system. The iPhone 11 has a longer battery life, while the Google Pixel 4 has a faster processor.\n", "llmj": { "question": "Below are two responses for a given task. The task is defined by the Instruction with an Input that provides further context. Evaluate the responses and generate a reference answer for the task.\n\n### Instruction:\nCompare the given products.\n\n### Input:\niPhone 11 and Google Pixel 4\n\n### Response 1:\nThe iPhone 11 has a larger screen size and a longer battery life than the Google Pixel 4.\n\n### Response 2:\nThe iPhone 11 and Google Pixel 4 are both flagship smartphones released in 2018. The iPhone 11 has a 6.1-inch LCD display, while the Google Pixel 4 has a 5.7-inch OLED display. The iPhone 11 has an A13 Bionic chipset, while the Google Pixel 4 has a Qualcomm Snapdragon 845 chipset. The iPhone 11 has a dual-camera system, while the Google Pixel 4 has a single camera system. The iPhone 11 has a longer battery life, while the Google Pixel 4 has a faster processor.\n\n### Evaluation:\n", "ground_truth": "2\n\n### Reason: Response 2 provides a more detailed and comprehensive comparison of the two products, including their specifications and features. Response 1 only mentions two aspects of the products and does not provide as much information.\n\n### Reference: The iPhone 11 and Google Pixel 4 are both flagship smartphones released in 2018. The iPhone 11 has a 6.1-inch LCD display, while the Google Pixel 4 has a 5.7-inch OLED display. The iPhone 11 has an A13 Bionic chipset, while the Google Pixel 4 has a Qualcomm Snapdragon 845 chipset. The iPhone 11 has a dual-camera system, while the Google Pixel 4 has a single camera system. The iPhone 11 has a longer battery life, while the Google Pixel 4 has a faster processor.", "document_in_context": null }, "sample_size": 1980 } } ``` ## RLAIF - Rantai Pemikiran **Masukan Dataset** Sumber Data: [https://huggingface. co/datasets/thesven/gsm8k-reasoning/tree/main/data](https://huggingface.co/datasets/thesven/gsm8k-reasoning/tree/main/data) **Satu Sampel** ``` { "data_source": "openai/gsm8k", "prompt": [ { "role": "system", "content": "\nYou are an AI assistant that uses a Chain of Thought (CoT) approach with reflection to answer queries.\n\nFollow these steps:\n- Think through the problem step by step within the ‹thinking> tags.\n- Reflect on your thinking\n\nto check for any errors or improvements within the ‹reflection› tags.\n- Make any necessary adjustments based on your reflection.\n- Provide your final, concise answer within the ‹output> tags.\n\nImportant: The and ‹reflection› sections are for your internal reasoning process only.\nDo not include any part of the final answer in these sections.\nThe actual response to the query must be entirely contained within the ‹output› tags.\n\nUse the following format for your response:\n\n[Your initial thought process goes here]\n\n[Your step-by-step reasoning goes here. This is your internal thought process, not the final answer. You can create as many reasoning steps as necessary in your process.]\n\n\n‹reflection>\n[Your reflection on your reasoning, checking for errors or improvements. You can create as many reflection steps as necessary in your process.]\n\n\n\n[Any adjustments to your thinking based on your reflection]\n\n\n\n[Your final, concise answer to the query. This is the only part that will be shown to the user.]\n\n" }, { "role": "user", "content": "A craft store makes a third of its sales in the fabric section, a quarter of its sales in the jewelry section, and the rest in the stationery section. They made 36 sales today. How many sales were in the stationery section?" } ], "ability": "chain-of-thought", "reward_model": { "style": "llmj-cot", "ground_truth": "Thus, there were 36 - 12 - 9 = <<36-12-9=15>>15 sales in the stationery section." }, "extra_info": { "split": "train", "index": 0, "question": "A craft store makes a third of its sales in the fabric section, a quarter of its sales in the jewelry section, and the rest in the stationery section. They made 36 sales today. How many sales were in the stationery section?", "short_answer": "15", "model_output": "\nTo find the number of sales in the stationery section, I need to first calculate the number of sales in the fabric and jewelry sections. The store makes a third of its sales in the fabric section, which is 1/3 of 36 sales. I can calculate this as 36 * 1/3 = 12 sales in the fabric section. The store makes a quarter of its sales in the jewelry section, which is 1/4 of 36 sales. I can calculate this as 36 * 1/4 = 9 sales in the jewelry section. Since the rest of the sales are in the stationery section, I can subtract the sales from the other two sections from the total number of sales to find the number of sales in the stationery section.\n\n\n\n1. Calculate the number of sales in the fabric section: 36 * 1/3 = 12\n2. Calculate the number of sales in the jewelry section: 36 * 1/4 = 9\n3. Calculate the number of sales in the stationery section: 36 - 12 - 9 = 15\n\n\n‹reflection>\nUpon reviewing my reasoning, I noticed that my calculations seem correct, but I should ensure that I understand the question correctly. The question asks for the number of sales in the stationery section, and I believe I have correctly calculated that. However, I should double-check my math to make sure it's accurate.\n\n\n\nUpon re-examining my math, I am confident that my calculations are correct. The fabric section makes up 1/3 of the sales, the jewelry section makes up 1/4, and the rest is the stationery section. My math checks out, and I am confident in my answer.\n\n\n\n15\n" } } ``` ## RLAIF - Faithfulnesss **Set Data Masukan** Sumber: [https://huggingface. co/datasets/rajpurkar/squad\$1v2/blob/main/squad\$1v2/train-00000-dari-00001.parket](https://huggingface.co/datasets/rajpurkar/squad_v2/blob/main/squad_v2/train-00000-of-00001.parquet) **Satu Sampel** ``` { "data_source": "squad_v2", "prompt": [ { "role": "system", "content": "You are a helpful assistant that answers questions based on the provided context. Only use information from the context." }, { "role": "user", "content": "Context: Beyoncé Giselle Knowles-Carter (/biːˈjɒnseɪ/ bee-YON-say) (born September 4, 1981) is an American singer, songwriter, record producer and actress. Born and raised in Houston, Texas, she performed in various singing and dancing competitions as a child, and rose to fame in the late 1990s as lead singer of R&B girl-group Destiny's Child. Managed by her father, Mathew Knowles, the group became one of the world's best-selling girl groups of all time. Their hiatus saw the release of Beyoncé's debut album, Dangerously in Love (2003), which established her as a solo artist worldwide, earned five Grammy Awards and featured the Billboard Hot 100 number-one singles \"Crazy in Love\" and \"Baby Boy\".\n\nQuestion: When did Beyonce start becoming popular?" } ], "ability": "faithfulness", "reward_model": { "style": "llmj-faithfulness", "ground_truth": "Beyoncé Giselle Knowles-Carter (/biːˈjɒnseɪ/ bee-YON-say) (born September 4, 1981) is an American singer, songwriter, record producer and actress. Born and raised in Houston, Texas, she performed in various singing and dancing competitions as a child, and rose to fame in the late 1990s as lead singer of R&B girl-group Destiny's Child. Managed by her father, Mathew Knowles, the group became one of the world's best-selling girl groups of all time. Their hiatus saw the release of Beyoncé's debut album, Dangerously in Love (2003), which established her as a solo artist worldwide, earned five Grammy Awards and featured the Billboard Hot 100 number-one singles \"Crazy in Love\" and \"Baby Boy\"." }, "extra_info": { "question": "When did Beyonce start becoming popular?", "split": "train", "index": 0 } } ``` ## RLAIF - Ringkasan **Set Data Masukan** [Sumber: Dataset gsm8k yang dibersihkan https://huggingface. co/datasets/thesven/gsm8k-reasoning/tree/main/data](https://huggingface.co/datasets/thesven/gsm8k-reasoning/tree/main/data) **Satu Sampel** ``` { "data_source": "cnn_dailymail", "prompt": [ { "role": "system", "content": "You are a helpful assistant that creates concise, accurate summaries of news articles. Focus on the key facts and main points." }, { "role": "user", "content": "Summarize the following article:\n\nLONDON, England (Reuters) -- Harry Potter star Daniel Radcliffe gains access to a reported £20 million ($41.1 million) fortune as he turns 18 on Monday, but he insists the money won't cast a spell on him. Daniel Radcliffe as Harry Potter in \"Harry Potter and the Order of the Phoenix\" To the disappointment of gossip columnists around the world, the young actor says he has no plans to fritter his cash away on fast cars, drink and celebrity parties. \"I don't plan to be one of those people who, as soon as they turn 18, suddenly buy themselves a massive sports car collection or something similar,\" he told an Australian interviewer earlier this month. \"I don't think I'll be particularly extravagant. \"The things I like buying are things that cost about 10 pounds -- books and CDs and DVDs.\" At 18, Radcliffe will be able to gamble in a casino, buy a drink in a pub or see the horror film \"Hostel: Part II,\" currently six places below his number one movie on the UK box office chart. Details of how he'll mark his landmark birthday are under wraps. His agent and publicist had no comment on his plans. \"I'll definitely have some sort of party,\" he said in an interview. \"Hopefully none of you will be reading about it.\" Radcliffe's earnings from the first five Potter films have been held in a trust fund which he has not been able to touch. Despite his growing fame and riches, the actor says he is keeping his feet firmly on the ground. \"People are always looking to say 'kid star goes off the rails,'\" he told reporters last month. \"But I try very hard not to go that way because it would be too easy for them.\" His latest outing as the boy wizard in \"Harry Potter and the Order of the Phoenix\" is breaking records on both sides of the Atlantic and he will reprise the role in the last two films. Watch I-Reporter give her review of Potter's latest » . There is life beyond Potter, however. The Londoner has filmed a TV movie called \"My Boy Jack,\" about author Rudyard Kipling and his son, due for release later this year. He will also appear in \"December Boys,\" an Australian film about four boys who escape an orphanage. Earlier this year, he made his stage debut playing a tortured teenager in Peter Shaffer's \"Equus.\" Meanwhile, he is braced for even closer media scrutiny now that he's legally an adult: \"I just think I'm going to be more sort of fair game,\" he told Reuters. E-mail to a friend . Copyright 2007 Reuters. All rights reserved.This material may not be published, broadcast, rewritten, or redistributed." } ], "ability": "summarization", "reward_model": { "style": "llmj-summarization", "ground_truth": "Harry Potter star Daniel Radcliffe gets £20M fortune as he turns 18 Monday .\nYoung actor says he has no plans to fritter his cash away .\nRadcliffe's earnings from first five Potter films have been held in trust fund ." }, "extra_info": { "question": "LONDON, England (Reuters) -- Harry Potter star Daniel Radcliffe gains access to a reported £20 million ($41.1 million) fortune as he turns 18 on Monday, but he insists the money won't cast a spell on him. Daniel Radcliffe as Harry Potter in \"Harry Potter and the Order of the Phoenix\" To the disappointment of gossip columnists around the world, the young actor says he has no plans to fritter his cash away on fast cars, drink and celebrity parties. \"I don't plan to be one of those people who, as soon as they turn 18, suddenly buy themselves a massive sports car collection or something similar,\" he told an Australian interviewer earlier this month. \"I don't think I'll be particularly extravagant. \"The things I like buying are things that cost about 10 pounds -- books and CDs and DVDs.\" At 18, Radcliffe will be able to gamble in a casino, buy a drink in a pub or see the horror film \"Hostel: Part II,\" currently six places below his number one movie on the UK box office chart. Details of how he'll mark his landmark birthday are under wraps. His agent and publicist had no comment on his plans. \"I'll definitely have some sort of party,\" he said in an interview. \"Hopefully none of you will be reading about it.\" Radcliffe's earnings from the first five Potter films have been held in a trust fund which he has not been able to touch. Despite his growing fame and riches, the actor says he is keeping his feet firmly on the ground. \"People are always looking to say 'kid star goes off the rails,'\" he told reporters last month. \"But I try very hard not to go that way because it would be too easy for them.\" His latest outing as the boy wizard in \"Harry Potter and the Order of the Phoenix\" is breaking records on both sides of the Atlantic and he will reprise the role in the last two films. Watch I-Reporter give her review of Potter's latest » . There is life beyond Potter, however. The Londoner has filmed a TV movie called \"My Boy Jack,\" about author Rudyard Kipling and his son, due for release later this year. He will also appear in \"December Boys,\" an Australian film about four boys who escape an orphanage. Earlier this year, he made his stage debut playing a tortured teenager in Peter Shaffer's \"Equus.\" Meanwhile, he is braced for even closer media scrutiny now that he's legally an adult: \"I just think I'm going to be more sort of fair game,\" he told Reuters. E-mail to a friend . Copyright 2007 Reuters. All rights reserved.This material may not be published, broadcast, rewritten, or redistributed.", "split": "train", "index": 0, "source_id": "42c027e4ff9730fbb3de84c1af0d2c50" } } ``` ## RLAIF - Prompt Kustom Dalam contoh ini, kita gunakan [RLAIF - Rantai Pemikiran](#model-customize-open-weight-samples-rlaif2) untuk membahas bagaimana prompt jinja kustom dapat menggantikan salah satu petunjuk yang telah ditetapkan. **Di bawah ini adalah contoh prompt khusus untuk CoT:** ``` You are an expert logical reasoning evaluator specializing in Chain-of-Thought (CoT) analysis. Given: A problem prompt and a model's reasoning-based response. Goal: Assess the quality and structure of logical reasoning, especially for specialized domains (law, medicine, finance, etc.). Scoring rubric (start at 0.0, then add or subtract): Core Components: Structural Completeness (0.3 max) - Clear problem statement: +0.05 - Defined variables/terminology: +0.05 - Organized given information: +0.05 - Explicit proof target: +0.05 - Step-by-step reasoning: +0.05 - Clear conclusion: +0.05 Logical Quality (0.4 max) - Valid logical flow: +0.1 - Proper use of if-then relationships: +0.1 - Correct application of domain principles: +0.1 - No logical fallacies: +0.1 Technical Accuracy (0.3 max) - Correct use of domain terminology: +0.1 - Accurate application of domain rules: +0.1 - Proper citation of relevant principles: +0.1 Critical Deductions: A. Invalid logical leap: -0.3 B. Missing critical steps: -0.2 C. Incorrect domain application: -0.2 D. Unclear/ambiguous reasoning: -0.1 Additional Instructions: - Verify domain-specific terminology and principles - Check for logical consistency throughout - Ensure conclusions follow from premises - Flag potential domain-specific compliance issues - Consider regulatory/professional standards where applicable Return EXACTLY this JSON (no extra text): { "score": , "component_scores": { "structural_completeness": , "logical_quality": , "technical_accuracy": }, "steps_present": { "problem_statement": , "variable_definitions": , "given_information": , "proof_target": , "step_reasoning": , "conclusion": }, "reasoning": "", "domain_flags": [""] } ### (Prompt field from dataset) Problem Prompt: {{ prompt }} Model's Response: {{ model_output }} ### Ground truth (if applicable): {{ ground_truth }} ``` ## Pembelajaran Penguatan dari Hadiah yang Dapat Diverifikasi (RLVR) - Pencocokan Tepat **Set Data Masukan** Sumber: [https://huggingface. co/datasets/openai/gsm8k](https://huggingface.co/datasets/openai/gsm8k) **Sampel** ``` { "data_source": "openai/gsm8k", "prompt": [ { "content": "Natalia sold clips to 48 of her friends in April, and then she sold half as many clips in May. How many clips did Natalia sell altogether in April and May? Let\'s think step by step and output the final answer after \\"####\\".", "role": "user" } ], "ability": "math", "reward_model": { "ground_truth": "72", "style": "rule" }, "extra_info": { "answer": "Natalia sold 48\\/2 = <<48\\/2=24>>24 clips in May.\\nNatalia sold 48+24 = <<48+24=72>>72 clips altogether in April and May.\\n#### 72", "index": 0, "question": "Natalia sold clips to 48 of her friends in April, and then she sold half as many clips in May. How many clips did Natalia sell altogether in April and May?", "split": "train" } } ``` ## RLVR - Eksekusi Kode **Set Data Masukan** Sumber: [https://huggingface. co/datasets/open-r1/codeforces](https://huggingface.co/datasets/open-r1/codeforces) **Sampel** ``` { "data_source": "codeforces", "prompt": [ { "content": "\nWhen tackling complex reasoning tasks, you have access to the following actions. Use them as needed to progress through your thought process.\n\n[ASSESS]\n\n[ADVANCE]\n\n[VERIFY]\n\n[SIMPLIFY]\n\n[SYNTHESIZE]\n\n[PIVOT]\n\n[OUTPUT]\n\nYou should strictly follow the format below:\n\n[ACTION NAME]\n\n# Your action step 1\n\n# Your action step 2\n\n# Your action step 3\n\n...\n\nNext action: [NEXT ACTION NAME]\n\n", "role": "system" }, { "content": "Title: Zebras\n\nTime Limit: None seconds\n\nMemory Limit: None megabytes\n\nProblem Description:\nOleg writes down the history of the days he lived. For each day he decides if it was good or bad. Oleg calls a non-empty sequence of days a zebra, if it starts with a bad day, ends with a bad day, and good and bad days are alternating in it. Let us denote bad days as 0 and good days as 1. Then, for example, sequences of days 0, 010, 01010 are zebras, while sequences 1, 0110, 0101 are not.\n\nOleg tells you the story of days he lived in chronological order in form of string consisting of 0 and 1. Now you are interested if it is possible to divide Oleg's life history into several subsequences, each of which is a zebra, and the way it can be done. Each day must belong to exactly one of the subsequences. For each of the subsequences, days forming it must be ordered chronologically. Note that subsequence does not have to be a group of consecutive days.\n\nInput Specification:\nIn the only line of input data there is a non-empty string *s* consisting of characters 0 and 1, which describes the history of Oleg's life. Its length (denoted as |*s*|) does not exceed 200<=000 characters.\n\nOutput Specification:\nIf there is a way to divide history into zebra subsequences, in the first line of output you should print an integer *k* (1<=\u2264<=*k*<=\u2264<=|*s*|), the resulting number of subsequences. In the *i*-th of following *k* lines first print the integer *l**i* (1<=\u2264<=*l**i*<=\u2264<=|*s*|), which is the length of the *i*-th subsequence, and then *l**i* indices of days forming the subsequence. Indices must follow in ascending order. Days are numbered starting from 1. Each index from 1 to *n* must belong to exactly one subsequence. If there is no way to divide day history into zebra subsequences, print -1.\n\nSubsequences may be printed in any order. If there are several solutions, you may print any of them. You do not have to minimize nor maximize the value of *k*.\n\nDemo Input:\n['0010100\\n', '111\\n']\n\nDemo Output:\n['3\\n3 1 3 4\\n3 2 5 6\\n1 7\\n', '-1\\n']\n\nNote:\nnone\n\nWrite Python code to solve the problem. Present the code in \n```python\nYour code\n```\nat the end.", "role": "user" } ], "ability": "code", "reward_model": { "ground_truth": "{\"inputs\": [\"0010100\", \"111\", \"0\", \"1\", \"0101010101\", \"010100001\", \"000111000\", \"0101001000\", \"0000001000\", \"0101\", \"000101110\", \"010101010\", \"0101001010\", \"0100101100\", \"0110100000\", \"0000000000\", \"1111111111\", \"0010101100\", \"1010000\", \"0001110\", \"0000000000011001100011110101000101000010010111000100110110000011010011110110001100100001001001010010\", \"01010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010\", \"0010011100000000\"], \"outputs\": [\"3\\n1 1\\n5 2 3 4 5 6\\n1 7\", \"-1\", \"1\\n1 1\", \"-1\", \"-1\", \"-1\", \"3\\n3 1 6 7\\n3 2 5 8\\n3 3 4 9\", \"4\\n5 1 2 3 4 5\\n3 6 7 8\\n1 9\\n1 10\", \"8\\n1 1\\n1 2\\n1 3\\n1 4\\n1 5\\n3 6 7 8\\n1 9\\n1 10\", \"-1\", \"-1\", \"1\\n9 1 2 3 4 5 6 7 8 9\", \"2\\n5 1 2 3 4 5\\n5 6 7 8 9 10\", \"2\\n5 1 2 3 8 9\\n5 4 5 6 7 10\", \"-1\", \"10\\n1 1\\n1 2\\n1 3\\n1 4\\n1 5\\n1 6\\n1 7\\n1 8\\n1 9\\n1 10\", \"-1\", \"2\\n3 1 8 9\\n7 2 3 4 5 6 7 10\", \"-1\", \"-1\", \"22\\n1 1\\n1 2\\n1 3\\n1 4\\n1 5\\n1 6\\n1 7\\n1 8\\n7 9 24 25 26 27 28 29\\n7 10 13 14 17 18 23 30\\n11 11 12 15 16 19 22 31 32 33 34 35\\n3 20 21 36\\n3 37 46 47\\n9 38 39 40 45 48 57 58 75 76\\n17 41 42 43 44 49 50 51 54 55 56 59 72 73 74 77 80 81\\n9 52 53 60 71 78 79 82 83 84\\n7 61 64 65 66 67 70 85\\n5 62 63 68 69 86\\n3 87 88 89\\n3 90 91 92\\n5 93 94 95 96 97\\n3 98 99 100\", \"1\\n245 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 ...\", \"8\\n3 1 8 9\\n5 2 3 4 7 10\\n3 5 6 11\\n1 12\\n1 13\\n1 14\\n1 15\\n1 16\"]}", "style": "rule" }, "extra_info": { "index": 49, "split": "train" } } ``` **Fungsi Hadiah** Fungsi Hadiah: [https://github.com/volcengine/verl/tree/main/verl/utils/reward\$1score/prime\$1code](https://github.com/volcengine/verl/tree/main/verl/utils/reward_score/prime_code) ## RLVR - Jawaban Matematika **Set Data Masukan** [Sumber: Dataset gsm8k yang dibersihkan https://huggingface. co/datasets/thesven/gsm8k-reasoning/tree/main/data](https://huggingface.co/datasets/thesven/gsm8k-reasoning/tree/main/data) **Sampel** ``` [ { "content": "\nYou are an AI assistant that uses a Chain of Thought (CoT) approach with reflection to answer queries...", "role": "system" }, { "content": "A craft store makes a third of its sales in the fabric section, a quarter of its sales in the jewelry section, and the rest in the stationery section. They made 36 sales today. How many sales were in the stationery section?", "role": "user" }, { "content": "\nTo find the number of sales in the stationery section, I need to first calculate the number of sales in the fabric and jewelry sections...\n\n\n\n1. Calculate the number of sales in the fabric section: 36 * 1/3 = 12\n2. Calculate the number of sales in the jewelry section: 36 * 1/4 = 9\n3. Calculate the number of sales in the stationery section: 36 - 12 - 9 = 15\n\n\n\nUpon reviewing my reasoning, I noticed that my calculations seem correct...\n\n\n\nUpon re-examining my math, I am confident that my calculations are correct...\n\n\n\n15\n", "role": "assistant" } ] ``` **Perhitungan Hadiah** Fungsi Hadiah: [https://github.com/volcengine/verl/blob/main/verl/utils/reward\$1score/gsm8k.py](https://github.com/volcengine/verl/blob/main/verl/utils/reward_score/gsm8k.py) ## RLVR - Lambda Kustom **Set Data Masukan** [Sumber: Dataset gsm8k yang dibersihkan https://huggingface. co/datasets/thesven/gsm8k-reasoning/tree/main/data](https://huggingface.co/datasets/thesven/gsm8k-reasoning/tree/main/data) **Sampel** ``` [ { "content": "\nYou are an AI assistant that uses a Chain of Thought (CoT) approach with reflection to answer queries.\n\nFollow these steps:\n- Think through the problem step by step within the ‹thinking> tags.\n- Reflect on your thinking\n\nto check for any errors or improvements within the ‹reflection› tags.\n- Make any necessary adjustments based on your reflection.\n- Provide your final, concise answer within the ‹output> tags.\n\nImportant: The and ‹reflection› sections are for your internal reasoning process only.\nDo not include any part of the final answer in these sections.\nThe actual response to the query must be entirely contained within the ‹output› tags.\n\nUse the following format for your response:\n\n[Your initial thought process goes here]\n\n[Your step-by-step reasoning goes here. This is your internal thought process, not the final answer. You can create as many reasoning steps as necessary in your process.]\n\n\n‹reflection>\n[Your reflection on your reasoning, checking for errors or improvements. You can create as many reflection steps as necessary in your process.]\n\n\n\n[Any adjustments to your thinking based on your reflection]\n\n\n\n[Your final, concise answer to the query. This is the only part that will be shown to the user.]\n\n", "role": "system" }, { "content": "A craft store makes a third of its sales in the fabric section, a quarter of its sales in the jewelry section, and the rest in the stationery section. They made 36 sales today. How many sales were in the stationery section?", "role": "user" }, { "content": "\nTo find the number of sales in the stationery section, I need to first calculate the number of sales in the fabric and jewelry sections. The store makes a third of its sales in the fabric section, which is 1/3 of 36 sales. I can calculate this as 36 * 1/3 = 12 sales in the fabric section. The store makes a quarter of its sales in the jewelry section, which is 1/4 of 36 sales. I can calculate this as 36 * 1/4 = 9 sales in the jewelry section. Since the rest of the sales are in the stationery section, I can subtract the sales from the other two sections from the total number of sales to find the number of sales in the stationery section.\n\n\n\n1. Calculate the number of sales in the fabric section: 36 * 1/3 = 12\n2. Calculate the number of sales in the jewelry section: 36 * 1/4 = 9\n3. Calculate the number of sales in the stationery section: 36 - 12 - 9 = 15\n\n\n‹reflection>\nUpon reviewing my reasoning, I noticed that my calculations seem correct, but I should ensure that I understand the question correctly. The question asks for the number of sales in the stationery section, and I believe I have correctly calculated that. However, I should double-check my math to make sure it's accurate.\n\n\n\nUpon re-examining my math, I am confident that my calculations are correct. The fabric section makes up 1/3 of the sales, the jewelry section makes up 1/4, and the rest is the stationery section. My math checks out, and I am confident in my answer.\n\n\n\n15\n", "role": "assistant" } ] ``` **Contoh Perhitungan Hadiah** ``` # RLVR Evaluator for OSS # lambda_grader.py import json import re import uuid from typing import Any, Dict, List def custom_reward(assistant_answer: str, ground_truth: str) -> float: """ Add custom reward computation here Example:- Reward = fraction of ground-truth words that are correct in the correct position. Example: gt: "the cat sat" ans: "the dog sat" word-by-word: "the" == "the" -> correct "dog" != "cat" -> wrong "sat" == "sat" -> correct correct = 2 out of 3 -> reward = 2/3 ≈ 0.67 """ ans_words = assistant_answer.strip().lower().split() gt_words = ground_truth.strip().lower().split() if not gt_words: return 0.0 correct = 0 for aw, gw in zip(ans_words, gt_words): if aw == gw: correct += 1 return correct / len(gt_words) # Lambda utility functions def _ok(body: Any, code: int = 200) -> Dict[str, Any]: return { "statusCode": code, "headers": { "content-type": "application/json", "access-control-allow-origin": "*", "access-control-allow-methods": "POST,OPTIONS", "access-control-allow-headers": "content-type", }, "body": json.dumps(body), } def _assistant_text(sample: Dict[str, Any]) -> str: """Extract assistant text from sample messages.""" for m in reversed(sample.get("messages", [])): if m.get("role") == "assistant": return (m.get("content") or "").strip() return "" def _sample_id(sample: Dict[str, Any]) -> str: """Generate or extract sample ID.""" if isinstance(sample.get("id"), str) and sample["id"]: return sample["id"] return str(uuid.uuid4()) def _ground_truth(sample: Dict[str, Any]) -> str: """Extract ground truth from sample or metadata if available""" if isinstance(sample.get("reference_answer"), str) and sample["reference_answer"]: return sample["reference_answer"].strip() md = sample.get("metadata") or {} gt = md.get("reference_answer", None) or md.get("ground_truth", None) if gt is None: return "" return str(gt).strip() def _score_and_metrics(sample: Dict[str, Any]) -> Dict[str, Any]: sid = _sample_id(sample) solution_text = _assistant_text(sample) # Extract ground truth gt = _ground_truth(sample) metrics_list: List[Dict[str, Any]] = [] # Custom rlvr scoring if solution_text and gt: # Compute score reward_score = custom_reward( assistant_answer=solution_text, ground_truth=gt ) # Add detailed metrics metrics_list.append({ "name": "custom_reward_score", "value": float(reward_score), "type": "Reward" }) # The aggregate reward score is the custom reward score aggregate_score = float(reward_score) else: # No solution text or ground truth - default to 0 aggregate_score = 0.0 metrics_list.append({ "name": "default_zero", "value": 0.0, "type": "Reward" }) print("detected score", { "id": sid, "aggregate_reward_score": float(aggregate_score), "metrics_list": metrics_list, }) return { "id": sid, "aggregate_reward_score": float(aggregate_score), "metrics_list": metrics_list, } def lambda_handler(event, context): """AWS Lambda handler for custom reward lambda grading.""" # CORS preflight if event.get("requestContext", {}).get("http", {}).get("method") == "OPTIONS": return _ok({"ok": True}) # Body may be a JSON string (API GW/Function URL) or already a dict (Invoke) raw = event.get("body") or "{}" try: body = json.loads(raw) if isinstance(raw, str) else raw except Exception as e: return _ok({"error": f"invalid JSON body: {e}"}, 400) # Accept top-level list, {"batch":[...]}, or single sample object if isinstance(body, dict) and isinstance(body.get("batch"), list): samples = body["batch"] else: return _ok({ "error": "Send a sample object, or {'batch':[...]} , or a top-level list of samples." }, 400) try: results = [_score_and_metrics(s) for s in samples] except Exception as e: return _ok({"error": f"Custom scoring failed: {e}"}, 500) return _ok(results) ``` **Contoh kode fungsi hadiah** ``` # RLVR Evaluator for OSS # lambda_grader.py import json import re import uuid from typing import Any, Dict, List def custom_reward(assistant_answer: str, ground_truth: str) -> float: """ Add custom reward computation here Example:- Reward = fraction of ground-truth words that are correct in the correct position. Example: gt: "the cat sat" ans: "the dog sat" word-by-word: "the" == "the" -> correct "dog" != "cat" -> wrong "sat" == "sat" -> correct correct = 2 out of 3 -> reward = 2/3 ≈ 0.67 """ ans_words = assistant_answer.strip().lower().split() gt_words = ground_truth.strip().lower().split() if not gt_words: return 0.0 correct = 0 for aw, gw in zip(ans_words, gt_words): if aw == gw: correct += 1 return correct / len(gt_words) # Lambda utility functions def _ok(body: Any, code: int = 200) -> Dict[str, Any]: return { "statusCode": code, "headers": { "content-type": "application/json", "access-control-allow-origin": "*", "access-control-allow-methods": "POST,OPTIONS", "access-control-allow-headers": "content-type", }, "body": json.dumps(body), } def _assistant_text(sample: Dict[str, Any]) -> str: """Extract assistant text from sample messages.""" for m in reversed(sample.get("messages", [])): if m.get("role") == "assistant": return (m.get("content") or "").strip() return "" def _sample_id(sample: Dict[str, Any]) -> str: """Generate or extract sample ID.""" if isinstance(sample.get("id"), str) and sample["id"]: return sample["id"] return str(uuid.uuid4()) def _ground_truth(sample: Dict[str, Any]) -> str: """Extract ground truth from sample or metadata if available""" if isinstance(sample.get("reference_answer"), str) and sample["reference_answer"]: return sample["reference_answer"].strip() md = sample.get("metadata") or {} gt = md.get("reference_answer", None) or md.get("ground_truth", None) if gt is None: return "" return str(gt).strip() def _score_and_metrics(sample: Dict[str, Any]) -> Dict[str, Any]: sid = _sample_id(sample) solution_text = _assistant_text(sample) # Extract ground truth gt = _ground_truth(sample) metrics_list: List[Dict[str, Any]] = [] # Custom rlvr scoring if solution_text and gt: # Compute score reward_score = custom_reward( assistant_answer=solution_text, ground_truth=gt ) # Add detailed metrics metrics_list.append({ "name": "custom_reward_score", "value": float(reward_score), "type": "Reward" }) # The aggregate reward score is the custom reward score aggregate_score = float(reward_score) else: # No solution text or ground truth - default to 0 aggregate_score = 0.0 metrics_list.append({ "name": "default_zero", "value": 0.0, "type": "Reward" }) print("detected score", { "id": sid, "aggregate_reward_score": float(aggregate_score), "metrics_list": metrics_list, }) return { "id": sid, "aggregate_reward_score": float(aggregate_score), "metrics_list": metrics_list, } def lambda_handler(event, context): """AWS Lambda handler for custom reward lambda grading.""" # CORS preflight if event.get("requestContext", {}).get("http", {}).get("method") == "OPTIONS": return _ok({"ok": True}) # Body may be a JSON string (API GW/Function URL) or already a dict (Invoke) raw = event.get("body") or "{}" try: body = json.loads(raw) if isinstance(raw, str) else raw except Exception as e: return _ok({"error": f"invalid JSON body: {e}"}, 400) # Accept top-level list, {"batch":[...]}, or single sample object if isinstance(body, dict) and isinstance(body.get("batch"), list): samples = body["batch"] else: return _ok({ "error": "Send a sample object, or {'batch':[...]} , or a top-level list of samples." }, 400) try: results = [_score_and_metrics(s) for s in samples] except Exception as e: return _ok({"error": f"Custom scoring failed: {e}"}, 500) return _ok(results) ``` **Contoh prompt hadiah** ``` You are an expert RAG response evaluator specializing in faithfulness and relevance assessment. Given: Context documents, a question, and response statements. Goal: Evaluate both statement-level faithfulness and overall response relevance to the question. Scoring rubric (start at 0.0, then add or subtract): Core Components: Faithfulness Assessment (0.6 max) Per statement evaluation: - Direct support in context: +0.2 - Accurate inference from context: +0.2 - No contradictions with context: +0.2 Deductions: - Hallucination: -0.3 - Misrepresentation of context: -0.2 - Unsupported inference: -0.1 Question Relevance (0.4 max) - Direct answer to question: +0.2 - Appropriate scope/detail: +0.1 - Proper context usage: +0.1 Deductions: - Off-topic content: -0.2 - Implicit/meta responses: -0.2 - Missing key information: -0.1 Critical Flags: A. Complete hallucination B. Context misalignment C. Question misinterpretation D. Implicit-only responses Additional Instructions: - Evaluate each statement independently - Check for direct textual support - Verify logical inferences - Assess answer completeness - Flag any unsupported claims Return EXACTLY this JSON (no extra text): { "statements_evaluation": [ { "statement": "", "verdict": <0 or 1>, "reason": "", "context_support": "" } ], "overall_assessment": { "question_addressed": <0 or 1>, "reasoning": "", "faithfulness_score": <0.0-1.0>, "relevance_score": <0.0-1.0> }, "flags": [""] } ## Current Evaluation Task ### Context {{ ground_truth }} ### Question {{ extra_info.question }} ### Model's Response {{ model_output }} ```