# Submit a model distillation job in Amazon Bedrock
<a name="submit-model-distillation-job"></a>

You can perform model distillation through the Amazon Bedrock console or by sending a [CreateModelCustomizationJob](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_CreateModelCustomizationJob.html) request with an [Amazon Bedrock control plane endpoint](https://docs.aws.amazon.com/general/latest/gr/bedrock.html#br-cp).

## Prerequisites
<a name="submit-model-distillation-job-prerequisites"></a>
+ Create an IAM service role with the required permissions. For comprehensive security and permissions information, see [Model customization access and security](custom-model-job-access-security.md).
+ (Optional) Encrypt input and output data, your customization job, or inference requests made to custom models. For more information, see [Encryption of custom models](encryption-custom-job.md).
+ (Optional) Create a Virtual Private Cloud (VPC) to protect your customization job. For more information, see [(Optional) Protect your model customization jobs using a VPC](custom-model-job-access-security.md#vpc-model-customization).

For more information on setting up on demand inference, see [Set up inference for a custom model](model-customization-use.md).

## Submit your job
<a name="submit-model-distillation-job-how-to"></a>

------
#### [ Console ]

1. Sign in to the AWS Management Console with an IAM identity that has permissions to use the Amazon Bedrock console. Then, open the Amazon Bedrock console at [https://console.aws.amazon.com/bedrock](https://console.aws.amazon.com/bedrock).

1. From the left navigation pane, choose **Custom models** under **Tune**.

1. Choose **Create distillation job**.

1. For **Distilled model details**, do the following:

   1. For **Distilled model name**, enter a name for your distilled model.

   1. (Optional) For **Model encryption**, select the checkbox if you want to provide a KMS key for encrypting your job and its related artifacts. 

      For more information, see [Encryption of custom models](encryption-custom-job.md).

   1. (Optional) Apply **Tags** to your distilled model.

1. For **Job configuration**, do the following:

   1. For **Job name**, enter a name for your distillation job.

   1. (Optional) For **Model encryption**, select the checkbox if you want to provide a KMS key for encrypting your job and its related artifacts. 

      For more information, see [Encryption of custom models](encryption-custom-job.md).

   1. (Optional) Apply **Tags** to your job.

1. For **Teacher model – Student model details**, choose the teacher and student models for creating your distilled model.

   For more information, see [Prerequisites for model distillation](prequisites-model-distillation.md).

1. For **Synthetic data generation**, do the following:

   1. For **Max response length**, specify the maximum length of the synthetic responses generated by the teacher model.

   1. For **Distillation input dataset**, choose one of the following options:
      + **Directly upload to S3 location** – Specify the S3 location where you're storing the input dataset (prompts) that'll be used for distillation. For more information, see [Option 1: Provide your own prompts for data preparation](distillation-data-prep-option-1.md).
      + **Provide access to invocation logs** – Specify the S3 location where you're storing the invocation logs with the input dataset (prompts) that'll be used for distillation. For more information, see [Option 2: Use invocation logs for data preparation](distillation-data-prep-option-2.md).
        + (Optional) For **Request Metadata Filters**, specify filters if you want Amazon Bedrock to only use certain prompts in your logs for distillation.
        + Choose **Read prompts** or **Read prompt-response pairs** depending on what you want Amazon Bedrock to access from your logs. Keep in mind that responses are read only if your teacher model matches the model in your logs.

1. For **Distillation output**, specify the S3 location where you want to upload the metrics and reports about your distillation job.

   For more information, see [Analyze the results of a model customization jobAnalyze model customization job results](model-customization-analyze.md).

1. For **VPC settings**, choose a VPC configuration for accessing the S3 bucket with your training data.

   For more information, see [(Optional) Protect your model customization jobs using a VPC](custom-model-job-access-security.md#vpc-model-customization).

1. For **Service access**, specify the IAM role for accessing the S3 bucket with your training data. Unless you use a Cross Region inference profile or VPC configurations, you can create the role in the Amazon Bedrock console with the correct permissions automatically configured. Or you can use an existing service role. 

    For a job that has Amazon VPC configurations or uses a Cross Region inference profile, you must create a new service role in IAM that has the required permissions. 

   For more information, see [Create an IAM service role for model customization](custom-model-job-access-security.md#custom-model-job-service-role).

1. Choose **Create distillation job** to start the distillation job. After you customize a model, you can set up inference for the model. For more information, see [Set up inference for a custom model](model-customization-use.md). 

------
#### [ API ]

At minimum, you must provide the following fields to submit your model distillation job when using the Amazon Bedrock API.


****  

| Field | Description | 
| --- | --- | 
| baseModelIdentifier | The model identifier of the student model | 
| customModelName | The name of the new distilled model | 
| jobName | The name of the model distillation job | 
| roleArn | Role that gives Amazon Bedrock permissions to read training and validation files and write to the output path | 
| trainingDataConfig | The Amazon S3 path that has your training data | 
| outputDataConfig | The Amazon S3 path that contains your training and validation metrics | 
| distillationConfig | Inputs required for distillation job | 
| customModelKmsKeyId | To encrypt the custom model | 
| clientRequestToken | Token to prevent the request from completing more than once | 

The following fields are optional:


****  

| Field | Description | 
| --- | --- | 
| customizationType | Set to DISTILLATION  by default for distillation jobs | 
| validationDataConfig | List of validation data Amazon S3 paths | 
| jobTags | To associate tags with the job | 
| customModelTags | To associate tags with the resulting custom model | 
| vpcConfig | VPC to protect your training data and distillation job | 

To prevent the request from completing more than once, include a `clientRequestToken`.

You can include the following optional fields for extra configurations.
+ `jobTags` and/or `customModelTags` – Associate [tags](tagging.md) with the customization job or resulting custom model.
+ `vpcConfig` – Include the configuration for a [virtual private cloud (VPC) to protect your training data and customization job](custom-model-job-access-security.md#vpc-model-customization).

The following is an example snippet from [CreateModelCustomizationJob](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_CreateModelCustomizationJob.html) API. This example uses the prompt-response pairs in the invocation log as the input data source and specifies the filter for selecting prompt-response pairs. 

```
"trainingDataConfig": {
    "invocationLogsConfig": {
        "usePromptResponse": true,
        "invocationLogSource": {
            "s3Uri": "string"
        },
        "requestMetadataFilters": {
            "equals": {
                "priority": "High"
            }
        }
    }
}
```

**Response**

The response returns a `jobArn` of the model distillation job.

------

## Next steps
<a name="submit-model-distillation-next-steps"></a>
+ [Monitor your distillation job](model-customization-monitor.md). For more information on setting up on demand inference, see [Set up inference for a custom model](model-customization-use.md).