Reinforcement fine-tuning access and security

Before you begin reinforcement fine-tuning, make sure that you understand what kind of access Amazon Bedrock needs for RFT-specific operations. RFT requires additional permissions beyond standard fine-tuning due to its reward function execution capabilities.

For basic model customization security setup including trust relationships, Amazon S3 permissions, and KMS encryption, see Create an IAM service role for model customization.

Prerequisites

Before adding RFT-specific IAM permissions, you must add the following IAM service roles:

RFT-specific IAM permissions

Add these permissions to your existing model customization service role for RFT functionality.

Lambda permissions for reward functions

You must add Lambda invocation permissions. The following shows an example policy you can use:

{
    "Version": "2012-10-17",		 	 	 
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "lambda:InvokeFunction"
            ],
            "Resource": [
                "arn:aws:lambda:*:*:function:reward-function-name"
            ]
        }
    ]
}

Invocation log access

To use existing Amazon Bedrock model invocation logs as training data, add permissions to access your Amazon S3 bucket where invocation logs are stored.

You need to provide Amazon S3 bucket access permissions for the input bucket. The following shows an example policy you can use:

{
    "Version": "2012-10-17",		 	 	 
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:GetObject",
                "s3:ListBucket"
            ],
            "Resource": [
                "arn:aws:s3:::s3-invocation-logs-bucket",
                "arn:aws:s3:::s3-invocation-logs-bucket/*"
            ]
        }
    ]
}

For security setup including basic IAM roles, Amazon S3 permissions, and encryption, see Create an IAM service role for model customization.

Grader Lambda function permissions for RLAIF

If you create your own Lambda function for Reinforcement Learning from AI Feedback (RLAIF) reward functions, you need to add specific permissions to the Lambda execution role.

Bedrock permissions for LLM judges

For LLM-as-Judge reward functions (RLAIF), add permissions to invoke foundation models. The following shows an example policy you can use for your Lambda execution role.

The following is an example for bedrock LLM as judge invocation using foundation model:

{
    "Version": "2012-10-17",		 	 	 
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "bedrock:InvokeModel"
            ],
            "Resource": [
                "arn:aws:bedrock:*:*:foundation-model/*"
            ]
        }
    ]
}

The following is an example for bedrock LLM as judge invocation using inference profile:

{
    "Version": "2012-10-17",		 	 	 
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "bedrock:InvokeModel"
            ],
            "Resource": [
                "arn:aws:bedrock:us-east-1::foundation-model/amazon.nova-premier-v1:0",
                "arn:aws:bedrock:us-east-2::foundation-model/amazon.nova-premier-v1:0",
                "arn:aws:bedrock:us-west-2::foundation-model/amazon.nova-premier-v1:0"
            ],
            "Condition": {
                "StringLike": {
                    "bedrock:InferenceProfileArn": "arn:aws:bedrock:us-east-1:111122223333:inference-profile/us.amazon.nova-premier-v1:0"
                }
            }
        },
        {
            "Effect": "Allow",
            "Action": [
                "bedrock:InvokeModel"
            ],
            "Resource": [
                "arn:aws:bedrock:us-east-1:111122223333:inference-profile/us.amazon.nova-premier-v1:0"
            ]
        }
    ]
}

For information about inference profile prerequisites, see Prerequisites for inference profiles.