Configuring Lambda permissions for Amazon MSK event source mappings - AWS Lambda
Required permissionsOptional permissions

Configuring Lambda permissions for Amazon MSK event source mappings

To access the Amazon MSK cluster, your function and event source mapping need permissions to perform various Amazon MSK API actions. Add these permissions to the function's execution role. If your users need access, add the required permissions to the identity policy for the user or role.

The AWSLambdaMSKExecutionRole managed policy contains the minimum required permissions for Amazon MSK Lambda event source mappings. To simplify the permissions process, you can:

  • Attach the AWSLambdaMSKExecutionRole managed policy to your execution role.

  • Let the Lambda console generate the permissions for you. When you create an Amazon MSK event source mapping in the console, Lambda evaluates your execution role and alerts you if any permissions are missing. Choose Generate permissions to automatically update your execution role. This doesn't work if you manually created or modified your execution role policies, or if the policies are attached to multiple roles. Note that additional permissions may still be required in your execution role when using advanced features such as On-Failure Destination or AWS Glue Schema Registry.

Required permissions

Your Lambda function execution role must have the following required permissions for Amazon MSK event source mappings. These permissions are included in the AWSLambdaMSKExecutionRole managed policy.

CloudWatch Logs permissions

The following permissions allow Lambda to create and store logs in Amazon CloudWatch Logs.

MSK cluster permissions

The following permissions allow Lambda to access your Amazon MSK cluster on your behalf:

We recommend using kafka:DescribeClusterV2 instead of kafka:DescribeCluster. The v2 permission works with both provisioned and serverless Amazon MSK clusters. You only need one of these permissions in your policy.

VPC permissions

The following permissions allow Lambda to create and manage network interfaces when connecting to your Amazon MSK cluster:

Optional permissions

Your Lambda function might also need permissions to:

  • Access cross-account Amazon MSK clusters. For cross-account event source mappings, you need kafka:DescribeVpcConnection in the execution role. An IAM principal creating a cross-account event source mapping needs kafka:ListVpcConnections.

  • Access your SCRAM secret, if you're using SASL/SCRAM authentication. This lets your function use a username and password to connect to Kafka.

  • Describe your Secrets Manager secret, if you're using SASL/SCRAM or mTLS authentication. This allows your function to retrieve the credentials or certificates needed for secure connections.

  • Access your AWS KMS customer managed key, if your AWS Secrets Manager secret is encrypted with an AWS KMS customer managed key.

  • Access your schema registry secrets, if you're using a schema registry with authentication:

    • For AWS Glue Schema Registry: Your function needs glue:GetRegistry and glue:GetSchemaVersion permissions. These allow your function to look up and use the message format rules stored in AWS Glue.

    • For Confluent Schema Registry with BASIC_AUTH or CLIENT_CERTIFICATE_TLS_AUTH: Your function needs secretsmanager:GetSecretValue permission for the secret containing the authentication credentials. This lets your function retrieve the username/password or certificates needed to access the Confluent Schema Registry.

    • For private CA certificates: Your function needs secretsmanager:GetSecretValue permission for the secret containing the certificate. This allows your function to verify the identity of schema registries that use custom certificates.

  • Access Kafka cluster consumer groups and poll messages from the topic, if you're using IAM authentication for the event source mapping.

These correspond to the following required permissions:

Additionally, if you want to send records of failed invocations to an on-failure destination, you'll need the following permissions depending on the destination type:

  • For Amazon SQS destinations: sqs:SendMessage - Allows sending messages to an Amazon SQS queue

  • For Amazon SNS destinations: sns:Publish - Permits publishing messages to an Amazon SNS topic

  • For Amazon S3 bucket destinations: s3:PutObject and s3:ListBucket - Enables writing and listing objects in an Amazon S3 bucket

For troubleshooting authentication and authorization errors, see Troubleshooting Kafka event source mapping errors.