CloudWatch pipelines IAM policies and permissions - Amazon CloudWatch
API caller permissionsSource-specific IAM policiesTrust relationshipsResource policiesManaging resource policies

CloudWatch pipelines IAM policies and permissions

This section provides detailed IAM requirements for CloudWatch pipelines, including permissions for API callers, source-specific policies, trust relationships, and resource policies.

API caller permissions

Any role specified in the pipeline configuration that calls the CreateTelemetryPipeline API (such as S3 source roles, Secrets Manager access roles, or CloudWatch Logs source roles) must have specific permissions to pass roles.

PassRole permissions

Required for any roles specified in the pipeline configuration (S3 source roles, Secrets Manager access roles, or CloudWatch Logs source roles).

Example IAM policy for S3 sources
{
    "Version": "2012-10-17",		 	 	 
    "Statement": [
        {
            "Sid": "PassRoleForS3Source",
            "Effect": "Allow",
            "Action": "iam:PassRole",
            "Resource": "arn:aws:iam::account-id:role/s3-source-role"
        }
    ]
}
Example IAM policy for Secrets Manager sources
{
    "Version": "2012-10-17",		 	 	 
    "Statement": [
        {
            "Sid": "PassRoleForSecretsManagerSource",
            "Effect": "Allow",
            "Action": "iam:PassRole",
            "Resource": "arn:aws:iam::account-id:role/secrets-manager-role"
        }
    ]
}
Example IAM Policy for CloudWatch Logs Sources
{
    "Version": "2012-10-17", 		 	 	 
    "Statement": [
        {
            "Sid": "PassRoleForCloudWatchLogsSource",
            "Effect": "Allow",
            "Action": "iam:PassRole",
            "Resource": "arn:aws:iam::account-id:role/cloudwatch-logs-role""
        }
  ]
}

Pipeline rule permissions

When using cloudwatch_logs source for Create/Update operations (logs:PutPipelineRule) and Delete operations (logs:DeletePipelineRule) the role must also have permissions to perform those operations.

Example IAM policy for CloudWatch Logs pipeline rules
{
    "Version": "2012-10-17",		 	 	 
    "Statement": [
        {
            "Sid": "PipelineRuleForCloudWatchLogs",
            "Effect": "Allow",
            "Action": [
                "logs:PutPipelineRule",
                "logs:DeletePipelineRule"
            ],
            "Resource": "*"
        }
    ]
}

Reducing scope with condition keys

To scope down the permission policy to telemetry pipelines, you can specify Condition Keys as shown in the following examples:

Example IAM policy for S3 sources (basic)
{
    "Version": "2012-10-17",		 	 	 
    "Statement": [
        {
            "Sid": "PassRoleForS3Source",
            "Effect": "Allow",
            "Action": "iam:PassRole",
            "Resource": "arn:aws:iam::YOUR-ACCOUNT-ID:role/S3-SOURCE-ROLE"
        }
    ]
}
Example IAM policy for S3 sources (scoped down with condition keys)
{
    "Version": "2012-10-17",		 	 	 
    "Statement": [
        {
            "Sid": "PassRoleForS3Source",
            "Effect": "Allow",
            "Action": "iam:PassRole",
            "Resource": "arn:aws:iam::YOUR-ACCOUNT-ID:role/S3-SOURCE-ROLE",
            "Condition": {
              "StringEquals": {
                "iam:PassedToService": [
                  "telemetry-pipelines.observabilityadmin.amazonaws.com"
                ],
                "iam:AssociatedResourceARN": [
                  "arn:aws:observabilityadmin:YOUR-REGION:YOUR-ACCOUNT-ID:telemetry-pipeline/*"
                ]
              }
            }
        }
    ]
}
Example IAM policy for Secrets Manager sources (basic)
{
    "Version": "2012-10-17",		 	 	 
    "Statement": [
        {
            "Sid": "PassRoleForSecretsManagerSource",
            "Effect": "Allow",
            "Action": "iam:PassRole",
            "Resource": "arn:aws:iam::YOUR-ACCOUNT-ID:role/SECRETS-MANAGER-ROLE"
        }
    ]
}
Example IAM policy for Secrets Manager sources (scoped down with condition keys)
{
    "Version": "2012-10-17",		 	 	 
    "Statement": [
        {
          "Sid": "PassRoleForSecretsManagerSource",
          "Effect": "Allow",
          "Action": "iam:PassRole",
          "Resource": "arn:aws:iam::YOUR-ACCOUNT-ID:role/SECRETS-MANAGER-ROLE",
          "Condition": {
            "StringEquals": {
              "iam:PassedToService": [
                "telemetry-pipelines.observabilityadmin.amazonaws.com"
              ],
              "iam:AssociatedResourceARN": [
                "arn:aws:observabilityadmin:YOUR-REGION:YOUR-ACCOUNT-ID:telemetry-pipeline/*"
              ]
            }
          }
        }
    ]
}
Example IAM policy for CloudWatch Logs sources (scoped down with condition keys)
{
    "Version": "2012-10-17",		 	 	 
    "Statement": [
        {
          "Sid": "PassRoleForCloudWatchLogsSource",
          "Effect": "Allow",
          "Action": "iam:PassRole",
          "Resource": "arn:aws:iam::YOUR-ACCOUNT-ID:role/CLOUDWATCH-LOGS-ROLE",
          "Condition": {
            "StringEquals": {
              "iam:PassedToService": [
                "logs.amazonaws.com"
              ],
              "iam:AssociatedResourceARN": [
                "arn:aws:observabilityadmin:YOUR-REGION:YOUR-ACCOUNT-ID:telemetry-pipeline/*"
              ]
            }
          }
        }
    ]
}

Source-specific IAM policies

Different source types require specific IAM permissions to access their respective data sources.

CloudWatch Logs sources

For CloudWatch Logs sources, any IAM role specified in the pipeline configuration must have a trust relationship with logs.amazonaws.com.

Example IAM role trust policy for CloudWatch Logs sources (basic)
{
    "Version": "2012-10-17",		 	 	 
    "Statement": [
        {
            ""Effect": "Allow",
            "Principal": {
                "Service": "logs.amazonaws.com"
            },
            "Action": "sts:AssumeRole"
        }
    ]
}

S3 sources

For S3 sources, customers must provide an IAM role with permissions to access S3 objects and SQS queues.

Example IAM policy for S3 sources
{
    "Version": "2012-10-17",		 	 	 
    "Statement": [
        {
            "Sid": "s3-access",
            "Effect": "Allow",
            "Action": [
                "s3:GetObject"
            ],
            "Resource": "arn:aws:s3:::bucket-name/*"
        },
        {
            "Sid": "sqs-access",
            "Effect": "Allow",
            "Action": [
                "sqs:ReceiveMessage",
                "sqs:DeleteMessage",
                "sqs:ChangeMessageVisibility"
            ],
            "Resource": "arn:aws:sqs:region:account-id:queue-name"
        },
        {
            "Sid": "kms-access",
            "Effect": "Allow",
            "Action": "kms:Decrypt",
            "Resource": "arn:aws:kms:region:account-id:key/key-id",
            "Condition": {
                "Comment": "Only required if S3 buckets and/or SQS queue uses KMS encryption"
            }
        }
    ]
}

Sources using AWS Secrets Manager

For sources that reference AWS Secrets Manager (Microsoft Office 365, Microsoft Entra ID, Palo Alto NGFW), customers must provide an IAM role with Secrets Manager access.

Example IAM policy for Secrets Manager sources
{
    "Version": "2012-10-17",		 	 	 
    "Statement": [
        {
            "Sid": "secrets-manager-access",
            "Effect": "Allow",
            "Action": [
                "secretsmanager:GetSecretValue"
            ],
            "Resource": "arn:aws:secretsmanager:region:account-id:secret:secret-name*"
        },
        {
            "Sid": "kms-access",
            "Effect": "Allow",
            "Action": "kms:Decrypt",
            "Resource": "arn:aws:kms:region:account-id:key/key-id",
            "Condition": {
                "Comment": "Only required if Secrets Manager uses KMS encryption"
            }
        }
    ]
}

Trust relationships

Any IAM role specified in the pipeline configuration must have a trust relationship with the CloudWatch pipelines service principal.

Pipeline role trust policy

All pipeline roles must trust the telemetry-pipelines.observabilityadmin.amazonaws.com service principal.

Example Trust policy for pipeline roles
{
    "Version": "2012-10-17",		 	 	 
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "Service": "telemetry-pipelines.observabilityadmin.amazonaws.com"
            },
            "Action": "sts:AssumeRole"
        }
    ]
}

Resource policies

CloudWatch Logs resource policies are required for pipelines that write to log groups, except for pipelines using the cloudwatch_logs source.

CloudWatch Logs resource policy

After calling CreateTelemetryPipeline API, you will receive a pipeline ARN. For pipelines where the source is not cloudwatch_logs, customers must call logs:PutResourcePolicy to allow the CloudWatch pipelines service principal to write to the configured log group.

Resource policy limits

CloudWatch Logs has a maximum of 10 resource policies per account per region. If you reach this limit, remove the Condition section and Resource restriction from the policy.

Timing constraint

You have a limited time window (less than 5 minutes) to create the resource policy after receiving the pipeline ARN. If the pipeline becomes active before the policy is in place, data will be dropped.

Example logs:PutResourcePolicy request
{
    "policyName": "ATPTelemetryPipelinePolicy",
    "policyDocument": {
        "Version": "2012-10-17",		 	 	 
        "Statement": [
            {
                "Effect": "Allow",
                "Principal": {
                    "Service": "telemetry-pipelines.observabilityadmin.amazonaws.com"
                },
                "Action": [
                    "logs:CreateLogStream",
                    "logs:PutLogEvents"
                ],
                "Resource": "arn:aws:logs:region:account-id:log-group:log-group-name:*",
                "Condition": {
                    "StringEquals": {
                        "aws:SourceArn": "arn:aws:observabilityadmin:region:account-id:telemetry-pipeline/pipeline-id"
                    }
                }
            }
        ]
    }
}

The following AWS CLI example should how to create a log-groups scope resource policy. Each log group is limited to a single resource policy.

aws logs put-resource-policy --resource-arn "arn:aws:logs:us-east-1:123456789012:log-group:testGroup" --policy-document file://~/example_resource_policy.json

Managing resource policies

This guide provides steps for creating or updating a CloudWatch Logs resource policy for telemetry pipelines using the AWS CLI.

Check for existing policies:

aws logs describe-resource-policies --region <YOUR-REGION>

This returns all existing resource policies. Look for any policy that might already be associated with your log group.

If no resource policy exists, create a new one:

aws logs put-resource-policy \
        --region <YOUR-REGION> \
        --policy-name ATPTelemetryPipelinePolicy \
        --policy-document '{
    "Version": "2012-10-17",		 	 	 
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "Service": "telemetry-pipelines.observabilityadmin.amazonaws.com"
            },
            "Action": [
                "logs:CreateLogStream",
                "logs:PutLogEvents"
            ],
            "Resource": "arn:aws:logs:<YOUR-REGION>:<YOUR-ACCOUNT-ID>:log-group:<YOUR-LOG-GROUP>:*",
            "Condition": {
                "StringEquals": {
                    "aws:SourceArn": "arn:aws:observabilityadmin:<YOUR-REGION>:<YOUR-ACCOUNT-ID>:telemetry-pipeline/<PIPELINE-ID>"
                }
            }
        }
    ]
}'

Replace the following placeholders:

  • <YOUR-REGION> - Your AWS region (e.g., us-east-1)

  • <YOUR-ACCOUNT-ID> - Your 12-digit AWS account ID

  • <YOUR-LOG-GROUP> - Your CloudWatch Logs log group name

  • <PIPELINE-ID> - Your telemetry pipeline ID

If a resource policy already exists, merge the new statement with it:

  1. Retrieve the existing policy:

    aws logs describe-resource-policies \
        --region <YOUR-REGION> \
        --query "resourcePolicies[?policyName=='ATPTelemetryPipelinePolicy'].policyDocument" \
        --output text > existing-policy.json

  2. Open existing-policy.json and add the new statement to the existing Statement array: 

    {
    "Version": "2012-10-17",		 	 	 
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "Service": "existing-service.amazonaws.com"
            },
            "Action": [
                "logs:SomeAction"
            ],
            "Resource": "arn:aws:logs:region:account:log-group:existing-group:*"
        },
        {
            "Effect": "Allow",
            "Principal": {
                "Service": "telemetry-pipelines.observabilityadmin.amazonaws.com"
            },
            "Action": [
                "logs:CreateLogStream",
                "logs:PutLogEvents"
            ],
            "Resource": "arn:aws:logs:<YOUR-REGION>:<YOUR-ACCOUNT-ID>:log-group:<YOUR-LOG-GROUP>:*",
            "Condition": {
                "StringEquals": {
                    "aws:SourceArn": "arn:aws:observabilityadmin:<YOUR-REGION>:<YOUR-ACCOUNT-ID>:telemetry-pipeline/<PIPELINE-ID>"
                }
            }
        }
    ]
}

  3. Update the policy:

    aws logs put-resource-policy \
        --region <YOUR-REGION> \
        --policy-name ATPTelemetryPipelinePolicy \
        --policy-document file://existing-policy.json

Confirm the policy was created or updated successfully:

aws logs describe-resource-policies \
        --region <YOUR-REGION> \
        --query "resourcePolicies[?policyName=='ATPTelemetryPipelinePolicy']"