Processor compatibility and restrictions - Amazon CloudWatch
Processor-specific restrictions

Processor compatibility and restrictions

General processor rules
Maximum count

A pipeline can have at most 20 processors.

Parser placement

Parser processors (OCSF, CSV, Grok, etc.) must be the first processor in a pipeline.

Unique processors

The following processors can appear only once per pipeline:

  • add_entries

  • copy_values

Source compatibility matrix
Processor Type CloudWatch Logs Source S3 Source API-based Sources
OCSF Compatible with CloudTrail only Fully compatible Compatible with specific schemas
parse_vpc Must be first processor Not applicable Not applicable
parse_route53 Must be first processor Not applicable Not applicable
parse_json Must be first processor Fully compatible Fully compatible
grok Must be first processor Fully compatible Fully compatible
csv Must be first processor Not compatible Not compatible
key_value Must be first processor Fully compatible Fully compatible
add_entries Compatible (max 1) Compatible (max 1) Compatible (max 1)
copy_values Compatible (max 1) Compatible (max 1) Compatible (max 1)
String processors (lowercase, uppercase, trim) Fully compatible Fully compatible Fully compatible
Field processors (move_keys, rename_keys) Fully compatible Fully compatible Fully compatible
Data transformation (date, flatten) Fully compatible Fully compatible Fully compatible
Compatibility definitions
Fully compatible

Can be used without restrictions with the source type

Must be first processor

When used, must be the first processor in the pipeline configuration

Compatible with restrictions

Can be used but has specific limitations or requirements

Not compatible

Cannot be used with this source type

Not applicable

Processor is not relevant for this source type

Processor-specific restrictions

Processor restrictions by source type
Processor Source Type Restrictions
OCSF CloudWatch Logs with CloudTrail

  • Only allowed when data_source_name is aws_cloudtrail

  • Must use CloudTrail-specific schema version

  • Cannot be combined with other processors
OCSF API-based Sources

  • Must use source-specific schema (e.g., microsoft_office365_management_activity for Office 365)

  • Requires specific mapping version for each source type

  • Must be first processor in pipeline
parse_vpc CloudWatch Logs

  • Only valid for VPC Flow Logs

  • Must be first processor

  • Source field must contain raw VPC Flow Log format
parse_route53 CloudWatch Logs

  • Only valid for Route 53 Resolver Query Logs

  • Must be first processor

  • Source field must contain Route 53 Resolver query log format
add_entries All Sources

  • Maximum one instance per pipeline

  • Key names must be valid according to field naming rules
copy_values All Sources

  • Maximum one instance per pipeline

  • Source fields must exist in the event
Important

When using processors with restrictions:

  • Always validate your pipeline configuration using the ValidateTelemetryPipelineConfiguration API before deployment

  • Test the pipeline with sample data using the TestTelemetryPipeline API to ensure proper processing

  • Monitor pipeline metrics after deployment to ensure events are being processed as expected