Data sources
CloudWatch pipelines supports three types of data:
-
AWS services logs through CloudWatch Logs
When an AWS service source is selected, CloudWatch pipelines intercepts logs ingested into CloudWatch Logs for processing. Processing of vended AWS service data occurs at the Log Source and Source Type level through CloudWatch Logs data sources. To get started enable logging of vended logs for the supported AWS services using the service’s console and then select the data source and type in the CloudWatch pipelines creation wizard. For more information about CloudWatch Logs data sources, see Data source discovery and management.
AWS vended sources provide native integration with AWS services for automatic data collection. For a list of data source names and types, see Supported AWS services for data sources.
-
Third-party log data through integration
Third-party log data can extend monitoring capabilities to support identity and access management as well as endpoint and network security. CloudWatch pipelines manages collection of data from third-party platforms through either direct API integration or an S3 bucket integration. For more details, see Third-party data sources integration. For a list of data source names and types, see Supported third-party sources for data sources.
-
Custom logs through CloudWatch Logs or S3 buckets
CloudWatch pipelines can process custom Logs stored in CloudWatch Logs or S3 buckets. Custom sources accommodate unique organizational requirements:
-
Application-specific logs – Custom application telemetry from EC2 instances with specialized logging formats
-
File-based ingestion – S3-based log files from legacy systems or batch processing workflows
-
Serverless integration – Lambda function logs and custom serverless application telemetry
For more details, see Custom log data from CloudWatch Logs or an Amazon S3 bucket.
-
Topics
