Central Logging and Analytics in Hybrid Environments
Publication date: July 21, 2020 (Diagram history)
This architecture shows how to build a central logging and analytics solution. You can analyze Amazon CloudWatch logs from a hybrid environment, including VMware Cloud on AWS.
Central Logging and Analytics in Hybrid Environments
-
Install the Amazon CloudWatch agent on VMs on-premises.
-
Send CloudWatch metrics and logs to the customer central logging VPC through and AWS Transit Gateway.
-
Send CloudWatch data from VMware Cloud on AWS to the CloudWatch endpoint in the customer-owned VPC through an elastic network interface.
-
Export log data from your log group to load onto other systems such as ISV solutions.
-
Create a subscription to deliver logs from a specific CloudWatch log group to Amazon Data Firehose in the logging account destination.
-
Use Amazon Data Firehose to continuously stream the log data to https://docs.aws.amazon.com/opensearch-service/latest/developerguide/what-is.html.
-
Use Amazon Data Firehose to deliver log data to the Amazon S3 bucket.
-
Use AWS Glue to build the Data Catalog and to create the ETL jobs for data processing.
-
Amazon Athena uses the AWS Glue Data Catalog to discover and query data in Amazon S3.
-
Amazon Redshift reads and loads data from multiple data files stored in Amazon S3 buckets.
-
Build visualizations and dashboards by using Amazon QuickSight with Amazon Athena and Amazon Redshift.
Further reading
For additional information, refer to
Diagram history
To be notified about updates to this reference architecture diagram, subscribe to the RSS feed.
| Change | Description | Date |
|---|---|---|
Initial publication | Reference architecture diagram first published. | July 21, 2020 |
Note
To subscribe to RSS updates, you must have an RSS plugin enabled for the browser you are using.
