View a markdown version of this page

Centralized Log Analytics - Centralized Log Analytics
Centralized Log AnalyticsFurther readingDiagram history

Centralized Log Analytics

Publication date: November 7, 2022 (Diagram history)

This architecture shows how to search, analyze, and visualize machine data by using for operational insights.

Centralized Log Analytics

Architecture diagram showing centralized log analytics with .

  1. Collectors such as FluentBit, Amazon Kinesis Agent, and the Amazon CloudWatch Agent (or services such as AWS CloudTrail) collect log lines and store them in Amazon S3.

  2. Amazon S3 sends an object create event to Amazon SQS.

  3. Amazon SQS invokes an AWS Lambda function that transforms the log lines from strings to structured JSON (if necessary).

  4. The Lambda function uses the OpenSearch _bulk API to deliver the JSON-formatted log lines to https://docs.aws.amazon.com/opensearch-service/latest/developerguide/what-is.html.

  5. The user logs into OpenSearch Dashboards to perform interactive log analytics, build visualizations or notebooks, and monitor dashboards.

Further reading

For additional information, refer to

Diagram history

To be notified about updates to this reference architecture diagram, subscribe to the RSS feed.

ChangeDescriptionDate

Initial publication

Reference architecture diagram first published.

November 7, 2022

Note

To subscribe to RSS updates, you must have an RSS plugin enabled for the browser you are using.