View a markdown version of this page

Manufacturing Data Lake on AWS - Manufacturing Data Lake on AWS
Manufacturing data lake architecture diagramFurther readingDiagram history

Manufacturing Data Lake on AWS

Publication date: June 29, 2020 (Diagram history)

With this architecture, you can build a manufacturing data lake that combines operational technology (OT) data from Industrial Internet of Things (IIoT) devices and factory applications with enterprise application data. You can then run manufacturing analytical use cases and predictions with machine learning (ML) models. This architecture uses AWS Lake Formation, Amazon Simple Storage Service (Amazon S3), Amazon Redshift, AWS IoT Core, Amazon Data Firehose, and Amazon SageMaker AI.

Manufacturing data lake architecture diagram

Reference architecture diagram for building a manufacturing data lake on AWS that combines IIoT and enterprise data for analytics and ML predictions.

The following steps describe the architecture:

  1. Create the data lake structure with Lake Formation or Amazon S3 and Amazon Redshift. Store raw and processed data in Amazon S3 buckets organized by data source and processing stage.

  2. For structured relational data, create an Amazon Relational Database Service (Amazon RDS) instance within an Amazon VPC.

  3. For compatible enterprise applications, use Amazon API Gateway and AWS Lambda to create an interface into Amazon RDS.

  4. For compatible databases, use AWS Database Migration Service to create copies or export structured data sets into the data lake.

  5. For IIoT and automation equipment, use AWS IoT Core and Amazon Data Firehose to stream data into the data lake. AWS IoT SiteWise collects and organizes industrial equipment data.

  6. For external or batch interfaces, use AWS Transfer Family to put data into the data lake.

  7. Use AWS Glue to build the data catalog and create extract, transform, and load (ETL) jobs.

  8. Use Amazon SageMaker AI to train and deploy ML models against data in the data lake.

  9. Use AWS AppSync to build a GraphQL endpoint with Lambda resolvers. This endpoint provides a flexible API for consuming applications.

  10. Build visualizations with Amazon Quick Sight by using Amazon Athena and Amazon Redshift.

Further reading

For additional information, see the following resources:

Diagram history

To be notified about updates to this reference architecture diagram, subscribe to the RSS feed.

ChangeDescriptionDate

Initial publication

Reference architecture diagram first published.

June 29, 2020

RSS subscription

To subscribe to RSS updates, you must have an RSS plugin enabled for the browser that you are using.