View a markdown version of this page

Supply Chain Data Lake Solution - Supply Chain Data Lake Solution
Supply chain data lake diagramFurther readingDiagram history

Supply Chain Data Lake Solution

Publication date: May 18, 2022 (Diagram history)

With this architecture, you can build a supply chain data lake on AWS. Ingest data from planning, execution, and real-time shipment status providers. Build cross-category scorecards for analysts and planners. The solution uses AWS Lake Formation for the data lake and AWS Glue for extract, transform, and load (ETL). Amazon SageMaker AI provides machine learning (ML) capabilities.

Supply chain data lake diagram

Reference architecture diagram showing how to build a supply chain data lake by using AWS Lake Formation, AWS Glue, SageMaker AI, and Amazon Neptune.

The following steps describe the data ingestion and analytics pipeline for this architecture:

  1. Collect supply chain data from multiple enterprise sources. Include ERP/CRM SaaS applications, manufacturing edge devices, logs, streaming media, and social networks.

  2. Ingest data by using AWS Database Migration Service, AWS DataSync, Amazon Kinesis, Amazon MSK, AWS IoT Core, and Amazon AppFlow.

  3. Integrate third-party data (such as weather data) into the data lake by using AWS Data Exchange.

  4. Build the scalable supply chain data lake with AWS Lake Formation.

  5. Use Amazon Simple Storage Service for data lake storage.

  6. Use AWS Glue to extract, transform, catalog, and ingest data across multiple data stores such as ERP, planning, and shipment visibility systems.

  7. Analyze data in Amazon S3 by using standard SQL with Amazon Athena as a serverless interactive query service.

  8. Build dashboards with Amazon Quick Sight to help planners drill down from planning to execution to real-time shipment status.

  9. Use Amazon Redshift as a cloud data warehouse.

  10. Process vast amounts of data by using open-source tools with Amazon EMR as the cloud big data platform.

  11. Build, train, and deploy ML models with SageMaker AI. Add intelligence to your supply chain applications.

  12. Optimize network queries for speed and accuracy by using the Amazon Neptune graph database.

Further reading

For additional information, see the following resources:

Diagram history

To receive updates about this reference architecture diagram, subscribe to the RSS feed.

ChangeDescriptionDate

Initial publication

Reference architecture diagram first published.

May 18, 2022

RSS subscription requirement

To subscribe to RSS updates, you must have an RSS plugin enabled for the browser you are using.