Consolidate product information from databases, ERP systems, and supplier files into a single governed data lake. Deliver consistent, enriched catalogs to your e-commerce, point-of-sale, and marketing systems simultaneously.
Overview
This Guidance demonstrates how to consolidate fragmented product data from multiple retail sources—including supplier systems, IoT devices, and social media—into a unified, governed product catalog on AWS. The system ingests product information from diverse sources in near real-time and batch modes, then transforms and enriches the data by standardizing formats, validating pricing rules, and enhancing product attributes. Retailers can query the unified catalog using serverless analytics, build AI-powered dashboards with natural language queries, and use generative AI to automatically enrich product descriptions. The enriched product information flows seamlessly to downstream systems like e-commerce platforms, point-of-sale terminals, and marketing tools to ensure consistent customer experiences across all channels. You can accelerate time-to-market for new products, reduce manual data entry errors, and deliver consistent omnichannel experiences by automating product data management with built-in governance and AI-powered enrichment.
Benefits
Unify product data across channels
Enrich catalogs with generative AI
Auto-generate product descriptions and classify items using foundation models and custom machine learning. Reduce manual catalog enrichment effort while improving product content quality at scale.
Eliminate infrastructure management overhead
Deploy a fully serverless pipeline that ingests, transforms, and serves product data without provisioning servers. Scale automatically with your catalog size while reducing operational costs.
How it works
These architecture diagrams show how product data moves through the solution—from ingestion across diverse retail sources, through transformation and enrichment on AWS, to serving a unified catalog to business users and downstream systems. Choose a tab to view the full data flow or how a business user accesses the product information management (PIM) system.
Download the architecture diagram.Full data flow
Step 1
Retailers collect product data from multiple sources including traditional databases, supplier file shares, IoT devices, ERP systems, social media feeds, and product image repositories to create a unified product catalog.
AWS DataSync, Amazon Kinesis, AWS Transfer Family, and AWS Database Migration Service securely ingest product information from diverse retail systems into S3, enabling near real time inventory updates and batch product catalog synchronization.
Amazon Simple Storage Service provides scalable storage for product catalogs, pricing data, and digital assets, supporting server-side encryption with SSE-S3 and SSE-KMS to protect data at rest. AWS Lake Formation enforces unified governance policies so you can maintain data quality and compliance across retail operations.
AWS Glue and AWS Glue DataBrew transform and enrich product data by standardizing SKU formats, validating pricing rules, and enriching product attributes for consistent omnichannel experiences and write results back to S3.
You query enriched product catalogs using Amazon Athena, analyze trends with Amazon Redshift, and build AI-powered dashboards with natural language querying in Amazon QuickSight using Generative BI capabilities.
You build generative AI applications with Amazon Bedrock, a fully managed service with built-in security and responsible AI features, to enrich product descriptions using RAG with your product knowledge base and foundation models. Amazon SageMaker AI trains and deploys custom ML models for product classification.
Enriched product information flows to downstream retail systems including ERP systems, point-of-sale systems, marketing automation tools, and e-commerce websites for consistent customer experiences.
Business accessing the PIM system
Step 1
You open the web application URL over HTTPS and sign in through the React frontend hosted on AWS Amplify, authenticating via Amazon Cognito user pools with MFA.
Authenticated API requests route through Amazon API Gateway with Amazon Cognito authorizer to AWS Lambda functions for product operations.
AWS Lambda queries product data from Apache Iceberg tables stored in Amazon S3 using Amazon Athena for serverless SQL analytics.
AWS Lambda caches query results in Amazon DynamoDB, encrypted at rest by default using AWS owned keys (with optional SSE-KMS for customer-managed key control), with TTL-based expiration to reduce query latency and API response time.
Raw product data lands in Amazon S3, which triggers S3 event notifications that invoke AWS Step Functions workflows to orchestrate the multi-stage ETL pipeline.
You transform, validate, and load data into Iceberg tables using AWS Glue jobs with automated data quality checks and schema evolution.
AWS Lambda and AWS Glue retrieves seed user credentials from AWS Secrets Manager at runtime. Amazon CloudWatch provides centralized monitoring, alarms, and logging across all services.
Deploy with confidence
Everything you need to launch this Guidance in your account is right here.
Let's make it happen
Ready to deploy? Review the sample code on GitHub for detailed deployment instructions to deploy as-is or customize to fit your needs.