View a markdown version of this page

Architecture overview - Guidance for Deploying a Prebid Server on AWS
Architecture diagram

Architecture overview

This section provides a reference implementation architecture diagram for the components deployed with this solution and a summary of AWS Well-Architected design considerations.

Architecture diagram

Deploying this solution with the default parameters deploys the following components in your AWS account.

Guidance for Deploying a Prebid Server on AWS architectural overview

aws solution for prebid server architecture
Note

AWS CloudFormation resources are created from AWS Cloud Development Kit (AWS CDK) constructs.

The high-level process flow for the solution, including the components deployed with the AWS CDK template, is as follows:

  1. A user browses to a page on a website that hosts ads.

  2. The publisher site returns the page source to the browser with resources, and one or more script modules (also called wrappers) that enable real-time bid requests and responses for ads of given dimensions, types, topics, and other criteria.

  3. The bid requests are received from the browser at the Amazon CloudFront endpoint integrated with AWS Web Application Firewall (AWS WAF) for entry into the solution. This step helps validate legitimate traffic from malicious requests, such as penetration or denial-of-service attempts. Traffic can be received here as HTTP or HTTPS.

  4. The request is forwarded to Application Load Balancer (ALB). ALB determines which container running Prebid Server in the cluster is at a utilization level that can accept more requests. ALB has a network interface on the public internet and one in each private subnet where containers are hosted within Amazon Virtual Private Cloud (Amazon VPC).

  5. The request arrives at an Amazon Elastic Container Service (Amazon ECS) container, is parsed and validated, and requests to different bidding services are sent concurrently. The connectivity method depends on the deployment configuration:

    1. External bidders: Requests are sent through the NAT gateway and internet gateway to bidders over the public internet.

    2. Bidder simulator with VPC peering: Requests are sent directly through the VPC peering connection to the bidder simulator in its own VPC.

    3. Bidder simulator with RTB Fabric: Requests are sent through the AWS RTB Fabric requester gateway to the responder gateway in the bidder simulator VPC, utilizing the private RTB Fabric network.

  6. The NAT gateway and internet gateway allow containers to initiate outbound requests to the internet and receive responses. These resources are primarily used for Prebid Server containers to request and gather bids for ad auctions from external bidders.

  7. Bidders receive one or more bid requests (either over the internet, through VPC peering, or via RTB Fabric) from a Prebid Server container. Bidders respond with zero or more bids for the various requests. The response, including the body of the winning creative(s), is sent back to the browser.

  8. For cache operations, Prebid Server containers are configured to use the publicly accessible endpoint (CloudFront domain or external ALB DNS) to store and retrieve cached bid responses. Cache requests from containers flow through the same entry point as client requests, reaching the cache Lambda function via ALB routing rules. This unified cache endpoint ensures both server-side and client-side cache access use the same path.

  9. During normal operation, Amazon CloudWatch metrics are collected from various resources involved in handling requests and responses through the solution. As the load changes throughout the cluster, CloudWatch alarms are used to determine when to scale-out or scale-in the container cluster.

  10. An ECS service definition (Prebid ECS service) is responsible for tracking the health of the cluster, performing scale-out and scale-in operations, and managing the collection of containers available for ALB. The Prebid ECS service uses AWS Fargate instances.

  11. Metrics log files for each container are stored to a shared Amazon Elastic File System (Amazon EFS) using NFS protocol. This file system is mounted to each Prebid Server container during start-up. A single metrics log file is written for a limited time and then closed and rotated, so that it can be included in the next stage of processing. EFS is treated as a temporary location as log data is generated and moved to longer-term storage on Amazon Simple Storage Service (Amazon S3) and into AWS Glue.

  12. AWS DataSync replicates rotated log files from EFS to S3 on a recurring schedule. DataSync verifies each transferred file and provides a report of the completed work to an AWS Lambda function.

  13. The DataSyncLogsBucket S3 bucket receives the replicated log files from EFS using the same folder structure. Log files arrive in this bucket as a result of the DataSync process.

  14. The delete_efs_files Lambda function runs after the DataSync process completes in step 12 and removes transferred and verified log file data from EFS.

  15. An AWS Glue job performs an ETL operation on the metrics data in the DataSyncLogsBucket S3 bucket. The ETL operation structures the metric data into a single database with several tables, partitions the physical data, and writes it to an S3 bucket.

  16. The MetricsEtlBucket S3 bucket contains the metric log data transformed and partitioned through ETL. The data in this bucket is made available to AWS Glue clients for queries.

  17. Many different types of clients use AWS Glue Data Catalog to access the Prebid Server metric data.