View a markdown version of this page

Logging - Guidance for Deploying a Prebid Server on AWS
Prebid Server operation and metrics logs

Logging

The log files generated by this solution are stored in four separate buckets depending on their context. Logs containing operational data, such as CloudFront access logs, are stored in one bucket, while logs containing business intelligence, such as Prebid Server performance metrics, are stored in another bucket. This organization is intended to facilitate data pipelines for context-specific log analysis.

The following S3 buckets are used for log storage.

CloudFront access logs bucket

Contains detailed information about every request received by the CloudFront distribution.

ALB access logs bucket

Contains detailed information about every request received by the Application Load Balancer (ALB). Elastic Load Balancing provides access logs that capture detailed information about requests sent to your load balancer. Each log contains information such as the time the request was received, the client’s IP address, latencies, request paths, and server responses. You can use these access logs to analyze traffic patterns and troubleshoot issues. For more information, see Access logs for your Application Load Balancer in Elastic Load Balancing.

DataSync logs bucket

  • Contains stdout and stderr log streams from the Prebid Server processes running in the ECS cluster.

  • Each log stream is saved in a unique directory named according to its container identifier in ECS.

    To learn more about these logs, see the Prebid Server operation and metrics logs section.

DataSync metrics bucket

  • Contains metric streams for Prebid Server transactions running in the ECS cluster.

  • Each metric stream is saved in a unique directory named according to its container identifier in ECS.

    To learn more about these logs, see the Prebid Server operation and metrics logs section.

Prebid Server containers in Amazon ECS send log information to CloudWatch Logs. These logs can be found within the AWS::Logs::LogGroup resource shown in the CloudFormation stack. The log group includes output from the commands specified in the Dockerfile that is used to assemble Prebid Server containers. Since the Prebid Server process redirects both stdout and stderr to EFS, the CloudWatch logs will be empty unless ECS encountered a problem running the Prebid Server process.

Prebid Server operation and metrics logs

This solution uses EFS to store logs from every Prebid Server task running in ECS. Each task writes runtime logs to /mnt/efs/logs/<CONTAINER_ID> and metrics logs to /mnt/efs/metrics/<CONTAINER_ID> . The container identifier in the path ensures that each Prebid Server instance stores logs without interfering with the logs from other instances.

The logging policy for the prebid process is defined in the prebid-logging.xml file. It includes the following customizations:

  • Info level logging of stdout/stderr output.

  • Rolling policy to archive logs when they exceed 100 MB or at a predetermined frequency of once per hour.

  • Asynchronous logging to minimize the computational overhead of logging. For more information, see Benchmarking synchronous and asynchronous logging.

The log files are archived periodically (every 60 minutes) and moved to the corresponding archive folders /mnt/efs/logs/<CONTAINER_ID>/archived and /mnt/efs/metrics/<CONTAINER_ID>/archived.

DataSync moves the archived log files from EFS to S3 every 60 minutes. Runtime logs are saved to s3://<STACK_NAME>-datasynclogsbucket-<RANDOM_STRING>/<CONTAINER_ID>/archived. Metrics logs are saved to s3://<STACK_NAME>-datasyncmetricsbucket-<RANDOM_STRING>/<CONTAINER_ID>/archived.

If you need to access the Prebid Server logs before they are copied by AWS DataSync, then mount the EFS filesystem directly using the procedure described in Accessing Prebid Server logs from EFS.