# Detection
**Topics**
+ [
# SEC 4 How do you detect and investigate security events?
](sec-04.md)
# SEC 4 How do you detect and investigate security events?
Capture and analyze events from logs and metrics to gain visibility. Take action on security events and potential threats to help secure your workload.
**Topics**
+ [
# SEC04-BP01 Configure service and application logging
](sec_detect_investigate_events_app_service_logging.md)
+ [
# SEC04-BP02 Analyze logs, findings, and metrics centrally
](sec_detect_investigate_events_analyze_all.md)
+ [
# SEC04-BP03 Automate response to events
](sec_detect_investigate_events_auto_response.md)
+ [
# SEC04-BP04 Implement actionable security events
](sec_detect_investigate_events_actionable_events.md)
# SEC04-BP01 Configure service and application logging
Configure logging throughout the workload, including application logs, resource logs, and AWS service logs. For example, ensure that AWS CloudTrail, Amazon CloudWatch Logs, Amazon GuardDuty and AWS Security Hub CSPM are enabled for all accounts within your organization.
A foundational practice is to establish a set of detection mechanisms at the account level. This base set of mechanisms is aimed at recording and detecting a wide range of actions on all resources in your account. They allow you to build out a comprehensive detective capability with options that include automated remediation, and partner integrations to add functionality.
In AWS, services that can implement this base set include:
+ [AWS CloudTrail](http://aws.amazon.com/cloudtrail) provides event history of your AWS account activity, including actions taken through the AWS Management Console, AWS SDKs, command line tools, and other AWS services.
+ [AWS Config](http://aws.amazon.com/config) monitors and records your AWS resource configurations and allows you to automate the evaluation and remediation against desired configurations.
+ [Amazon GuardDuty](http://aws.amazon.com/guardduty) is a threat detection service that continuously monitors for malicious activity and unauthorized behavior to protect your AWS accounts and workloads.
+ [AWS Security Hub CSPM](http://aws.amazon.com/security-hub) provides a single place that aggregates, organizes, and prioritizes your security alerts, or findings, from multiple AWS services and optional third- party products to give you a comprehensive view of security alerts and compliance status.
Building on the foundation at the account level, many core AWS services, for example [Amazon Virtual Private Cloud Console (Amazon VPC)](http://aws.amazon.com/vpc), provide service-level logging features. [Amazon VPC Flow Logs](https://docs.aws.amazon.com/vpc/latest/userguide/flow-logs.html) enable you to capture information about the IP traffic going to and from network interfaces that can provide valuable insight into connectivity history, and trigger automated actions based on anomalous behavior.
For Amazon Elastic Compute Cloud (Amazon EC2) instances and application-based logging that doesn’t originate from AWS services, logs can be stored and analyzed using [Amazon CloudWatch Logs](http://aws.amazon.com/cloudwatch). An [agent](https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/CWL_GettingStarted.html) collects the logs from the operating system and the applications that are running and automatically stores them. Once the logs are available in CloudWatch Logs, you can [process them in real-time](https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/Subscriptions.html), or dive into analysis using [CloudWatch Logs Insights](https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/AnalyzingLogData.html).
Equally important to collecting and aggregating logs is the ability to extract meaningful insight from the great volumes of log and event data generated by complex architectures. See the *Monitoring* section of the [Reliability Pillar whitepaper](https://docs.aws.amazon.com/wellarchitected/latest/reliability-pillar/monitor-workload-resources.html) for more detail. Logs can themselves contain data that is considered sensitive–either when application data has erroneously found its way into log files that the CloudWatch Logs agent is capturing, or when cross-region logging is configured for log aggregation and there are legislative considerations about shipping certain kinds of information across borders.
One approach is to use AWS Lambda functions, triggered on events when logs are delivered, to filter and redact log data before forwarding into a central logging location, such as an Amazon Simple Storage Service (Amazon S3) bucket. The unredacted logs can be retained in a local bucket until a reasonable time has passed (as determined by legislation and your legal team), at which point an Amazon S3 lifecycle rule can automatically delete them. Logs can further be protected in Amazon S3 by using [Amazon S3 Object Lock](https://docs.aws.amazon.com/AmazonS3/latest/dev/object-lock.html), where you can store objects using a write-once-read-many (WORM) model.
**Level of risk exposed if this best practice is not established:** High
## Implementation guidance
+ Enable logging of AWS services: Enable the logging of AWS services to meet your requirements. Logging capabilities include the following: Amazon VPC Flow Logs, Elastic Load Balancing (ELB) logs, Amazon S3 bucket logs, CloudFront access logs, Amazon Route 53 query logs, and Amazon Relational Database Service (Amazon RDS) logs.
+ [AWS Answers: native AWS security-logging capabilities ](https://aws.amazon.com/answers/logging/aws-native-security-logging-capabilities/)
+ Evaluate and enable logging of operating systems and application-specific logs to detect suspicious behavior.
+ [ Getting started with CloudWatch Logs ](http://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/CWL_GettingStarted.html)
+ [ Developer Tools and Log Analysis ](https://aws.amazon.com/marketplace/search/results?category=4988009011)
+ Apply appropriate controls to the logs: Logs can contain sensitive information and only authorized users should have access. Consider restricting permissions to Amazon S3 buckets and CloudWatch Logs log groups.
+ [ Authentication and Access Control for Amazon CloudWatch ](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/auth-and-access-control-cw.html)
+ [Identity and access management in Amazon S3 ](https://docs.aws.amazon.com/AmazonS3/latest/dev/s3-access-control.html)
+ Configure [Amazon GuardDuty](https://docs.aws.amazon.com/guardduty/latest/ug/what-is-guardduty.html): GuardDuty is a threat detection service that continuously looks for malicious activity and unauthorized behavior to protect your AWS accounts and workloads. Enable GuardDuty and configure automated alerts to email using the lab.
+ [Configure customized trail in CloudTrail](http://docs.aws.amazon.com/awscloudtrail/latest/userguide/cloudtrail-create-and-update-a-trail.html): Configuring a trail enables you to store logs for longer than the default period, and analyze them later.
+ Enable [AWS Config](https://docs.aws.amazon.com/config/latest/developerguide/WhatIsConfig.html): AWS Config provides a detailed view of the configuration of AWS resources in your AWS account. This view includes how the resources are related to one another and how they were previously configured so that you can see how the configurations and relationships change over time.
+ Enable [AWS Security Hub CSPM](https://docs.aws.amazon.com/securityhub/latest/userguide/what-is-securityhub.html): Security Hub CSPM provides you with a comprehensive view of your security state in AWS and helps you check your compliance with the security industry standards and best practices. Security Hub CSPM collects security data from across AWS accounts, services, and supported third-party partner products and helps you analyze your security trends and identify the highest priority security issues.
## Resources
**Related documents:**
+ [ Amazon CloudWatch ](https://aws.amazon.com/cloudwatch/)
+ [Amazon EventBridge ](https://aws.amazon.com/eventbridge)
+ [ Getting started: Amazon CloudWatch Logs ](https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/CWL_GettingStarted.html)
+ [Security Partner Solutions: Logging and Monitoring](https://aws.amazon.com/security/partner-solutions/#logging-monitoring)
**Related videos:**
+ [ Centrally Monitoring Resource Configuration and Compliance ](https://youtu.be/kErRv4YB_T4)
+ [Remediating Amazon GuardDuty and AWS Security Hub CSPM Findings ](https://youtu.be/nyh4imv8zuk)
+ [ Threat management in the cloud: Amazon GuardDuty and AWS Security Hub CSPM](https://youtu.be/vhYsm5gq9jE)
**Related examples:**
+ [ Lab: Automated Deployment of Detective Controls ](https://wellarchitectedlabs.com/Security/200_Automated_Deployment_of_Detective_Controls/README.html)
# SEC04-BP02 Analyze logs, findings, and metrics centrally
Security operations teams rely on the collection of logs and the use of search tools to discover potential events of interest, which might indicate unauthorized activity or unintentional change. However, simply analyzing collected data and manually processing information is insufficient to keep up with the volume of information flowing from complex architectures. Analysis and reporting alone don’t facilitate the assignment of the right resources to work an event in a timely fashion.
A best practice for building a mature security operations team is to deeply integrate the flow of security events and findings into a notification and workflow system such as a ticketing system, a bug or issue system, or other security information and event management (SIEM) system. This takes the workflow out of email and static reports, and allows you to route, escalate, and manage events or findings. Many organizations are also integrating security alerts into their chat or collaboration, and developer productivity platforms. For organizations embarking on automation, an API-driven, low-latency ticketing system offers considerable flexibility when planning what to automate first.
This best practice applies not only to security events generated from log messages depicting user activity or network events, but also from changes detected in the infrastructure itself. The ability to detect change, determine whether a change was appropriate, and then route that information to the correct remediation workflow is essential in maintaining and validating a secure architecture, in the context of changes where the nature of their undesirability is sufficiently subtle that their execution cannot currently be prevented with a combination of AWS Identity and Access Management (IAM) and AWS Organizations configuration.
Amazon GuardDuty and AWS Security Hub CSPM provide aggregation, deduplication, and analysis mechanisms for log records that are also made available to you via other AWS services. GuardDuty ingests, aggregates, and analyzes information from sources such as AWS CloudTrail management and data events, VPC DNS logs, and VPC Flow Logs. Security Hub CSPM can ingest, aggregate, and analyze output from GuardDuty, AWS Config, Amazon Inspector, Amazon Macie, AWS Firewall Manager, and a significant number of third-party security products available in the AWS Marketplace, and if built accordingly, your own code. Both GuardDuty and Security Hub CSPM have an Administrator-Member model that can aggregate findings and insights across multiple accounts, and Security Hub CSPM is often used by customers who have an on- premises SIEM as an AWS-side log and alert preprocessor and aggregator from which they can then ingest Amazon EventBridge through a AWS Lambda-based processor and forwarder.
**Level of risk exposed if this best practice is not established:** High
## Implementation guidance
+ Evaluate log processing capabilities: Evaluate the options that are available for processing logs.
+ [Use Amazon OpenSearch Service to log and monitor (almost) everything ](https://d1.awsstatic.com/whitepapers/whitepaper-use-amazon-elasticsearch-to-log-and-monitor-almost-everything.pdf)
+ [Find an AWS Partner that specializes in logging and monitoring solutions ](https://aws.amazon.com/security/partner-solutions/#Logging_and_Monitoring)
+ As a start for analyzing CloudTrail logs, test Amazon Athena.
+ [ Configuring Athena to analyze CloudTrail logs ](https://docs.aws.amazon.com/athena/latest/ug/cloudtrail-logs.html)
+ Implement centralize logging in AWS: See the following AWS example solution to centralize logging from multiple sources.
+ [Centralize logging solution ](https://aws.amazon.com/solutions/centralized-logging/)
+ Implement centralize logging with partner: APN Partners have solutions to help you analyze logs centrally.
+ [ Logging and Monitoring ](https://aws.amazon.com/security/partner-solutions/#Logging_and_Monitoring)
## Resources
**Related documents:**
+ [AWS Answers: Centralized Logging ](https://aws.amazon.com/answers/logging/centralized-logging/)
+ [AWS Security Hub CSPM](https://docs.aws.amazon.com/securityhub/latest/userguide/what-is-securityhub.html)
+ [Amazon CloudWatch](https://aws.amazon.com/cloudwatch/)
+ [Amazon EventBridge ](https://aws.amazon.com/eventbridge)
+ [ Getting started: Amazon CloudWatch Logs ](https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/CWL_GettingStarted.html)
+ [Security Partner Solutions: Logging and Monitoring](https://aws.amazon.com/security/partner-solutions/#logging-monitoring)
**Related videos:**
+ [ Centrally Monitoring Resource Configuration and Compliance ](https://youtu.be/kErRv4YB_T4)
+ [Remediating Amazon GuardDuty and AWS Security Hub CSPM Findings ](https://youtu.be/nyh4imv8zuk)
+ [ Threat management in the cloud: Amazon GuardDuty and AWS Security Hub CSPM](https://youtu.be/vhYsm5gq9jE)
# SEC04-BP03 Automate response to events
Using automation to investigate and remediate events reduces human effort and error, and enables you to scale investigation capabilities. Regular reviews will help you tune automation tools, and continuously iterate.
In AWS, investigating events of interest and information on potentially unexpected changes into an automated workflow can be achieved using Amazon EventBridge. This service provides a scalable rules engine designed to broker both native AWS event formats (such as AWS CloudTrail events), as well as custom events you can generate from your application. Amazon GuardDuty also allows you to route events to a workflow system for those building incident response systems (AWS Step Functions), or to a central Security Account, or to a bucket for further analysis.
Detecting change and routing this information to the correct workflow can also be accomplished using AWS Config Rules and [Conformance Packs](https://docs.aws.amazon.com/config/latest/developerguide/conformance-packs.html). AWS Config detects changes to in-scope services (though with higher latency than EventBridge) and generates events that can be parsed using AWS Config Rules for rollback, enforcement of compliance policy, and forwarding of information to systems, such as change management platforms and operational ticketing systems. As well as writing your own Lambda functions to respond to AWS Config events, you can also take advantage of the [AWS Config Rules Development Kit](https://github.com/awslabs/aws-config-rdk), and a [library of open source](https://github.com/awslabs/aws-config-rules) AWS Config Rules. Conformance packs are a collection of AWS Config Rules and remediation actions you deploy as a single entity authored as a YAML template. A [sample conformance pack template](https://docs.aws.amazon.com/config/latest/developerguide/operational-best-practices-for-wa-Security-Pillar.html) is available for the Well-Architected Security Pillar.
**Level of risk exposed if this best practice is not established:** Medium
## Implementation guidance
+ Implement automated alerting with GuardDuty: GuardDuty is a threat detection service that continuously monitors for malicious activity and unauthorized behavior to protect your AWS accounts and workloads. Enable GuardDuty and configure automated alerts.
+ Automate investigation processes: Develop automated processes that investigate an event and report information to an administrator to save time.
+ [ Lab: Amazon GuardDuty hands on ](https://hands-on-guardduty.awssecworkshops.com/)
## Resources
**Related documents:**
+ [AWS Answers: Centralized Logging ](https://aws.amazon.com/answers/logging/centralized-logging/)
+ [AWS Security Hub CSPM](https://docs.aws.amazon.com/securityhub/latest/userguide/what-is-securityhub.html)
+ [ Amazon CloudWatch ](https://aws.amazon.com/cloudwatch/)
+ [Amazon EventBridge ](https://aws.amazon.com/eventbridge)
+ [ Getting started: Amazon CloudWatch Logs ](https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/CWL_GettingStarted.html)
+ [Security Partner Solutions: Logging and Monitoring](https://aws.amazon.com/security/partner-solutions/#logging-monitoring)
+ [ Setting up Amazon GuardDuty ](https://docs.aws.amazon.com/guardduty/latest/ug/guardduty_settingup.html)
**Related videos:**
+ [ Centrally Monitoring Resource Configuration and Compliance ](https://youtu.be/kErRv4YB_T4)
+ [Remediating Amazon GuardDuty and AWS Security Hub CSPM Findings ](https://youtu.be/nyh4imv8zuk)
+ [ Threat management in the cloud: Amazon GuardDuty and AWS Security Hub CSPM](https://youtu.be/vhYsm5gq9jE)
**Related examples:**
+ [Lab: Automated Deployment of Detective Controls ](https://wellarchitectedlabs.com/Security/200_Automated_Deployment_of_Detective_Controls/README.html)
# SEC04-BP04 Implement actionable security events
Create alerts that are sent to and can be actioned by your team. Ensure that alerts include relevant information for the team to take action. For each detective mechanism you have, you should also have a process, in the form of a [runbook](https://wa.aws.amazon.com/wat.concept.runbook.en.html) or [playbook](https://wa.aws.amazon.com/wat.concept.playbook.en.html), to investigate. For example, when you enable [Amazon GuardDuty](http://aws.amazon.com/guardduty), it generates different [findings](https://docs.aws.amazon.com/guardduty/latest/ug/guardduty_findings.html). You should have a runbook entry for each finding type, for example, if a [trojan](https://docs.aws.amazon.com/guardduty/latest/ug/guardduty_trojan.html) is discovered, your runbook has simple instructions that instruct someone to investigate and remediate.
**Level of risk exposed if this best practice is not established:** Low
## Implementation guidance
+ Discover metrics available for AWS services: Discover the metrics that are available through Amazon CloudWatch for the services that you are using.
+ [AWS service documentation](https://aws.amazon.com/documentation/)
+ [Using Amazon CloudWatch Metrics](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/working_with_metrics.html)
+ Configure Amazon CloudWatch alarms.
+ [Using Amazon CloudWatch Alarms](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/AlarmThatSendsEmail.html)
## Resources
**Related documents:**
+ [ Amazon CloudWatch ](https://aws.amazon.com/cloudwatch/)
+ [Amazon EventBridge ](https://aws.amazon.com/eventbridge)
+ [Security Partner Solutions: Logging and Monitoring](https://aws.amazon.com/security/partner-solutions/#logging-monitoring)
**Related videos:**
+ [ Centrally Monitoring Resource Configuration and Compliance ](https://youtu.be/kErRv4YB_T4)
+ [Remediating Amazon GuardDuty and AWS Security Hub CSPM Findings ](https://youtu.be/nyh4imv8zuk)
+ [ Threat management in the cloud: Amazon GuardDuty and AWS Security Hub CSPM](https://youtu.be/vhYsm5gq9jE)