# Building a scalable web crawling system on AWS
<a name="building"></a>

This section describes how to build the web crawler described in the [Architecture](architecture.md) section. It includes a systematic approach to creating a robust dataset of companies and their associated web properties. This dataset serves as the foundation for your crawling activities. Then, this section describes how to build an ethical web crawler in Python.

**Topics**
+ [Preparing a dataset](building-prepare.md)
+ [Building the web crawler](building-crawler.md)
+ [Building the AWS infrastructure](building-infrastructure.md)