Connection scaling
The most common technique for scaling web service capacity is to add or remove application servers (instances) in response to changes in user traffic. Each application server can use a database connection pool.
This approach causes the total number of database connections to grow proportionally with the number of application instances. For example, 20 application servers configured with 200 database connections each would require a total of 4,000 database connections. If the application pool scales up to 200 instances (for example, during peak hours), the total connection count will reach 40,000. Under a typical web application workload, most of these connections are likely idle. In extreme cases, this can limit database scalability: idle connections do take server resources, and you’re opening significantly more of them than you need. Also, the total number of connections is not easy to control because it’s not something you configure directly, but rather depends on the number of application servers.
You have two options in this situation:
-
Tune the connection pools on application instances. Reduce the number of connections in the pool to the acceptable minimum. This can be a stop-gap solution, but it might not be a long-term solution as your application server fleet continues to grow.
-
Introduce a connection proxy between the database and the application. On one side, the proxy connects to the database with a fixed number of connections. On the other side, the proxy accepts application connections and can provide additional features such as query caching, connection buffering, query rewriting/routing, and load balancing.
