September 10, 2015

Running Austria's Most Visited Web Site on a Software Load Balancer? Impossible!

- Nah, it's totally doable. With HAProxy.


Background

When a website reaches a certain size, it's necessary to distribute load between multiple server instances. Incoming requests are spread via a load balancer which, in turn, represents the most critical part of the network infrastructure. Among the best known hardware products are probably the BIG-IP F5 series. Unfortunately, using a hardware solution does not provide the flexibility we need. We constantly want to change stuff like pool members, create new pools, manage IP quotas and kick unwanted or abusive traffic from our servers. Plus, if you plan to switch from HTTP to HTTPS entirely, performance also becomes an issue.

So we decided to go for it and implement our own solution. We chose HAProxy and did not regret it. The reasons are manifold but mostly we were looking for open source, speed, and scalability.