Load balancers and reverse proxies
One hostname is not one machine. Behind www.example.com there is a tier of caches, a
tier of balancers, a tier of proxies, several application servers, and — almost always —
one database that cannot simply be cloned.
Balancing, at two different layers
A load balancer spreads incoming connections across a pool and stops sending traffic to the sick ones. It can work at either of two layers, and the difference decides what it is capable of:
| Layer 4 | Layer 7 | |
|---|---|---|
| Sees | TCP connections | HTTP requests |
| Can route on | Address and port | Path, host, headers, cookies |
| Terminates TLS | No | Yes |
| Cost per connection | Very low | Higher, and it can do far more |
Health checks are the half people forget, and they are what turns a crashed server into a non-event: the balancer keeps asking each backend whether it is well, and quietly removes the ones that stop answering.
Reverse proxy: the same box, wearing a different hat
A reverse proxy accepts requests on behalf of the real application and forwards them on. The client believes it is talking to the site; the application never sees the raw Internet.
That position is so convenient that one process usually ends up doing all of it:
terminating TLS, caching, compressing, rate limiting, rewriting paths, and adding the
Forwarded header that tells the application who the real client was — because from the
application's point of view, every request now comes from the proxy.
Where TLS ends, and how many times
Something worth tracing in the diagram: TLS is often terminated at the edge, re-encrypted to the load balancer, and terminated again at the proxy. Each hop is a place the traffic exists in plaintext, inside somebody's trust boundary — which is fine when that boundary is yours, and is exactly the question to ask when it is not.
Key takeaways
- One hostname is a stack of tiers: edge cache, balancer, reverse proxy, application servers, and a data tier.
- A layer-4 balancer forwards connections; a layer-7 balancer reads the HTTP request and can route on its contents.
- Health checks are what make a crashed backend a non-event, and they are the part most often left unconfigured.
- A reverse proxy is the natural place for TLS, caching, compression and rate limiting — which is why one process usually does all five.
- Stateless tiers scale by adding copies. The data tier does not, which is why doubling application servers so often changes nothing.