Rather than define a load balancer in the abstract, let us follow one request all the way through. The setting is an imaginary online shop, shop.example.in, running on three identical application servers behind a load balancer. A customer in Pune taps a product link.

Hop one: the name becomes an address

The phone asks DNS for the address of shop.example.in. The answer is not the address of any application server. It is the address of the load balancer, the single public door. This is the first benefit and the one people forget: clients know one address, while the number of servers behind it can change freely. Add a fourth server tomorrow and no customer needs to know.

Hop two: the door picks a server

The phone connects to the load balancer. Now it has to decide which of the three servers will handle this. Common strategies:

  • Round robin: server 1, then 2, then 3, then back to 1. Simple and fair when requests cost about the same.
  • Least connections: send the request to whichever server currently has the fewest open connections. Better when some requests are slow and some are quick.
  • Weighted variants: if one server has twice the CPU, give it twice the share.
  • Hashing on something like the client address, so the same client tends to reach the same server.

Here is a worked number. Suppose requests take 100 ms on average, but one in ten is a heavy report taking 2 seconds. Under pure round robin, a server can by bad luck collect several heavy ones in a row while another sits idle. Least connections notices the pile-up, because the busy server holds more open connections, and steers new work elsewhere.

Layer 4 or layer 7: how much does the door look at?

The “layers” refer to the OSI networking model. A layer 4 balancer works at the transport level. It sees IP addresses and ports, and forwards TCP connections without understanding what is inside. It is fast and simple, and works for any protocol, not only web traffic.

A layer 7 balancer works at the application level, for HTTP in our case. It reads the request: the path, the headers, cookies. That unlocks smarter behaviour. Requests for /api can go to one group of servers and /images to another. It can add or rewrite headers, compress responses, and route a small percentage of traffic to a new version for testing. The price is more CPU per request and more complexity, since it must understand and sometimes terminate the protocol.

Hop three: who handles the encryption?

The shop uses HTTPS. A layer 7 balancer will typically perform TLS termination: it holds the certificate, decrypts the traffic, and forwards plain or re-encrypted HTTP to the servers. This means the certificate lives in one place instead of three, and the servers are spared the cryptographic work. If your compliance rules demand encryption all the way to the application, you configure the balancer to re-encrypt on the inside, or use layer 4 pass-through so the balancer never sees the plaintext.

One side effect catches many teams. The application server now sees the balancer’s address as the client. The real customer address arrives in a header, conventionally X-Forwarded-For. Logging, rate limiting and geolocation all need to read that header, and only trust it when it was set by your own balancer.

Hop four: the response, and what the balancer was doing meanwhile

Server 2 builds the page and the response returns the same way, through the balancer. While this was happening, the balancer was also running health checks. Every few seconds it calls something like a /health endpoint on each server. If a server fails, say, three checks in a row, it is taken out of rotation; once it passes a couple of times, it is put back. That is how a crashed server disappears from the customer’s view without anyone doing anything.

The quality of the health check matters. A check that only confirms the process is running will keep sending traffic to a server whose database connection is dead. A check that tests every downstream dependency can take all servers out at once when the shared database blips. Most teams settle on a check that confirms the server can do useful work, while keeping deeper dependency checks for monitoring.

The customer’s second request

The customer adds the product to a cart. This request may land on server 3. If server 1 had kept the cart in its own memory, the cart appears empty. Two fixes exist. Sticky sessions make the balancer send the same user to the same server, typically through a cookie, which is quick to set up but uneven (one server collects a heavy user) and loses the session if that server dies. The better long-term answer is to make the servers stateless, keeping carts and sessions in a shared store, so any server can serve any request.

The door itself can fail

A load balancer removes the individual servers as single points of failure, but it can become one itself. Production setups therefore run balancers in pairs or as managed cloud services that spread across zones, and DNS or anycast addressing directs traffic to healthy ones. Cloud offerings such as the application and network load balancers from the large providers map neatly onto the layer 7 and layer 4 split described above.

Figures and names in the examples are illustrative.

Leave a Reply

Trending