Networking & Cloud Computing

How Load Balancers Work: Layer 4 vs Layer 7, Algorithms and Health Checks

The traffic cop behind every busy website, and how it decides which server answers your request

Zarton EditorsNov 8, 2026 · 7 min read
Listen to this article

A popular website is never really one computer. Behind the single address you type sit a handful, or a few thousand, identical servers, and something has to decide which one answers each request. That traffic cop is the load balancer, and it is the reason a site can lose a server at 3 a.m. without anyone noticing.

This guide explains how load balancers work: where they sit, the difference between Layer 4 and Layer 7 balancing, the algorithms that pick a server, how health checks quietly remove broken machines, and the settings that trip up teams in production.

What a load balancer actually does

A load balancer accepts connections on behalf of a group of servers (often called a pool, upstream, backend or target group) and forwards each one to a member of that group. Clients only ever see the balancer's address. That simple indirection buys several things at once:

  • Scale. Add servers to the pool and capacity grows without changing anything clients see.
  • Availability. If a server fails its health checks, the balancer stops sending it traffic.
  • Safe deployments. You can drain one server, update it and put it back while the others carry the load.
  • A single front door. TLS certificates, request routing and security rules can live in one place.

Getting to the balancer itself usually starts with a DNS lookup that returns the balancer's address rather than any individual server's.

Layer 4 vs Layer 7: how load balancers work at each level

The "layer" refers to how much of the traffic the balancer reads before deciding where to send it.

Layer 4: transport-level balancing

A Layer 4 balancer looks only at connection details such as IP addresses, ports and protocol. It never parses the HTTP request, so it is fast and protocol-agnostic. AWS documents that its Network Load Balancer picks a target with a flow hash built from the protocol, source and destination IPs and ports, and the TCP sequence number, and then keeps each TCP connection on that one target for its whole life.

Layer 7: application-level balancing

A Layer 7 balancer terminates the connection, reads the HTTP request, and can route on the host name, path, headers or cookies. That lets it send /api to one pool and /images to another, add headers, and reuse backend connections. AWS's Application Load Balancer evaluates listener rules in priority order, adds X-Forwarded-For, X-Forwarded-Proto and X-Forwarded-Port headers so your app can see the real client, and can multiplex requests from many clients over one backend connection.

Layer 4 (transport)Layer 7 (application)
Decides usingIPs, ports, protocolHost, path, headers, cookies
Reads HTTP?NoYes, terminates and re-sends
Typical usesDatabases, game servers, raw TCP/UDP, very high throughputWebsites, APIs, microservice routing
TLSUsually passed through or terminated without inspectionTerminated so the request can be inspected
ExamplesAWS NLB, HAProxy mode tcp, NGINX streamAWS ALB, HAProxy mode http, NGINX http

The algorithms that choose a server

Cloudflare's learning center splits balancing algorithms into two families: static ones that ignore the servers' current state, and dynamic ones that look at it.

Static algorithms

  • Round robin. Hand requests to each server in turn. It is NGINX's default and AWS ALB's default. It works best when servers are similar and requests cost about the same.
  • Weighted round robin. Give bigger servers a larger share. In the NGINX docs, a server with weight=5 next to one with the default weight of 1 receives five of every six requests.
  • Hashing. Hash something about the request, such as the client IP or URL, so the same input always lands on the same server. HAProxy recommends balance uri for cache servers, because each URL keeps hitting the cache that already holds it.

Dynamic algorithms

  • Least connections. Send new work to the server with the fewest open connections. HAProxy's docs recommend it for long-lived sessions such as SQL, gRPC streams or LDAP, and caution that it fits short HTTP requests less well.
  • Least outstanding requests / least time. Prefer the server with the fewest requests in flight or the fastest recent responses. AWS ALB offers least_outstanding_requests; NGINX offers least_time.
  • Power of two choices. Pick two servers at random and use the less busy one. It is cheap, avoids herding every balancer onto the same "least loaded" server, and HAProxy made balance random (which uses two draws by default) its default algorithm starting with version 3.3.
upstream app_servers {
    least_conn;                          # dynamic: fewest active connections
    server app1.internal:8080 weight=2;  # twice the share of app2
    server app2.internal:8080 max_fails=3 fail_timeout=30s;
    server app3.internal:8080 backup;    # only used if the others are down
}

server {
    listen 80;
    location / {
        proxy_pass http://app_servers;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    }
}
An NGINX upstream that combines least connections, weights, passive health checks and a backup server.

Health checks: how broken servers disappear

An algorithm is only useful if it chooses among servers that work. Balancers find out in two ways.

  1. Passive checks watch real traffic. In NGINX open source, if a server fails max_fails times within fail_timeout, it is marked unavailable for a while.
  2. Active checks probe each server on a schedule, typically by requesting a path like /health. AWS ALB's defaults are an HTTP request to / every 30 seconds with a 5-second timeout, expecting a 200. A target needs 5 consecutive successes to be marked healthy and 2 failures to be marked unhealthy.

Make your health endpoint honest. It should check that the app can actually serve requests, but avoid making it fail just because one optional downstream service is slow, or a single hiccup can pull every server out at once.

Production details that bite

Sticky sessions

Some apps keep login state in server memory, so a user must keep returning to the same machine. Balancers support this with affinity cookies or IP hashing. AWS ALB stickiness is off by default and uses either a balancer-generated or an application cookie. It works, but it unbalances load and breaks when that server dies, so storing sessions in a shared database or cache is usually the better fix.

Draining and slow start

When you remove a server, in-flight requests need time to finish. AWS calls this the deregistration delay and sets it to 300 seconds by default. The reverse problem is a freshly started server getting a flood of traffic before its caches warm up; ALB's optional slow start ramps a new target's share up linearly over 30 to 900 seconds.

Long-lived connections

Round robin balances connections, not work. A server holding thousands of WebSocket connections may look idle by request count while it is actually the busiest box in the pool. Least connections is often the better choice there, and a per-client rate limit in front protects the whole pool from one noisy client.

Key takeaways

  • A load balancer hides a pool of servers behind one address and spreads connections across them.
  • Layer 4 balancers route on IPs and ports; Layer 7 balancers read HTTP and can route on paths, hosts and headers.
  • Round robin is the common default; least connections and power-of-two-choices adapt to real load.
  • Health checks, passive and active, are what make balancing improve availability.
  • Plan for draining, slow start and long-lived connections before they surprise you.

Frequently asked questions

What is the difference between a load balancer and a reverse proxy?

A reverse proxy sits in front of servers and forwards requests to them. A load balancer is a reverse proxy, or a network device, whose main job is spreading traffic across several servers. Tools like NGINX and HAProxy do both.

Should I use a Layer 4 or Layer 7 load balancer?

Use Layer 7 for websites and APIs when you want routing by path or host, header handling and HTTP-aware health checks. Use Layer 4 for non-HTTP protocols or when you need raw throughput and minimal processing.

Which load balancing algorithm is best?

Round robin is fine when servers and requests are uniform. Choose least connections for long-lived connections, hashing when the same client or URL should hit the same server, and power of two choices for large, frequently changing pools.

Is the load balancer itself a single point of failure?

It can be. Managed cloud balancers run redundantly across availability zones, and self-hosted setups typically run two or more balancer nodes with a floating IP or DNS failover between them.

Sources

Comments

No comments yet — be the first to share your thoughts.

Join the discussion

Comment

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

Keep reading

All posts

Run your whole business from one conversation.

Get startedTalk to us