How Load Balancers Decide Where Your Request Goes

A load balancer spreads requests across interchangeable backends, notices when one dies, and stops sending traffic to it. Most people accept whatever default their provider set, which holds up until request durations get uneven or a backend dies at 2am.

Round robin cycles through the pool and assumes every request costs the same and every server is equally strong. Least connections picks whoever has the fewest open connections, so a box stuck on slow work naturally stops receiving new requests. Consistent hashing exists because naive hash(ip) % N reshuffles nearly every client when the pool size changes, wrecking cache locality. Random with two choices samples two backends and takes the less loaded one, approximating least-connections without any shared state. L4 balancing forwards by IP and port without reading the payload; L7 parses HTTP and can route on path, host, or cookie.

Health checks are the sharp edge: one covering a shared dependency can fail every backend at once and turn a partial outage into a total one.