A logistics company’s tracking service receives GPS pings from delivery vehicles and writes them to a database. On a normal afternoon it receives about 1,000 pings a second and processes about 1,000 a second. During the festive-season rush, traffic rises to 1,200 a second while processing stays at 1,000. The difference looks small. It is 200 pings a second piling up somewhere: 720,000 an hour. If each ping takes around 2 KB in memory, that is roughly 1.4 GB of extra backlog every hour. Somewhere a buffer grows until the process runs out of memory, or latency climbs until a “live” map shows positions from forty minutes ago.

Every pipeline where one stage feeds another has this problem waiting for it. Backpressure is the general name for mechanisms that let a slow consumer push back on a fast producer, so the mismatch is handled by design rather than by a crash.

The only four options

When input persistently exceeds capacity, there are only a few things a system can do. Every real design is some combination of them.

  1. Buffer. Store the excess and process it later. This works for bursts: a five-minute spike followed by a lull drains naturally. It does not work for a sustained mismatch, because a buffer only delays the moment of reckoning.
  2. Slow the producer down. Tell it to wait, block its writes, or reject its requests so it retries later. This is backpressure in the strict sense: the pressure propagates upstream.
  3. Drop work. Discard some input deliberately, ideally the least valuable. This is load shedding.
  4. Increase capacity. Add consumers, batch writes, or make processing cheaper. This is the real fix for a sustained mismatch, but it takes time, so you still need one of the first three while it happens.

The dangerous design is the one that makes none of these choices explicitly. An unbounded in-memory queue is a decision to buffer forever, which in practice means a decision to crash later. In Java, for example, the fixed-size thread pools created through the Executors factory methods use an unbounded queue by default, so a slow pool quietly accumulates tasks rather than refusing them.

Backpressure is already everywhere

TCP has had it for decades. The receiver advertises a window: how much data it is willing to accept. If the receiving application stops reading, its buffer fills, the advertised window shrinks to zero, and the sender stops sending. Nobody has to write code for this; it is why a slow download does not overflow your machine’s memory.

Many programming models make it explicit. In Node.js streams, writing to a stream returns false when its internal buffer exceeds a threshold (the highWaterMark), and a well-behaved producer pauses until the stream emits a drain event. Piping streams together handles this automatically, which is one reason piping is preferred over hand-written read-and-write loops. In Go, sending on a full buffered channel blocks the sender until a receiver takes something off. The Reactive Streams specification, implemented by libraries such as Project Reactor and RxJava, is built around the subscriber requesting a specific number of items, so the publisher never sends more than was asked for.

Between services, the mechanism is usually rejection. An overloaded API returns HTTP 429 Too Many Requests or 503 Service Unavailable, optionally with a Retry-After header. Clients that respect it, with exponential backoff, effectively slow down. Clients that ignore it and retry immediately make the overload worse.

Pull-based queues change the shape of the problem

Putting a durable queue or log such as Kafka or SQS between producer and consumer is a form of buffering, but a well-behaved one. Consumers pull at their own pace, so they cannot be overwhelmed, and the backlog lives on disk rather than in a process’s memory. For the GPS example, writing pings to Kafka means the tracking service simply falls behind during the rush, and the growing consumer lag becomes a measurable signal.

The buffer is still finite, though. Kafka retains data for a configured period; if consumers fall behind by more than that, the oldest unconsumed data is deleted. And a large lag means stale results, which for a live tracking map may be as bad as no results. A queue converts a memory problem into a latency problem. Whether that is acceptable depends on what the data is for.

Choosing what to drop

Load shedding sounds alarming, but for some data it is clearly the right answer. For vehicle positions, only the latest ping per vehicle matters for the live map. Under pressure the service can keep only the newest ping for each vehicle and discard older ones still waiting, a technique sometimes called conflation. Ten thousand vehicles then produce at most ten thousand pending updates, however far behind the service is. Metrics pipelines often sample or pre-aggregate under load for the same reason.

For other data, dropping is unacceptable. Payment events, orders and audit logs must not be shed; they must be buffered durably and the producer slowed if necessary. In mixed systems, prioritisation helps: under overload an API might keep serving checkout requests while rejecting recommendation requests, so the parts of the system that earn money keep working.

Design rules worth keeping

  • Bound every queue. In-memory queues, thread pool queues and connection pools should have a maximum size and a defined behaviour when full: block, reject, or drop oldest.
  • Measure the gap, not just the throughput. Queue depth, consumer lag and the age of the oldest unprocessed item reveal a mismatch long before memory or latency alarms do.
  • Make rejection cheap. Refusing work should cost far less than doing it. An overloaded service that spends significant effort on requests it then rejects can still collapse.
  • Propagate the signal. Backpressure only works if each stage passes it upstream. A middle service that buffers unboundedly breaks the chain.

Underneath all of this is a simple piece of arithmetic: if arrivals exceed service rate for long enough, any buffer fills. Backpressure is how you decide, in advance and on purpose, what happens next.

Written by the RP Strativa Team. Ping rates, sizes and the logistics scenario are illustrative.

Leave a Reply

Trending