There is a belief that performance testing is the activity of firing lots of users at a system and checking that it stays fast. Put that way, it is not wrong. It is just one quarter of the picture, and the other three quarters are where the embarrassing production incidents come from.
The names get used loosely, so it helps to define each by the question it answers. Same system, same tools (JMeter, k6, Gatling and others all do these), different shape of traffic over time.
Load test: can we carry the traffic we expect?
You apply the realistic peak, hold it steady for long enough to reach a stable state, and compare response times, error rates and resource usage with agreed targets. A ticket-booking site expecting 3,000 concurrent users on a normal Saturday evening runs 3,000, maybe with a margin on top. The question is “does it meet its service levels at the load we planned for?”. If you have never agreed what “meet” means, such as 95th percentile under two seconds, error rate under one percent, you can run this test forever and not know whether it passed.
Stress test: where does it break, and how?
Now go past the plan. Ramp up in steps, 3,000, 4,500, 6,000, 9,000, until something gives. The headline number, the load at which latency goes off a cliff, is useful for capacity planning, but the more valuable finding is the manner of failure. Does the system slow down gracefully and return clear “busy” errors? Or do threads pile up, the database connection pool runs dry, and the whole thing stops answering even health checks? And after you cut the load back, does it recover by itself, or does somebody have to restart servers at 2 a.m.? A system that fails politely and recovers alone is in far better shape than one that is merely a bit faster but collapses messily.
Spike test: what happens when it arrives all at once?
Real traffic is not a smooth ramp. A flash sale opens at noon, an IPL match ends and everyone reopens the app, a push notification goes to two million phones. A spike test jumps from low load to a high one within seconds, and then often drops back. It exposes things steady tests cannot: autoscaling that needs several minutes to add capacity while users are already failing, caches that are empty at the start, connection pools that open connections slowly, a login service that every request depends on. It also shows whether queues drain afterwards. Test the drop as well as the jump; some systems have trouble scaling down cleanly.
Soak test: does it still work after a long time?
A soak, or endurance, test applies a moderate and realistic load for hours, sometimes a day or more. Time itself is the variable. Slow problems only show here: memory leaks, file handles or connections not released, log files filling a disk, a daily batch job colliding with peak traffic, database tables growing until a query plan changes.
A small example with illustrative numbers. A service has a 2 GB heap and normally uses 1.5 GB, leaving 0.5 GB (about 500 MB) of headroom. A leak of 50 MB per hour sounds trivial, and a 30-minute load test will never see it. The headroom is gone after 500 / 50 = 10 hours, at which point garbage collection thrashes and then the process fails with an out-of-memory error. In production, with a deployment every fortnight, that means an outage every fortnight at an odd hour. The only way to see it before release is a test long enough to watch the memory trend, so graph used memory after each garbage collection and look at the slope, not the peaks.
How they fit together
| Test | Traffic shape | Duration | Typical finding |
|---|---|---|---|
| Load | Expected peak, steady | 30 to 60 minutes | Misses the agreed response time or error target |
| Stress | Stepped increase beyond plan | Until failure, then recovery | Breaking point, failure mode, recovery behaviour |
| Spike | Sudden jump, then drop | Minutes | Slow autoscaling, empty caches, pool exhaustion |
| Soak | Moderate, steady | Many hours | Leaks, growth, scheduled jobs colliding with traffic |
You will see other terms: scalability testing (how does capacity change as you add servers?), volume testing (how does it behave with a huge amount of data?), capacity testing. They are variations on the same idea of choosing what to vary and what to hold still.
Things that spoil all four
A test environment much smaller than production, with a database holding 10,000 rows instead of 50 million, gives numbers that mean little. Either make it comparable or say clearly what the result does and does not predict. The load generator itself can be the bottleneck; watch its CPU and network. Test scripts that use one user and one search term hit caches unrealistically, so vary the data. And third-party dependencies like a payment gateway should be simulated, with realistic delays, rather than hammered, since you probably do not have permission to load-test someone else’s service.
Not every system needs all four on every release. A back-office tool used by thirty people may need a quick load check and nothing else. But the choice should be deliberate. “We ran a load test” answers one question; before launch day, ask which of the other three your system’s real life is likely to pose.
Figures and names in the examples are illustrative.
Leave a Reply