Farhan tests an orders API for a quick-commerce startup in Hyderabad. His test calls the list endpoint, sees status 200, sees 50 orders, checks the first one’s fields, and passes. It has passed for a year. Last week a finance analyst exporting orders through the same API noticed her total was short by 12 orders. Nobody had ever looked at page five.
Let us use Farhan’s API to see what pagination can get wrong, and then his rate limiter, which has a similar blind spot.
The setup
Suppose the test account has 237 orders and the page size is 50. Pages one to four hold 50 each, which is 200, and page five holds the remaining 37. That arithmetic is the first thing to check: the total of all pages equals 237, with no gaps and no repeats. Farhan never summed anything.
The API has two common shapes. With offset pagination, the client asks for a page number or an offset and a limit. With cursor pagination, each response contains an opaque token pointing to the next chunk, and the client passes it back. The tests differ a little, so find out which one you have. Developers who want the reasoning behind the choice can read about offset versus keyset on this site; for a tester the interest is in how each one breaks.
What to try on the edges
- The last page. Does it return 37 items and signal there is no more, whether through a missing next link, a null cursor, or a has-more flag set to false? A common bug is a final empty page, or a next link that points forever to an empty result.
- Exact multiples. With 200 orders and page size 50, is there a fifth page that is empty? Boundaries again: 199, 200, 201.
- Beyond the end. Page 99. A sensible API returns an empty list or a clear error, not a server error.
- Bad parameters. Page zero, page negative, a size of zero, a size of 100,000, a size of “abc”. Is there a maximum page size, and is it enforced? An unbounded size is a performance hazard.
- Sort stability. Is the order deterministic? If sorting is by creation time and several orders share the same timestamp, rows can swap between pages and appear twice or vanish. The fix is a unique tiebreaker such as the id.
- Data changing while paging. The famous one. Fetch page one, then a new order arrives, then fetch page two. With offset paging, everything shifts by one and the last order of page one reappears on page two. Delete one instead and an order gets skipped. Cursor-based designs largely avoid this, which is a reason to prefer them, but you should still test it.
- Filters with paging. The total count and the pages must reflect the filter, and the filter must survive in the next link.
Farhan’s twelve missing orders turned out to be the sort-stability case. Orders created in the same second were being split across a page boundary inconsistently.
An automated check that earns its keep
Write one test that walks every page by following the next link until it ends, collects all ids, and asserts three things: the count equals the known total, there are no duplicate ids, and the set matches the ids you seeded. For this to work you need control over the test data, not just whatever the shared environment happens to hold. Seed 237 orders in setup, and clean them afterwards.
Now the rate limiter
The API promises 100 requests per minute per API key. Farhan’s team has a test that says “send a request, get 200”. It tells you nothing about the limit. What should the checks be?
Send 100 quick requests: all should succeed. The 101st within the same window should return status 429 Too Many Requests. The response should usually carry a Retry-After header, and many APIs also send headers showing the limit and the remaining count. Header names are not standardised across providers, so check your own API’s documentation instead of assuming. Then wait for the window to reset and confirm requests work again.
The subtle cases are the interesting ones. Is the limit per key, per user or per IP? Two different keys should not share one budget if the design says per key. Is it a fixed window, where a client can send 100 at 10:00:59 and 100 more at 10:01:00, or a sliding window or token bucket that smooths this out? You can design a test for that burst at the boundary. Does a rejected request still count against the limit? Does the limit apply to every endpoint or only some? Under parallel load from several clients, does the counter hold up, or can a race let 130 through?
Be practical about the environment. Triggering a real limit in a shared QA system can disrupt other teams, and time-based tests are fragile around window edges. Ideally the limit is configurable, so the test environment uses a limit of, say, five per ten seconds and the tests finish quickly. Use a dedicated key for these tests so you do not lock out colleagues.
Farhan added the page-walking test and a small limit-breaching test that evening. The first one failed immediately, which is exactly what it was for.
Figures and names in the examples are illustrative.
Leave a Reply