An API is a promise to people whose code you cannot see. Change it carelessly and an app on someone’s phone, a partner’s billing job or a mobile build that users have not updated in eight months stops working, with no error on your side at all. Testing backward compatibility is the discipline of checking that promise before you ship. There are two broad ways to evolve an API, and they ask different things of a tester. I will compare them and then say which I prefer.
What counts as breaking
Start with a definition, because it is looser than people expect. A change is breaking if a client written against the old behaviour can fail without being modified. The obvious cases: removing a field, renaming a field, changing a type (a number becoming a string), making an optional request field mandatory, tightening validation so previously accepted input is rejected, changing an error code or status, and changing the meaning of a value while keeping its name.
The less obvious ones are where the bugs hide. Adding a new value to an enumeration, such as a new order status “PARTIALLY_SHIPPED”, can break a client with a switch statement that does not expect it. Changing the sort order of results, the default page size, or the precision of a decimal can all break someone. Even response time can: a client with a two-second timeout is broken by a call that now takes three.
Approach one: explicit versions
Here the version is part of the address or a header, so old and new live side by side, for example a version 1 and a version 2 of the same endpoint. A breaking change means a new version number, and clients migrate when they are ready.
For testers the appeal is clarity. Each version has its own contract, and you can keep the old suite running against the old version untouched. The cost is multiplication. Two live versions mean two sets of tests, shared code that must not leak new behaviour into the old version, and a bug fix that must be checked in both. Three live versions is three times the surface. The danger is accidental coupling: a developer fixes something in a shared function for version 2, and version 1 quietly changes. So the most valuable test here is a frozen regression suite for each old version, run on every build, with expected results that nobody is allowed to casually update.
Approach two: additive change, one live version
Here there is effectively one API that only grows. New fields are added, never removed or repurposed. Clients are expected to ignore what they do not recognise, an idea often called the tolerant reader. Features are retired through a long, announced deprecation period.
The upside is a single codebase and a single set of tests. The downside is discipline: nothing but review and tests stops someone from sneaking in a removal. So the key test becomes a compatibility check against a stored description of the contract. If the API publishes an OpenAPI document, a tool can compare the old and new descriptions and flag removed fields, changed types and newly required parameters automatically in the pipeline. Several such diff tools exist. They catch structural breaks but not behavioural ones, such as changed meaning, so they supplement tests and do not replace them.
The clients you will never meet
Whichever approach, the hard part is clients you do not control. Mobile apps are the classic case. When a bank in Lucknow releases a new app build, a share of customers will run the old build for months, and force-update screens only help so much. The backend must serve both.
The practical test: keep recorded sample requests from each supported client version, and replay them against the new backend on every release. Check responses against what that version expects. Better still, run the actual old app builds in the test environment for key journeys. Where consumers are known services, consumer-driven contract testing makes each consumer’s expectations explicit and checks the provider against them before release.
Side effects deserve their own attention
Compatibility is not only about the wire format. A new version often changes stored data, for instance adding a mandatory column or a new status. Check that records created by the old version can still be read and updated by the new one, and the reverse during a rolling deploy or rollback, when both versions run against the same database for a while. Test the rollback path too: can you go back without corrupting what the new version wrote?
Where I land
For most internal and partner APIs, I lean towards additive change with a deprecation policy, and an explicit new version only when a real break is unavoidable. Multiplying live versions multiplies testing and support forever, and in my experience old versions are the ones nobody retests properly. But I would flip the answer for a public API with thousands of unknown consumers, or in a regulated setting where a client cannot upgrade on your timetable. There, explicit versions with a published sunset date are the honest choice.
Either way, the testing question is the same one: what does an old client expect, and have we checked it against what we are about to ship?
Figures and names in the examples are illustrative.
Leave a Reply