Two ways of giving testers somewhere to test. In the first, the company keeps one QA environment, lives with it for years and lets every team deploy into it. In the second, every pull request or test run gets its own freshly built copy of the application and its dependencies, which is thrown away afterwards. I will argue for the second as the default for application-level testing, and then explain why you still need a version of the first.

What the shared environment does well

Do not dismiss it; it exists for reasons. It is simple to understand: everyone knows the URL. It is cheap in the sense that there is one set of servers to pay for. It holds data that has accumulated over time, which is useful because real systems rarely run on pristine data. And it connects to everything: the payment sandbox, the SMS gateway, the other teams’ services, because someone spent weeks wiring that up. When a test needs “the whole company’s software together”, this is often the only place that has it.

What it does to a team over time

The costs are less visible and larger. Suppose five squads share one QA environment. Squad A deploys a half-finished build at 11 a.m. Squad B’s regression suite, started at 10:45, begins failing, and B’s engineers spend the afternoon investigating failures caused by A. A tester refreshes a database to get clean data and wipes out another squad’s prepared test accounts. Somebody changes a configuration flag “just for a quick check” and forgets. Within months, nobody can say with confidence what version is deployed or what state the data is in, so a failing test is ambiguous: is it the product, the script, or the environment?

That ambiguity is the real cost. Teams respond with booking spreadsheets, deployment freezes and “QA is down” messages in chat. Queuing for an environment is waiting time, and waiting time is invisible in most dashboards. Tests also become order-dependent and flaky because the data underneath them drifts.

What an ephemeral environment changes

On-demand environments are created by the pipeline from a definition kept in version control: containers or a Kubernetes namespace, infrastructure scripts, a seed data set. A pull request triggers a build, the environment appears, tests run against it, and it is destroyed when the PR merges or after a time limit. The key properties are:

  • Known state. The build is exactly your branch, the data is exactly the seed. A red test is far more likely to be about your change.
  • Isolation. Nobody else’s deployment can break your run, and your tests cannot damage anyone’s data, so parallel work stops colliding.
  • Reproducibility. If a bug is found, anyone can recreate the same environment from the same definition. “Works on my machine” shrinks to “works in this definition”.
  • The definition is code, so changes to the environment are reviewed, versioned and visible.

Cost, counter-intuitively, is often not worse. A shared environment runs 24 hours a day, 7 days a week, even though it is used mainly for about 10 hours on weekdays. Ephemeral environments exist only while they are being used. An environment that lives for two hours per pull request, for forty PRs a week, runs for 80 environment-hours, though it is a smaller stack and several may overlap. The comparison depends on how heavy the stack is, so measure rather than assume.

Where on-demand environments struggle

Honesty matters here, because this is not a free lunch.

The hard part is dependencies. Your service talks to twelve others, a mainframe adapter and a licensed third-party product that costs money per instance. You cannot spin up everything per branch. The usual answer is to run your own service for real and replace the rest with stubs, mocks or contract-verified fakes, which means the environment tests your service in isolation, not the whole chain. Data is the second problem: a seed set that is small and clean is quick to load but will not show the strange cases that years of real usage produce. Third, build time. If standing up the environment takes 25 minutes, developers will resent it, so invest in caching, small images and parallel start-up. Fourth, the skills: someone must own the infrastructure definitions, and that is real engineering effort rather than a spare afternoon.

Taking a side

My position, stated as opinion: make ephemeral environments the default for functional, API and UI tests at the level of one team’s service, and keep exactly one stable, well-governed shared environment for the checks only it can do. That second environment is for end-to-end flows across teams, integration with real partner sandboxes, performance rehearsals and user acceptance testing. The important word is governed. It has a named owner, deployments happen through the pipeline only, versions are visible on a status page, and nobody pushes experimental builds into it.

Why this split? Because the two answer different questions. The ephemeral one asks “does this change work, in isolation, with known data?” and should answer quickly and reliably on every change. The shared one asks “do all the pieces still work together?” and can afford to answer slowly, a few times a day.

If you are starting from a single shared QA, you need not rebuild everything. Begin with one service that is easy to containerise, give it a seed script and per-PR deployment, and watch how many “is it the environment?” conversations disappear. Count those conversations before and after. If the number does not fall, you have learned something too.

Figures and names in the examples are illustrative.

Leave a Reply

Trending