It is 9:10 on a Tuesday morning in Bengaluru. Kavya, a QA engineer on an invoicing product, opens the dashboard and sees the nightly regression: 412 tests run, 375 passed, 37 failed. Her manager has already pinged: “Is the build good?” Thirty-seven red lines. The tempting reply is “looking into it”. The better move is to spend ten minutes sorting before replying, and here is how that ten minutes goes.

Minute one to three: group before you read

Kavya does not open failure number one. She groups all 37 by error message and by the stage where they failed. Reading them one by one is how an hour disappears; grouping turns 37 problems into a handful.

Three clusters appear. Thirty tests fail with a connection timeout while setting up data, before the test even reaches the application. Five fail with “element not found” on the same button. Two fail with an assertion: the invoice total expected 11,800 and the actual was 11,500.

30 + 5 + 2 = 37. Every failure belongs to a cluster, which is itself reassuring. A failure that fits no cluster is the one to read first.

Three questions that separate the causes

For each cluster she asks the same short questions, in this order.

First: did it fail before the test touched the product? Errors during setup, login, database connection, network or a missing test account point at the environment or the framework, not at the feature under test. Second: when did it start? Comparing with the previous night’s run, and with the time of the last deployment, narrows suspects fast. Third: does the evidence show the application misbehaving, or only the test failing to find what it expected? A screenshot or response body settles this in seconds if the framework captured one.

Cluster one: thirty timeouts

All thirty fail during data setup, and all started at 2:47 a.m. Kavya looks at the environment’s health page, then the infrastructure channel. The test database was restarted for a patch window at that time and came back slowly. The tests did not run against a broken product; they never reached it. She reruns two of them manually now: both pass. Verdict: environment, owner is the platform team, action is to rerun the whole cluster after confirming the database is stable.

Note what she did not do. She did not add a retry to the framework to “make it green” and move on. Retrying can be a legitimate tool for known transient issues, but a retry added in frustration hides the next outage from everybody.

Cluster two: five missing buttons

Five tests cannot find the element with the identifier “btn-download”. The screenshot shows the invoice screen, fully loaded, with a button labelled Download PDF sitting exactly where it should. She checks the commit history: a developer renamed the identifier yesterday as part of a UI clean-up. The product works; the locator is stale. Verdict: script issue, owner is the test team. She fixes the locator in one place because the tests share a page object, and the cluster is cleared with one change. This is why centralised locators pay off.

One more thought here: should the developer have told QA about the rename? Perhaps. That is a conversation about process, to be had kindly and without blame, perhaps with an agreement to use stable test attributes on elements that automation depends on.

Cluster three: the two wrong totals

Now the one that matters. Both failing tests create an invoice for ₹10,000 of goods with 18 percent GST, which should give 10,000 + 1,800 = ₹11,800. The application shows ₹11,500, which is 10,000 plus 15 percent. The failure appears in the response body, not just on screen, so it is not a display glitch. Both tests use a customer in a particular state with a special tax setting. Previous night, they passed. A tax-configuration change went in yesterday afternoon.

Everything here points to the product: the setup worked, the element was found, the expected value comes from the tax rules, and the actual value is wrong in the API itself. She still checks her own expectation: she confirms with the requirement that 18 percent applies to this item category, so she is not asserting something that changed legitimately. Verdict: product defect. She writes it up with the two test names, the input, expected and actual figures, the build number and the suspect change, and assigns it to the developer who owns tax rules. Severity is high, since wrong tax on invoices is a compliance problem.

The reply she sends at 9:20

“Of 37 failures: 30 environment (database restart overnight, rerun pending), 5 script (locator change, fixed), 2 genuine product defect in GST calculation for one customer type, raised as a high-severity defect. Build is not releasable until the GST defect is resolved.” That message is useful because it is specific, it names owners, and it gives the manager a decision.

Making the routine cheaper next time

The routine can be partly automated. Have the framework attach a screenshot, page source or response body, and timestamps on failure. Tag failures with a category when they match known patterns, such as connection errors during setup. Run an environment health check before the suite starts and abort with one clear message if it fails, instead of generating thirty confusing ones. And keep a running note of recurring causes: if the same cluster appears every Tuesday after the patch window, the fix is to change the schedule, not to get better at triage.

Figures and names in the examples are illustrative.

Leave a Reply

Trending