Imagine a personal loan eligibility screen with four inputs: age, monthly income, existing EMIs and credit score. A tester writes a script that fills the four fields, clicks Check, and compares the message with an expected result. Then the business adds twenty more scenarios. Copying the script twenty times would work for about a week, until someone renames the Check button and twenty-one scripts break at once. Data-driven testing is the simple idea that fixes this: keep one set of steps, and feed it a table of values.

The separation at the heart of it

Every check has two parts that change at different speeds. The procedure (open the form, enter values, submit, read the result) changes when the screen or API changes. The data (which values, and what outcome they should produce) changes when business rules change or when someone finds a case that was missed. Data-driven testing stores these apart so that each can change without touching the other.

For the loan screen, the data might be a table like this:

CaseAgeMonthly income (₹)Existing EMIs (₹)Credit scoreExpected
Minimum age accepted2140,0000760Eligible
Below minimum age2040,0000760Not eligible: age
EMI burden too high3550,00030,000780Not eligible: obligations
Score just below cut-off3580,0000699Not eligible: score

A single test method runs once per row. In TestNG this is usually a DataProvider; in JUnit 5 a ParameterizedTest with a source annotation; in pytest the parametrize marker; in Cucumber a Scenario Outline with an Examples table. The mechanics differ, but each runner reports every row as its own test result, so one bad row does not hide the others.

Notice the first column. Naming each row with its intent (“score just below cut-off”) turns a wall of numbers into documentation. When a row fails, the report says which rule broke, not just “iteration 4 failed”.

Choosing rows deliberately

Data-driven testing makes adding cases cheap, which tempts teams into adding hundreds of rows that test the same thing. Fifty rows with incomes between ₹40,000 and ₹90,000 and no other differences give almost no more confidence than one. The useful rows come from test design techniques: one representative per equivalence class, values on each side of every boundary (age 20 and 21, score 699 and 700 if the cut-off is 700), and combinations that trigger different rules. The table is where those techniques become executable, not a substitute for them.

A good test for any row: if you deleted it, would a realistic bug slip through? If not, it is noise that costs run time.

Where the data should live

Teams store test data in many places, and each has a cost.

  • Inline in the test code, as an array or annotation. Easiest to read alongside the test, versioned with the code, reviewed in pull requests. Best for small, stable tables.
  • CSV or JSON files in the repository. Still versioned and diffable, easy for non-programmers to edit, and fine for tables of dozens or hundreds of rows. JSON handles nested data, such as API request bodies, far better than CSV.
  • Excel workbooks. Familiar to business analysts and common in Indian service projects where clients supply scenarios in spreadsheets. The downside is that binary files do not diff cleanly in Git, so reviewing a change means opening the file and hunting. Reading them also adds a library such as Apache POI or openpyxl to the framework.
  • A database or test data service. Useful when data must be generated fresh, such as unique PAN-format IDs or accounts in a particular state. It adds a dependency that can itself fail.

There is no single right answer, but the default worth defending is: keep data next to the test, in a text format, until there is a concrete reason not to.

How data-driven suites go wrong

Logic creeping into the data. A column named “action” with values like “skip_otp” or “use_old_flow” means the test procedure now branches based on data. At that point you no longer have one procedure; you have several hidden inside one method, and a reader cannot tell what any row does without tracing the code. If rows need different steps, they are different tests.

Expected values computed by the test. If the test calculates the expected EMI using the same formula as the application, both can be wrong together and the test still passes. Expected results should come from an independent source: a worked example from the business, a spreadsheet signed off by the product owner, or a calculation done a different way.

Rows that depend on each other. If row 3 assumes row 2 created a customer, running rows in parallel or in a different order breaks the suite. Each row should set up what it needs.

Shared, mutable data. Pointing many rows at the same pre-existing account in a shared QA environment works until someone else’s test changes that account’s balance. Generating or resetting data per row is slower but far more reliable.

Huge tables nobody owns. A 3,000-row spreadsheet inherited from a previous vendor, with no one able to say why each row exists, is a liability. Prune it using the “would a bug slip through” question above.

Related ideas that are easy to confuse

Keyword-driven testing goes a step further and moves the steps themselves into a table, using verbs such as “Login”, “SearchProduct” or “VerifyText”. Robot Framework is a well-known example. It lets non-programmers assemble tests, at the cost of another abstraction layer to maintain.

Property-based testing also runs one procedure many times, but the inputs are generated randomly within rules, and the check is a general property (“the total never goes negative”) rather than a fixed expected value per row. It finds cases you did not think of; data-driven testing confirms the cases you did.

Used well, a data-driven test reads like a business rule table that happens to execute. Used badly, it is a script with a spreadsheet attached that nobody dares to change. The difference is almost entirely in how carefully the rows are chosen and named.

Written by the RP Strativa Team. The loan rules, cut-offs and figures in the examples are illustrative.

Leave a Reply

Trending