testdatatools
Practical decision guide

Test data tool PoC: a reusable acceptance checklist

Give shortlisted tools the same small business scenario and acceptance criteria before comparing speed. This payments fixture catches a common failure: a successful generation job that still returns the wrong business rows. Download the brief and use it with any candidate.

Editorial review: · Sources and scope below

Download the PoC brief (.md)

What must the example contain?

Create four customer IDs, 1–4, and six payment IDs, 101–106. The table below fixes customer, status and amount in integer cents. Customer 4 deliberately has no payment. Reporting must select only settled payments worth at least 10,000 cents. Expected IDs are exactly 101, 104 and 106, with a total of 60,000 cents. The settled payment 103 is deliberately below the boundary; pending and rejected payments must stay excluded.

A small fixture with an exact expected result

Illustrative payments; amounts in integer cents
payment_idcustomer_idstatusamount_cents
1011settled10000
1021rejected50000
1032settled9999
1042settled20000
1053pending40000
1063settled30000

The customers table must contain IDs 1, 2, 3 and 4. Customer 4 deliberately has no payment. Use this query against the generated payments table; inspect relationships separately.

SELECT payment_id FROM payments
WHERE status = 'settled' AND amount_cents >= 10000
ORDER BY payment_id;

Expected IDs: 101, 104, 106. Total amount: 60,000 cents.

What counts as a pass?

Require four distinct customer IDs, six distinct payment IDs, no orphan payments and exact agreement with every scenario row. Check the reporting IDs and total independently. Row count alone is insufficient: swapping a settled and rejected status can preserve six rows while changing the business result. A generator log that reports success is supporting execution evidence, not the business oracle.

How do you compare the real effort?

Use a fresh target for each candidate. Record exact product and edition, model or script, runtime versions, manual edits and time to first correct result. Separately record a changed business rule, failed-run recovery and a second engineer repeating the setup. Keep an unsupported or untested requirement visible rather than turning it into an invented zero score.

What should you add for your own system?

Replace this small schema with one representative part of your system after the basic check works. Add your connector, composite keys, target cleanup and required error cases. If source data is involved, add permitted selection and transformation assertions. For multi-system delivery, check each target and cross-system identifiers. This fixture does not benchmark any product or establish privacy, load capacity or protocol conformance.

Illustrative scenarios and editorial acceptance criteria; not measured product benchmarks.

Choose your next step