Test data tool PoC: a reusable acceptance checklist

Editorial review: 2026-10-11 · Sources and scope below

Download the PoC brief (.md)

What must the example contain?

Create four customer IDs, 1–4, and six payment IDs, 101–106. The table below fixes customer, status and amount in integer cents. Customer 4 deliberately has no payment. Reporting must select only settled payments worth at least 10,000 cents. Expected IDs are exactly 101, 104 and 106, with a total of 60,000 cents. The settled payment 103 is deliberately below the boundary; pending and rejected payments must stay excluded.

A small fixture with an exact expected result

The customers table must contain IDs 1, 2, 3 and 4. Customer 4 deliberately has no payment. Use this query against the generated payments table; inspect relationships separately.

SELECT payment_id FROM payments
WHERE status = 'settled' AND amount_cents >= 10000
ORDER BY payment_id;

Expected IDs: 101, 104, 106. Total amount: 60,000 cents.

What counts as a pass?

Require four distinct customer IDs, six distinct payment IDs, no orphan payments and exact agreement with every scenario row. Check the reporting IDs and total independently. Row count alone is insufficient: swapping a settled and rejected status can preserve six rows while changing the business result. A generator log that reports success is supporting execution evidence, not the business oracle.

How do you compare the real effort?

Use a fresh target for each candidate. Record exact product and edition, model or script, runtime versions, manual edits and time to first correct result. Separately record a changed business rule, failed-run recovery and a second engineer repeating the setup. Keep an unsupported or untested requirement visible rather than turning it into an invented zero score.

What should you add for your own system?

Replace this small schema with one representative part of your system after the basic check works. Add your connector, composite keys, target cleanup and required error cases. If source data is involved, add permitted selection and transformation assertions. For multi-system delivery, check each target and cross-system identifiers. This fixture does not benchmark any product or establish privacy, load capacity or protocol conformance.

Executed customer-label fixture

The local verification harness ran Faker, DATAMIMIC CE and SDV on 11 October 2026 in Linux with Python 3.12. Each product supplies four customer labels. The common Python adapter assigns IDs and the six fixed payment rows. SQLite checks this assembled fixture. The products do not build its relationships.

Faker uses seed_instance(42); CE person/v1 uses seed 42 and a fixed clock. SDV fits a GaussianCopulaSynthesizer on 256 synthetic-only label/score rows. Only its label column enters the fixture. There is no production data, privacy score or distribution-quality evaluation. Cleanup and rollback are checked by the neutral SQLite script.

MOSTLY AI SDK 6.1.5 with LOCAL dependencies was installed without an account. Its actual training attempt was blocked when the environment denied AF_UNIX socket creation (Operation not permitted). No generated output or failed business result is attributed to the product.

This does not test product-managed relationships, Enterprise editions, database connectors, subsetting, privacy, throughput or portability. The exact tested versions are not described as the latest releases.

Download scripts, dependency lock and results (.zip) · Read the run record (.json)

python run_fixture.py
python3.12 -m venv .venv
.venv/bin/python -m pip install -r requirements-poc.lock
.venv/bin/python compare_generators.py --model-dir .fixture-model --output results-local.json

Illustrative scenarios and editorial acceptance criteria; not measured product benchmarks.

Choose your next step

• Define requirements and a proof of concept
• Reproducible test data for CI/CD: what to freeze and verify
• Open-source test data tools: Faker, DATAMIMIC CE and alternatives
• Compare the documented capabilities of your shortlist