Every modernization status deck eventually shows the same slide: a test suite, all green, hundreds or thousands of cases passing. It reads as proof. Mostly it isn't. A test suite that has never once failed hasn't demonstrated that the conversion is correct — it has only demonstrated that nothing in the suite has yet been wrong in a way the suite would notice. Those are different claims, and the gap between them is exactly where a negative control belongs.
What a negative control is
Borrowed from lab science, a negative control is a case you already know should fail, deliberately run through the same pipeline as everything else, specifically to confirm the pipeline is capable of detecting failure at all. In a modernization project that means: take a known-bad input, or the legacy system's output with a deliberately introduced discrepancy, and run it through your comparison harness. If the harness reports a match anyway, you haven't proven the conversion works — you've proven the comparison can't tell the difference between right and wrong, which is a much worse thing to discover after cutover than before it.
This sounds obvious once stated. It is skipped constantly, because building a comparison harness and building a broken input to feed it are two separate pieces of work, and only the first one shows up as progress on a project plan. Nobody demos a test that's supposed to fail.
How a comparison harness quietly stops testing anything
A byte-exact comparison sounds unforgiving by design, but the harness that performs it is itself code, and code has bugs. Some of the most common ways a comparison silently goes toothless:
- A parser that swallows the field it should be checking. If the extraction step for a given field fails silently and returns a default or empty value on both sides, the comparison will report a match — both sides agree on nothing, which trivially passes.
- A normalization step that erases the actual difference. Trimming whitespace, rounding to a tolerance "just for the comparison," or reformatting dates before comparing can mask exactly the class of discrepancy the test exists to catch.
- A test harness that only re-runs the happy path. If the harness was built against a handful of representative records and never touched again, it will keep validating those same shapes forever while the rest of the schema goes uncovered, green the whole time.
- An exception path that gets treated as a pass. A comparison that errors out on a malformed or unexpected record and is coded to log-and-continue rather than fail-and-flag will report a clean run while quietly skipping the exact records most likely to expose a real bug.
None of these require anyone to be dishonest. They are the ordinary failure modes of test infrastructure that was written once, under deadline, and never adversarially tested against its own blind spots.
The check costs almost nothing and answers a real question
Verifying a comparison harness is not expensive relative to building the conversion itself. Take a handful of legitimate outputs and mutate them in ways that should matter: flip a digit in a dollar amount, drop a trailing record, shift a date by one day, truncate a string field, swap two fields that happen to be the same data type. Run each mutated case through the harness and confirm it fails, with a message that correctly identifies what's wrong. If any of those mutations sail through as a match, you've found a hole in the thing that's supposed to be catching every other hole — and you found it before it mattered, not after a customer noticed a wrong number on a statement.
The question a negative control answers is blunt: has this test suite ever actually failed, on purpose, in front of the people relying on it? If the answer is no, the suite's all-green history proves less than it appears to. A suite that has run for months without a single red result is consistent with a correct conversion — and equally consistent with a comparison that's been quietly broken since week one and nobody had reason to look.
Where this fits with everything else on this blog
Byte-exact parity, production-data testing, a defined exit condition for the parallel run — all of it depends on trusting the mechanism that reports pass or fail. A vendor who can show you the test suite passing and also show you the negative controls that prove the suite is capable of failing is demonstrating something categorically different from a vendor who can only show you green. Ask for the second thing. If a program can't produce a handful of deliberately broken test cases and show its own harness catching every one of them, treat every other green result from that harness as unverified until it can.