Reading a modernization RFP like an engineer: 10 questions that expose the bid

Most people read a modernization RFP response the way they read a restaurant menu: for the parts that sound appetizing. Case studies, methodology diagrams, a slide with a maturity model on it, references to "agile" and "AI-accelerated" and "proven frameworks." None of that tells you whether the vendor can actually reproduce the behavior of the system you're paying them to replace. An engineer reads the same document looking for the ten or so questions it is carefully not answering — because those are the questions that predict the change orders, the year-three rescue, and the parallel run that never converges.

Below are the questions worth asking before a signature, not after the first missed milestone. None of them are exotic. All of them are routinely dodged, because a bid that answered them honestly would be less competitive than a bid that stays vague.

The questions that expose the bid

1. What does "done" mean, in writing, before day one? Ask for the itemized parity target — the specific behaviors, against specific reference data, checked in a specific way. If the answer is a phase plan with milestones like "core functionality delivered" instead of a testable definition, the definition of done will be written later, by whoever needs it to be smaller.
2. What proves parity, and against whose data? A demo against sample records proves the demo works. Ask specifically whether verification runs against real production data volumes and edge cases, or synthetic data the vendor generated themselves. The difference is where the actual risk in the system has always hidden.
3. Is the comparison deterministic, byte-for-byte, or "reasonably close"? "Reasonably close" is a phrase that turns into an open-ended reconciliation project the moment cutover approaches. Byte-exact output against the legacy system, on the same inputs, is a claim you can actually check. Ask for it in the contract language, not the sales deck.
4. Who owns the discovered-requirements risk? Every legacy estate has undocumented behavior that only surfaces during conversion. Ask directly whether that discovery triggers a change order against you or is absorbed inside the fixed price. If the answer is vague, assume every discovery becomes billable.
5. What is the AI doing, specifically, in the conversion path? "AI-accelerated" is now a standard slide. Ask whether a model is generating the replacement logic that will run in production, or assisting a human engineer's comprehension of unfamiliar code. Those are different risk profiles wearing the same marketing language, and only one of them belongs anywhere near money-handling logic without a deterministic proof step after it.
6. What happens to scope that gets descoped? Ask what the process is for removing a committed capability mid-project — who signs off, what gets documented, whether there's a cost attached. If descoping is administratively free, it will happen quietly and often; that's how "done" ends up meaning less than what was promised.
7. Can they show a reference where the parallel run actually ended? Not a reference who is happy with the relationship — a reference where the legacy system was actually decommissioned on the date it was supposed to be, and can say so. An open-ended parallel run is a cost center dressed up as due diligence, and it's the default outcome when "matching" was never precisely defined at the start.
8. What is the exit path if this vendor fails or the relationship ends? Ask who owns the converted source, the test harness, and the parity evidence — and whether any of it depends on a proprietary runtime that only this vendor can operate. A modernization that trades one hostage situation for another hasn't reduced your risk, it's relocated it.
9. How is the price structured against the incentive to actually finish? Time-and-materials pricing pays the same whether the project is efficient or not. Fixed price without a locked scope invites the change-order machine. Ask what happens to the vendor's margin if the project runs long — if the answer is "nothing," the schedule risk is entirely yours.
10. What do the public post-mortems on similar programs say? Government modernization failures are unusually well documented because audit offices and inspectors general publish what happened. Reading a few before writing the RFP — what was promised, what shipped, what the retrospective blamed — tells you which of these ten questions that program's buyer failed to ask.

Why the dodge is predictable, not accidental

None of this is a claim that vendors are uniquely dishonest. It's a claim about incentives: a fixed-price bid that commits to byte-exact parity, a locked scope, and a hard decommission date is harder to win against a bid that leaves those things soft, because the soft bid can promise a lower number and a shorter timeline up front. The buyer who doesn't ask these questions is the buyer who ends up comparing two numbers that were never actually comparable — one priced against real behavior and real risk ownership, one priced against a demo.

The fix isn't more skepticism in the room during the sales pitch. It's putting these ten questions into the RFP itself, as required answers with contractual weight, before a single vendor gets shortlisted. A bid that can't answer question one in writing — what "done" means, testable, before day one — shouldn't advance to question two. Everything downstream, from the change orders to the descope spiral to the parallel run that never converges, is what happens when that first question goes unanswered and everyone finds out what "done" meant after the money is already spent.