Evaluation harnesses for models you cannot roll back

Rolling a model back is quick. Repoint the serving endpoint at the previous artefact, wait for the caches to clear, done inside twenty minutes. What does not roll back is everything the model already did. The transactions it declined have been declined. The maintenance crew it sent to a substation has been sent. A readmission score either prompted a follow-up call or it did not, and the discharge happened on Tuesday.

That asymmetry is what an evaluation harness exists to manage. The question it has to answer is how many irreversible actions a bad model takes before somebody notices it has become one.

Offline evaluation proves the least, and is still mandatory

A held-out set demonstrates that the model has learned the historical data. It says nothing about the population shift that arrived last quarter, the upstream feature that silently changed units, or the feedback loop where blocking a fraud pattern removes that pattern from next year's training data. We run offline evaluation as a regression gate rather than as evidence of readiness. It answers one question well: is this candidate worse than the model in production on cases we already understand? A clean answer there is necessary and nowhere near sufficient.

Shadow mode is the cheapest honest test available

The candidate scores live traffic and its decisions go nowhere. We compare it against the incumbent on the same requests, in the same order, under the same latency budget. On the fraud platform we built for a financial institution, shadow running is where feature pipeline defects surfaced, and they surface as a score distribution that looks nothing like the training set rather than as an error in a log. A model whose inputs are subtly wrong will produce confident scores all day and pass every unit test in the repository.

Shadow mode also produces the disagreement set: the cases where candidate and incumbent reach different conclusions. That set is small, all of the risk lives inside it, and it is short enough for a human being to read line by line before anyone signs the release.

Stage by consequence, not by traffic share

Ten per cent of traffic is the wrong unit when the actions are irreversible. Ten per cent of traffic on a payments platform is still thousands of declines an hour. We stage by what the model is allowed to do instead. Lowest tier: the score is logged and shown to an analyst, who decides. Next: the model decides, and a sample is reviewed within the hour by a person who can reverse it while reversal still means something. Only then does it decide unattended, and only inside the value bands where a wrong answer is recoverable.

The metric that actually gates the release

Teams argue about area under the curve for six weeks and then release on the strength of a slide. The gating metric has to be the one the business will be judged on when this goes wrong, measured on the population that will be affected, with a threshold agreed in writing before the candidate exists.

In fraud work that has usually meant the rate of blocked transactions belonging to legitimate customers at a fixed detection level, because a missed fraud is a number and a false decline is a customer relationship. On network prediction for a telecom estate it was callouts that found nothing wrong, because an engineering team dispatched to a healthy site three times stops believing the fourth alert, and at that point the model has degraded the operation it was bought to improve. Both are false-positive costs. Both are the number a model owner is least keen to see on the gate.

Agree it before training starts. A threshold chosen after seeing the results is a description of the results.

None of this removes the need for a rollback path. Keep the previous artefact warm, keep the switch one command away, rehearse it quarterly and make sure the person on call at three in the morning has done it once. Treat it as damage control rather than as a safety net, though, because by the time it is used the damage has already left the building. The harness is the safety net, and it earns its cost in the releases it stops.