Putting a model into production inside a regulated bank

The model took six weeks. Everything after the model took the rest of the year, and the parts that took the rest of the year are the reason it is still running.

That is the shape of most machine learning work inside a regulated institution, and it surprises people who have only shipped models into consumer products. The trial is honest enough. Held-out performance is good, the sponsor is pleased, and the model looks finished. What follows is a set of questions the trial never had to answer, asked by people with the authority to stop the project.

The questions a trial never has to answer

The first is about the features. In the trial they were computed in a notebook from warehouse tables that land overnight. At decision time the same features have to be available in an operational store, inside a latency budget measured in milliseconds, computed from data that exists at that moment. A feature that depends on a nightly aggregate is unavailable for the transaction happening now, and rebuilding it as a streaming calculation is a project of its own with its own failure modes.

The second is point-in-time correctness. Warehouse tables get corrected. A record written a year ago and amended six months later reads today as though the amendment was always there. Training on that data teaches the model to rely on information that will never exist at the moment of a live decision. It is the most boring form of leakage and it is extremely common, because the correction is invisible unless the warehouse keeps history.

The third question is who is accountable when the model is wrong, and it arrives early. An automated decision in a bank has an owner in the first line, a challenge function in the second and an audit function in the third. All three will want to know what the model does in language that does not require them to read the code.

The fourth lands hardest on teams that did well in the trial. The data used to build the model was made available under one arrangement, often a one-off extract signed off for analysis. Using it in an operational system is a different legal basis, sometimes a different consent position and occasionally a different physical location. We have watched a finished model wait three months for a data-sharing agreement that was never wrong, only slow.

Lineage is the first deliverable

The first months of one of these programmes are plumbing, and that is the correct allocation of the budget. Every feature needs a documented source, an owner, a refresh cadence and a definition agreed with the business. Definitions are where the arguments are. Customer tenure measured from account opening and customer tenure measured from the first funded transaction give different answers for exactly the population the model cares about, and both are defensible right up to the point where somebody has to pick one and write it down.

The artefact that makes audit possible is the stored decision. At the moment the system decides, it writes the input vector as it stood, the score, the model version, the threshold in force and any rule applied after scoring, keyed to the decision and retained for as long as the decision is retained. Recomputing the features later produces a different vector, because the world has moved and, in some cases, because the decision itself is now part of the customer's history.

Storage is cheap and this is still where teams cut corners, usually because the vector is wide and nobody has asked for one yet. Somebody asks eventually, and by then the window has closed.

Evaluation the bank can run without us

The evaluation has to survive our departure. That means a frozen dataset with a documented split, a scoring script the bank owns and can run unaided, and metrics agreed with the business before the numbers are known. Choosing the metric after seeing the results is not evaluation. It is decoration.

The threshold is a business decision dressed as a technical one. It should be set against a cost model expressed in currency: what a false positive costs in handling and in customer damage, what a false negative costs in loss and in regulatory exposure. When those numbers are written down, the threshold conversation takes an afternoon. When they are not, it takes a quarter and ends with somebody picking a round number.

One more thing about evaluation. The business will ask for a single number and the honest answer has at least two. Report the operating point, the cost model behind it and the uncertainty around it, and resist a summary metric that hides which population the model is worst on. Subgroup performance gets asked about at audit, and measuring it in advance is far easier than producing it under time pressure with a deadline attached.

Then shadow mode. The model runs against live traffic, records the decision it would have made, and takes no action. This is where distribution problems appear, and they always appear: the live population does not match the training population, a feature is null more often than the warehouse suggested, an upstream system sends a status code that was not in the sample. Shadow mode for a full business cycle is cheap insurance and it is usually the difference between a controlled launch and an incident. After that, a canary on a defined slice, with an agreed rollback and somebody reading the score distribution daily rather than weekly.

The review queue is a component, not an overflow

A model that flags cases produces a queue, and the queue is a designed component with its own service level, staffing plan and interface. It gets treated as an operational detail. It decides whether the whole system works.

The reviewer needs the reason for the flag in language they can act on, the contributing factors, the customer history that is relevant to the decision, and a small number of clearly worded outcomes. Those outcomes are labels. If the interface captures them cleanly, the next version of the model has supervision from the people who know the domain best. If the interface offers a free-text box, the next version has nothing.

The staffing is a real cost and it belongs in the business case. A model that flags 1% of a large transaction volume is a hiring plan. A model that flags 5% is a different company. Moving the threshold moves the headcount, which is another reason the cost model matters more than the last two points of precision.

Rollback means the previous decision, not the previous container

Redeploying the old container is not a rollback. If the feature pipeline has moved on, the old model now sees inputs it was never validated against. A real rollback restores the model, the feature definitions, the threshold and the post-processing rules as one versioned set, and the system can answer what it would have decided under the previous version for any transaction inside the window.

We test that path before launch and then on a schedule, in the same way a failover is tested. A rollback that has never been executed is a plan, and plans behave differently at three in the morning.

A model that cannot be switched off in an afternoon is not in production. It is in charge.

There is also a kill switch, which is a separate thing from a rollback. Somebody in operations, without an engineer and without a release, must be able to route every decision to the previous rule set or to manual handling. That control gets used perhaps twice in the life of a system, and both times it justifies every hour spent building it.

Procurement, model risk and the documents nobody enjoys

The model risk function will ask for a document in their own format, covering purpose, data sources, methodology, assumptions, limitations, monitoring plan and evidence of independent validation. That document is the gate, and writing it takes weeks nobody put in the plan. We now budget the writing alongside the engineering and start it while the model is still being trained, because the act of writing the limitations section reliably finds two more of them.

Procurement adds its own set: where the model runs, whether training data leaves the institution, who may inspect the weights, what happens to the model if the contract ends, and what the notice period means for a system making live decisions. Security review is a scheduled stage with a named reviewer and a date, treated as delivery work rather than as a surprise before release.

Day two

Outcome labels arrive late. A fraud label can take weeks and a credit label can take years, so a monitoring plan that waits for outcomes is blind for exactly as long as it takes a problem to become expensive. Input distribution and score distribution are available immediately, and they move first. Monitor those daily, with a threshold that pages somebody.

Drift also arrives through the door rather than through the data. A change to a fee schedule, a new product, a campaign that brings in a different population: each shifts the input distribution without anything technical having changed. The monitoring should be read by somebody who knows what the business did last month, which argues for a short standing review rather than a dashboard nobody opens.

Retraining is a change and goes through the same gate as the original release: evaluation, documentation, sign-off. Automatic retraining straight into production is a thing we do not do inside a regulated institution. It is a fine idea in a recommendation engine and an unpleasant one in a system that declines payments.

Counted honestly, the model is a small part of the work, and it is the part that changes least once the system is live. The lineage, the stored decisions, the evaluation harness, the review queue, the rollback path and the monitoring are what the bank operates for the next decade, usually with a different team than the one that built it. That is the test we care about: whether somebody who has never met us can explain, in 2029, why the system declined a payment last Tuesday.