Fraud detection that survives an audit

A customer was declined at a till on a Tuesday in March. Eleven months later, in a room with the regulator, somebody has to explain that decision. Whether a fraud model survives that conversation is what decides whether it goes live at all.

Answering it requires four things that have to exist before the decline happens: the feature vector as it stood at that moment, the score, the model version and threshold in force, and any rule applied after the model returned. Recomputing the features today gives a different answer, partly because the customer's history has moved on and partly because the decline itself is now part of it.

So the first design decision in a fraud system has nothing to do with the model. It is to persist the decision: inputs, score, versions and outcome, keyed to the transaction and retained for as long as the transaction is retained. Teams skip this because the vector is wide and nobody has asked for one yet. Somebody asks eventually, and by then the window has closed.

Feature provenance

Every feature needs a written definition, a named owner and a traceable source. What an auditor asks about is what the feature encodes. Anything derived from a postcode, a device fingerprint or a name will attract the question of whether it acts as a proxy for a protected characteristic, and the answer has to be a written position with evidence behind it. Saying that the model does not use protected fields directly is not a position. It does not need to use them directly.

Third-party data brings a second layer. A bureau feed or a device intelligence service usually comes with contractual terms about its use in an adverse decision, and those terms are the sort of thing that surfaces after go-live if nobody read them during procurement.

Simpler models at the boundary

A gradient boosted ensemble over several hundred features will beat a scorecard on a validation set. The question is where in the system that advantage is worth its opacity.

Our answer, repeatedly, has been to put the complex model where opacity costs nothing and something simple where the decision is made. Ranking a review queue, prioritising cases for an investigator, generating candidate signals: use whatever performs best, because a human sees the output and no customer feels the ordering. At the point where a payment is declined, use a model whose reason can be read out in a sentence, over features whose definitions fit on a page.

The lift given up at the boundary is usually small, and it buys the ability to defend the decision to a customer, a complaints team and a supervisor. Defensibility is what keeps the system switched on, and a fraud system suspended after a complaint saves nothing at all.

There is a second reason to prefer the simpler model at the boundary, which is monotonicity. A supervisor can be told that a higher value of a given feature can only raise the score, and that statement can be enforced in the model rather than hoped for. An ensemble will happily learn a non-monotonic relationship in a sparse region of the data and produce a decline that nobody can explain to the person it happened to.

A false positive here is a person standing at a till with a card that has stopped working, in front of other people. That cost is real, it lands on a customer who did nothing wrong, and it belongs in the cost model next to the fraud loss rather than in a footnote. None of this makes the audit an adversary. The audit is the condition under which a bank is allowed to let a model decline a payment at all, and a system built to answer it is the same system that can be handed to a different team in five years and still be operated with confidence.