Large language models behind a compliance boundary

The first question on these programmes is almost never about the model. It is about where the tokens go. A risk officer wants to know which machine holds the prompt, who can read the inference logs, and what happens when a customer account number appears in a support ticket the system is about to summarise.

Answering that properly is most of the work. The modelling part is short by comparison.

Where the weights actually run

Three positions cover nearly everything we have built. An open-weights model on hardware the client owns, inside their own network. The same class of model on a cloud tenancy pinned to a named region, with a contractual commitment that data stays in it. Or a commercial API under a zero-retention agreement. All three are defensible. They are defensible to different regulators, which is the whole of the decision.

The strictest clause in the client's existing obligations usually settles it, and benchmark scores come a distant second. On one programme a data residency term buried in a supplier agreement signed years earlier determined the entire architecture, and we found it in week five. We now read those contracts before the architecture review rather than after.

Pseudonymisation, and what it breaks

Stripping identifiers before a prompt crosses the boundary is straightforward until the answer has to make sense. Replace every account number with a token and the model can no longer tell a customer which of their two accounts went overdrawn. Replace names with placeholders and the drafted reply comes back subtly wrong in tone.

Reversible pseudonymisation held outside the model has worked best for us. A mapping table in the client's own store, entities swapped out on the way in and swapped back on the way out. The model never sees the real value and the user still gets a complete answer. That mapping table then becomes infrastructure in its own right, with a retention policy, an access log and a named owner, which surprises people who assumed redaction was a preprocessing step in the ingest code.

The retrieval index inherits nothing

The most common leak we find in these systems has nothing to do with the model. Documents get embedded into a vector index, and the index does not carry the permissions the source repository enforced. Every chunk is now equally readable by every caller, so an assistant that was meant to answer policy questions will happily quote a document the questioner was never cleared to see.

Permissions have to be reapplied at query time against the identity of the person asking, and revocations have to propagate to the index rather than only to the source. That is a reindexing pipeline with its own service level. It is dull, and it is the part a penetration tester will find first.

Evaluation an auditor will accept

A test set the team assembled last month, scored once, showing good numbers, carries very little weight in a formal review. What carries weight is a frozen set, a documented sampling method, a record of every model and prompt version scored against it, and a named person who signed off the threshold. Prompt versions matter as much as model versions here, because a wording change in a system prompt can move behaviour further than a model upgrade.

We build the harness before the first prototype and keep the results for the versions we rejected. Auditors ask about the rejected ones, and a recollection that something did not perform well is a weaker answer than a scored comparison.

The story told afterwards

Every one of these systems eventually produces an output somebody disputes. An answer given to a customer was wrong, a case went to the wrong queue, and someone senior wants to know how it happened. If the only record is a chat transcript, that conversation goes badly.

Reconstruction is the design point. Prompt, retrieved context and its source documents, model version, prompt version, pseudonymisation map reference, output, and whoever reviewed it, all recorded against one identifier. It is unglamorous plumbing and it costs a visible fraction of the build. It also decides whether the system is still running in two years or was quietly switched off after a single incident.

The model will be replaced, probably twice, and the replacement will be cheaper and better. The boundary, the pseudonymisation service, the evaluation harness and the audit trail are what has to survive, and should be built to outlive whatever is generating text this quarter.