Documentation that stays true after the second refactor

Every long-running system we have inherited comes with a wiki space, and every wiki space has the same shape. A dozen pages that were accurate on the day they were written, three that are still true, and one that is confidently wrong in a way that will cost somebody an afternoon. Nobody was careless. The pages simply described things that then changed, and nothing anywhere noticed.

The useful way to think about this is ownership. If a document restates a fact that the code already owns, the code will change and the document will not, because only one of them is executed. That is not a discipline problem to be solved with a review checklist. It is a structural property, and after two refactors it is close to certain.

The three kinds that survive

Documentation lasts when it records something the code cannot express, or when its falsehood breaks something. Almost everything durable falls into one of three groups.

Decisions with their context. An architecture decision record captures the constraint, the options considered, the one chosen and the date. It is a historical statement, so refactoring cannot make it false. The reader in five years learns why the obvious option was rejected, which is exactly the information they would otherwise waste a week rediscovering. Never edit an old record when the decision changes; write a new one that supersedes it and leave the original in place. The chain of superseded records is more valuable than any single entry in it.

Documents that execute. A README whose setup commands are run by the pipeline cannot drift, because drift is a failed build. Schema documentation generated from the schema is the schema. Integration tests written so the test names read as statements about behaviour do more explanatory work than a page describing the same behaviour, and they fail when they lapse. The general form: attach the document to something that runs.

Documents that get exercised. A runbook used during a quarterly failover drill stays true, because a wrong step gets found by somebody following it under mild pressure. A runbook filed at handover and opened for the first time during a real incident is a work of fiction with a serious tone. The exercise is what maintains it, not the writing.

What to deliberately leave undocumented

Deciding what not to write is most of the work, and it is unpopular, because the request usually arrives as a reasonable ask from someone senior. Four categories we push back on:

• How a function works internally. The code is the record, it is one click away, and the prose version will be wrong first.
• The current schema, endpoint list or configuration reference. Generate it or omit it. A hand-maintained copy is a second source of truth that nothing validates.
• Step-by-step walkthroughs of a user interface that changes every quarter, unless a regulator requires them, in which case version them with the release.
• Architecture as intended rather than as built. Aspirational diagrams are the most damaging documents in any estate, because they look authoritative and describe a system that does not exist.

The diagram question comes up on every programme. Our position is that one diagram of the boundaries is worth maintaining, because boundaries change rarely and slowly. Diagrams of what happens inside a boundary are best drawn on demand, from the code, and thrown away afterwards.

The test

Before writing anything, ask what will happen when this becomes false. If the answer is that a build fails, a test goes red or a drill stalls, the document will be maintained by that mechanism and the effort is worth spending. If the answer is that a reader is misled at some unknown future date, either attach it to something that runs or accept that it is a note with a shelf life and mark the date on it.

That test cuts most documentation programmes down considerably, which is uncomfortable to propose in a review where somebody has asked for comprehensive documentation. The trade is worth making anyway. A short set of documents that are known to be true is more useful at three in the morning than a comprehensive set where the reader has to work out, page by page, which parts they are allowed to believe.