Show Your Work

Show notes

An auditor has a discipline most of us never have to develop: they are not allowed to take the decider at its word. When you audit a decision, you are reconstructing it from the outside — assembling, from records and artifacts, an account of what happened that does not depend on the deciding party's own story about itself. The reason is not suspicion of bad faith. It is that a self-report is generated after the fact, and an account generated after the fact can be accurate, can be a plausible reconstruction, or can be a confabulation — and from the outside you have no way to tell which. So the auditor sets the testimony aside and goes looking for something that does not depend on it.

That is the line this episode draws and keeps redrawing: the difference between an explanation and a record you can re-run. An explanation is testimony — the system's opinion about its own reasoning, offered when you ask. A replayable record is closer to physical evidence: it was created at the moment of decision, not in response to your question, and you can take it apart and put the decision back together yourself. Courts accept both kinds of thing, but they weight them very differently, and for good reason. An auditor lives inside that distinction. The system said so is the one answer they are professionally forbidden to accept, because it is exactly the answer that cannot be checked.

Here is the uncomfortable part, and the reason most automated decisions fail an audit before it even starts. The typical record of an automated decision is a log line: a timestamp, a label, maybe a confidence score. A log line tells you what was output. It does not tell you what the system actually consulted, what rules were active at that moment, or whether the same inputs would produce the same output tomorrow. There is no trail to walk back across — and a decision with no reconstructable trail cannot survive the one test an auditor exists to apply. It is not that the decision was wrong. It is that the question of whether it was right or wrong can never be opened.

So we end on the constructive question: what would a record actually have to carry to be reconstructable by a stranger? At minimum, the evidence that was really consulted, in the order it was consulted; the full text of the rules that were active at decision time, frozen as they stood rather than referenced by a version number that may since have changed; and enough captured state — the inputs, the model and settings — that someone could replay the decision and confirm it lands in the same place. That is a far higher bar than a log line, and it is roughly the argument the book Admissible Reality makes at length: that consequential automated decisions should be built to be audited, not merely explained. This conversation tests that argument against someone whose job is to do the auditing.