The Denied

Show notes

Most thinking about explainable algorithms quietly assumes the wrong audience. The reports get written for auditors, regulators, and the institution's own risk committee — the people who already hold the model and can already see inside it. But the person who actually needs the explanation is the one who can see the least: the applicant told the loan is declined, the household whose benefit was recalculated downward, the customer whose account was flagged and frozen. The decision went against them, and they are the party with the strongest claim on an account of it and the weakest access to one. This episode starts from a simple reordering of who the record is for. Accountability is owed first to the person the decision went against — not last, after the institution has satisfied everyone it answers to laterally.

What that person is owed is more than a notification and more than a reason code. A short list of contributing factors — "credit utilization," "insufficient history," "recent inquiries" — tells you which dials the system watched, but not what it actually saw in your case, which rule it applied, or how to change the outcome. A reconstructable account is different in kind. It says: here is the evidence we used about you, here is the rule we applied to that evidence, here is the specific way the two produced a no, and here is the path by which you can challenge any of it — correct a wrong input, contest the rule's relevance, demand a human look again. An adverse automated decision that cannot be reconstructed and cannot be contested is not a decision a person can answer; it is a verdict delivered without standing to appeal.

This is exactly where "the model declined you" fails the standard. It names an actor and supplies no account. It cannot be checked against the facts of the person's situation, because it does not state which facts were used. It cannot be challenged, because there is no articulated rule to dispute and no surface to push back on. And it launders responsibility: the institution made a decision and chose to run it through a model, but "the model" is offered as though the decision belonged to no one. A person on the receiving end of a consequential no is owed a step they can put a finger on and say this one is wrong — and a reason code, by design, gives them nothing to point at.

That is the warrant frame this show keeps returning to, and the admissibility frame underneath it: a conclusion earns its authority not because a system produced it but because the path to it can be reconstructed and contested by the party it binds. Evidence used, rule applied, route to challenge — that is the structure of an account that could survive scrutiny, and it is the structure an adverse decision owes the person it falls on. The book this series accompanies, Admissible Reality, argues for exactly this standard: not machines that merely output correct-looking answers, but decisions whose grounds the affected person could actually examine and argue with. The denied person is the test of whether any of it is real.