New on SSRN: Ariadne's Thread, a measurement-theoretic method for legal openness. Read the paper
20
Legal Knowledge Engineering

A system is structurally reliable only for the errors its representation records.

First stated 15 April 2026 in The Promise Fulfilled

Defense

Call a research tool structurally reliable for a class of error when the information needed to catch that error is recorded in the tool's data, in a form that a defined and repeatable check actually consults. Call it opportunistically reliable when correct answers depend on whether the material a given query happens to retrieve happens to carry the warning. Opportunistic reliability can be real and substantial. It is also unstable: the same tool can be right on Monday and incomplete on Tuesday for the same question, and nothing in the output reveals which day it is.

The distinction turns an argument about quality into a question about representation. A system with no field for the procedural stage of a holding cannot filter on stage, however capable its model. A system that does not record subsequent history cannot know that a case was reversed last month unless the retrieved text happens to say so. A system that does not index rejected arguments cannot find the courts that refused an inference.

This is why agentic retrieval, longer context and better prompts help only in part. They convert some failures from systematic to opportunistic by raising the chance that the needed context turns up. They do not touch failures that are gaps in representation, among them authority health, direction and negative space. Those require the information to be extracted, typed and stored as first-class objects before any query runs.

For evaluators, the thesis supplies a question that can be asked of any product without access to its internals: for this class of error, where is the information that catches it stored, and what check consults it? If the answer is that the model notices, the reliability is opportunistic.

Supporting pieces

Revision history

1 Sep 2026Added to the Theses.

How to cite this thesis

Computational Law Institute (2026, September 1). Thesis 20: A system is structurally reliable only for the errors its representation records.. https://institute.legawrite.ai/agenda/theses/20

Cite the thesis by number and the date of the revision you relied on.