Openness is the mechanism of trust.
First stated 10 July 2026 in We Ran a Blind eDiscovery Review with No Humans in the Loop
Legal AI asks lawyers to trust outputs they cannot fully inspect, produced by systems they cannot audit, evaluated on benchmarks they cannot rerun. That is backwards for a profession that answers to judges. When a party describes its own document review, courts want to know how it was done, not how confident the party feels.
The blind eDiscovery review we published was designed around that expectation. The review engine, isResponsive, was released under the Apache 2.0 license with its pipeline architecture, execution logs, metrics, failure analysis and reproducibility records. The rubric was frozen by cryptographic hash before classification, thresholds were fixed before execution, and the answer key was hidden. The report published what went right, a recall of 0.8071 (95 percent interval 0.789 to 0.824) across 8,665 documents, and what went wrong: a failed elusion gate, a failed calibration gate, and misses concentrated in implicit responsiveness. It then stated which uses the evidence supports and which it does not.
Publishing failures is not a courtesy. It is what makes the successes credible. A reader who can see the gates that failed can believe the gates that passed, and a reader who can rerun the method does not need to believe anything.
The principle governs every claim the Institute makes. Benchmarks should come with rubrics fixed before the run and published after it. Frameworks should come with version numbers and known limitations. Vendors, including those connected to us, should publish measured error rates per error class, evaluated by someone other than themselves. Openness is not a marketing posture. It is the mechanism by which a standard becomes something anyone can check.
Supporting pieces
Revision history
| 1 Sep 2026 | Added to the Theses. |
How to cite this thesis
Computational Law Institute (2026, September 1). Thesis 26: Openness is the mechanism of trust.. https://institute.legawrite.ai/agenda/theses/26