isResponsive
An Open TAR Methodology for Court-Defensible Responsiveness Review
Court-defensible technology-assisted review does not require a proprietary black box. This Article describes isResponsive, an open TAR methodology for responsiveness review built from four controls: per-request rubrics locked by cryptographic hash before classification begins, a calibrated classification cascade optimized under explicit recall constraints, a deterministic privilege screen that routes every flagged document to attorney-only review, and a formal validation stopping rule that issues a counsel-signed Defensibility Certificate only when per-request recall lower bounds, elusion upper bounds, and active-learning convergence are all satisfied. The claim is procedural: under Rule 26, Daubert, and the TAR case law, defensibility turns on a measurable, documented, counsel-supervised process that an opposing party, special master, or court-appointed expert can reproduce, not on trust in any particular model. The Article makes no production-results claim; the system runs in live matters, but matter-level claims issue only through signed validation certificates, and aggregate production telemetry is reserved for a forthcoming structured report. On a synthetic five-million-document fixture at approximately 0.8 percent prevalence, the worked cascade configuration yields an estimated end-to-end recall lower bound of 0.821 at an estimated total cost of $47,297, a synthetic-fixture result offered as proof of pipeline arithmetic rather than of production performance. If that forthcoming telemetry confirms the engineering target, the 75 percent recall reference that practice draws from Sedona TAR guidance deserves re-examination, not as a mandatory floor, but as a soft benchmark that a reproducible open process may clear at higher levels and proportionate cost.
- Authors
- Ross Brodskiy
- Posted
- 28 July 2026 (SSRN)
- SSRN abstract
- 7092721
- Length
- 34 pages
- DOI
- Not yet assigned
- Keywords
- technology-assisted review, eDiscovery, responsiveness review, validation, defensibility, recall and elusion
- Court-defensible TAR does not require a proprietary black box. Defensibility turns on a measurable, documented, counsel-supervised process that an opponent, special master or court-appointed expert can reproduce.
- The methodology rests on four controls: per-request rubrics locked by SHA-256 hash, a calibrated classification cascade optimized under a recall constraint, a deterministic privilege screen, and a validation stopping rule that issues a counsel-signed Defensibility Certificate.
- Completion is declared only when three independent gates pass for every request for production: a one-sided 95 percent recall lower bound at or above the floor, an elusion upper bound below 5 percent, and active-learning convergence. Any failure sends the matter back for labeling and a fresh sample.
- Measured axis by axis against the In re Broiler Chicken validation order, the protocol is stricter wherever the order's text reaches, yet four of the order's requirements have no current analog: blind coding of the validation sample, privilege coding within it, turnover of responsive documents found in the discard pile, and a qualitative inquiry into what was missed.
- A synthetic five-million-document fixture yields an estimated end-to-end recall lower bound of 0.821 at an estimated total cost of $47,297, offered as proof of pipeline arithmetic, not of production performance.
- If production telemetry confirms the engineering target, the 75 percent recall reference drawn from Sedona guidance deserves re-examination: not as a mandatory floor, but as a soft benchmark that a reproducible open process may clear at higher levels and proportionate cost.
- 1. Introduction
- 2. Background and Prior Work
- 3. The Four Laws
- 4. System Architecture
- 5. Rubric Extraction
- 6. The Classification Cascade
- 7. Active Learning
- 8. Validation and the Defensibility Certificate
- 9. The Privilege Screen
- 10. Empirical Results
- 11. Limitations and Open Questions
- 12. Conclusion
The paper in brief
Discovery cost decides which claims can be pursued and which defenses can be held, and the pressure is sharpest in mid-market litigation, where data volumes look like enterprise cases and budgets do not. This paper describes isResponsive, a technology-assisted review engine for responsiveness review developed by Legawrite.AI and built for open release. Its claim is narrow and procedural. A court should not be asked to trust a model because the model is good. It should be shown a process for declaring completion that is measurable, documented, supervised by counsel, and reproducible by the other side.
1 and 2. Why TAR, and what the law already says
The paper places the method in the lineage of Grossman and Cormack's empirical work, continuous active learning and the TREC Legal Track, then walks the case law. Da Silva Moore approved computer-assisted review; Rio Tinto described its acceptability as effectively settled; Hyles held that a requesting party cannot force a producing party to use it. Later decisions moved the fight to protocol, transparency and validation. In re Broiler Chicken Antitrust Litigation entered a detailed validation order. Livingston allowed TAR after keyword culling. In re Diisocyanates found a validation unreasonable because it estimated recall for the TAR stage alone and ignored documents culled by search terms. In re Insulin Pricing declined to fix a recall target in advance while accepting a commitment of seventy percent or higher. The generative AI cases, from Mata v. Avianca onward, confirm the premise: when legal work uses AI, the defensible unit is a reliable method with a record, not a fluent output. Sedona's guidance emphasizes reasonableness, and the 75 percent recall figure used in practice is a soft reference, not a universal floor.
3. The Four Laws
The engine is written to the Four Laws of System Design for Computational Law, which the paper is careful to call a software design canon, not legal authority. Each law maps to a control. The Zeroth Law (never present more confidence than the system can defend) means the system refuses to certify when recall fails. The First Law (explainability) is implemented as hash-pinned artifacts and a tamper-evident audit chain. The Second Law (grounded authority) means the model classifies against counsel-approved rubrics and never becomes a source of law. The Third Law (independently testable components) produces an eight-service architecture.
4. System architecture
Eight services (Loader, Embedder, Reranker, Classifier with Aggregator, Calibrator, Sampler, Validator, Orchestrator) talk only through the Orchestrator, so every material transition lands in one audit chain. The separation is a legal design choice: a challenge to extraction, lost recall, privilege or tampering each has a specific unit of inspection. The same code runs as a single-process pilot or as containers for a large matter. A state machine blocks classic defensibility failures: classification cannot start before the rubric is locked, and a certificate cannot be signed if its rubric hash does not match.
5. Rubric extraction
Responsiveness is request-specific, so the case documents are turned into one structured rubric per request for production: inclusion and exclusion criteria, time limits, actors, exemplars and edge cases. Attorneys review and lock the rubric before classification, because rubric error is upstream of model error. Locking is by SHA-256 hash over canonical JSON, and every classification records the hash. If the rubric changes, earlier classifications must be rerun or carried forward with explicit provenance.
6. The classification cascade
Three stages (embedding retrieval, a reranker cutoff and language-model classification) are tuned to minimize total cost subject to a recall constraint. Every positive call must cite spans in the document. Raw model confidence is not treated as a probability; Platt scaling, with isotonic regression as a fallback, converts it into a matter-specific probability that sets the auto-discard, human-review and auto-responsive bands. The binding margin is the Wilson lower bound on recall, which the paper calls the statistical expression of the Zeroth Law.
7. Active learning
A cold-start regime seeds review from rubric-driven harvest, uncertain documents and a random reserve; a steady-state regime balances uncertainty, stage disagreement and diversity. The validation fold is walled off from training without exception, and a failed validation is retried on a fresh sample. Gold-set documents get at least two attorney coders and senior adjudication.
8. Validation and the Defensibility Certificate
Completion requires three independent conditions for every request: a one-sided 95 percent recall lower bound, computed by stratified bootstrap across accept and reject pools, that meets the matter floor; a Clopper-Pearson upper bound on elusion in the reject pool below 5 percent; and converged active learning. Documents culled at retrieval or reranking stay in the reject pool, so the bounds price in every stage's losses. The resulting certificate, in JSON and PDF, records sample sizes, seeds, per-request results, model versions, artifact hashes and counsel's signature, and it refuses to generate if any hash invariant fails.
The paper then measures this protocol against the Broiler Chicken order point by point. It is stricter where the order's text reaches: a confidence bound instead of a point estimate, per-request instead of matter-level measurement, mechanical fresh-sample retry, and a standing tamper-evident artifact. It also names four gaps where the order demands more, and sets out a version 1.1 remedy for each; until then, a party under a similar order should add those steps by agreement.
9. The privilege screen
Privilege is not treated as another probability. The version 1 screen is deterministic and conservative, using patterns, lists and metadata with no language model at all. Any flagged document goes to attorney-only review and can never be auto-produced. The paper is candid that the screen misses implicit privilege and non-United States regimes.
10 to 12. Results, limitations and conclusion
The paper separates production telemetry, synthetic fixtures and methodological invariants, and refuses to mix them. On a synthetic five-million-document fixture at about 0.8 percent prevalence, the stage lower bounds of 0.977, 0.951 and 0.885 multiply to an estimated end-to-end recall lower bound of 0.821, at an estimated $47,297 dominated by human review rather than GPU cost. Limitations include very low prevalence, correlated errors across cascade stages, privilege gaps, long-document aggregation and the Broiler Chicken deltas. The conclusion states the citable contribution as structure, not a model: define responsiveness per request, lock the criteria, calibrate, protect validation from training, measure with confidence bounds, refuse to certify on failure, and leave a record.
The paper treats openness as a dependency of its own claim. Legawrite.AI has since released the engine under the Apache 2.0 license, as reported in the author's case study of a blind review; it installs with pip install legawrite-tar.
Disclosure. The paper states that Legawrite.AI develops and commercially licenses isResponsive and that the author holds a financial interest in the company and the system; the methodology is published in open form so that its claims can be verified rather than taken on trust.
Where to go next
- We Ran a Blind eDiscovery Review with No Humans in the Loop: the engine on a real corpus, including the gates it failed.
- The Four Laws of System Design: the design canon the engine is written to.
- What a Discovery Framework Must Do: the practitioner's view of the same problem.
Frameworks in this piece
Terms in this piece
Revision history
| 28 Jul 2026 | Paper page created. |
How to cite
Brodskiy, R. (2026, July 28). isResponsive: An Open TAR Methodology for Court-Defensible Responsiveness Review [Working paper]. SSRN. https://papers.ssrn.com/sol3/papers.cfm?abstract_id=7092721
Related pieces
New papers, frameworks and essays. No marketing. Or use RSS.