A Public Record of Citation Failures

Why it matters

Policy on AI in legal practice is being written now, in standing orders, ethics opinions, sanctions decisions and carrier questionnaires. Most of it responds to the failures that are easiest to see. If the record contains only fabricated cases, policy will be tuned to fabrication, and the failures that grounded systems still make, such as the wrong stage, the missing controlling case and the misrepresented holding, will stay invisible until they are expensive.

State of the art

Damien Charlotin's AI Hallucination Cases Database, maintained at HEC Paris, lists more than two thousand decisions in which a court or tribunal has addressed AI use, with classes that include fabricated case law, false quotations, misrepresented case law and fabricated exhibits. It is the best available record, and it is growing. Its limits are structural, not editorial: it records what a court noticed and wrote about. A posture mismatch or an unretrieved controlling case rarely produces a written decision about AI, and when it costs a client a motion, nothing marks it as a tool failure at all. Professional guidance, including ABA Formal Opinion 512 (2024), frames the duty as verification by counsel, whatever the tool.

Our partial answers

The Institute's charter commits us to maintaining a public record of AI citation failures in court. Our approach is to classify beyond fabrication, using the vocabularies developed in our programs: posture mismatch by standard, record, burden and forum; the omission classes; errors of direction; and failures of currency. We also separate failures of the document from failures of process, because process is what courts and carriers increasingly ask about.

Open: how to record failures that never generate a decision; how to attribute a failure to a tool rather than to its user without vendor cooperation; and how to publish such a record in a way that informs policy without singling out individual lawyers.