ChatGPT fake citations: what courts are doing about them, and how not to be next

A language model is a text predictor. Asked for authority that supports your position, it produces something shaped exactly like authority: a plausible case name, a realistic docket number, a quotation in flawless judicial prose. Peer-reviewed testing published in Scientific Reports found GPT-3.5 fabricated 55% of the academic references it generated, and GPT-4 still fabricated 18%.

The danger is that fabricated citations are not obviously wrong. They are formatted correctly, they sound right, and they arrive alongside genuine ones. Nothing in the output distinguishes the real from the invented. That distinction only exists in the sources themselves, which is why every safe workflow runs through them.

Mata v. Avianca (S.D.N.Y. 2023) set the template: two lawyers submitted a brief citing six cases ChatGPT had invented, and Judge P. Kevin Castel imposed a $5,000 sanction. The court's point was not that they used AI, but that they signed a filing without checking it.

The pattern has escalated since. In 2025, a California federal judge imposed roughly $31,000 in sanctions on two firms whose brief contained AI-generated research that no one verified before filing. The Utah Court of Appeals sanctioned a lawyer whose brief cited "Royer v. Nelson," a case that exists only in a ChatGPT conversation. Trackers of court decisions now document hundreds of filings with AI-fabricated citations across multiple countries, and courts increasingly treat an unverified citation as a candor problem, not a technology problem.

One more development is worth knowing: in early 2026, a federal court ruled that a litigant's exchanges with a public chatbot were not privileged. What you type into a general-purpose AI tool may be discoverable.

The duty is the same even where the headline sanction has not happened yet. The Code of Professional Responsibility and Accountability requires candor toward the courts: a lawyer certifies the filings they sign. A hallucinated G.R. number or an invented Supreme Court quotation breaches that duty the same way a fabricated federal case does in the United States.

Philippine legal research has an additional trap: the authoritative sources are scattered across the official Gazette, the Supreme Court E-Library, and decades of scanned decisions. When primary sources are hard to search, the temptation to accept a chatbot's confident summary grows. That is exactly the situation where verification discipline matters most, and where research grounded in the actual corpus, like Tatsulok's curated Philippine law collections, removes the guesswork.

Every sanctioned filing shares one omission: nobody opened the cited source before relying on it. The counter-workflow is short.

1. Treat every AI-supplied citation as unverified until you have opened the source itself, not a summary of it.

2. Verify the proposition, not just the existence. Real cases get cited for things they do not say; an existing case with an invented quotation has also drawn sanctions.

3. Prefer tools that answer from a defined corpus and link each claim to the passage it came from, so verification is one click instead of a separate research task.

4. Keep the human signature meaningful. The lawyer, researcher, or analyst who signs the output owns every line of it, including the ones the AI wrote.

The audits and the sanctions point to the same design conclusion: citations bolted onto a chatbot are decoration; citations that open the source are infrastructure. In Tatsulok, every sentence of an answer links to the exact passage in the document it came from. If the sources do not support a claim, the answer says so instead of inventing support.

That converts the verification workflow from a chore into the default reading experience. You are not cross-checking the AI against the record; you are reading the record through the AI. For legal work, compliance, and any domain where a fabricated reference carries professional consequences, that inversion is the entire point.

If you adopt nothing else, adopt this: AI may draft, summarize, and locate, but no citation goes into anything filed, published, or relied upon until a human has opened the cited source and confirmed it says what the draft claims. Tools that ground answers in your own documents and link every claim to its passage make that policy nearly free to follow. Tools that do not, make it your unpaid second job.