AI document review: what it actually does well, and how to use it without getting burned

Document review is the reading layer of knowledge work: finding the clause that governs termination, checking a report against last year's, confirming what was actually agreed. AI document review points a language model at those files so you can ask in plain language and get answers, summaries, extracted terms, and flagged risks.

Two architectures hide behind one label. Chat-with-a-file tools reason about one document per conversation. Grounded library tools index your whole corpus, answer across documents, and cite where each claim came from. For anything beyond a quick single-file check, the second architecture is what the workflows below assume.

The LawGeex study remains the cleanest head-to-head. Twenty experienced US corporate lawyers and an AI system reviewed five NDAs totaling 153 paragraphs, marking legal risks. The AI averaged 94% accuracy; the lawyers averaged 85%. The AI finished in 26 seconds; the lawyers averaged 92 minutes. The spread among humans is the underrated finding: the best lawyer matched the AI at 94%, the weakest scored 67%.

Read it carefully, though. The task was narrow, one document type with well-known risk patterns, and the study predates modern general-purpose models, which are more capable and also more willing to improvise. The honest conclusion is not "AI replaces review" but "AI makes the first pass fast and consistent, and consistency was the humans' weak point."

The failures are not random noise; they cluster where verification is hardest. Models paraphrase a clause slightly wrong, attribute a term to the wrong party, or answer from general knowledge when the document is silent. Columbia's Tow Center measured even the best general assistant citing sources incorrectly 37% of the time, and fabricated legal citations have already drawn court sanctions, including the $5,000 fine in Mata v. Avianca.

All of these share one property: the output reads fluently either way. You cannot detect the error by looking at the answer. You can only detect it by looking at the source, which is why the tooling question below matters more than the model question.

The setup that captures the 200x speed gain without inheriting the failure modes has three parts.

1. Let the AI do the exhaustive pass: every document, every clause, every cross-reference. Exhaustiveness is what humans are worst at and machines are best at.

2. Route every finding through its citation. A flagged risk that links to the exact clause takes seconds to confirm or dismiss. A flagged risk with no anchor takes as long as doing the review yourself.

3. Keep the judgment human. Whether a clause is acceptable for this deal, this client, this risk appetite is not a retrieval problem, and the person who signs still owns every line.

Grounding: answers must come from your documents, not the model's general knowledge, and the tool should say when the documents do not contain an answer.

Passage-level citations: every claim links to the exact text it came from, one click away. Page-level or document-level pointers quietly reintroduce the manual search you were trying to eliminate.

Library scope: review questions span documents, so the tool must too: all five supplier contracts at once, this year's report against last year's.

Access control: review material is usually confidential. You need private-by-default storage and explicit, auditable sharing.

This is the shape Tatsulok is built around: a private library, questions across everything in it, and every sentence of every answer linked to the passage that supports it, including across curated public collections like Philippine law.