When AI says "I don't know", it may be the most trustworthy thing it ever tells you
An AI that says "the sources do not contain this" is usually working correctly, not failing. Language models are built to produce an answer, so the honest gap is the harder behavior, and the one that protects you. This post teaches you to read refusals: how to tell an honest evidence gap from a broken one, and how to turn "insufficient information" into a sharper question or a missing source, so the gap becomes the start of better research instead of a dead end.
Updated

The answer nobody celebrates
You ask a careful question, and the tool comes back with: the provided sources do not contain enough information to answer this. There is a flicker of disappointment, maybe irritation. The other AI would have given you three confident paragraphs by now.
Hold that thought, because it is exactly backwards. The other AI would have given you three confident paragraphs assembled from pattern and plausibility, and you would have carried them into a meeting. The tool that stopped did something genuinely difficult: it noticed the edge of its evidence and told you where it was. In work where being wrong has a cost, that sentence is not a failure message. It is the product keeping its promise.
Why saying "I don't know" is the hard behavior
A language model's native instinct is to continue the text plausibly. Ask it for an answer and something answer-shaped comes out, whether or not truth was available; that is how fabricated citations are born. Stanford's benchmark of AI legal research tools found products marketed as hallucination-free still hallucinating on 17% to 33% of queries, precisely because filling the gap is the default and admitting it is the exception.
An honest refusal means someone engineered against the default: the system checked its retrieved evidence, found it insufficient, and chose the awkward truth over the fluent guess. When you see "not in your sources," you are seeing safeguards working in daylight.
Honest gap or broken refusal? Three questions
Not every refusal deserves applause. Tell them apart with three questions.
1. Is the refusal specific? An honest gap names its scope: "the uploaded contracts do not address early termination." A broken refusal is vague or global, refusing things the sources plainly contain.
2. Can it show you what it did find? A working system can list the nearest relevant passages even when they fall short. Nothing found in sources that you know are relevant suggests a retrieval problem, not an evidence gap.
3. Does a narrower question change the outcome? Honest gaps respond to precision. If no reformulation ever succeeds against sources that contain the answer, the refusal is broken, and it is fair to switch tools.
Turning the gap into your next move
An honest "I don't know" hands you a map of what is missing, and there are only three moves on it.
Narrow the question: "what does the contract say about termination?" may fail where "which clause governs termination notice periods?" succeeds, because precision helps retrieval find the right passage.
Add the missing source: the gap often means exactly what it says. The policy update, the amended schedule, the newer decision is not in the pile. Add it and ask again.
Or accept the finding: sometimes "the sources are silent" IS the answer, and a valuable one. Knowing that your contracts do not address a scenario is the discovery that starts the real work.
Choose tools that are allowed to say it
Here is the uncomfortable part: most AI tools are not designed to tell you when they do not know, because confident answers feel better in a demo. A tool can only be honest about its gaps if it is grounded in a defined set of sources and engineered to stop at their edge.
That is the standard worth demanding, and the one Tatsulok is built to: answers come only from your documents and curated collections, every claim links to its passage, and when the sources run out, it says so plainly instead of improvising. The refusal is not a limitation we tolerate. It is the feature that makes every other answer worth trusting.
A practice drill for this week, no signup needed: take any AI tool you use and ask it a question you know your materials cannot answer. Watch what it does. If it answers anyway, you have learned what its confident answers are worth. If it stops and says so, you have found a tool that respects the edge of its evidence, and you now know what that looks like.
FAQ
- Why does AI say there is insufficient information?
- In a grounded tool, it means the retrieved sources genuinely do not contain the answer, and the system chose honesty over improvisation. That is engineered behavior working against the model's default instinct to produce something answer-shaped regardless.
- Is an AI that refuses to answer worse than one that always answers?
- No, usually the opposite. Stanford's benchmark found tools marketed as hallucination-free still hallucinating on 17% to 33% of legal queries. A tool that admits its gaps gives you a truthful map; a tool that always answers hands you fluent text you must independently verify every time.
- How do I tell an honest gap from a broken refusal?
- Ask three things: is the refusal specific about what is missing, can the tool show the nearest passages it did find, and does a narrower question change the outcome? Vague, global refusals against sources that plainly contain the answer indicate a retrieval problem, not an evidence gap.
- What should I do when AI cannot answer from my documents?
- Make one of three moves: narrow the question so retrieval can find the right passage, add the source that is actually missing, or accept that the silence itself is the finding. All three are progress; only re-asking the same question and hoping is not.
- Can I make ChatGPT admit when it does not know?
- You can ask it to answer only from provided sources and to say when they are insufficient, and it helps, but it is a request, not a guarantee. Tools architected around grounding enforce the behavior instead of hoping for it.
Sources
Related guides

Verify AI answers
To verify an AI answer, demand a citation to the exact source passage and check the highlighted text against the original document. Here is how.
Read the article
AI hallucinations
AI hallucinations are confident but false AI answers. Learn why they happen, what they cost in the real world, and how cited answers help you catch them.
Read the article
AI slop
AI slop is low-effort, mass-produced AI content. Learn what it is, what 'workslop' costs teams, and how cited, verifiable AI creates value not noise.
Read the article