
What does AI do if it doesn't know the answer?
Key Facts
- AI models are 34% more likely to use confident language like "definitely" when hallucinating than when stating facts, per MIT research.
- Hallucination rates by model vary wildly: Claude 4.1 Opus fabricates 0% of the time, while Gemini 3 Pro fabricates 88%, per benchmark data.
- Eliminating AI hallucination is mathematically impossible given how large language models generate text, according to formal research.
- Air Canada was ordered to pay C$812 in damages after its chatbot invented a bereavement fare policy that didn't exist, per court records.
- Court rulings involving AI hallucinations surged from 10 in 2023 to 73 in just the first five months of 2025, per tracked data.
- Even when summarizing a document right in front of them, modern AI models still hallucinate 1% to 30% of the time, per Vectara testing.
- Purpose-built legal AI tools hallucinate 17% to 33% of the time, with GPT-4 hitting 43%, per Stanford RegLab research.
The Default Fallback Is a Confident Guess
Ask an AI agent a question it can't answer and you probably won't hear "I don't know." You'll hear a fluent, confident answer that may be entirely invented, and the customer has no way to tell the difference.
That behavior is architectural. Large language models generate text by predicting what should come next, so when a query lands outside what the model actually knows, the system fills the gap with the most plausible continuation it can produce. OpenAI describes hallucination as "a tendency to invent facts in moments of uncertainty," and Anthropic's 2025 interpretability work found that models contain internal circuits that decline to answer unless they know the answer, with hallucinations occurring when that inhibition fails incorrectly.
The result is a confidence paradox. MIT research from January 2025 found that models are 34% more likely to use confident language like "definitely" and "without doubt" when hallucinating than when stating facts, according to the same benchmark analysis tracking model fabrication rates. Tone is an anti-signal. A hedged sentence and a flat assertion can carry the same level of certainty, and a customer has no reliable way to sort them.
Refusal is possible, but it behaves like a model-level trait rather than a guarantee. Benchmark data on AA-Omniscience shows how wide the spread runs:
- Claude 4.1 Opus: 0% fabrication, because it refuses to guess
- Claude Sonnet 4.6: roughly 38%
- Grok 4: 64%
- GPT-5.1: 81%
- Gemini 3 Pro: 88%
Each of those responses reaches a customer as a finished answer. In Moffatt v. Air Canada (2024), the airline's chatbot invented a bereavement fare policy, and the British Columbia Civil Resolution Tribunal awarded C$812 in damages, calling the airline's argument that the chatbot was a separate legal entity "remarkable." The business was held responsible for what its AI said.
No amount of tuning removes the risk entirely. Xu et al. (2024) showed that eliminating hallucination is mathematically impossible given how these systems generate text, and Karpowicz (2025) found no model can simultaneously be factually correct, preserve source meaning, surface relevant knowledge, and stay within what it knows. You can optimize for three of those four.
For the agents we build at Agents by AIQ, that is why the design question is never whether an agent will ever hit an unknown. It will. The question is what the agent is configured to do in that moment, which is why our done-for-you AI receptionist, phone answering, voice, and support agent builds define confidence thresholds, no-go topics, and escalation paths before the agent goes live, since a customer will never be able to spot the difference between a confident answer and a fabricated one.
Why You Can't Engineer the Problem Away
Every mitigation helps. None of them finishes the job. If you're deploying AI agents against real customers, you need to know exactly where the engineering stops.
The hard ceiling comes from Xu et al.'s 2024 formal result: hallucination is mathematically impossible to eliminate given how LLMs generate text. OpenAI has publicly acknowledged the same, citing epistemic uncertainty and computational intractability as permanent factors, per research on hallucination rates and rankings. This isn't a maturity problem that another model version will fix. It's how the architecture works.
Grounding was supposed to be the cure. Retrieval-augmented generation connects a model to your actual documents, and IBM describes it as producing more accurate answers than the model alone could manage. But Stanford/Vectara testing from 2024 to 2025 found that every grounded tool tested still hallucinated, because summarization is still generation. Modern models hallucinate roughly 1% to 30% of the time even when summarizing a document sitting right in front of them. Purpose-built legal research tools hit 17–33%.
So when a vendor claims their system is "hallucination-free," treat that as a red flag. Stanford RegLab's verdict on such claims is "overstated." The more useful framing comes from Karpowicz: no language model can simultaneously deliver factually correct output, preserve source meaning, surface relevant knowledge, and stay within what it actually knows. You can optimize for any three. You cannot get all four.
There's also a failure mode that grounding makes worse, not better. A practitioner in a small-business discussion describes AI that "isn't necessarily 'making something up'" but is "confidently answering from incomplete or stale context." The knowledge base gets updated; the retrieval layer keeps pulling the old answer. The response sounds completely reasonable, which is exactly why nobody catches it. The highest-risk categories:
- Return and refund policies after a change
- Expired promotions still being quoted to customers
- Product compatibility answers pulled from outdated specs
- Delivery promises that create direct costs when wrong
This is why we at Agents by AIQ scope agents around failure modes rather than around the promise of a clean model. The realistic goal is an agent that escalates the right queries at the right moments, with context intact, and that verifies freshness on the answers most likely to cost you money. Anything a vendor tells you beyond that deserves your skepticism.
The Real Answer: Escalation Architecture
So what does a well-designed agent actually do at that moment of uncertainty? It doesn't guess. It routes. The documented operational answer across contact centers and enterprise deployments is confidence-threshold escalation: the AI evaluates its own certainty on every interaction and automatically hands the query to a human when confidence drops below a defined threshold, according to contact-center research.
That routing happens inside a tiered oversight structure. The same research describes three models. Human-in-the-loop means the AI pauses and requires human approval before acting, reserved for high-stakes or unknown complex queries. Human-on-the-loop lets the AI act autonomously within guardrails while humans monitor dashboards and intervene on threshold breaches. Human-out-of-the-loop is full autonomy with post-hoc review, used only for low-risk, well-bounded tasks where unknowns are rare.
Escalation also needs explicit triggers, and Decagon's documented framework lists four:
- Low confidence detection, the baseline trigger
- An explicit customer request for a human
- A sentiment threshold, such as detected frustration
- Policy-based rules, including legal disputes, fraud claims, and VIP customers
Those policy-based rules matter more than they look. CX industry guidance documents that some organizations define "no-go zones": topics the AI is never allowed to handle, which default straight to humans. NICE, for instance, programs its AI to flag uncertain responses for human review before they ever reach a customer. Given that hallucination is mathematically impossible to eliminate, as formal research shows, these boundaries are the real safety mechanism.
Then there's the part most deployments get wrong: the handoff itself. Decagon calls it "the point where automation meets human judgment," and the loop only works if the AI's work becomes the human's starting point. That means the human receives a full package: conversation summary, detected intent, account data, and any actions already taken. In voice contexts, a real-time transcript and a sentiment flag come along too.
Skip that, and you've solved nothing. If the customer has to re-explain their problem from scratch, the time savings evaporate and satisfaction drops. A dropped-context handoff doesn't just waste the escalation, it actively damages the experience you automated to improve. This is why at Agents by AIQ, scoping the escalation path and the context package is part of the build itself, not a setting bolted on later.
What This Means for Your Business (and Your Liability)
When your AI makes something up, the legal bill lands on you, not the model. That's the hard lesson from Moffatt v. Air Canada, where a tribunal ordered the airline to pay C$812 in damages after its chatbot fabricated a bereavement fare policy that didn't exist. Air Canada argued the chatbot was "a separate legal entity" responsible for its own words; the tribunal called that argument "remarkable" and rejected it outright, as court records show.
And this is no longer an edge case. Court rulings involving AI hallucinations climbed from 10 in 2023 to 37 in 2024, then hit 73 in just the first five months of 2025, with over 50 cases in July 2025 alone. Courts are catching up fast, and the pattern is consistent: you own what your agent says.
What this means in practice is that the build question changes. Since hallucination can't be engineered out (researchers have shown it's mathematically impossible given how LLMs generate text), you're choosing which failure mode you're exposed to, not chasing a hallucination-free model. Here's how that maps to a real agent build:
- Define no-go topics per vertical. Legal advice, medical questions, refund decisions, and delivery promises should route to a human by default. Practitioners flag refunds, product compatibility, and delivery commitments as the highest-risk categories because they "directly create a cost for the business."
- Set confidence thresholds and escalation paths at build time. Contact centers use confidence-based routing, sending queries to human oversight whenever the AI's certainty drops below a defined threshold. This is design work, not an afterthought.
- Verify knowledge freshness. A distinct failure mode isn't fabrication at all: the agent confidently answering from stale or incomplete context, like quoting a return policy you changed last month.
- Monitor escalation rates. If escalations spike, your knowledge base or thresholds need attention, not the model.
When we scope agent builds at Agents by AIQ, this routing and escalation design is the core of the work, because it's what actually determines your exposure. The model will always have gaps. Your job is to make sure that when it hits one, it hands off instead of guessing.
Frequently Asked Questions
What does an AI chatbot actually do when it doesn't know the answer?
Can you tell when an AI is making something up by how confident it sounds?
Don't newer models or better training fix AI hallucinations?
Doesn't grounding the AI in my own documents (RAG) stop it from making things up?
Who is legally responsible if my business's AI gives a customer wrong information?
How should a well-designed AI agent handle questions it can't answer?
The Question Isn't Whether Your AI Will Guess, It's What Happens When It Does
Here's the uncomfortable truth this article has been building toward: when an AI agent doesn't know the answer, it usually doesn't say so. It guesses, fluently, confidently, and often wrongly. That's not a bug waiting on a smarter model; researchers have shown it's mathematically baked into how these systems generate text. Grounding, retrieval, and better prompts all help, but none of them finishes the job. The documented operational answer is escalation architecture: confidence thresholds, no-go topics, and handoffs that carry full context to a human. And the stakes are real, court rulings involving AI hallucinations jumped from 10 in 2023 to 73 in the first five months of 2025, and businesses are being held responsible for what their agents say. So before you deploy an AI receptionist, voice agent, or support agent against real customers, ask a different question: not "can it answer everything?" but "what does it do when it can't?" That's the design decision that actually determines your exposure. If you'd like help scoping escalation paths, confidence thresholds, and no-go topics for your business, book a call with Agents by AIQ, we'll build the agent around the failure modes, not the fantasy of a perfect model.