AI hallucinations: why your AI lies to you and how I catch it

TL;DR: Your AI lies to you with complete confidence, and it lies most reliably when you ask it to back up something you’ve already written. Those invented citations, numbers, and facts are called AI hallucinations, and they haven’t gone away. I’m Julie Kaiser, a science writer who’s been fact-checking AI on real client work since 2023, and what catches them is a reflex, not a tool: treat every specific as unverified until you’ve checked it, and ask one question out loud, every time. Is that true?

The first time I caught my AI lying to me

Nine days into using ChatGPT for the first time, June 2023, I was working on a technical writing job. I asked for a disease related to a particular gene dysfunction. The answer was good. Accurate, on point, correctly cited – and not interesting.

So I asked for a different example, thinking a sexier disease angle might be more interesting for my reader. ChatGPT delivered: a more interesting disease, complete with the mechanism, the signaling pathways involved, and a citation – a real scientist’s name, a real journal, a DOI link, a plausible-sounding title.

The article doesn’t exist. I searched and searched, found nothing, and typed back exactly this: “i can’t find that article.” Not proud of the lowercase. It was a fast, casual message, the kind I used to type before I started dictating. And ChatGPT folded instantly – no defense of the source, no clarification, back to the original correct answer as if the invented one had never happened.

Nine days. That’s how long it took to learn the most useful thing I know about these tools. The single most reliable way to make an LLM lie: ask it for a citation for a claim you’ve already written.

What is an AI hallucination, really?

An AI hallucination is a chatbot stating something false as if it were established fact – an invented citation, a wrong number, an event that never happened – in the same fluent, confident tone it uses for everything true.

The mechanism is unglamorous. A language model generates the most statistically likely next words, period. It isn’t checking claims against reality, because there is no reality inside the model to check against. When the truth is well covered in its training data, plausible and true mostly coincide. When you ask for something that doesn’t exist, the model does the only thing it can do: it produces something answer-shaped. “Lying” isn’t quite fair, since lying takes intent. But from where you’re sitting, the difference doesn’t matter. The text is confident either way.

Why does ChatGPT hallucinate?

ChatGPT hallucinates because it’s built to produce the most plausible answer, not the most verified one – and when no true answer exists, a plausible one gets assembled on the spot from real-sounding parts.

That’s why my day-9 citation contained a real scientist’s name and a real journal. The name and the topic sit near each other in the training data, and a real name inside a fake reference is exactly what plausible looks like.

Notice what my request actually did, though. I had an accurate example in hand and asked for a better-sounding one. I wasn’t asking what the evidence says. I was handing the model a conclusion and requesting support, and it obliged – these models usually do. Ask for a citation for a claim you’ve already written, and you’ve ordered the lie in advance.

Does AI still hallucinate in 2026?

Yes. Less than in 2023, and less obviously – which is its own problem.

Today’s models invent fewer sources out of whole cloth. The misses run subtler now: a real paper attached to a claim it doesn’t quite support, a real number from the wrong year, a quote that’s ninety percent right.

An obviously fake source filters itself. A nearly-right one sails through. Hallucinations didn’t disappear as models improved – they got harder to spot.

How do you know if your AI is hallucinating?

Mostly, you can’t tell by reading. Confidence and fluency look identical whether the content is true or invented, so you find out by checking the specifics: the citation, the number, the name, the date.

Three checks do most of the work for me:

• Search for the source. A real article surfaces within ten seconds of searching. If I can’t find it fast, I treat it as fiction until proven otherwise.
• Push back, plainly. “i can’t find that article” was all it took on day 9 – but invented sources don’t always fold on the first challenge. Sometimes the model doubles down with more detail instead: a link to where you’d find it, a title, an author list. So I ask directly: did you make this up? Did you find it somewhere, or did you guess from your training data? Often enough the answer comes back – I guessed, sorry, let me look it up now. Real sources survive the questioning.
• Distrust the convenient answer. The response that fits your argument perfectly, on the first try, is the one to check hardest.

Whether ChatGPT is accurate overall is a bigger question, and it has its own post.

What I do now: the fact-check reflex

My verification system is mostly one question, asked every time an answer sounds even slightly convenient: is that true?

This isn’t a vibe. I went back through three years of my ChatGPT archive and counted: I challenged the AI 88 times across 72 conversations, including “is that true” 30 times, “is this accurate” 20 times, “is that real” 13 times, and – twice, flatly – “did you make this up”. Not a vibe. A documented three-year habit. And that’s ChatGPT alone – I was using Perplexity and Gemini over the same years, and none of those challenges are in the count.

The question isn’t magic. What it changes is my posture: every AI answer is a draft claim, not a fact, until something outside the conversation confirms it. The 88 challenges are what that posture looks like in practice, and each one costs about five seconds.

They also mean everything I didn’t challenge. I’ve missed things – I don’t know how many. None of us do.

I do this fact-checking for biotech and pharma teams too – it’s part of the writing work on my Work with me page.

What about AI detectors?

AI detectors are interesting because they have the trust problem running in the other direction: instead of an AI inventing facts about the world, the detection software invents facts about writers.

Same failure mode – confident output nobody verified – but now the false claim is about you. One writer who interviewed colleagues in r/freelanceWriters about detector false positives reported: “Many of the writers I spoke with have faced dire consequences, including loss of assignments, unpaid invoices, and even termination from freelance positions after their work was evaluated by an AI checker.”

A hallucinated citation and a false AI-writing accusation teach the same lesson: fluent, confident, machine-generated output is not evidence. It’s a claim, and somebody still has to check it. AI writing detectors and their false positives have a full post of their own.

Which AI has the most hallucinations?

There’s no stable answer, and a ranking wouldn’t help you much anyway. Leaderboards reshuffle with every model release, and hallucination rates depend more on what you ask than on which model you ask. The same model that summarizes a document faithfully will invent a reference the moment you request support for a shaky claim.

If you want a museum of AI hallucination examples, the AI Hallucination Cases database tracks court cases where fabricated citations made it into real legal filings. Lawyers keep learning the day-9 lesson in public.

The hallucination rate that matters is the one in front of you, and your prompt controls it more than the model does. Hand any of them a conclusion and ask for support, and you’ve invited the lie.

The one thing to take from this

Treat every specific your AI hands you – citation, number, name, date, quote – as unverified until you’ve checked it somewhere the AI can’t touch. That’s the entire method, and the question that starts it costs five seconds: is that true?

I’m not quitting these tools. They’re how I work now. But the reason I check everything – the reflex, the 88 challenges – is that I refuse to put slop out under my name.

And if the worry underneath this one is bigger than citations – whether using AI counts as cheating at all – I’ve answered the six versions of that fear straight: Is using AI cheating?