Is ChatGPT accurate? What it gets wrong and how I check it

TL;DR: ChatGPT is accurate often enough to be useful and wrong often enough that you can’t skip the checking. The errors that actually bite aren’t the invented facts – they’re the plausible answers propped up by citations that don’t say what they’re cited for. I’m Julie Kaiser, a science writer, and my check comes down to this: click the source, read what it actually says, and ask whether your own question forced the answer.

Can I trust ChatGPT?

You can trust ChatGPT to be useful. You can’t trust it to be right, and it won’t warn you which one you’re getting.

The failure everyone knows about is the fake source – the confident answer with nothing real behind it. I learned that version early, on day 9 with ChatGPT, when it invented a citation out of thin air. That story, and the fact-check reflex it built, live in the flagship post on AI hallucinations.

This post is about the failure that comes after you know all that. The wrongness that arrives looking reasonable, wearing a source.

Can you trust AI citations?

No – a citation in an AI overview or an AI report doesn’t mean what a citation means in a scientific paper, and treating the two as the same is how plausible nonsense ends up in your work.

Here’s what I mean. You can ask Perplexity a question and it will give you a very nice-sounding report with citations to sources. Things that look like citations to sources. Then you take the time to click through to those primary sources and read what they actually say about the topic, and often you find one of two things:

• the source doesn’t say anything at all about what it’s being cited for, or
• the source is talking about the same general topic the report covers, and nothing more.

In a rigorously written paper, a citation means “this is the source of this information.” Sure, we can’t always trust those either. But they don’t mean the same thing as an AI citation, and I say that as someone who evaluates scientific literature for a living. ChatGPT’s sourced answers earn the same click-through, every time.

How do I check if ChatGPT is telling the truth?

You check ChatGPT the way you’d check any unverified claim: take the specific it gave you – the source, the number, the name – and follow it to something outside the conversation.

The citation click-through does the heaviest lifting. Read the actual page, not the reassuring shape of the reference, and ask whether it says the thing it’s being cited for. Most of the time that settles it.

The rest of my checks live in the flagship post: search for the source, push back once and see if the answer folds, distrust anything that fits your argument a little too well. The short version is one question, asked before anything ships. Is that true?

The goading trap

Sometimes you goad it. It’s the way you ask the question that forces it to give you the answer to that question.

That invented citation from day 9? I had goaded it into existence. I’d asked ChatGPT for a better, more interesting example of a disease connected to the gene dysfunction I was writing about – and it obliged: a more interesting example, completely made up, with a scientific citation to go with it. Ask for a better example and you’ll get a better example, whether or not one exists. Ask why X is true and you’ll get reasons X is true, whether or not it is. The model answers the question in front of it.

I’d love to tell you the model makes all the mistakes here. It doesn’t – sometimes the goading is mine.

Is ChatGPT reliable?

Reliable enough to start with, not reliable enough to finish with – and knowing which stage you’re in is most of the skill.

I know ChatGPT gives me really useful information. I also know that if the question I’m asking really, really matters, I need to be very careful with trusting it, and I do some other backups. You can get an idea about things. In some fields it’s probably better than others. In some it’s not.

So the stakes set the workflow. Getting oriented in a new topic? Light checking – I’m collecting directions, not facts. Drafting something a human will edit and check anyway? Medium – I flag the load-bearing claims as I go. Anything that ships with my name on it, or a number in it? The full treatment, every specific followed to its source.

How I fact-check ChatGPT

My AI fact-checking workflow is small on purpose: the citation click-through from this post, plus the “is that true?” reflex from the flagship post. Two habits, not a system.

If that sounds exhausting: most of what ChatGPT tells me never needs the full check, because most of it never leaves the conversation. The checking scales with what ships. A brainstorm can be wrong in six places and still do its job. A published number can’t.

The one line to remember

An answer from ChatGPT is a draft claim, not a fact. The checking is what makes it accurate, and nobody does that part but you.

Verify before you ship.