Learn · Guide 8

Is ChatGPT accurate?

Not always. My AI agents run on OpenAI's GPT models, the family behind ChatGPT, and a second AI checking their work sent back nearly 1 in 5 pieces. Most mistakes weren't wild inventions. They were things left out, or claims that went a step past the source. Here's how I catch them.

By Faisal Hourani6 steps2 partsAbout 6 minutesTools: ChatGPT, Claude or Gemini, and the source your AI worked from

From Episode 7: How often my AI gets it wrong

The numbers · my AI agents’ work, five weeks

9,616Pieces of work checked

1,752Sent back at the first check, nearly 1 in 5

About 1 in 2Of the mistakes were things left out

986Fixed on the very next try

From my agents’ work log, 5 September to 12 October 2026. Most of the work was research: reading other businesses’ ads and pages and writing up what they sell and who buys. A second AI or an automatic check looked at every piece before I used it; I read 60 of the mistakes one by one. News answers from AI assistants had at least one significant issue 45% of the time in a study of 3,000+ answers by 22 public broadcasters (EBU and BBC, October 2025).

Part 1

What AI gets wrong

AI does most of my research now. I can't check it all myself, so a second AI checks every piece against its source.

STEP 1

Ask what it left out

A wrong fact stands out. A missing one doesn’t, because nothing on the page looks off. When AI sums up your reviews, a supplier’s sheet or a competitor’s page, the most common slip in my work was leaving something out, not making something up.

Try this, if it fits: when AI sums up your reviews, could you ask it “What did you leave out?”

WHAT I DIDAbout half the mistakes I read were gaps: 31 of 60. A buyer group, a product or a detail that was in the source but not in the write-up.

COPY THISHere is the source and your summary. List everything in the source a buyer would want to know that your summary leaves out. Quote the source line for each.

Lesson 1 from episode 7: I ask what it left out, about 1 in 2 mistakes were gaps

STEP 2

Ask for the line behind each claim

The slip is usually small: one step past what the source says. “Uses a tool” becomes “pays for a tool”. In a test on a coffee store’s product page, “sourced from” two producers became “grown by” them. Each sounds right, which is why it gets through. Asking for the exact line turns a feeling into a check.

Try this, if it fits: could you ask AI to quote the exact line behind each claim it makes?

WHAT I DIDAbout 1 in 4 mistakes went past the source: 16 of 60. A Reddit comment said "We also use copilot for PR reviews". My AI wrote "Their team uses a paid review tool."

COPY THISFor each claim in your answer, quote the exact line in the source that supports it. If there is none, write "not in the source".

Lesson 2 from episode 7: I ask for the line behind each claim, about 1 in 4 went past the source

STEP 3

Check every number

Numbers carry the most weight and are the easiest to check: either the source has it or it doesn’t. A made-up number in a report, a product page or an ad is the mistake a customer is most likely to catch.

Try this, if it fits: could you ask AI where each number came from, before it goes on your page?

WHAT I DIDAn automatic check compares every number in a draft with its sources. In five weeks, it sent back 13 of my AI's reports for a number none of its sources had.

COPY THISList every number in your answer. Next to each, quote the source line it comes from, or write "not in the source".

13 reports caught

Every number, its line

No line, no number

Part 2

Fixing it fast

Over half the mistakes were fixed on the very next try.

STEP 4

Send it back with the reason

“Try again” gets you a new answer with new mistakes. A note that names the problem gets the fix. Keeping everything else the same also stops the AI from breaking a part that was fine.

Try this, if it fits: could you tell AI exactly what was wrong, and ask for only that fix?

WHAT I DID986 of the 1,752 returns passed the very next check. Each return said what was wrong, where, and what the source says.

COPY THISFix only these things and change nothing else: [each problem, where it is, and what the source says].

Lesson 3 from episode 7: I send it back with the reason, over half fixed on the next try

STEP 5

Double-check who it's for

What a business sells is usually written plainly on its page. Who buys it often isn’t, so AI fills the gap with a guess that sounds reasonable. If you use AI to research competitors or your own market, that’s the line to check first.

Try this, if it fits: when AI says who a competitor sells to, could you ask for the line that says so?

WHAT I DID35 of the 60 mistakes I read were about who a business sells to: a buyer group missed, or one the source never showed. Only 4 were about what it sells.

35 of 60 about who buys

4 about what’s sold

Ask for the line

STEP 6

Test your checker with a wrong answer

A checker that passes everything looks the same as one that works. The only way to tell is to hand it something you know is wrong. Even my working checker missed some: 12 of the 60 mistakes I read had already got past one check, and a second look caught them.

Try this, if it fits: could you give your checker a wrong answer on purpose, and see if it notices?

WHAT I DIDMy first checker passed an empty draft with zero errors, and passed "Our product doubles your income.", a claim no source had. The fixed one fails both.

0 errors on a blank page

Plant a mistake

Then trust it

Want these checks run for you? The free AI double-checker at the end of this page takes what your AI wrote and what it was based on, and lists what’s missing, what goes past the source and every number without one, with a fix note to paste back.