AI hallucinations: checking the sourcesbefore you publish an article

The text is clean, the tone is right, and it cites a study with a perfectly plausible title. You click the link: page not found. Or worse, the page exists but says nothing like what the article claims. This is the quietest form of AI hallucination, and the most expensive one once the article goes live under your name.

I write this blog with a content machine I built, and I run others, including the one for Moonworkers, my company. Checking the sources of a text produced by an AI is not a final proofread there: it is a set of rules put in place before the writing even starts. Here is why these errors happen, where they hide, and the method I apply, with its limits.

01Why an AI invents sources that look real

A language model does not consult a library. Google says so plainly in its documentation: generative models do not retrieve facts, they predict a likely sequence of words based on their training data, which is why their output may contain inaccuracies known as hallucinations (Google Search Central). A bibliographic reference has a very regular shape. The model can produce that shape even when it knows nothing about the substance.

OpenAI adds an explanation I find helpful. Standard evaluations mostly score accuracy, a bit like a multiple choice test where a blank answer scores zero: a model that takes a guess gets a better score than one that admits it does not know. And rare, arbitrary facts, such as a birthday or the title of a thesis, follow no pattern the model could learn. Their example speaks for itself: asked for the title of one of their researchers' theses, a chatbot gave three different titles, all wrong (OpenAI, why language models hallucinate).

In an explainer video, someone from the Anthropic team points out a perverse effect: because hallucinations are getting rarer, people get into the habit of not checking (Anthropic, Why do AI models hallucinate?). I am wary of this in myself: the better the text, the more your attention drops.

02Where hallucinations slip into an article

From what I see, they rarely sit in the general reasoning. They sit in the precise details, the ones a busy reader does not check. The Anthropic video names the same risk areas: specific facts, statistics, citations, obscure or very recent topics, dates and names (Anthropic, Why do AI models hallucinate?).

In a blog article, that means:

• the round figure with no source ("most projects fail…") that comes from nowhere;
• the study with a credible title and a real author that was never published;
• the real source cited for a claim it does not contain, or whose conclusion has been turned around;
• the internal link to a page on your own site that never existed;
• the metadata: SEO title, description, image alt text.

That last point often surprises people. Yet Google asks you to fact-check all AI-generated content before publishing, including the title element, the meta description, structured data and image alt text (Google Search Central).

And "plugging the AI into documents" is not enough on its own. Stanford researchers tested legal research tools built on that principle (RAG, where the model reads documents before answering), which their vendors described as hallucination-free. Two of them gave false information in more than 17% of cases, a third in more than 34%. That was still better than general-purpose chatbots, which were wrong between 58% and 82% of the time on these questions (Stanford HAI, AI on Trial).

03My rule: no source gets in without being read

In my machine, checking starts before the writing. Every article starts from a brief, which I describe in my article on the content machine and human review. That brief holds a source register, and it decides what the text is allowed to cite.

Read, with what it supports

A source only enters the register with two fields: a note that it has been read, and its contribution, meaning the precise claim it supports. A page that cannot be read, behind a paywall or refusing access, is not cited. For this article, I opened every page cited, and the Anthropic video was read through its transcript, not guessed from its title.

An extract copied word for word

On the strictest version of my machine, each source also carries an extract copied word for word, which the tool then finds in the page. A paraphrased extract blocks publication. So does a conclusion that has been turned around: if the study says the effect is nil and the text says "modest", the check fails. This rule catches errors of interpretation, not only dead links.

Named titles, and no AI as a source

Each source has a title in the form "Organisation, page title". A title paraphrased from memory is often wrong, and a list of bare links tells the reader nothing. My blocklist also excludes chat assistants as sources: what a model says about itself or about a topic proves nothing. On AI, I cite the vendor's documentation.

I apply the same logic outside articles. At Moonworkers, the free calculators on the site were checked one by one against the official tools. And an "oracle" written from HMRC's own texts replayed hundreds of contribution cases: it found four one-penny errors, caused by a rounding rule. The source of truth is the official text, not the model's output.

04What the tool checks on its own, and what it cannot see

Part of the work automates very well. My final check, which I call the gate, rereads the article against its brief:

• any external link missing from the source register blocks publication: you read the page, add it, and validate again;
• in network mode, the tool opens every link: a 404 page blocks, a page that refuses robots has to be justified;
• a percentage or a headcount with no link in the same paragraph is flagged;
• internal links are compared with the list of pages that actually exist on the site.

Here is the limit, and I would rather say it. On most of my machines, the "read" note remains a declaration. The network check proves a page exists, not that it says what the article claims. Only the copied extract, where it is required, comes close, and it checks a single sentence. So judgement stays human: rereading each source's contribution against the sentence in the article. That is why every published article then goes through my own review.

On the model side, Claude's documentation suggests techniques I use too: allow the model to say it does not know, have it extract exact quotes before writing, and ask it to withdraw any claim it cannot find a quote for. Anthropic notes that these techniques reduce hallucinations without eliminating them (Anthropic, Claude documentation). I go into more detail in my article writing prompt.

05The checklist I run before publishing

You do not need a machine to apply the main part. Here is the order I follow, whether the article was written by an AI or by hand.

1. List every source cited, external and internal links included.
2. Open every link yourself. An error page, a paywall or an unrelated page: the source goes.
3. For each source, find the sentence in the page that supports the claim. If you cannot find it, rewrite or delete the claim.
4. Check that the conclusion has not been hardened or softened ("may" turned into "will", "nil" turned into "modest").
5. Every figure keeps its link in the same paragraph. A figure from a software vendor is presented as such.
6. Replace each source's title with the one actually on the page, with the organisation.
7. Reread the SEO title, the description and the image alt text with the same care as the body.
8. Remove any citation whose only origin is a conversation with an assistant.

This list takes time, often more than the model's first draft. I do not know an honest shortcut.

06Where to start on Monday

Take the last article you published with help from an AI and run points 2 and 3 of the checklist on it. If a single source does not hold up, you know where the next rule belongs: in the brief, before the writing, rather than in a last-minute proofread.

If you want to go further, the most useful step is often to write these rules down somewhere a tool can reread them. That is what I do on my engagements, and you will find a few example projects in the portfolio. To talk it through from your own situation, everything is on the services page. And if you use Claude day to day, my article on Claude for marketing goes with this one.