Guides · 7 min read

AI-Hallucinated Citations: Why a Working DOI Doesn't Mean the Paper Is Real

There is a comfortable assumption behind most citation checking: if the DOI resolves, the reference is real. Paste the DOI into a browser, watch a publisher page load, tick the box. That assumption is now the single biggest blind spot in reference checking — because the most common kind of AI-fabricated citation resolves perfectly.

How big the problem actually is

The rate of published papers containing at least one fabricated reference rose roughly six-fold in two years: from about 1 in 2,828 papers in 2023 to about 1 in 458 in 2025. Those figures come from the audit by Topaz and colleagues published in The Lancet, as reported by STAT News in May 2026.

The same team audited roughly 2.5 million biomedical papers and found that about 1 in 277contains at least one fabricated reference. Nature's own analysis of the phenomenon concluded that tens of thousands of 2025 publications may carry invalid AI-generated references.

This is not confined to low-visibility venues. A GPTZero scan of 4,841 accepted NeurIPS 2025 papers — a top-tier machine-learning conference, peer reviewed — identified at least 100 fabricated citations spread across 51 papers.

Two things follow. First, the problem is now common enough that “my co-authors would never” is not a defence: a single reference pulled from a chatbot by one collaborator is enough. Second, and more practically, the fabrications have changed shape.

The dominant pattern: right DOI, wrong paper

Early language models invented citations wholesale — fake author, fake journal, fake DOI. Those were easy to catch, because nothing resolved. The pattern that dominates now is different, and it is the one Topaz and colleagues single out:

A real, resolvable DOI is attached to a paper title that was never written. The DOI belongs to some genuine article — usually in a plausible journal, often in roughly the right field — but it is not the article the sentence claims to be citing.

Consider a reference like this, which is the shape of the fabrication rather than a real example:

Smith J, Roberts L. A randomized trial of mindfulness training
in adolescent athletes. Nature. 2021;596:583-589.
doi:10.1038/s41586-021-03819-2

The DOI is genuine and resolves instantly. It belongs to a real, heavily cited 2021 Nature paper. It has nothing to do with mindfulness training in adolescent athletes. A reviewer who clicks the link and sees nature.com load has confirmed nothing at all; a reviewer who reads the resulting page title has caught it in three seconds. Almost nobody does the second thing for all 60 references in a manuscript.

Why automated checks miss it too

Most citation-checking tools verify a reference by asking one question: does this identifier exist?They send the DOI to Crossref, get a 200 response, and mark the reference verified. Against the current fabrication pattern that check has a near-zero catch rate by construction — it is testing the one thing the fabrication gets right.

Catching it requires comparing the metadata behind the DOI against the title and authors written in the reference, and deciding whether they describe the same paper. That is a different and harder operation than a lookup, because it has to tolerate legitimate variation — preprint versus version of record, abbreviated journal names, transliterated author names, a title the authors themselves changed at proof stage — without waving through a genuine mismatch.

How to check a reference list properly

Whether you do this by hand or with a tool, these are the four checks that matter, in order of how much they catch:

  1. Does the DOI's registered title match the cited title?Not “does the DOI resolve.” This is the check that catches the dominant pattern. Crossref's public API returns the registered title for any DOI, so this is verifiable without a subscription.
  2. If there is no DOI, does the paper exist at all?Search the exact title in more than one index. A title that appears in no scholarly database — not Crossref, not OpenAlex, not Semantic Scholar, not PubMed — is very likely fabricated. One database missing it means little; four missing it means a great deal.
  3. Has the paper been retracted? Real, correctly cited papers can still be retracted, and citing retracted work uncritically is its own integrity problem. The Retraction Watch database is the reference source here.
  4. Do the author names line up? A fabricated reference frequently keeps a plausible first author from the field while inventing the rest. Compare the full author list, not just the first name.

The uncomfortable arithmetic: done properly, this is four lookups per reference across several databases. For a 60-reference manuscript that is a few hours of tedious work that nobody budgets for — which is precisely why the fabrication rate keeps climbing.

What PubScope does about it

Citation Checkcross-references each reference against Crossref, OpenAlex, DataCite, Semantic Scholar and arXiv, then against a local copy of the Retraction Watch database. Crucially, it does not stop at “the DOI exists.” When a DOI resolves but its registered metadata describes a different paper than the one cited, the reference is reported as a DOI mismatchrather than as verified — the Topaz pattern, reported as what it is.

It also reports the compound case, which is the one most likely to embarrass an author: the cited DOI belongs to a different paper and that paper has been retracted. Both facts are surfaced, with the Retraction Watch record.

We think claims like this should be checkable rather than asserted, so the test cases are published. Our validation pagelists every reference in a public 13-case set — including a DOI-mismatch case and several deliberately fabricated references — with the checker's measured result on each, so you can paste them in and reproduce the run yourself. Alongside it we run a larger holdout that we deliberately do notpublish, because a published set can be optimised against. The page states plainly what those numbers do and do not establish — including that building the holdout exposed a real bug and that one case counts as passed only under a scoring rule we changed. Thirteen published cases is a small sample, and we would rather hand you the cases than a headline accuracy percentage you cannot check.

See it for any journal on PubScope

PubScope brings the public signals together for tens of thousands of journals: Web of Science / Scopus / DOAJ indexing, SJR quartile, APC, a 0–100 Trust Score and predatory-risk flags — each linking out so you can confirm it at the source.

Check a journal Find journals for my paper

If you find a fabricated reference in your own draft

Sources

Figures on this page are quoted from the sources listed above and are not PubScope estimates. Media aggregations of the fabricated-citation count that we could not trace to a named primary study are deliberately excluded.