citation-rot
A monthly re-check of 1,991 URLs cited in a public dataset of AI search answers, recording dead, moved and changed pages. Every record is in git.
- Status
- Live
- Launched
- Licence
- MIT
- Language
- Python
One command, copied from the repo.
Install citation-rot: git clone https://github.com/rishsadh/citation-rot && cd citation-rot && python -m citation_rot report
rishsadh/citation-rot
What it does
It draws 2,000 URLs from a published dataset of AI search answers (Search Arena, from LMArena), removes nine that a privacy audit flagged, freezes the remaining 1,991 as a panel, and re-checks the same URLs every month. Each month's result is one JSONL file in git, with a status, the redirect chain, the final URL, the title and a simhash of the visible text for every URL. The page itself is never stored. It needs Python 3.10 or newer and has no dependencies.
The run on 30 September 2026 found 62.9% of the URLs live, 6.5% moved, 4.4% dead, 2.7% unreachable or server error, 22.0% blocked and 1.5% other. Of the 1,523 pages that could be judged, 17.7% were not live (95% interval 15.9% to 19.7%). The URLs were cited in answers from 18 March to 13 April 2025.
About one URL in five was blocked, so its state is unknown. The panel has equal quotas per engine family, so quote the per-engine rows, not the combined figures. The figures are not a replication of the 19.3% in the study named below: the engines, dates, sample and treatment of redirects and blocked pages all differ. A single check is noisy, and repeat passes disagreed on about 1% of URLs. It names itself in the user agent, reads robots.txt, waits at least 2 seconds between requests to one host, and never fetches what robots.txt disallows.
The seed URLs come from model output, so the providers' terms apply to them and CC-BY-4.0 does not. 849 of the URLs in the panel are in the google engine family, and METHOD.md has a note on Google's terms for grounded results. Read it before reusing the data. This is not legal advice.
Disclosure: Rish Sadh runs Reidify, a studio that sells AEO services. This is published as data with its method, not as a pitch.
Why it exists
On 24 July 2026 OtterlyAI reported that 19.3% of the URLs cited by seven AI search engines in one month did not return a live page, a share that counts redirects and unreachable responses as not live, and said its figures cover roughly a third of a corpus of about 20 million URLs. It is one snapshot, and the URL list is not published there. A search of GitHub, npm and the web on 30 September 2026 did not find a tool that re-checks a fixed set of cited URLs every month and publishes every record.
This is a small version of that. Each month's result is one file in git, so the decay, the recoveries and the pages that changed under the same URL can be read from the history.