What it is
The AI Page Audit Tool is a free utility designed to assess a webpage’s suitability for being read and quoted by large language models such as ChatGPT, Claude, Perplexity, and Google AI. It checks the page against three core criteria—Reach, Read, and Trust—to identify specific issues that prevent AI crawlers from successfully processing and citing the content.
How it works
The tool evaluates a page based on three pillars: Reach, Read, and Trust. Reach checks if AI crawlers can fetch the page, looking for issues like robots.txt rules, firewalls, noindex, or nosnippet tags. Read assesses the content quality by analyzing the HTML text without JavaScript, headings, and response speed, as most AI crawlers do not execute JavaScript. Trust evaluates the page’s citation quality by checking for structured data (JSON-LD), the updated date, and whether the page directly answers buyer questions.
TWSC experience
Not yet tested by TWSC.
Known limitations
The tool can only audit public pages, such as homepages, pricing pages, or blog posts. It cannot audit pages that are behind a login or on private networks. Furthermore, the audit does not determine whether AI models will name the user; that depends on the entire web.
Related
Sources
The Web Scraping Wiki is compiled from the tests and research published on The Web Scraping Club, the newsletter for web scraping professionals. Subscribe to get the next article in your inbox.