Summary of an article published on The Web Scraping Club on 2026-01-11 (free to read). Read the full article

A new wave of LLM-based scraping services is changing how we get data out of pages, and AnyCrawl is one of them. It is an open source service under the MIT license that turns websites into structured, LLM-ready output. It covers SERP collection, single-page extraction and full-site crawling, and you can self-host it with Docker. That makes it a candidate for data pipelines and AI workflows where you want control over infrastructure.

I went through what AnyCrawl is, why you might pick it, and how its three endpoints work: scrape, search and crawl. I also covered templates, the Playground and the pricing models. Then I built a Node.js example against the Quotes to Scrape sandbox. It chains a Google search with a scrape call that uses AI extraction driven by a prompt and a JSON schema. I close with the main pros and cons.

What’s inside

  • Scrape and search are synchronous, while crawl runs as an asynchronous job you create, poll and fetch in pages, and can cancel.
  • The scrape endpoint supports three engines: Cheerio for static HTML, Playwright for JavaScript-heavy pages and Puppeteer for deep Chrome integration.
  • Search currently supports only Google, and results can optionally be scraped one by one.
  • Templates let you reuse a configuration through a template_id. We compare the Template Gallery, which has only a few entries, with the Apify Store.

In the full article: The complete Node.js script, the AI extraction setup with prompt and JSON schema, the pricing tables, the parameter tables for each endpoint and my final list of pros and cons.

Read the full article on The Web Scraping Club