What it is

Feedstock is a high-performance web crawler and scraper for TypeScript, powered by Bun and Playwright. It supports single and multi-page crawling with concurrent execution, deep crawling using various strategies, and content extraction with multiple methods. The tool also includes features like smart caching, URL filtering, rate limiting, and built-in stealth mode to help avoid detection.

How it works

Feedstock uses Bun for fast and efficient execution and Playwright for browser automation. It supports multiple browser backends, including Playwright’s Chromium, Firefox, and WebKit, as well as generic CDP and Lightpanda. The tool can handle single and multi-page crawling with concurrent execution, deep crawling using BFS, DFS, BestFirst, Bandit, and Focused strategies, and content extraction using CSS selectors, regex, XPath, table, accessibility tree, and composite multi-strategy extraction.

TWSC experience

Not yet tested by TWSC.

Sources


The Web Scraping Wiki is compiled from the tests and research published on The Web Scraping Club, the newsletter for web scraping professionals. Subscribe to get the next article in your inbox.