Summary of an article published on The Web Scraping Club on 2025-11-22 (free to read). Read the full article
The Lab articles that usually go out on Thursdays have dropped sharply in frequency over the past two months. Two things happened at once: daily work sprints to catch deadlines, and the birth of Edoardo, my first son. Lab pieces take time and careful study to write, and they are the main benefit for paying subscribers, so I wanted to explain the gap directly.
I am also announcing a new project for 2026, The State of Web Scraping report, which we are producing with Apify. We want it to be detailed and data-driven, built on what people who build, break and run scrapers every day tell us. For that we opened a short industry survey on proxies, infrastructure, anti-bot systems, AI-powered scraping and the tools that matter in 2026. Answers are anonymous.
What’s inside
- The slowdown has two causes, deadline-driven work sprints and a newborn, and not a change in editorial direction.
- The report is a joint effort between The Web Scraping Club and Apify, meant to map real trends in the industry.
- The survey is aimed at practitioners who run scrapers daily and covers five areas: proxies, infrastructure, anti-bot systems, AI-powered scraping and tooling.
- The form takes only a few minutes, and responses stay anonymous.
In the full article: The survey link, the form call to action and my full personal note on the slower Lab schedule are only in the full article.