Summary of an article published on The Web Scraping Club on 2026-09-21 (free to read). Read the full article

In November 2023 I concluded that you buy a scraper if you need it once and build it if you need a feed running for years, because a developer’s hourly rate was the biggest cost. Three years later, with a properly set up coding environment, the cost of writing and even maintaining a scraper is close to zero. That pushes toward building everything in-house again, and I wanted to check whether the numbers back it up.

We split the question into three decisions: code production, infrastructure (proxies, stealth browsers, anti-bot bypass, captcha solving) and execution. I describe the skills library I built for my own scraping agents. Then I redid the 2023 exercise at the same $20 an hour, adding LLM token costs and a traffic column. For traffic, I measured page weight on four ecommerce sites: ikea.it, mediaworld.it, etsy.com and asos.com. The comparison covers three tiers of anti-bot protection, plus maintenance, hidden costs and the sites no vendor sells.

What’s inside

  • Why the code layer is only near zero for teams that already built a harness of skills for their scraping agents, and what keeping those skills current takes.
  • How the pricing models differ: proxies billed per GB, unblockers and scraping browsers billed per request, and why that makes the total project price hard to predict.
  • What changes in maintenance, and why it does not change on the hardest targets, where the work is working out the bypass again.
  • Why building is cheap to start and expensive to leave, and the rule I suggest for deciding in advance how long an internal attempt gets before you buy.

In the full article: The cost table with break-even points for the three anti-bot tiers, the token and traffic assumptions behind it, the skills library details, and my verdict on which layer to build and which to buy.

Read the full article on The Web Scraping Club