Summary of an article published on The Web Scraping Club on 2023-04-11 (free to read). Read the full article

The proxy market is crowded and getting more so. For anyone scraping at scale, the useful questions are what a vendor’s unblocking products actually do, how the IP pool is built, and where AI is going in this space. Those are the points that decide whether a proxy provider fits a real scraping stack.

For our monthly interview I spoke with Aviv Besinsky, Director of Proxy products at Bright Data. I asked him how the company differs from other players and how Web Unlocker works, a product I had already tested on the newsletter. We also covered the recent Scraping Browser, which plugs into Playwright or Puppeteer, and the challenges of running a proxy infrastructure. I then asked whether AI will soon handle anti-bot, pagination and extraction from a simple description of the data. We closed with the ethics of sourcing residential and mobile IPs, and a fun fact from the early days.

What’s inside

  • How Bright Data positions itself against other proxy providers, including its compliance onboarding and its claim about the lag before competitors catch up.
  • What Web Unlocker automates behind a single request: proxy selection, fingerprints, retries and captcha solving.
  • What Scraping Browser handles for you, and what is left for you to write when you control it from Playwright or Puppeteer.
  • How residential and mobile peers are recruited through an SDK inside apps, with opt-in and opt-out for the device owner.

In the full article: Aviv’s complete answers to all seven questions, including his views on AI-driven scraping, the two infrastructure challenges he names and the early-days anecdote, plus the links to my Web Unlocker review.

Read the full article on The Web Scraping Club