Summary of an article published on The Web Scraping Club on 2023-01-08 (free to read). Read the full article

Proxy providers sit at the base of almost every scraping stack, yet few of us know how they get their IPs, how they check quality, or why a proxy company ends up selling scraping APIs. For anyone choosing or combining providers, those answers matter more than a feature list.

For this monthly interview I talked with Martin Ganchev, VP of Enterprise Partnerships at Smartproxy. We covered how the company moved from self-service residential and datacenter proxies to a scraping infrastructure business, and how the proxy market works from the inside. We also went through how IPs are procured and tested, what a customer should check when picking a provider, and the current API lineup. I also brought him a market research figure on in-house scraping versus bought datasets, and asked where scraping tools are heading. It ends with a story from his first day at the company.

What’s inside

  • How Smartproxy sources exit nodes, including wholesalers and the consent requirement for end users’ devices.
  • The quality checks applied to IPs: stability, speed, location in databases, stress testing and an internal selection algorithm.
  • Why Martin says large clients often use several proxy providers, and how he ties price to reputation and quality.
  • His view on why big retailers scrape in-house, why smaller companies buy datasets, and why some do both.

In the full article: Martin’s complete answers to all ten questions, including his comments on the API portfolio, the market research figure I quoted, and the first-day anecdote.

Read the full article on The Web Scraping Club