Summary of an article published on The Web Scraping Club on 2022-03-12 (free to read). Read the full article
Sales teams still spend a lot of time on manual research to find prospects. Web scraping automates the collection of business and contact data from many online sources, so a team can build and refresh lead lists without adding headcount. For a practitioner, the useful question is where that data lives and how to use it without creating legal or technical trouble.
I go through how scraping fits into lead generation and what it brings to prospecting: precise targeting by industry, role, company size or location, scalability, up-to-date data, competitive intelligence and lower costs. Then I list the sources we most often see used, from LinkedIn and business directories to company websites, professional networks like Crunchbase and AngelList, and social media. I also cover best practices on privacy law, Terms of Service, robots.txt and server load, and close with a B2B example from a cybersecurity firm.
What’s inside
- Which five kinds of sources feed a lead list, and what each one is good for.
- How competitor websites and client lists can be used as a source of prospects.
- Three best practices: legal compliance, respecting Terms of Service and robots.txt, and limiting server load.
- A cybersecurity firm example showing how scraped data turns into targeted outreach campaigns.
In the full article: Source-by-source descriptions of what each platform offers for prospecting, the full list of benefits, and the cybersecurity firm example are only in the full article.