Summary of an article published on The Web Scraping Club on 2023-02-06 (free to read). Read the full article
Most headless browsers were built for testing, not for scraping. Splash is one of the few designed for data extraction from the start, and that matters when the target renders content with JavaScript and a plain HTTP request returns an empty shell.
In this short piece I introduce Splash and what sets it apart from other headless browsers: a Lua scriptable interface, a fast JavaScript rendering engine and an HTTP API. I then go through the jobs it is used for. These are extracting text, images and other content from websites, automating tasks such as filling out forms and clicking links, and scraping dynamic sites that depend on JavaScript. It is an overview for people who have not used the tool yet and want to know what it does before deciding whether to try it.
What’s inside
- The three features that separate Splash from general-purpose headless browsers
- Why the HTTP API matters when you plug a browser into a scraping pipeline
- Which tasks Splash can automate besides extraction, such as forms and link clicks
- The kinds of users the tool targets, from data scientists to businesses collecting marketing data
In the full article: Descriptions of each Splash feature and the full list of use cases, with the examples for data scientists, businesses and developers.