Summary of an article published on The Web Scraping Club on 2023-02-06 (free to read). Read the full article
Browser automation is the default answer when a target renders content with JavaScript or needs a logged-in session, and Playwright is one of the names that comes up first. Before picking it for a project, you want to know what it is, what it runs on and where it fits in a scraping stack.
This is a short introduction to the library. I describe what Playwright is and which browser engines it drives: Chromium, WebKit and Firefox. Then I go through the reasons to use it for scraping: realistic interaction with pages such as clicks, form submissions and scrolling, handling of browser sessions, cookies and local storage, running the same script on different browsers and versions, and running many scraping tasks at once. It is a starting point, not a tutorial, and it does not cover anti-bot systems.
What’s inside
- Which browser engines Playwright drives and why that matters for keeping scripts on current browser versions.
- How session, cookie and local storage management applies to targets that require a logged-in user.
- What running the same scraping script across several browsers means for compatibility.
- How Playwright handles several scraping tasks at once on larger projects.
In the full article: The full text of the introduction, with the complete reasoning behind each of the four reasons to choose Playwright and my closing verdict on it for scraping projects.