What it is

Zenrows Batch is an asynchronous scraping API designed to manage large URL lists as a single, managed job. It abstracts away the need for users to build the infrastructure required for large-scale scraping, including queuing, concurrency management, retry logic, and result delivery.

How it works

Batch functions as an async scraping API where users submit a list of URLs and a scraping configuration in a single request to receive a job ID. The system queues each URL, handles protected access using mode=auto, and automatically retries any failures that are transient.

Upon completion, the job notifies the user either by triggering a webhook or by allowing the user to poll a status endpoint. Submissions can be handled in three ways: inline submission for closed jobs, batching URLs over time for open jobs, or uploading a CSV file to reference the URLs.

TWSC experience

Not yet tested by TWSC.

Sources


The Web Scraping Wiki is compiled from the tests and research published on The Web Scraping Club, the newsletter for web scraping professionals. Subscribe to get the next article in your inbox.