Summary of an article published on The Web Scraping Club on 2026-08-23 (free to read). Read the full article
Scrapy owes its success to a structured way of building spiders. If you work in Go, you usually give that up or add a Python or JavaScript stack, with another runtime and another set of dependencies to maintain. GoScrapy tries to bring the same philosophy to Go, with its performance, goroutine concurrency and single-binary deployment.
I looked at what GoScrapy is, how its engine is built and where it differs from Scrapy. The core pieces are the signal-driven architecture, the built-in middlewares and pipelines, and Gosm, a declarative mapping engine that describes extraction logic in struct tags. Then I built a complete spider on the ScrapingCourse.com e-commerce demo site. It goes from project scaffolding to extracting product data across all pagination pages and exporting it to CSV. I close with my view of the project’s maturity, its small community and its licensing.
What’s inside
- How the central signal bus decouples the engine from its components and which lifecycle events it exposes.
- Why Gosm collects each field as a separate array across all matching elements, and what that means when you rebuild individual records.
- How the generated project layout maps to Scrapy’s, and the role of the spider, record and settings files.
- The caveats I found: a small community, no real official documentation, little LLM knowledge of the framework, a v0.x API and the BSL license.
In the full article: The full Go code for the Gosm models, the spider and the CSV export, the commands to scaffold and run the project, the target page selectors, the proxy setting from the FAQ and my final assessment.