Summary of an article published on The Web Scraping Club on 2025-09-21 (free to read). Read the full article
Every new model release brings back the same question: how do I use LLMs in my scraping workflows? As of today, they mostly belong to the parsing step, after you get the HTML and before you save the data. They are not replacing traditional scrapers. Pre-trained models are generalists, and they struggle with long pages, raw HTML and clean JSON output, which is a problem when your target has industry-specific quirks.
I explain what fine-tuning really is, using Chip Huyen’s definition, and why it is not the same as RAG. Then I look at instruct fine-tuning and at industry-specific cases, with Zillow listings and Airbnb amenities as examples. I compare two ways to do it, custom code and automation tools like n8n, based on my own experience. The real question is ROI, and most of it sits in the dataset.
What’s inside
- Why pasting raw HTML into an LLM does not work: token limits, weak HTML understanding, and unreliable JSON on large outputs, plus the HTML-to-markdown workaround.
- Fine-tuning changes how a model follows instructions and formats output. Adding knowledge about a field is the job of RAG.
- Real estate shows where a generic model fails: abbreviations like bds and ba, ambiguous land descriptions, and amenities a host does not offer.
- The coding route and the automation route are compared on required skills, time, initial cost and flexibility. The costly part is building the labeled dataset, which you can often get by scraping.
In the full article: Zillow and Airbnb screenshots with the parsing problems marked, the side-by-side comparison of coding versus automation tools on skills, time, cost and flexibility, and links to my Llama4 and n8n fine-tuning walkthroughs.