Summary of an article published on The Web Scraping Club on 2026-04-19 (free to read). Read the full article
Next.js is one of the most widely used full-stack frameworks, and it sends a lot of structured data inside the HTML the server returns. Hydration needs that data to rebuild the React component tree in the browser, so it has to be serialized into the page. For scraping, that means you can often skip DOM parsing and browser automation, and sometimes get more than what the page displays.
I show a trick that works on virtually any Next.js site, and I explain why it works. I start with how hydration works and what ends up in the HTML. Then I cover the two cases you run into, depending on whether the site uses the Pages Router or the App Router. I use a Nike product page for the first and an OpenAI page for the second. I combine both into one short JavaScript script, explain how to move it from DevTools into a normal HTTP client with Beautiful Soup or Cheerio, and list pros and cons.
What’s inside
- Why the same rule applies to Remix, Nuxt, Angular Universal and SvelteKit: if a framework hydrates, it must serialize data into the HTML.
- Why the React Flight format used by the App Router is not plain JSON, and why the client decoder does not work outside the original build.
- The caveat that the hydration data can be partial, so you may still need traditional scraping to complete it.
- Why the hydration scripts are in the server HTML, so a plain HTTP request is enough and no client-side rendering is needed.
In the full article: The JavaScript snippets for both approaches, the combined script, the DevTools screenshots of the Nike and OpenAI pages, and the Gemini-parsed React Flight example are only in the full article.