Summary of an article published on The Web Scraping Club on 2025-02-16 (free to read). Read the full article
The travel industry was one of the first to go digital, and it still produces a huge amount of data: schedules, prices, seat availability, delays, cancellations. Airlines, OTAs, flight trackers and airports all publish part of it. Some OTAs and tracking services also sell API access, but in some cases they reserve it for travel industry operators. For investors, analysts and operators, knowing where each piece of data lives, and how often it changes, decides whether a scraping project is worth building.
We map the main actors in the flight industry and sort the data into three categories: routes, flights and bookings. We then look at what each category is good for, from dynamic pricing to market analysis. Three use cases get a closer look: estimating the fill rate of a flight from booking data, measuring cancellations and delays to judge how an airline or airport operates, and using flight volumes as a soft signal of economic activity. For each one we say how often the data needs to be collected.
What’s inside
- How a booking site’s limit on tickets per seating class can hint at how many seats are left on a flight
- Why the fill rate matters for listed airlines, and why short-distance routes are a special case
- Which refresh frequency fits each use case: weekly for routes, daily for delays and cancellations, a few hours before takeoff for seat availability
- Why flight volumes are only a soft signal of the economy and need a long collection period to mean something
In the full article: The details on each data category, the screenshots of the booking and flight status pages used in the examples, and the full reasoning behind the three use cases are only in the full article.