Summary of an article published on The Web Scraping Club on 2025-08-10 (free to read). Read the full article
Scraping reviews is only the first step. Companies collect them to get something out of the data, and sentiment analysis is the usual way: it turns unstructured customer opinions into numbers you can compare across products. Reviews are a goldmine, but nobody reads thousands of them by hand, so the method you pick decides how useful the output is.
This is the theory half of a two-part series, and the code comes in Part 2. I go through the five approaches to sentiment analysis: lexicon-based, machine learning, deep learning, hybrid, and cloud APIs. For each one I cover how it works, what it needs and where it breaks. We then run three sample reviews through the logic, from a plain positive one to mixed sentiment and sarcasm. Last, I explain why, for Amazon reviews, I prefer commercial scraping APIs over custom code, and I list the providers: Bright Data, ScraperAPI, Oxylabs and Decodo.
What’s inside
- Lexicon tools such as VADER and TextBlob are fast, but sarcasm and domain jargon confuse them.
- The machine learning pipeline runs from labeling to prediction, with bag-of-words and TF-IDF for features, and needs a good amount of labeled data.
- Hybrid setups send only the ambiguous reviews to heavy models, which saves compute.
- Four Amazon scraping obstacles: pagination and login, blocks, HTML changes, and scraping many products at once.
In the full article: The full tokenization and scoring steps for each method, the trade-offs of Google Cloud Natural Language and Amazon Comprehend, the worked review examples with their expected classifications, and the provider-by-provider list of Amazon scraping APIs.