Summary of an article published on The Web Scraping Club on 2024-11-21 (for paid subscribers). Read the full article

Telegram’s public channels and groups are a good source for brand perception, OSINT and AI training data. I’m in several of them myself, from local and global news channels to a matched betting group I joined only out of curiosity about the math. The catch is knowing what you can legitimately touch. Public channels and groups are in scope, private ones are not, and the legal and ethical frame comes before any code.

We start from how Telegram is built: public channels, public groups, private groups and bots. Then we compare the tools: the official Telegram API, Telethon, Pyrogram, and BeautifulSoup or Selenium on the web interface. We chose the API with Telethon. From there we get API credentials, install Telethon, and build a first scraper that pulls messages from a public group and retrieves information about its members.

What’s inside

  • Why the official API is the most reliable and scalable route, while scraping the web interface is less efficient and more prone to automation blocks.
  • Where the API credentials come from: the API Development Tools section on my.telegram.org, after logging in with your phone number.
  • How Pyrogram compares with Telethon, and why bots are a separate case handled through Telegram’s Bot API.
  • A caveat on legality: staying on public data and not storing personal data keeps you on safer ground, while private groups without consent do not.

In the full article: Telethon scraper script from the GitHub repository, the message extraction code for public groups, and the code for retrieving member information.

Read the full article on The Web Scraping Club