What it is

CrowdSec is a Web Application Firewall (WAF) solution that has incorporated bot detection capabilities. This feature allows the WAF to react not only to user actions but also to the identity of the user, distinguishing between legitimate human users and automated bots. It is designed to efficiently deal with distributed attacks, scraping, scalping, and other forms of aggressive automation.

How it works

The bot detection approach combines several techniques, most notably fingerprinting and proof of work. This combination forces attackers to engage with anti-detection measures while simultaneously increasing the computational cost of botting, thereby raising the economics of automated attacks.

CrowdSec also features a dedicated Kubernetes datasource that interacts directly with the Kubernetes API using pods/logs. This allows users to deploy a single CrowdSec instance to monitor multiple pods, simplifying the build, run, and architecture of Kubernetes deployments.

TWSC experience

Not yet tested by TWSC.

Sources


The Web Scraping Wiki is compiled from the tests and research published on The Web Scraping Club, the newsletter for web scraping professionals. Subscribe to get the next article in your inbox.