2 weeks ago
p3k dots

A thought on JavaScript "proof of work" anti-scraper systems.

Letting your scraper run JavaScript means that it can also run JavaScript for other purposes, for example for people who would like to exploit your scraper's CPU to do some cryptocurrency mining, or simply have you run JavaScript for as long as you'll let it keep going (perhaps because they've recognized you as a LLM scraper and want to waste as much of your CPU as possible).

Relevant: Arsen und Spitzenhäubchen.