😠

Perplexity is using stealth, undeclared crawlers to evade website no-crawl directives (via lobste.rs).

Perplexity is repeatedly modifying their user agent and changing IPs and ASNs to hide their crawling activity, in direct conflict with explicit no-crawl preferences expressed by websites.