Anubis is awesome! Stopping (AI)crawlbots
Daniel Quinn @ danielquinn @lemmy.ca Posts 27Comments 846Joined 2 yr. ago

Daniel Quinn @ danielquinn @lemmy.ca
Posts
27
Comments
846
Joined
2 yr. ago
Are there any repair/Linux cafés in Cambridge that might want to contribute to "EndOf10"?
The number of lines for each character by percentage of the series
Boy on quad bike arrested on Chisholm Trail for 'dangerous driving'... carrying a machete
This all appears to be based on the user agent, so wouldn't that mean that bad-faith scrapers could just declare themselves to be typical search engine user agent?