Welcome to DU! The truly grassroots left-of-center political community where regular people, not algorithms, drive the discussions and set the standards. Join the community: Create a free account Support DU (and get rid of ads!): Become a Star Member Latest Breaking News Editorials & Other Articles General Discussion The DU Lounge All Forums Issue Forums Culture Forums Alliance Forums Region Forums Support Forums Help & Search

Computer Help and Support

(18,181 posts)

2. Good for you.

Sun May 25, 2025, 07:47 PM

May 25

I don't have a site to slam, but I admire what people are doing.

Tarpits
https://arstechnica.com/tech-policy/2025/01/ai-haters-build-tarpits-to-trap-and-trick-ai-scrapers-that-ignore-robots-txt/

Cloudflare Using Devilish Trick to Trap AI Scrapers in Infinite Maze of AI-Generated Content
https://futurism.com/cloudflare-trap-ai-scrapers-maze

How to block AI web crawlers: challenges and solutions
https://stytch.com/blog/how-to-block-ai-web-crawlers/

I fart in their general direction.

Edit history

Please sign in to view edit histories.

0 members have recommended this reply (displayed in chronological order):

4 replies

= new reply since forum marked as read

Highlight:

Good for you. usonian May 25 #2

Good for you for adding these links, usonian. highplainsdem May 25 #3

I don't want to block web crawlers; I just don't want them spamming me William Seger May 27 #4