Ian Kluft ✅✈️🎈🌋 on Nostr: We were chatting this week at the Portland Linux Kernel Meetup about measures for ...

We were chatting this week at the Portland Linux Kernel Meetup about measures for misbehaving aggressive crawlers. Since some obviously use the robots.txt exclusion list as a checklist for what to read, it should start with a honeypot that feeds them bad data long enough to add them to a local or distributed blocklist.