Yesterday I got an outage on one of my Linode hosts because of AI scrapers.
It suddenly started triggering disk and bandwidth usage warnings.
I took a closer look, and my nginx logs were suddenly exploding with requests to my Wikiless instance (I didn't even know it was still running, let alone indexed by search engines).
And again the same pattern (or, better, lack of, with apparently legitimate IPs and user agents making hundreds of requests per minute).
It looks like stricter rules about scraping Wikipedia directly have redirected some of these scrapers to alternative frontend - such as Wikiless instances.
The solution for now has been to simply take down my Wikiless instance.
Whatever you guys are doing, please stop. You're breaking the web and doing a favour to nobody.
Fabio Manganiello
npub1s9…jdnmu
2025-04-30 10:52:35 UTC
Author Public Key
npub1s9uc08n58mxqk5umvapqulwzng0sja635q86r36d8n4rr9r9ygaskjdnmuPublished at
2025-04-30 10:52:35 UTCEvent JSON
{
"id": "870e1480450f14cd301872cffc2740a0ed3877dffabe1033f100e0d910491aac",
"pubkey": "8179879e743ecc0b539b67420e7dc29a1f097751a00fa1c74d3cea319465223b",
"created_at": 1746010355,
"kind": 1,
"tags": [
[
"proxy",
"https://manganiello.social/objects/6eeccc82-4816-408b-a691-af6bc5d621de",
"activitypub"
],
[
"L",
"pink.momostr"
],
[
"l",
"pink.momostr.activitypub:https://manganiello.social/objects/6eeccc82-4816-408b-a691-af6bc5d621de",
"pink.momostr"
],
[
"-"
]
],
"content": "Yesterday I got an outage on one of my Linode hosts because of AI scrapers.\n\nIt suddenly started triggering disk and bandwidth usage warnings.\n\nI took a closer look, and my nginx logs were suddenly exploding with requests to my Wikiless instance (I didn't even know it was still running, let alone indexed by search engines).\n\nAnd again the same pattern (or, better, lack of, with apparently legitimate IPs and user agents making hundreds of requests per minute).\n\nIt looks like stricter rules about scraping Wikipedia directly have redirected some of these scrapers to alternative frontend - such as Wikiless instances.\n\nThe solution for now has been to simply take down my Wikiless instance.\n\nWhatever you guys are doing, please stop. You're breaking the web and doing a favour to nobody.",
"sig": "aba97ebaaa573fbf3e05b2745ac3b1883818f67b909457175fbf8dc22fb50c0b1daafbf8a6f60b10896e15d259f8c240e87d6a21bfa9d20eee4aab9b3a532159"
}