The internet is largely built on two related principles:
- Hippy era libertarianism.
- "Badges? We ain't got no badges! We don't need no badges! I don't have to show you any stinking badges!"
One of the reasons we have phishing attacks is that back in the 1970's there was a strong antipathy against strong, reliable personal ID systems (e.g. national ID cards, etc).
And here we have an example of the fact that things like robots.txt files and hyperlink attributes in web pages have no legal status and may be, legally, ignored.
We are seeing this collision between libertarian "freedom" and real life in our "ethics? what ethics?" world in things like age verification and AI stealing the world's knowledge without positive permission or compensation.
"Perplexity is using stealth, undeclared crawlers to evade website no-crawl directives"
https://blog.cloudflare.com/perplexity-is-using-stealth-undeclared-crawlers-to-evade-website-no-crawl-directives/
Karl Auerbach
npub1rh…ps08d
2025-08-04 18:44:00 UTC
Author Public Key
npub1rhqu8mlfa6tr2up0whlkr79xmkvrkwyzs6jj6u728ykxsqrnpneqxps08dSeen on
wss://relay.momostr.pinkPublished at
2025-08-04 18:44:00 UTCEvent JSON
{
"id": "aae9cbd11392cdaaa88b3e757589ff84272d30f8481b8bce142e080b7215c959",
"pubkey": "1dc1c3efe9ee9635702f75ff61f8a6dd983b388286a52d73ca392c6800730cf2",
"created_at": 1754333040,
"kind": 1,
"tags": [
[
"proxy",
"https://sfba.social/@karlauerbach/114971970130700251",
"web"
],
[
"proxy",
"https://sfba.social/users/karlauerbach/statuses/114971970130700251",
"activitypub"
],
[
"L",
"pink.momostr"
],
[
"l",
"pink.momostr.activitypub:https://sfba.social/users/karlauerbach/statuses/114971970130700251",
"pink.momostr"
],
[
"-"
]
],
"content": "The internet is largely built on two related principles:\n\n- Hippy era libertarianism.\n\n- \"Badges? We ain't got no badges! We don't need no badges! I don't have to show you any stinking badges!\"\n\nOne of the reasons we have phishing attacks is that back in the 1970's there was a strong antipathy against strong, reliable personal ID systems (e.g. national ID cards, etc).\n\nAnd here we have an example of the fact that things like robots.txt files and hyperlink attributes in web pages have no legal status and may be, legally, ignored.\n\nWe are seeing this collision between libertarian \"freedom\" and real life in our \"ethics? what ethics?\" world in things like age verification and AI stealing the world's knowledge without positive permission or compensation.\n\n\"Perplexity is using stealth, undeclared crawlers to evade website no-crawl directives\"\n\nhttps://blog.cloudflare.com/perplexity-is-using-stealth-undeclared-crawlers-to-evade-website-no-crawl-directives/",
"sig": "88139a98ab6514614892a165e8f5e9731155966ce8ea38f5c5fcce0bc2111c664b94bf2011f8a4b2b2879caa6318cd3955dd21c840d22e4ca76f32bc42e5a16a"
}