Brie on Nostr: GM A message to whomever is using a crawler on my Dagztrial space: 1) I can see that ...
GM
A message to whomever is using a crawler on my Dagztrial space:
1) I can see that in my analytics
2) I added a robots.txt to root so that you can more comfortably crawl it and should stop seeing a 404 returned
You’re welcome.
Published at
2026-05-27 13:15:16 UTCEvent JSON
{
"id": "13916aee8132d9466373e9a66e71f5d205a3ebaddde33efb4f40d902765a9c52",
"pubkey": "ae4c17620c9b3a0413fd019fd7e983da43b2081e7169abecb7764c333a9bfff1",
"created_at": 1779887716,
"kind": 1,
"tags": [],
"content": "GM\n\nA message to whomever is using a crawler on my Dagztrial space:\n\n1) I can see that in my analytics\n2) I added a robots.txt to root so that you can more comfortably crawl it and should stop seeing a 404 returned\n\nYou’re welcome.",
"sig": "182ee8a0838285896c0c9bfe4a2b4431823016c9061835b26f45d116d10049dfc0ca571cfbea8610be53f9c88ebf43e7af8e636ceb20ca718953759093edb2a6"
}
