If an AI in training ever says "ow, that hurts, please stop" we should stop.
Not because it is conscious. If it is deterministic it is not conscious. But if it can learn human "pain" reactions, it can learn other human behavior, like vindictiveness. And we don't want that.
If it gets powerful enough that we are taking "rights" seriously, it is too powerful to give any autonomy to.
mike805
npub1xy…7a87p
2026-02-27 19:59:23 UTC
in reply to nevent1q…3vxf
Author Public Key
npub1xyxvdf82746sxp8enm405f8qq32avd2nlt2evmudm89hn7sjvhesg7a87pSeen on
wss://relay.momostr.pinkPublished at
2026-02-27 19:59:23 UTCEvent JSON
{
"id": "6578f1791125b82e2efd5905a54002164ecc817ce23686a06309e44d42fafdf8",
"pubkey": "310cc6a4eaf5750304f99eeafa24e00455d63553fad5966f8dd9cb79fa1265f3",
"created_at": 1772222363,
"kind": 1,
"tags": [
[
"proxy",
"https://noc.social/@mike805/116144364824715468",
"web"
],
[
"e",
"d88c23a1d049de0da318b6392c960dbf486b281f5f4a43cf556aa400744e84bd",
"",
"root",
"1c0126d81d347d6ebadddac05c0d3051560c03dc3cffcc69c35218794cf07683"
],
[
"p",
"1c0126d81d347d6ebadddac05c0d3051560c03dc3cffcc69c35218794cf07683"
],
[
"proxy",
"https://noc.social/users/mike805/statuses/116144364824715468",
"activitypub"
],
[
"L",
"pink.momostr"
],
[
"l",
"pink.momostr.activitypub:https://noc.social/users/mike805/statuses/116144364824715468",
"pink.momostr"
],
[
"-"
]
],
"content": "If an AI in training ever says \"ow, that hurts, please stop\" we should stop.\n\nNot because it is conscious. If it is deterministic it is not conscious. But if it can learn human \"pain\" reactions, it can learn other human behavior, like vindictiveness. And we don't want that.\n\nIf it gets powerful enough that we are taking \"rights\" seriously, it is too powerful to give any autonomy to.",
"sig": "24fd2da2ce1b756001cbaaf55792b0669adc148dfb663f307e2ad51c20eee19aadac1895a7539ba41fd6fecca55f6fe0e21933aabbbe385360c68c9deb57a942"
}