This is one of the funniest bad AI studies yet: "Evaluating large language models in theory of mind tasks"
https://www.nature.com/articles/s41467-024-55628-6
They reduced an inherently abstract metacognitive ability to producing plausible text, which they benchmark, then, with staggering irony, read theory of mind into.
(1/3)
The Luddite
npub1uc…90gd9
2025-01-16 16:06:25 UTC
Author Public Key
npub1uc33mtkfcg6lphwvzfnre0t6puxwgly577q40rgz6nvt2st0e7sqc90gd9Seen on
wss://relay.momostr.pinkPublished at
2025-01-16 16:06:25 UTCEvent JSON
{
"id": "8909c10763dc3e6b1d1e6bebe04eac4f3acd096c604eb8553ed2878ae8986f76",
"pubkey": "e6231daec9c235f0ddcc12663cbd7a0f0ce47c94f781578d02d4d8b5416fcfa0",
"created_at": 1737043585,
"kind": 1,
"tags": [
[
"proxy",
"https://assemblag.es/@theluddite/113838888448977161",
"web"
],
[
"proxy",
"https://assemblag.es/users/theluddite/statuses/113838888448977161",
"activitypub"
],
[
"L",
"pink.momostr"
],
[
"l",
"pink.momostr.activitypub:https://assemblag.es/users/theluddite/statuses/113838888448977161",
"pink.momostr"
],
[
"-"
]
],
"content": "This is one of the funniest bad AI studies yet: \"Evaluating large language models in theory of mind tasks\"\n\nhttps://www.nature.com/articles/s41467-024-55628-6\n\nThey reduced an inherently abstract metacognitive ability to producing plausible text, which they benchmark, then, with staggering irony, read theory of mind into.\n\n(1/3)",
"sig": "34038ed5cf4fe1d6b8d63a101f7c604f331e5cd949b7c8ee1879562bde84713733efc948fb69de15d5297e9892efa9672bf5a05e721891f87c75f99c6fd8d894"
}