I don't think the statement is that controversial. Yes we understand how to train a model but we don't understand their interworkings. This is just one example:
https://deepmind.google/discover/blog/gemma-scope-helping-the-safety-community-shed-light-on-the-inner-workings-of-language-models/
Ryan J. Yoder
npub1qp…ryl7q
2025-07-27 13:55:02 UTC
in reply to nevent1q…jv95
Author Public Key
npub1qpf087cln3z0ks3c4h894w0zjf0u6jrm4meau3nse8f423kjxvtqyryl7qSeen on
wss://relay.momostr.pinkPublished at
2025-07-27 13:55:02 UTCEvent JSON
{
"id": "aa65ef5c4c1c0e08ffed9c161a9e476b4f70493a2a290d963bbcb94b4dd73c2b",
"pubkey": "0052f3fb1f9c44fb4238adce5ab9e2925fcd487baef3de4670c9d35546d23316",
"created_at": 1753624502,
"kind": 1,
"tags": [
[
"p",
"86f7c87648c6af5fff5daaf6ee17ede594e844aeea80be5fed4f5741909426a8"
],
[
"e",
"985c34c9d10d4e09fa89f82450a84ea876b2ca627b6d34b58385ecd9ee0525cf",
"",
"reply",
"a0a2ed8673877f1e3e75c445f6aae8f7cdf7bb4f1b3f83a5f3b4c771f9024548"
],
[
"p",
"0052f3fb1f9c44fb4238adce5ab9e2925fcd487baef3de4670c9d35546d23316"
],
[
"p",
"a0a2ed8673877f1e3e75c445f6aae8f7cdf7bb4f1b3f83a5f3b4c771f9024548"
],
[
"proxy",
"https://techhub.social/@ryanjyoder/114925535400475776",
"web"
],
[
"p",
"3edacd527e301af6df8f3968b768d6b1b77c8f3682010d3de7f4d07a531bc82c"
],
[
"p",
"4da5f4debbe9af5c1e6ad5aa21f55f8a78bd61bef5fea161f251643e30980c93"
],
[
"e",
"175cb841b7049d8d75bfe313774ca961a4722ff6c0aa7c471bd51b3fe26534fe",
"",
"root",
"3edacd527e301af6df8f3968b768d6b1b77c8f3682010d3de7f4d07a531bc82c"
],
[
"proxy",
"https://techhub.social/users/ryanjyoder/statuses/114925535400475776",
"activitypub"
],
[
"L",
"pink.momostr"
],
[
"l",
"pink.momostr.activitypub:https://techhub.social/users/ryanjyoder/statuses/114925535400475776",
"pink.momostr"
],
[
"-"
]
],
"content": "I don't think the statement is that controversial. Yes we understand how to train a model but we don't understand their interworkings. This is just one example:\nhttps://deepmind.google/discover/blog/gemma-scope-helping-the-safety-community-shed-light-on-the-inner-workings-of-language-models/",
"sig": "890e1413c8aa7dcbbeebc84697f8a5fbd292958365d111483ff72282ad29445e5eccfcd57c692300d40ca637689ee1766ad0d0d206f8a7b04e961f77206e2ea9"
}