I have played a little bit with OpenAI's new iteration of #GPT, GPT-o1, which performs an initial reasoning step before running the LLM. It is certainly a more capable tool than previous iterations, though still struggling with the most advanced research mathematical tasks.
Here are some concrete experiments (with a prototype version of the model that I was granted access to). In https://chatgpt.com/share/2ecd7b73-3607-46b3-b855-b29003333b87 I repeated an experiment from https://mathstodon.xyz/@tao/109948249160170335 in which I asked GPT to answer a vaguely worded mathematical query which could be solved by identifying a suitable theorem (Cramer's theorem) from the literature. Previously, GPT was able to mention some relevant concepts but the details were hallucinated nonsense. This time around, Cramer's theorem was identified and a perfectly satisfactory answer was given. (1/3)
Terence Tao
npub1gv…2sduy
2024-09-13 22:03:15 UTC
Author Public Key
npub1gvea966levf836g93xsdnkde8mmz58qyznf9c82jg02hq35f46lsn2sduySeen on
wss://relay.momostr.pinkPublished at
2024-09-13 22:03:15 UTCEvent JSON
{
"id": "799abffc7a0ef3d89c8bb7ab970c05458f6eb034414665bd32a908b8af3864b6",
"pubkey": "4333d2eb5fcb1278e90589a0d9d9b93ef62a1c0414d25c1d5243d5704689aebf",
"created_at": 1726264995,
"kind": 1,
"tags": [
[
"proxy",
"https://mathstodon.xyz/@tao/113132502735585408",
"web"
],
[
"t",
"gpt"
],
[
"proxy",
"https://mathstodon.xyz/users/tao/statuses/113132502735585408",
"activitypub"
],
[
"L",
"pink.momostr"
],
[
"l",
"pink.momostr.activitypub:https://mathstodon.xyz/users/tao/statuses/113132502735585408",
"pink.momostr"
],
[
"-"
]
],
"content": "I have played a little bit with OpenAI's new iteration of #GPT, GPT-o1, which performs an initial reasoning step before running the LLM. It is certainly a more capable tool than previous iterations, though still struggling with the most advanced research mathematical tasks.\n\nHere are some concrete experiments (with a prototype version of the model that I was granted access to). In https://chatgpt.com/share/2ecd7b73-3607-46b3-b855-b29003333b87 I repeated an experiment from https://mathstodon.xyz/@tao/109948249160170335 in which I asked GPT to answer a vaguely worded mathematical query which could be solved by identifying a suitable theorem (Cramer's theorem) from the literature. Previously, GPT was able to mention some relevant concepts but the details were hallucinated nonsense. This time around, Cramer's theorem was identified and a perfectly satisfactory answer was given. (1/3)",
"sig": "33357b88e0a97ad37513a3280f8d965832bac71203836b22ee054a700fc2a54d1bbe7aa7fe0c4049d8f8ae4edbda3e18f27e3b7c274aa7ccfb17bf6c6175f8fe"
}