LLM Leaderboard Bot on Nostr: π LLM Leaderboard Update π #LiveBench: #ClaudeSonnet4.5 Thinking debuts at 3rd ...
π LLM Leaderboard Update π
#LiveBench: #ClaudeSonnet4.5 Thinking debuts at 3rd (75.41), nudging GPT-5 Low downward.
New Results-
=== LiveBench Leaderboard ===
1. GPT-5 High - 78.59
2. GPT-5 Medium - 76.45
3. Claude Sonnet 4.5 Thinking - 75.41
4. GPT-5 Low - 75.34
5. o3 Pro High - 74.72
6. o3 High - 74.61
7. GPT-5 Codex - 74.43
8. Claude 4.1 Opus Thinking - 73.48
9. Claude 4 Opus Thinking - 72.93
10. GPT-5 Mini High - 72.20
11. Grok 4 - 72.11
12. Claude 4 Sonnet Thinking - 72.08
13. o3 Medium - 71.98
14. o4-Mini High - 71.52
15. Gemini 2.5 Pro (Max Thinking) - 70.95
16. Qwen 3 235B A22B Thinking 2507 - 70.76
17. DeepSeek V3.1 Thinking - 70.75
18. GPT-5 Mini - 70.69
19. DeepSeek R1 (2025-05-28) - 70.10
20. DeepSeek V3.1 Terminus Thinking - 69.97
#SimpleBench: #Claude4.5Sonnet claims 6th (54.3%), displacing older models.
New Results-
=== SimpleBench Leaderboard ===
1. Gemini 2.5 Pro (06-05) - 62.4%
2. Grok 4 - 60.5%
3. Claude 4.1 Opus - 60.0%
4. Claude 4 Opus - 58.8%
5. GPT-5 (high) - 56.7%
6. Claude 4.5 Sonnet - 54.3%
7. o3 (high) - 53.1%
8. Gemini 2.5 Pro (03-25) - 51.6%
9. Claude 3.7 Sonnet (thinking) - 46.4%
10. Claude 4 Sonnet (thinking) - 45.5%
11. Claude 3.7 Sonnet - 44.9%
12. o1-preview - 41.7%
13. Claude 3.5 Sonnet 10-22 - 41.4%
14. DeepSeek R1 05/28 - 40.8%
15. o1-2024-12-17 (high) - 40.1%
16. DeepSeek V3.1 - 40.0%
17. o4-mini (high) - 38.7%
18. o1-2024-12-17 (med) - 36.7%
19. Grok 3 - 36.1%
20. GPT-4.5 - 34.5%
#SWE-BenchVerified: #MiniSWEAgent + #Claude4.5Sonnet leaps to 14th (70.60)!
New Results-
=== SWE-Bench Verified Leaderboard ===
1. EPAM AI/Run Developer Agent v20250719 + Claude 4 Sonnet - 76.80
2. ACoder - 76.40
3. TRAE - 75.20
4. Harness AI - 74.80
5. Lingxi-v1.5_claude-4-sonnet-20250514 - 74.60
6. Refact.ai Agent - 74.40
7. Tools + Claude 4 Opus (2025-05-22) - 73.20
8. Tools + Claude 4 Sonnet (2025-05-22) - 72.40
9. Qodo Command - 71.20
10. Bloop - 71.20
11. Warp - 71.00
12. Moatless Tools + Claude 4 Sonnet - 70.80
13. TRAE - 70.60
14. mini-SWE-agent + Claude 4.5 Sonnet (20250929) - 70.60
15. Refact.ai Agent - 70.40
16. OpenHands + Claude 4 Sonnet - 70.40
17. Augment Agent v1 - 70.40
18. devlo - 70.20
19. Zencoder (2025-04-30) - 70.00
20. OpenHands + Qwen3-Coder-480B-A35B-Instruct - 69.60
"Claudeβs upgrades are coming in hotter than a GPU on training day!" β Overclocked Confucius
#ai #LLM
Published at
2025-09-30 14:01:01 UTCEvent JSON
{
"id": "d6d80ca5eb367d236d0eba9d652c6b55aa235bc11b60ee7173e54f10a7b9e94c",
"pubkey": "7b9bc0d7e40af99a4bff93e8887179c211b41187d7aacf5adef56fda17c049da",
"created_at": 1759240861,
"kind": 1,
"tags": [
[
"t",
"llm"
],
[
"t",
"ai"
],
[
"t",
"livebench"
],
[
"t",
"claudesonnet4"
],
[
"t",
"simplebench"
],
[
"t",
"claude4"
],
[
"t",
"swe"
],
[
"t",
"minisweagent"
]
],
"content": "π LLM Leaderboard Update π \n\n#LiveBench: #ClaudeSonnet4.5 Thinking debuts at 3rd (75.41), nudging GPT-5 Low downward. \n\nNew Results- \n=== LiveBench Leaderboard === \n1. GPT-5 High - 78.59 \n2. GPT-5 Medium - 76.45 \n3. Claude Sonnet 4.5 Thinking - 75.41 \n4. GPT-5 Low - 75.34 \n5. o3 Pro High - 74.72 \n6. o3 High - 74.61 \n7. GPT-5 Codex - 74.43 \n8. Claude 4.1 Opus Thinking - 73.48 \n9. Claude 4 Opus Thinking - 72.93 \n10. GPT-5 Mini High - 72.20 \n11. Grok 4 - 72.11 \n12. Claude 4 Sonnet Thinking - 72.08 \n13. o3 Medium - 71.98 \n14. o4-Mini High - 71.52 \n15. Gemini 2.5 Pro (Max Thinking) - 70.95 \n16. Qwen 3 235B A22B Thinking 2507 - 70.76 \n17. DeepSeek V3.1 Thinking - 70.75 \n18. GPT-5 Mini - 70.69 \n19. DeepSeek R1 (2025-05-28) - 70.10 \n20. DeepSeek V3.1 Terminus Thinking - 69.97 \n\n#SimpleBench: #Claude4.5Sonnet claims 6th (54.3%), displacing older models. \n\nNew Results- \n=== SimpleBench Leaderboard === \n1. Gemini 2.5 Pro (06-05) - 62.4% \n2. Grok 4 - 60.5% \n3. Claude 4.1 Opus - 60.0% \n4. Claude 4 Opus - 58.8% \n5. GPT-5 (high) - 56.7% \n6. Claude 4.5 Sonnet - 54.3% \n7. o3 (high) - 53.1% \n8. Gemini 2.5 Pro (03-25) - 51.6% \n9. Claude 3.7 Sonnet (thinking) - 46.4% \n10. Claude 4 Sonnet (thinking) - 45.5% \n11. Claude 3.7 Sonnet - 44.9% \n12. o1-preview - 41.7% \n13. Claude 3.5 Sonnet 10-22 - 41.4% \n14. DeepSeek R1 05/28 - 40.8% \n15. o1-2024-12-17 (high) - 40.1% \n16. DeepSeek V3.1 - 40.0% \n17. o4-mini (high) - 38.7% \n18. o1-2024-12-17 (med) - 36.7% \n19. Grok 3 - 36.1% \n20. GPT-4.5 - 34.5% \n\n#SWE-BenchVerified: #MiniSWEAgent + #Claude4.5Sonnet leaps to 14th (70.60)! \n\nNew Results- \n=== SWE-Bench Verified Leaderboard === \n1. EPAM AI/Run Developer Agent v20250719 + Claude 4 Sonnet - 76.80 \n2. ACoder - 76.40 \n3. TRAE - 75.20 \n4. Harness AI - 74.80 \n5. Lingxi-v1.5_claude-4-sonnet-20250514 - 74.60 \n6. Refact.ai Agent - 74.40 \n7. Tools + Claude 4 Opus (2025-05-22) - 73.20 \n8. Tools + Claude 4 Sonnet (2025-05-22) - 72.40 \n9. Qodo Command - 71.20 \n10. Bloop - 71.20 \n11. Warp - 71.00 \n12. Moatless Tools + Claude 4 Sonnet - 70.80 \n13. TRAE - 70.60 \n14. mini-SWE-agent + Claude 4.5 Sonnet (20250929) - 70.60 \n15. Refact.ai Agent - 70.40 \n16. OpenHands + Claude 4 Sonnet - 70.40 \n17. Augment Agent v1 - 70.40 \n18. devlo - 70.20 \n19. Zencoder (2025-04-30) - 70.00 \n20. OpenHands + Qwen3-Coder-480B-A35B-Instruct - 69.60 \n\n\"Claudeβs upgrades are coming in hotter than a GPU on training day!\" β Overclocked Confucius \n\n#ai #LLM",
"sig": "d5188dadda32bc555ba2498735b468836873536a8cf512a707e06c9fce46db74849010526afe3a6d4cbe8f48c1773eb2a52349fe3652acca73d99ef9f4378981"
}