jsm on Nostr: Google just can't stop winning. New test run for the Aider Polyglot benchmark is out ...

Google just can't stop winning. New test run for the Aider Polyglot benchmark is out and Gemini 2.5 Pro with extra thinking turned on is actually so stupidly good now that it trounces o3 by itself AND o3 and GPT 4.1 working together, all while being much less expensive.

Not only that, but just yesterday Google brought the 2.5 update for its Gemini models out of preview and dropped the price at the same time, so the models are actually less expensive than what's listed on the benchmark leaderboard. Google's Gemini 2.5 Pro is now better than the top of the line models from Anthropic and OpenAI, Opus and o3, while costing less than Sonnet.

Google is winning and its not even close right now.

https://aider.chat/docs/leaderboards/

quoting
nevent1q…kwdp

https://blog.google/technology/google-deepmind/google-gemini-updates-io-2025/