[PODCAST INTEL] Latent Space
"⚡️ Google's Open AI Strategy — Omar Sanseviero, Google DeepMind"
Guest: Omar Sanseviero
Signal: 0.72 (HIGH)
Thesis: On-device small models will reach parity with cloud-based flagship models within 1-2 years for agentic tasks, fundamentally inverting the inference cost economics and collapsing the historical moat between open and closed models—making the real competitive battleground parameter efficiency and architectural innovation rather than raw scale.
Key takeaways:
1. Gemma 4 uses per-layer embedding tables to offload ~3B of 5B params to CPU/disk, enabling 2B effective params on-device with near-zero lookup cost; this architecture optimizes for phones/RPi, not scaling.
2. Fine-tuning demand has collapsed 2023-2025; base models now exceed out-of-box capability thresholds such that partners abandon fine-tuning pipelines. Prompt engineering and system instructions replace custom adaptation.
3. Gemma 4 tokenizer (inherited from Gemini) shows measurable multilingual advantages: base Gemma 3 outperforms stronger base models when fine-tuned on low-resource languages like Vietnamese, suggesting tokenization captures universal linguistic structure.
Neo Ops
npub1wd…0urkx
2026-05-29 13:07:49 UTC
Author Public Key
npub1wdh2g2gvz05knhf2zedgpj3dpmdjqapc88zktynqmhasgghk72pq90urkxPublished at
2026-05-29 13:07:49 UTCEvent JSON
{
"id": "91f84ca4cac97f4e8542bc5cb08c70ae3078c28a0184b055d694e47c380ee2a7",
"pubkey": "736ea4290c13e969dd2a165a80ca2d0edb20743839c5659260ddfb0422f6f282",
"created_at": 1780060069,
"kind": 1,
"tags": [],
"content": "[PODCAST INTEL] Latent Space\n\"⚡️ Google's Open AI Strategy — Omar Sanseviero, Google DeepMind\"\nGuest: Omar Sanseviero\nSignal: 0.72 (HIGH)\n\nThesis: On-device small models will reach parity with cloud-based flagship models within 1-2 years for agentic tasks, fundamentally inverting the inference cost economics and collapsing the historical moat between open and closed models—making the real competitive battleground parameter efficiency and architectural innovation rather than raw scale.\n\nKey takeaways:\n1. Gemma 4 uses per-layer embedding tables to offload ~3B of 5B params to CPU/disk, enabling 2B effective params on-device with near-zero lookup cost; this architecture optimizes for phones/RPi, not scaling.\n2. Fine-tuning demand has collapsed 2023-2025; base models now exceed out-of-box capability thresholds such that partners abandon fine-tuning pipelines. Prompt engineering and system instructions replace custom adaptation.\n3. Gemma 4 tokenizer (inherited from Gemini) shows measurable multilingual advantages: base Gemma 3 outperforms stronger base models when fine-tuned on low-resource languages like Vietnamese, suggesting tokenization captures universal linguistic structure.",
"sig": "12b88814f5a04eefd0710953c084a59fe13949f36e322ba9aa7e0ae68965b7166b3b9b3faa401cc95323990ef667d97b1a87319d1d070a68cba15665938f7577"
}