Haven’t ran qwen in a minute but it’s surprising you’re not getting higher throughput for gemma4 on your 3090s.
For what it’s worth if you use llama.cpp and disable reasoning you should see faster time to first byte at the cost of a slight degradation in quality. Haven’t used vllm so can’t comment there. I get 70-75 tok/s on a macbook m3 for comparison and only have 40 GPU cores
MachuPikacchu / Machu Pikacchu
npub1r6…4gmmd
2026-04-13 22:47:13 UTC
in reply to nevent1q…fue6
Author Public Key
npub1r6ggl0qazvwp02rlxgrf75lkfazuwhu35tmdg0u25eqsjax6243qh4gmmdPublished at
2026-04-13 22:47:13 UTCEvent JSON
{
"id": "d4759e2064a1ac469da7b13258d8700511e3814c2b4c6a4f9d365dccfa82380e",
"pubkey": "1e908fbc1d131c17a87f32069f53f64f45c75f91a2f6d43f8aa6410974da5562",
"created_at": 1776120433,
"kind": 1,
"tags": [
[
"e",
"0000f8b525efe71dafb0f256d0322f0c53199eb13c5bc875765eec7af5a7a77b",
"wss://nostrelites.org",
"root",
"e2ccf7cf20403f3f2a4a55b328f0de3be38558a7d5f33632fdaaefc726c1c8eb"
],
[
"p",
"e2ccf7cf20403f3f2a4a55b328f0de3be38558a7d5f33632fdaaefc726c1c8eb"
]
],
"content": "Haven’t ran qwen in a minute but it’s surprising you’re not getting higher throughput for gemma4 on your 3090s.\n\nFor what it’s worth if you use llama.cpp and disable reasoning you should see faster time to first byte at the cost of a slight degradation in quality. Haven’t used vllm so can’t comment there. I get 70-75 tok/s on a macbook m3 for comparison and only have 40 GPU cores",
"sig": "142f8ee652ac0c3c7d5dadad8c5c54be282e50163f86ca18f362a60695b9517dbff00f0ef030958dfaa79316bb0e79ddcfa9cda871ef03d6514d08191fe67747"
}