llama.cpp release b9235 added some new toys for boosting inference.
Benchmarked Qwen3.6 27B on an RTX 5090 with llama.cpp, using speculative n-gram tuning across 10k generated tokens tests.
Increasing --spec-ngram-map-k4v-size-m scaled decode throughput (predicted_per_second)
captjack 🏴☠️✨💜
npub1te…mjgwp
2026-05-21 05:32:32 UTC
in reply to nevent1q…wrnc
Author Public Key
npub1te0uzs6vj29umjaxlqqct82j8q6ppyefrxq06dhr8d6pvwfatgkqjmjgwpPublished at
2026-05-21 05:32:32 UTCEvent JSON
{
"id": "4fb9f9a40762e0d7c50de3a8c2933ec50d1233da36d85ab6465ef88e2042d32e",
"pubkey": "5e5fc1434c928bcdcba6f801859d5238341093291980fd36e33b7416393d5a2c",
"created_at": 1779341552,
"kind": 1,
"tags": [
[
"e",
"f6d0a2062c06868c5083ee5e8869d9c1490466afce6e208df2db24a546f15dd3",
"wss://intl-rocks-microphone-did.trycloudflare.com/",
"root",
"5e5fc1434c928bcdcba6f801859d5238341093291980fd36e33b7416393d5a2c"
]
],
"content": "llama.cpp release b9235 added some new toys for boosting inference.\n\nBenchmarked Qwen3.6 27B on an RTX 5090 with llama.cpp, using speculative n-gram tuning across 10k generated tokens tests.\n\nIncreasing --spec-ngram-map-k4v-size-m scaled decode throughput (predicted_per_second)",
"sig": "f97d90cef3884594b80588337f9d037a640062fcf0c49ee98af9b57d000ef8d56b173b500dc34acdfa8a407d823b8dc52a4213143619c70f4264c1d8a06ba509"
}