My favorite local model right now is a bit of surprise to me: I'm really enjoying the relatively tiny Qwen3-8B, running the 4bit quantized version on my Mac using MLX
It's surprisingly capable given it's a 4.3GB download and uses just 4-5GB of RAM while it's running
https://simonwillison.net/2025/May/2/qwen3-8b/
Simon Willison
npub1gv…utlwl
2025-05-02 23:45:12 UTC
Author Public Key
npub1gv26rplqyjqcfyhxryuqjwaxs0dwve3y6gpv6smn3hjm3wse3s8squtlwlPublished at
2025-05-02 23:45:12 UTCEvent JSON
{
"id": "2ff0058386dbe55854b600d7dc52b3f7b370d1e4e76d6e26e15e4bdb7bda3747",
"pubkey": "4315a187e024818492e61938093ba683dae66624d202cd43738de5b8ba198c0f",
"created_at": 1746229512,
"kind": 1,
"tags": [
[
"proxy",
"https://fedi.simonwillison.net/@simon/114440897356073827",
"web"
],
[
"proxy",
"https://fedi.simonwillison.net/users/simon/statuses/114440897356073827",
"activitypub"
],
[
"L",
"pink.momostr"
],
[
"l",
"pink.momostr.activitypub:https://fedi.simonwillison.net/users/simon/statuses/114440897356073827",
"pink.momostr"
],
[
"-"
]
],
"content": "My favorite local model right now is a bit of surprise to me: I'm really enjoying the relatively tiny Qwen3-8B, running the 4bit quantized version on my Mac using MLX\n\nIt's surprisingly capable given it's a 4.3GB download and uses just 4-5GB of RAM while it's running\n\nhttps://simonwillison.net/2025/May/2/qwen3-8b/",
"sig": "db3babc2a6523b57fd2cef055b5b268891d4cd8812593282b3b28f62f61b33b8a259fd4385b71f43ee8c7985ed6940d22bcd882b0b18ab42cdbb8d485fd58f83"
}