64 GB is going to get you the best of today. 128 GB is hard to justify, but it might give you a bit more runway if model sizes change. I'm not even sure how to predict that.
The latest LLama 3.2 models are fairly reasonably sized (1B to 11B for consumers) https://ollama.com/library/llama3.2
LLM + llm-ollama is a pretty nice combo plus the many other projects Simon writes about. Ollama can run hugging fact models too.
Jeff Triplett
npub1ns…8ahke
2024-11-02 00:52:04 UTC
in reply to nevent1q…6z42
Author Public Key
npub1nsp3hg75ge84dn5fttlkpxpxnj0dfl697239dtyz4js3mdw2jllqf8ahkeSeen on
wss://relay.momostr.pinkPublished at
2024-11-02 00:52:04 UTCEvent JSON
{
"id": "73c10cbff75892babf1226e7824a65c6630d330ee0a4d79304fc2bf926f87c88",
"pubkey": "9c031ba3d4464f56ce895aff6098269c9ed4ff45f2a256ac82aca11db5ca97fe",
"created_at": 1730508724,
"kind": 1,
"tags": [
[
"e",
"8ed0529c8621fa6d652770e8bbde1956fb74a3d87aae1fa77e4646fa3e50ef18",
"",
"reply",
"4315a187e024818492e61938093ba683dae66624d202cd43738de5b8ba198c0f"
],
[
"p",
"9c031ba3d4464f56ce895aff6098269c9ed4ff45f2a256ac82aca11db5ca97fe"
],
[
"p",
"ff0262f999f20c775c3fc40a80ab9712c4956b42d6a6500c463b3691df87a1f3"
],
[
"e",
"54e56e69bde9c6213c0a0a2ab64a9cf0cc06e3a63434d066ffe02849f5378892",
"",
"root",
"ff0262f999f20c775c3fc40a80ab9712c4956b42d6a6500c463b3691df87a1f3"
],
[
"proxy",
"https://mastodon.social/@webology/113410619758447777",
"web"
],
[
"p",
"4315a187e024818492e61938093ba683dae66624d202cd43738de5b8ba198c0f"
],
[
"proxy",
"https://mastodon.social/users/webology/statuses/113410619758447777",
"activitypub"
],
[
"L",
"pink.momostr"
],
[
"l",
"pink.momostr.activitypub:https://mastodon.social/users/webology/statuses/113410619758447777",
"pink.momostr"
],
[
"-"
]
],
"content": "64 GB is going to get you the best of today. 128 GB is hard to justify, but it might give you a bit more runway if model sizes change. I'm not even sure how to predict that. \n\nThe latest LLama 3.2 models are fairly reasonably sized (1B to 11B for consumers) https://ollama.com/library/llama3.2 \n\nLLM + llm-ollama is a pretty nice combo plus the many other projects Simon writes about. Ollama can run hugging fact models too.",
"sig": "749fcf2f4b98662f5bca42af1ffb3b88540d98dcdd211c2c563ecbc2faa76fae779941286ea840d1cabee2df10bf55428b10c02b69e2e94a7034f3d60f249fb6"
}