So I loaded Qwen2.5 7B and 14B this morning. 7B seems faster than LLaMa 3.1 8B (is that a thing?) and uses the same amount of RAM. Doesn't spin up my fans as soon for some reason. I don't like the answers about JS as much but I haven't tested it a lot. 14B barely runs on my computer, putting other things into swap. The answers are better but not that much better. I'm not really sure how I could test the difference.
This is all on CPU by the way.
#ollama #llm #llama #qwen #ai
Sunflower Björnskalle 🌻
npub1g2…w9ulx
2024-11-10 08:11:16 UTC
Author Public Key
npub1g2dywn37v0cymznw8elcscrmstpvz5h8trem7tlmdmfpyhus9p5s0w9ulxSeen on
wss://relay.momostr.pinkPublished at
2024-11-10 08:11:16 UTCEvent JSON
{
"id": "11d335ecf98ba478d0c5d83f5656c2b9744a8c5f375bda910f720b3f00370a05",
"pubkey": "429a474e3e63f04d8a6e3e7f88607b82c2c152e758f3bf2ffb6ed2125f902869",
"created_at": 1731226276,
"kind": 1,
"tags": [
[
"t",
"qwen"
],
[
"t",
"ollama"
],
[
"t",
"llama"
],
[
"t",
"llm"
],
[
"t",
"ai"
],
[
"proxy",
"https://mastodon.online/@apodoxus/113457645230463203",
"web"
],
[
"proxy",
"https://mastodon.online/users/apodoxus/statuses/113457645230463203",
"activitypub"
],
[
"L",
"pink.momostr"
],
[
"l",
"pink.momostr.activitypub:https://mastodon.online/users/apodoxus/statuses/113457645230463203",
"pink.momostr"
],
[
"-"
]
],
"content": "So I loaded Qwen2.5 7B and 14B this morning. 7B seems faster than LLaMa 3.1 8B (is that a thing?) and uses the same amount of RAM. Doesn't spin up my fans as soon for some reason. I don't like the answers about JS as much but I haven't tested it a lot. 14B barely runs on my computer, putting other things into swap. The answers are better but not that much better. I'm not really sure how I could test the difference.\n\nThis is all on CPU by the way.\n\n#ollama #llm #llama #qwen #ai",
"sig": "b2dfabcc7c3b0e85c00321e5c3dca336ebc553e108157bc471bf41f61061747cb11ced99ec5dea492c7abc1c38f7d1e264bf51e58485efd55aebfb0f4e7326c8"
}