rubberducky on Nostr: The models and context will work best when they run from memory (vram best, ram for ...

The models and context will work best when they run from memory (vram best, ram for cpu only). If you’re hurting for storage I’d consider well rated external spinner. The usefulness of models increases rapidly up to about 14b, but I don’t use bigger models because I’m not spoiled for vram.