{"type":"rich","version":"1.0","title":"Scott wrote","author_name":"Scott (npub1zg…gx9uk)","author_url":"https://yabu.me/npub1zgxfjvfyt9j59rnpxfay666njxa7wl4aye8d25jj2wmj9zp0m68q2gx9uk","provider_name":"njump","provider_url":"https://yabu.me","html":"Top of the line: several hundred GB of ram running locally. $20k-$100k of hardware before cost for beefing up electrical wiring setup at home. \nMiddle of the road: maybe a few grand on high end consumer grade GPUs\nShitty: 1 word per second or less and shit context but can run on a middle of the road PC \n\nThe \"magic\" of AI exists only at the top of the line running the latest large models. Model size and hardware costs started small and got big and I expect them to drop again dramatically over time and self hosting will get more practical, just as bitcoin mining has done (from usb block erupters, to massive 3 phase machines consuming 10kw+, to bitaxes running on 9v DC power). For now if you go middle of the road or shitty, the benefits of these tools will not be clear at all and your experience may make you conclude that AI is fake and gay when in reality it is immensely powerful.  \n\n"}