mike on Nostr: You can run an LLM on pretty much anything. Two easy ways, either LM Studio or ...

You can run an LLM on pretty much anything. Two easy ways, either LM Studio or Ollama. The main constraint is the size of the RAM on your machine.

There are two things AI uses:

GPU to process the prompt, and (V)RAM to hold the model and read through it constantly to construct the output.

If you don't have a good GPU, then prompt processing will take longer, if you have slow RAM, the token generated output will take longer.

The smaller your RAM, the dumber your model 😂

Small models work surprisingly well if you supply enough context and you allow web search so the model can compensate its lack of knowledge with live data from the Internet.

If you want to train an AI, however, that's a much harder job and you need something like an Nvidia DGX Spark, which is currently over £5,000 in the UK.