If you don't care about docker packages being used as installers and your home directory invisibly used to store massive weight files in exchange for not having to deal with learning any configuration: ollama or lmstudio.
If you just want to play for a bit: llamafile
If you want granular control with ease of execution in exchange for having to figure out what the settings mean and figure out which weights to download: koboldcpp. (check out bartowski on huggingface for the weights)
These are all based on llamacpp as a backend, by the way.
I run ollama off a symlink to an external volume. It just feels neater that way, and can run any GGUF off of HuggingFace. I would like to know what configuration I'm missing out on, though.
If you just want to play for a bit: llamafile
If you want granular control with ease of execution in exchange for having to figure out what the settings mean and figure out which weights to download: koboldcpp. (check out bartowski on huggingface for the weights)
These are all based on llamacpp as a backend, by the way.