Ollama v0.1.33 with Llama 3, Phi 3, and Qwen 110B
1–10 of 65 posts
Re: Ollama v0.1.33 with Llama 3, Phi 3, and Qwen 110B
#2this looks very awesome. can someone tell me why there is no chatter about this? is there something else out there that blows this out of the water in terms of ease of use and access to sample many LLM's ?
Re: Ollama v0.1.33 with Llama 3, Phi 3, and Qwen 110B
#3[Why] do models require a new version? It can already take arbitrary gguf; I assumed they just had a registry online
Re: Ollama v0.1.33 with Llama 3, Phi 3, and Qwen 110B
#4[Why] do models require a new version? It can already take arbitrary gguf; I assumed they just had a registry online
They do, and I was using the "new" models before the update. Perhaps there is tuning or bug fixes for them? Or they just want to confirm that these are supported. There are some new models that do have different architectures, so sometimes an update is necessary.
Re: Ollama v0.1.33 with Llama 3, Phi 3, and Qwen 110B
#5this looks very awesome. can someone tell me why there is no chatter about this? is there something else out there that blows this out of the water in terms of ease of use and access to sample many LLM's ?
Lola a has been brought up many times on HN. It’s a great tool!
Re: Ollama v0.1.33 with Llama 3, Phi 3, and Qwen 110B
#6[Why] do models require a new version? It can already take arbitrary gguf; I assumed they just had a registry online
Because the way they are quantized takes time to get bug-free when new architectures are released. If a model was quantized with a known bug in the quantizer, then it effectively makes those quantized versions buggy and they need to be requantized with a new version of llamacpp which has this fixed.
Re: Ollama v0.1.33 with Llama 3, Phi 3, and Qwen 110B
#7this looks very awesome. can someone tell me why there is no chatter about this? is there something else out there that blows this out of the water in terms of ease of use and access to sample many LLM's ?
Ollama is really organized - it relies on llama but the UX and organization it provides makes it legit. We recently made a one-click wizard to run Open WebUI and Ollama together, self hosted and remotely accessible but locally hosted [1]
Re: Ollama v0.1.33 with Llama 3, Phi 3, and Qwen 110B
#8this looks very awesome. can someone tell me why there is no chatter about this? is there something else out there that blows this out of the water in terms of ease of use and access to sample many LLM's ?
I use a mix of using llamacpp directly via my own python bindings and using it via llamacpp-python for function calling and full control over parameters and loading, but otherwise ollama is just great for ease of use. There's really not a reason not to use it, if just want to load gguf models and don't have any intricate requirements.
Re: Ollama v0.1.33 with Llama 3, Phi 3, and Qwen 110B
#9this looks very awesome. can someone tell me why there is no chatter about this? is there something else out there that blows this out of the water in terms of ease of use and access to sample many LLM's ?
HN isnt really the best space for LLM news - r/LocalLlama and twitter are much better. I think HN has some cultural issues with “AI” news
Re: Ollama v0.1.33 with Llama 3, Phi 3, and Qwen 110B
#10I wonder if Ollama will or plans to have other "Supported backends" than llama.cpp. It's listed on the very last line of their readme as if the llama.cpp dependency is just incidental and a very minor detail rather than Ollama as a deployment mechanism for llama.cpp and gguf based models.