111–120 of 235 posts
Earlier quoted context omitted.
It's fully open source. I mean yes it uses llama.cpp without giving it credit. But why run it in a VM?
> But why run it in a VM? Because you don't execute untrusted code in your machine without containerization/virtualization. Don't you?
Ollama is sketchy enough that I run it in a VM. Which is odd because it would probably take less effort to just run Llama.cpp directly, but VMs are pretty easy so just went that route. When I see people bring up the sketchiness most of the time the creator responds with the equivalent of shrugs , which imo increases the sketchiness.
ollama advertising llama.cpp features as their own is very dishonest in my opinion.
So many here are trashing on Ollama, saying it's "just" nice porcelain around llama.cpp and it's not doing anything complicated. Okay. Let's stipulate that. So where's the non-sketchy, non-for-profit equivalent? Where's the nice frontend for llama.cpp that makes it trivial for anyone who wants to play around with local LLMs without having to know much about their internals? If Ollama isn't doing anything difficult, w…
So many here are trashing on Ollama, saying it's "just" nice porcelain around llama.cpp and it's not doing anything complicated. Okay. Let's stipulate that. So where's the non-sketchy, non-for-profit equivalent? Where's the nice frontend for llama.cpp that makes it trivial for anyone who wants to play around with local LLMs without having to know much about their internals? If Ollama isn't doing anything difficult, w…
There are many flaws in Ollama but it makes many things much easier esp. if you don’t want to bother building and configuring. They do take a long time to merge any PRs though. One of my PRs has been waiting for 8 months and there was this another PR about KV cache quantization that took them 6 months to merge.
[1]: https://msty.app
So many here are trashing on Ollama, saying it's "just" nice porcelain around llama.cpp and it's not doing anything complicated. Okay. Let's stipulate that. So where's the non-sketchy, non-for-profit equivalent? Where's the nice frontend for llama.cpp that makes it trivial for anyone who wants to play around with local LLMs without having to know much about their internals? If Ollama isn't doing anything difficult, w…
this is such a low hanging fruit that it's silly how they are acting.
Ollama needs competition. I’m not sure what drives the people that maintain it but some of their actions imply that there are ulterior motives at play that do not have the benefit of their users in mind. However such projects require a lot of time and effort and it’s not clear if this project can be forked and kept alive.
The most recent one of the top of my head is their horrendous aliasing of DeepSeek R1 on their model hub, misleading users into thinking they are running the full model but really anything but the 671b alias is one of the distilled models. This has already led to lots of people claiming that they are running R1 locally when they are not.
Earlier quoted context omitted.
with vulkan it runs much much faster on consumer hardware, especially opn igpus like intel or amd.
For Intel, OpenVINO should be the preferred route. I dont follow AMD, but Vulkan is just the common denominator here.
I agree they should also support OpenVINO, but compared to Vulkan OpenVINO is a tiny market.
Earlier quoted context omitted.
Llama.cpp dropped support for multimodal vlms. That is why I am using ollama. I would happily switch back if I could.
llama.cpp readme still lists multimodal models.. Qwen2-VL and others. Is that inaccurate, or something different? [edit] Oh I see, here's an issue about it: https://github.com/ggerganov/llama.cpp/issues/8010
Earlier quoted context omitted.
the work involved is tiny compared to the work llama.cpp did to get vulkan up and running. this is not rocket science.
This sounds like it should be trivial to reproduce and extend - I look forward to trying out your repo!
This is going to sound like a troll, but it's an honest question: Why do people use Ollama over llama.cpp? llama.cpp has added a ton of features, is about as user-friendly as Ollama, and is higher-performance. Is there some key differentiator for Ollama that I'm missing?
Ollama - `brew install ollama` llama.cpp - Read the docs, with loads of information and unclear use cases. Question if it has API compatibility and secondary features that a bunch of tools expect. Decide it's not worth your effort when `ollama` is already running by the time you've read the docs