81–90 of 235 posts
Earlier quoted context omitted.
I did not assume the worst when submitting the post, but that is also my suspicion. The whole thing is very dodgy.
Are closer to the metal AI developers an under tracked bottle neck? AMD and Intel can barely get off the ground due to lagging software developers.
can someone please give a quick summary of the criticism towards ollama? as far as my intel goes it's a mozilla project shouldered mostly by one 10x programmer. i found ollama through hn and last time i didn't notice any lack of trust or suspected sketchiness ... so what changed?
In contrast, the ollama dev team is doing useful work (creating an easy interface) but otherwise mostly piggybacking off the already existing infrastructure
Ollama is sketchy enough that I run it in a VM. Which is odd because it would probably take less effort to just run Llama.cpp directly, but VMs are pretty easy so just went that route. When I see people bring up the sketchiness most of the time the creator responds with the equivalent of shrugs , which imo increases the sketchiness.
It's fully open source. I mean yes it uses llama.cpp without giving it credit. But why run it in a VM?
can someone please give a quick summary of the criticism towards ollama? as far as my intel goes it's a mozilla project shouldered mostly by one 10x programmer. i found ollama through hn and last time i didn't notice any lack of trust or suspected sketchiness ... so what changed?
That's completely off the mark.
Earlier quoted context omitted.
I am very technically inclined and use Ollama (in a VM, but still) because of all the steps and non-obviousness of how to run Llama.cpp. This framing feels a bit like the “Dropbox won’t succeed because rsync is easy” thinking.
> This framing feels a bit like the “Dropbox won’t succeed because rsync is easy” thinking. No this isn't. There are plenty of end user GUI apps that make it far easier than Ollama to download and run local LLMs (disclaimer: I build one of them). That's an entirely different market. IMO, the intersection between the set of people who use a command line tool, and the set of people who are incapable of running `brew in…
When I read the llama.cpp repo and see I have to build it, vs ollama where I just have to get it, the choice is already made.
I just want something I can quickly run and use with aider or mess around with. When I need to do real work I just use whatever OpenAI model we have running on Azure PTUs
I use Ollama because I am a casual user and can't be bothered to read the docs on how to setup llama.cpp. I just want to run a simple llm locally. Why would I care about Vulkan?
with vulkan it runs much much faster on consumer hardware, especially opn igpus like intel or amd.
Ollama needs competition. I’m not sure what drives the people that maintain it but some of their actions imply that there are ulterior motives at play that do not have the benefit of their users in mind. However such projects require a lot of time and effort and it’s not clear if this project can be forked and kept alive.
llama.cpp has supported vulkan for more than a year now. For more than 6 months now there has been an open PR to add vulkan backend support for Ollama. However, Ollama team has not even looked at it or commented on it. Vulkan backends are existential for running LLMs on consumer hardware (iGPUs especially). It's sad to see Ollama miss this opportunity.
The PR has been legitimately out-of-date and unmergeable for many months. It was forward-ported a few weeks ago, and is now still awaiting formal review and merging. (To be sure, Vulkan support in Ollama will likely stay experimental for some time even if the existing PR is merged, and many setups will need manual adjustment of the number of GPU layers and such. It's far from 100% foolproof even in the best-case scen…
Earlier quoted context omitted.
It's fully open source. I mean yes it uses llama.cpp without giving it credit. But why run it in a VM?
It severely over-permissions itself on my Mac.
Earlier quoted context omitted.
I did not assume the worst when submitting the post, but that is also my suspicion. The whole thing is very dodgy.
Are closer to the metal AI developers an under tracked bottle neck? AMD and Intel can barely get off the ground due to lagging software developers.