181–190 of 235 posts
Earlier quoted context omitted.
>So where's the non-sketchy, non-for-profit equivalent llama.cpp, kobold.cpp, oobabooga, llmstudio, etc. There are dozens at this point. And while many chalk the attachment to ollama up to a "skill issue", that's just venting frustration that all something has to do to win the popularity contest is to repackage and market it as an "app". I prefer first-party tools, I'm comfortable managing a build environment and cal…
> llama.cpp, kobold.cpp, oobabooga None of these three are remotely as easy to install or use. They could be, but none of them are even trying. > lmstudio This is a closed source app with a non-free license from a business not making money. Enshittification is just a matter of when.
Which part of the user experience did you have problems with when using it?
Earlier quoted context omitted.
I'm not sure that's fair, given that the distilled models are almost as good. Do you really think Deepseek's web interface is giving you access to 671b? They're going to be running distilled models there too.
It's simple enough to test the tokenizer to determine the base model in use (DeepSeek V3, or a Llama 3/Qwen 2.5 distill). Using the text "സ്മാർട്ട്", Qwen 2.5 tokenizes as 10 tokens, Llama 3 as 13, and DeepSeek V3 as 8. Using DeepSeek's chat frontend, both DeepSeek V3 and R1 returns the following response (SSE events edited for brevity): {"content":"സ","type":"text"},"chunk_token_usage":1 {"content":"്മ","type":"text…
Earlier quoted context omitted.
This sounds like it should be trivial to reproduce and extend - I look forward to trying out your repo!
the owner of that PR has already forked ollama. try it out. I did and it works great.
This is not a sarcastic comment. I'm genuinely happy that this was the outcome.
Earlier quoted context omitted.
Ok assuming what you said is correct, why wouldn't Ollama then be able to support Vulkan by default out of the box? Sorry I'm not sure what's the relationship exactly between the two projects. This is a genuine questions, not a troll question.
check the PR, it's a very short one. It's not more complicated than setting a compile time flag. I have no idea why they have been ignoring it. Ollama is just a friendly front end for llama.cpp. It doesn't have to do any of those things you mentioned. Llama.cpp does all that.
Shouldn't it just call Llama.cpp and let Llama.cpp handle the flags internally within Llama.cpp? I'm thinking from an abstraction layer perspective.
Earlier quoted context omitted.
What do you mean there is no such thing as R1-1.5b? DeepSeek released a distilled version based on a 1.5B Qwen model with the full name DeepSeek-R1-Distill-Qwen-1.5B, see chapter 3.2 on page 14 of their research article [0]. [0] https://arxiv.org/abs/2501.12948
Which is not the same model, it's not R1 it's R1-Distill-Qwen-1.5B....
Earlier quoted context omitted.
It's simple enough to test the tokenizer to determine the base model in use (DeepSeek V3, or a Llama 3/Qwen 2.5 distill). Using the text "സ്മാർട്ട്", Qwen 2.5 tokenizes as 10 tokens, Llama 3 as 13, and DeepSeek V3 as 8. Using DeepSeek's chat frontend, both DeepSeek V3 and R1 returns the following response (SSE events edited for brevity): {"content":"സ","type":"text"},"chunk_token_usage":1 {"content":"്മ","type":"text…
I’m not sure I understand what this comment is responding to. Wouldn’t a distilled Deepseek still use the same tokenizer? I’m not claiming they are using llama in their backend. I’m just saying they are likely using a lower-parameter model too.
Earlier quoted context omitted.
Additionally, Ollama makes model installation a single command. With llama.cpp, you have to download the raw models from Huggingface and handle storage for them yourself.
Not really, llama.cpp can download for quite some time, not as elegant as ollama but: llama-server --model-url "https://huggingface.co/bartowski/DeepSeek-R1-Distill-Qwen-32B-GGUF/resolve/main/DeepSeek-R1-Distill-Qwen-32B-IQ4_XS.gguf" Will get you up and running in one single command.
Earlier quoted context omitted.
This is great, I did not know about RamaLama and I'll be using and recommending that in future and if I see people using Ollama in instructions I'll recommend they move to RamaLama in the future. Cheers.
Yeah, I would love an actual alternative to Ollama, but RamaLama is not it unfortunately. As the other commenter said, onboarding is important. I just want one operation install and it needs to work and the simple fact RamaLama is written in Python, assures it will never be that easy, and this is even more true with LLM stuff when using AMD gpu. I know there will be people that disagree with this, that's ok. This is…
Earlier quoted context omitted.
It’s so hard to decipher the complaints about ollama in this comment section. I keep reading comments from people saying they don’t trust it, but then they don’t explain why they don’t trust it and don’t answer any follow up questions. As someone who doesn’t follow this space, it’s hard to tell if there’s actually something sketchy going on with ollama or if it’s the usual reactionary negativity that happens when a t…
they don’t explain why they don’t trust it We need to know a few things: 1) Show me the lines of code that log things and how it handles temp files and storage. 2) No remote calls at all. 3) No telemetry at all. This is the feature list I would want to begin trusting. I use this stuff, but I also don’t trust it.
The question is: Why is ollama considered “sketchy” but llama.cpp is not, given that both are open source?
I’m not trying to debate it. I’m trying to understand why people are saying this.