121–130 of 176 posts
Is llama.cpp (and thus llama.app) really that much better than Ollama? I've Only ever played with Ollama, so geniously curious to hear other's real-world experiences.
> Is llama.cpp (and thus llama.app) llama.app is just an URL (for the "advertisement" webpages of llama.cpp outside GitHub). > that much better than Ollama llama.cpp is the real thing, ollama was a fork that remained inferior.
Old news by now, but you might not be aware that llama-server can do multi-model for a while now, Meaning that you (and by that I mean your AI agent that has read the llama.cpp code) can write an ini file pointing to your models with parameters optimized for the specific model on your specific hardware. (Optimized by you through testing. Not that AI) Then, any api client can just select a model and the system does th…
Is that with vanilla llama.cpp or with the third-party llama-swap manager? Last time I checked llama-swap was still the go-to solution, although I admit I haven't looked into it further.
> but you might not be aware that llama-server can do multi-model for a while now
you will see that the sentence structure clearly implies both a change compared with a prior state and also lack of any third-party thing.
So the answer to the question has already been encoded as text available.
_
I can see the desire for explicit validation though. For that, I would propose a sentence structure like
> Oh cool! That means that llama-swap is now superseded/no longer needed?
That shows that you've read and understand the message, gives you the double-check and might on top spark a conversation about how these solutions compare. Plus that if the guy you're commenting too has spoken nonsense, they need to backpedal.
Llama.cpp team has failed to make their tech easy to install and use for years. Why can’t they figure it out???
But it is easy ... clone the git repo, make a build dir, cd into it, run cmake .., run build/bin/llama-server -m /path/to/model.gguf Browse to served web page with chat UI.... By "easy" do you mean "very lazy"?
It seems that llama.app is a direct competitor to ollama.com I can understand the desire for the llama.cpp project to want to own the end user relationship, it is true that previous to this they were a tool provider and not really owning the end user experience.
Earlier quoted context omitted.
I think it is due to the new website? it now looks like every other vibe coded site,the only upside is that is looks more saleable for people unfamiliar with it, e.g., explaining OSPO,IT the stack you are using. they should also add a pricing page for eenterprise where they promise 99.9% uptime for local models*.
Wow it’s aggressively vibe coded. Nothing inherently wrong with that, but it looks a bit amateurish which is funny. I’m still waiting on 98.css to become the standard for vibe coded sites. You don’t have to read docs anyway if you’re just using LLMs! All you have to do is say “use 98.css” and you have a 10/10 site https://jdan.github.io/98.css/
Earlier quoted context omitted.
But it is easy ... clone the git repo, make a build dir, cd into it, run cmake .., run build/bin/llama-server -m /path/to/model.gguf Browse to served web page with chat UI.... By "easy" do you mean "very lazy"?
VERY
I find it useful, but the models I run are pretty rubbish due to my lack of RAM, which is a pity.
Earlier quoted context omitted.
Is that with vanilla llama.cpp or with the third-party llama-swap manager? Last time I checked llama-swap was still the go-to solution, although I admit I haven't looked into it further.
If you revisit my comment and pay attention to the opener: > but you might not be aware that llama-server can do multi-model for a while now you will see that the sentence structure clearly implies both a change compared with a prior state and also lack of any third-party thing. So the answer to the question has already been encoded as text available. _ I can see the desire for explicit validation though. For that, I…
Does the llama.cpp UI provide the same? If not, it is too early to say that llama-swap is “superseded/no longer needed.”
Earlier quoted context omitted.
You're not wrong about the appearance of sketchiness of that, but.... dude, come on. "It's not hard" is only true because you already know how to do it.
It's literally three steps, assuming you have the equivalent of the debian "build-essential" dependencies installed on your system for cmake, compiler. It's the exact three rows of my post, pasted one at a time into the CLI. The llama-server build guide page is actually pretty good. I mean, sure, if there's people who can't figure that out, they're probably better off using a GUI that is a wrapper on top of somebody…
Anything that suggests curl into bash just plain sketches me out. (edit: I know, this isn't totally rational, it just seems weird to me. We download and trust a lot of software and run code from a bunch of package repositories as a regular activity...). Git clone llama.cpp and build it, it's not hard. https://github.com/ggml-org/llama.cpp/blob/master/docs/build... literally just a few steps for the basics: git clone…