171–180 of 299 posts
I've been using Open WebUI and have been blown away, it's a better ChatGPT interface than ChatGPT! https://github.com/open-webui/open-webui Curious how this compares to that, which has a ton of features and runs great
That’s what I came to say. I made a tool for my Mac where I can highlight any text then set a hotkey to use that text in a query to an LLM. Nice because it works on any text. Browser, IDE, email etc.
If you’re a power user of these LLMs and have coding experience, I actually recommend just whipping together your own bespoke chat UI that you can customize however you like. Grab any OpenAI compatible endpoint for inference and a frontend component framework (many of which have added standard Chat components) - the rest is almost trivial. I threw one together in a week with Gemini’s assistance and now I use it every…
Either directly or use it as a base for your own bespoke experience.
Earlier quoted context omitted.
No one should use ollama. A cursory search of r/localllama gives plenty of occassions where they've proven themselves bad actors. Here's a 'fun' overview https://www.reddit.com/r/LocalLLaMA/comments/1kg20mu/so_why_... There are multiple (far better) options - eg LM studio if you want GUI, llama.cpp if you want the CLI that ollama ripped off. IMO the only reason ollama is even in the conversation is it was easy to get…
This is just wrong. Ollama has moved off of llama.cpp and is working with hardware partners to support GGML. https://ollama.com/blog/multimodal-models
Not surprising; Ollama is set on becoming the standard interface for companies to deploy "open" models. The focus on "local" is incidental, and likely not long term. I'm sure Ollama is going to announce a plan to use "open" models through their own cloud-based API using this app.
Any whiff of a cloud service and the lawyers will freak out.
That's why we run models via Ollama on our laptops (M-series is crazy powerful) and a few servers on the intranet for more oomph.
LM Studio changed their license to allow commercial use without "call me" pricing, so we might look into that more too.
There's also Jan AI, which supports Linux, MCP, any Vulkan GPU, any Llama.cpp-compatible model, and optionally multiple cloud models as well. That seems like a better solution than this.
Choice is good but here is why prefer Ollama over others (I'm biased because I work on Ollama). Supporting multiple backends is HARD. Originally, we thought we'd just add multiple backends to Ollama - MLX, ROCm, TRT-LLM, etc. It sounds really good on paper. In practice, you get into the lowest common denominator effect. What happens when you want to release Model A together with the model creator, and backend B doesn…
You conceptually divide your product to "universal experience" and "conditional experience". You add platform-specific things to the conditional experience, while keeping universal experience unified. I mean, do you even have a choice? The backend limits you, the only alternative you have is to change the backend upstream, which often times is the same as no alternative.
The only case where this is a real problem is when the backends are so different that the universal experience is not the main experience. But I don't think this is the case here?
I tried Ollama once but immediately removed it, when I couldn't easily install models that are outside of the models they "support". LM Studio is by far the best tool out there in my humble opinion.
Looks like a big pivot on target audience from developers to regular users, at least on the homepage https://ollama.com/ as a product. Before, it was all about the CLI versions of Ollama for devs, now it's not even mentioned. At the bottom of the blog post it says: > For pure CLI versions of Ollama, standalone downloads are available on Ollama’s GitHub releases page. Nothing against that, just an observation. Previou…