Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ollama's new app

ollama.com

Post reply on HN
Print view View original

221–230 of 299 posts

Re: Ollama's new app

#221

Post by whitehexagon »

mchiang wrote: post #54
pentagrama wrote: post #44

Looks like a big pivot on target audience from developers to regular users, at least on the homepage https://ollama.com/ as a product. Before, it was all about the CLI versions of Ollama for devs, now it's not even mentioned. At the bottom of the blog post it says: > For pure CLI versions of Ollama, standalone downloads are available on Ollama’s GitHub releases page. Nothing against that, just an observation. Previou…

one of the maintainers for Ollama. I don't see it as a pivot. We are all developers ourselves, and we use it. In fact, there are many self-made prototypes before this from different individuals. We were hooked, so we built it for ourselves. Ollama is made for developers, and our focus in continually improving Ollama's capabilities.

This caught me out yesterday. I was trying to move models onto external disk, and it seems to require re-installation? but there was no sign of the simple CLI option that was previously presented and I gave up.

As a developer feature request, it would be great if ollama could support more than one location at once, so that it is possible to keep a couple models 'live' but have the option to plug in an external disk with extra models being picked up auto-magically based on the ollama_models path please. Or maybe the server could present a simple html interface next to the API endpoint?

And just to say thanks for making these models easily accessible. I am agAInst AI generally, but it is nice to be able to have a play with these models locally. I havent found one that covers Zig, but appreciate the steady stream of new models to try. Thanks.

Re: Ollama's new app

#222

Post by rubymamis »

pentagrama wrote: post #44

Looks like a big pivot on target audience from developers to regular users, at least on the homepage https://ollama.com/ as a product. Before, it was all about the CLI versions of Ollama for devs, now it's not even mentioned. At the bottom of the blog post it says: > For pure CLI versions of Ollama, standalone downloads are available on Ollama’s GitHub releases page. Nothing against that, just an observation. Previou…

> One missing feature that the ChatGPT desktop app has and I think is a good idea for these local LLM apps is a shortcut to open a new chat anytime (Alt + Space), with a reduced UI. It is great for quick questions.

This is exactly what I've implemented for my Qt C++ app: https://www.get-vox.com

Re: Ollama's new app

#223

Post by smahs »

bapak wrote: post #113
thorum wrote: post #97

If you’re a power user of these LLMs and have coding experience, I actually recommend just whipping together your own bespoke chat UI that you can customize however you like. Grab any OpenAI compatible endpoint for inference and a frontend component framework (many of which have added standard Chat components) - the rest is almost trivial. I threw one together in a week with Gemini’s assistance and now I use it every…

This is the most "just build your own Linux" comment I read this year. Just download some tool and be productive within seconds, I'd say.

I only did it once some 15 years back (in a happy memory) using LFS. It took about a week to get to a functional system with basic necessities. A code finetuned model can write a functional chat UI with all common features and a decent UX in under a minute.

Re: Ollama's new app

#224

Post by antimius »

apitman wrote: post #137
HumanOstrich wrote: post #71

Earlier quoted context omitted.

LM Studio is for hosting/serving local LLMs. Its chat UI is secondary and is pretty limited.

Good to know, thanks. What do people generally use to connect to it for chat?

Msty (msty.app). Currently they're working on Msty Studio which is only accessible to people with a license, but the desktop app is pretty good, it just doesn't have tool (MCP) support.

Re: Ollama's new app

#225

Post by jerieljan »

doctoboggan wrote: post #90
jonahbenton wrote: post #56

Earlier quoted context omitted.

OpenWebUI is what you are looking for from a usability perspective. Supports many models chat.

Last I tried OpenWebUI (A few months ago), it was pretty painful to connect non-OpenAI externally hosted models. There was a workaround that involved installing a 3rd party "function" (or was it a "pipeline"?), but it didn't feel smooth. Is this easier now? Specifically, I would like to easily connect anthropic models just by plugging in my API key.

The trick to this is to run a LiteLLM proxy that has all the connections to whatever you need to connect to and then point Open-WebUI to that.

I've been using this setup for several months now (over a year?) and it's very effective.

The proxy also benefits pretty much any other application you have that recognizes an OpenAI-compatible API. (Or even if it doesn't)

Re: Ollama's new app

#226

Post by ai_viewz »

I have been experimenting many LLMs in Ollama, but the opensource models are still behind paid versions like Cohere. Any model which gives onpar performance and quality of result compared to Cohere , please let me know

Re: Ollama's new app

#227

Post by ai_viewz »

mchiang wrote: post #163
whs wrote: post #142

Earlier quoted context omitted.

Seems that Google intend it to be that way - https://ai.google.dev/gemma/docs/capabilities/function-calli... . I suppose they are saying that the model is good enough that if you put the tool call format in prompt it should be able to handle any formats. I use PetrosStav/gemma3-tools and it seems that it only works half of the time - the rest the model call the tool but it doesn't get properly parsed by Ollama.

unfortunately, I don't think gemma 3 supports tool calling well. It's not trained into the model, and the 'support' for tool calling is post model training. We are working with Google, and trying to give the feedback on improving tool calling capabilities for future Gemma models. Fingers crossed!

We can do function calling with Google's Gemma 3, just need to follow the right pattern.

Re: Ollama's new app

#228

Post by smarx007 »

flux293m wrote: post #218
mchiang wrote: post #54

Earlier quoted context omitted.

one of the maintainers for Ollama. I don't see it as a pivot. We are all developers ourselves, and we use it. In fact, there are many self-made prototypes before this from different individuals. We were hooked, so we built it for ourselves. Ollama is made for developers, and our focus in continually improving Ollama's capabilities.

Congratulations on launching the front-end, but I don't see how it can be made for developers and not have a Linux version.

Whoah, are you telling me that there are devs on Linux who use anything else than a tiled WM? CLI or GTFO /s

Re: Ollama's new app

#229

Post by diggan »

hodgehog11 wrote: post #149

Not surprising; Ollama is set on becoming the standard interface for companies to deploy "open" models. The focus on "local" is incidental, and likely not long term. I'm sure Ollama is going to announce a plan to use "open" models through their own cloud-based API using this app.

> Ollama is set on becoming the standard interface for companies to deploy "open" models.

That's not what I've been seeing, but obviously my perspective (as anyone's) is limited. What I'm seeing is deployments of vLLM, SGLang, llama.cpp or even HuggingFace's Transformers with their own wrapper, at least for inference with open weight models. Somehow, the only place where I come across recommendations for running Ollama was on HN and before on r/LocalLlama but not even there as of late. The people who used to run Ollama for local inference (+ OpenWebUI) now seem to mostly be running LM Studio, myself included too.

Re: Ollama's new app

#230

Post by ddlsmurf »

coder543 wrote: post #95

I am somewhat surprised that this app doesn't seem to offer any way to connect to a remote Ollama instance. The most powerful computer I own isn't necessarily the one I'm running the GUI on.

I use ollama via the llm plugin: https://github.com/taketwo/llm-ollama?tab=readme-ov-file#oll...
Post reply on HN