I've been using LM Studio for a while, this is a nice update. For what I need, running a local model is more than adequate. As long as you have sufficient RAM, of course.
LM Studio 0.4
101–110 of 134 posts
Re: LM Studio 0.4
#102These days I don't feel the need to use anything other than llama.cpp server as it has a pretty good web UI and router mode for switching models.
MLX support on Macs was the main reason for me.
Re: LM Studio 0.4
#103My complaint is that LM Studio insists on installing as admin on my Mac. For no apparent reason, and they refuse to say why.
Re: LM Studio 0.4
#104Earlier quoted context omitted.
>What was the original core principle of ollama? Nothing, it was always going to be a rug pull. They leached off llama.cpp.
Everyone seems to be missing important piece here. Ollama is/was a one click solution for non technical person to launch a local model. It doesn’t need a lot of configuration, detects Nvidia GPU and starts model inferencing with single command. Core principle being your grandmother should be able to launch local AI model without needing to install 100 dependencies.
Exactly.
I can be in a non-technical team, and put the LLM code inside docker.
The local dev instruction is to install ollama and use it to pull the models and set some env vars.
The same code can point at bedrock when deployed there.
Using straight llamacpp at the time I wrote that it wasn't as straightforward.
Re: LM Studio 0.4
#105These days I don't feel the need to use anything other than llama.cpp server as it has a pretty good web UI and router mode for switching models.
I mostly use LM Studio for browsing and downloading models, testing them out quickly, but then actually integrating them is always with either llama.cpp or vLLM. Curious to try out their new cli though and see if it adds any extra benefits on top of llama.cpp.
Re: LM Studio 0.4
#106I’m really excited for lmster and to try it out. It’s essentially what I want from ollama. Ollama has deviated so much from their original core principles. Ollama has been broken and slow to update model support. There’s this “vendor sync” I’ve been waiting (essentially update ggml) for weeks.
LMStudio is great but its still not open source. I wish something better than Ollama can be created honestly similar to LMStudio (atleast its new CLI Part from what I can tell) and create an open source alternative. I think I am fairly technical but I still prefer how Ollama is simple but I know all the complaints about Ollama and I am really just wishing for a better alternative for the most part. Maybe just a direc…
> Maybe just a direct layer on top of vllm
My dream would be something like vLLM, but without all the Python mess, packaged as a single binary that has both HTTP server + desktop GUI, and can browse/download models. Llama.cpp is like 70% there, but large performance difference between llama.cpp and vLLM for the models I use.
Re: LM Studio 0.4
#107These days I don't feel the need to use anything other than llama.cpp server as it has a pretty good web UI and router mode for switching models.
Concurrency is an important use case when running multiple agents. vLLM can squeeze performance out of your GB10 or GPU that you wouldn't get otherwise.
Re: LM Studio 0.4
#108Earlier quoted context omitted.
>What was the original core principle of ollama? Nothing, it was always going to be a rug pull. They leached off llama.cpp.
Everyone seems to be missing important piece here. Ollama is/was a one click solution for non technical person to launch a local model. It doesn’t need a lot of configuration, detects Nvidia GPU and starts model inferencing with single command. Core principle being your grandmother should be able to launch local AI model without needing to install 100 dependencies.
> Ollama is/was a one click solution for non technical person to launch a local model
Maybe it is today, but initially ollama was only a cli, so obviously not for "non technical people" who would have no idea how to even use a terminal. If you hang out in the Ollama Discord (unlikely, as the mods are very ban-happy), you'd see constantly people asking for very trivial help, like how to enter commands in the terminal, and the community stringing them along, instead of just directing them to LM Desktop or something that would be much better for that type of user.
Re: LM Studio 0.4
#109My complaint is that LM Studio insists on installing as admin on my Mac. For no apparent reason, and they refuse to say why.
Is this possibly the same as this issue? https://github.com/lmstudio-ai/lmstudio-bug-tracker/issues/4...
I've only use LM Desktop on Linux and Windows, never seen anything asking for elevated permissions.
Re: LM Studio 0.4
#110These days I don't feel the need to use anything other than llama.cpp server as it has a pretty good web UI and router mode for switching models.
Concurrency is an important use case when running multiple agents. vLLM can squeeze performance out of your GB10 or GPU that you wouldn't get otherwise.
Also they've just spent more time optimizing vLLM than llama.cpp people done, even when you run just one inference call at a time. Best feature is obviously the concurrency and shared cache though. But on the other hand, new architectures are usually sooner available in llama.cpp than vLLM.
Both have their places and are complementary, rather than competitors :)