Does it work with NPUs ?
In the end it's llama.cpp doing the inference, so whatever llama.cpp supports, you should be able to use with LM Studio
LM Studio 0.4
91–100 of 134 posts
Re: LM Studio 0.4
#92Finally UI that is not so ugly. Now I'm only wondering if I somehow can setup that I can share the same LLM models between LM Studio and llamabarn/Ollama (so that I don't have to waste storage on duplicated models).
Ollama made the wonderful choice of trying to replicate Docker registries/layers for the model weights, so of course the models you download with Ollama cannot be easily reused with other tooling.
Compared to models downloaded with LM Studio, which are just the directories + the weights as made, you just point llama.cpp/$tool-of-choice and it works.
Re: LM Studio 0.4
#93What’s the main use-case for this? I get that I can run local models, but all the paid for (remote) models are superior. So is the use-case just for people who don’t want to use big tech’s models? Is this just for privacy conscious people? Or is this just for “adult” chats, ie porn bots? Not being cynical here, just wanting to understand the genuine reasons people are using it.
Reports of people getting hit by twitchy fingered banbots on cloud LLMs are starting to show up(Gemini bans apparently kill Gmail and GDrive too). Paranoid types like I am appreciate local options that won't get me banned.
Re: LM Studio 0.4
#94How does LM Studio differ from Ollama? Why would I use one rather than the other? The impression I get is that LM Studio is basically an Ollama-type of solution but with an IDE included -- is that a fair approximation? Things change so fast in the AI space that I really cannot keep up :(
Ollama is CLI/API "first". LM studio is a proper full blown gui with chat features etc. It's far easier to use than Ollama at least for non technical users (though they are increasingly merging in functionality, with LM studio adding CLI/API features and Ollama adding more UI).
Even as a technical person, when I wanted to play with running models locally, LM Studio turned it into a couple of button clicks.
Without much background, you’re finding models, chatting with them, have an OpenAI-compatible API w/logging. Haven’t seen the new version, but LM Studio was already pretty great.
Re: LM Studio 0.4
#95Why is it that there are ZERO truly prosumer LLM front ends from anyone you can pay? The closest thing we have to an LLM front end where you can actually CONTROL your model (i.e. advanced sampling settings) is oobabooga/sillytavern - both ultimately UIs designed mostly for "roleplay/cooming". It's the same shit with image gen and ComfyUI too!!! LM Studio purported to be something like those two, but it has NEVER prop…
You’re forgetting about Open WebUI.
Which is still WAY less feature complete than oobabooga/sillytavern and it's not even close.
Re: LM Studio 0.4
#96This release introduces parallel requests with continuous batching for high throughput serving, all-new non-GUI deployment option, new stateful REST API, and a refreshed user interface.
are parallel requests "free"? or do you half performance when sending two requests in parallel?
I have seen ~1,300 tokens/sec of total throughout with Llama 3 8B on a MacBook Pro. So no, you don’t halve the performance. But running batched inference takes more memory, so you have to use shorter contexts than if you weren’t batching.
Re: LM Studio 0.4
#97What’s the main use-case for this? I get that I can run local models, but all the paid for (remote) models are superior. So is the use-case just for people who don’t want to use big tech’s models? Is this just for privacy conscious people? Or is this just for “adult” chats, ie porn bots? Not being cynical here, just wanting to understand the genuine reasons people are using it.
I originally used local models as a somewhat therapeutic/advice thing. I didn't want to give openAI all my dirt. But then I decided I'm just a chemical reaction and a product of my environment, so I gave chatGPT all my dirt anyway. But before, I cared about my privacy.
> But then I decided I'm just a chemical reaction
That doesn’t address the practical significance of privacy, though. The real risk isn’t that OpenAI employees will read your chats for personal amusement. The risk is that OpenAI will exploit the secrets you’ve entrusted to them, to manipulate you, or to enable others to manipulate you.
The more information an unscrupulous actor has about you, the more damage they can do.
Re: LM Studio 0.4
#98These days I don't feel the need to use anything other than llama.cpp server as it has a pretty good web UI and router mode for switching models.
Re: LM Studio 0.4
#99Earlier quoted context omitted.
What was the original core principle of ollama? I had used oobabooga back in the day and found ollama unnecessary.
>What was the original core principle of ollama? Nothing, it was always going to be a rug pull. They leached off llama.cpp.
Everyone seems to be missing important piece here. Ollama is/was a one click solution for non technical person to launch a local model. It doesn’t need a lot of configuration, detects Nvidia GPU and starts model inferencing with single command.
Core principle being your grandmother should be able to launch local AI model without needing to install 100 dependencies.
Re: LM Studio 0.4
#100I’m really excited for lmster and to try it out. It’s essentially what I want from ollama. Ollama has deviated so much from their original core principles. Ollama has been broken and slow to update model support. There’s this “vendor sync” I’ve been waiting (essentially update ggml) for weeks.
LMStudio is great but its still not open source. I wish something better than Ollama can be created honestly similar to LMStudio (atleast its new CLI Part from what I can tell) and create an open source alternative.
I think I am fairly technical but I still prefer how Ollama is simple but I know all the complaints about Ollama and I am really just wishing for a better alternative for the most part.
Maybe just a direct layer on top of vllm or llama.cpp itself?