Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LM Studio – Discover, download, and run local LLMs

lmstudio.ai

Post reply on HN
Print view View original

81–90 of 157 posts

Re: LM Studio – Discover, download, and run local LLMs

#81

Post by SOLAR_FIELDS »

omneity wrote: post #40

I really like LM Studio and had it open when I came across this post. LM Studio is an interesting mixture of: - A local model runtime - A model catalog - A UI to chat with the models easily - An openAI compatible API And it has several plugins such as for RAG (using ChromaDB) and others. Personally I think the positioning is very interesting. They're well positioned to take advantage of new capabilities in the OS eco…

How does it compare with something like FastChat? https://github.com/lm-sys/FastChat

Feature set seems like a decent amount of overlap. One limitation of FastChat, as far as I can tell, is that one is limited to the models that FastChat supports (though I think it would be minor to modify it to support arbitrary models?)

Re: LM Studio – Discover, download, and run local LLMs

#82

Post by nigma »

reustle wrote: post #38

This looks great! If you're looking to do the same with open source code, you could likely run Ollama and a UI. https://github.com/jmorganca/ollama + https://github.com/ollama-webui/ollama-webui

I'm having a lot of fun chatting with characters using Faraday and koboldcpp. Faraday has a great UI that lets you adjust character profiles, generate alternative model responses, undo, or edit dialogue, and experiment with how models react to your input. There's also SillyTavern that I have yet to try out.

- https://faraday.dev/

- https://github.com/LostRuins/koboldcpp

- https://github.com/SillyTavern/SillyTavern

Re: LM Studio – Discover, download, and run local LLMs

#83

Post by cooperaustinj »

ekianjo wrote: post #58
kgeist wrote: post #16

For my experiments with new self-hostable models on Linux, I've been using a script to download GGUF-models from TheBloke on HuggingFace (currently, TheBloke's repository has 657 models in the GGUF format) which I feed to a simple program I wrote which invokes llama.cpp compiled with GPU support. The GGUF format and TheBloke are a blessing, because I'm able to check out new models basically on the day of their releas…

You dont need thebloke. Its trivial to make gguf files from bin models by yourself.

What a comment. Why do it the easy way when the more difficult and slower way works ok it to the same result‽ For people who just want to USE models and not back at them, TheBloke is exactly the right place to go.

Like telling someone interested in 3D printing minis to build a 3D printer instead of buying one. Obviously that helps them get to their goal of printing minis faster right?

Re: LM Studio – Discover, download, and run local LLMs

#84

Post by xcv123 »

0xEF wrote: post #79
pentagrama wrote: post #41

Curious about this and I just download it. Want to try uncensored models. I have a question, looking for the most popular "uncensored" model I just find "TheBloke/Luna-AI-Llama2-Uncensored-GGML", but it has 14 files to download between 2 to 7 GB, I just download the first one: https://imgur.com/a/DE2byOB I try the model and it works: https://imgur.com/a/2vtPcui I should download all the 14 files to get better results…

Honest question from someone new to exploring and using these models; why do you need uncensored? What are the use-cases that would call for it? Again, not questioning your motives or anything, just straight up curious. To use your example, any of us can find bomb building info online fairly easily, and has been a point of social contention since the Anarchist's cookbook. Nobody needs an uncensored LLM for that, of c…

For the entertainment value

Re: LM Studio – Discover, download, and run local LLMs

#85

Post by xcv123 »

cloudking wrote: post #60

Is anyone using open source models to actually get work done or solving problems in their software architecture? So far I haven't found anything near the quality of GPT-4.

> So far I haven't found anything near the quality of GPT-4

GPT-4 has an estimated 1.8 trillion parameters. Orders of magnitude beyond open source models and ~10x GPT-3.5 which has 175 billion parameters.

https://the-decoder.com/gpt-4-architecture-datasets-costs-an...

Re: LM Studio – Discover, download, and run local LLMs

#86

Post by lloydatkinson »

0xEF wrote: post #79
pentagrama wrote: post #41

Curious about this and I just download it. Want to try uncensored models. I have a question, looking for the most popular "uncensored" model I just find "TheBloke/Luna-AI-Llama2-Uncensored-GGML", but it has 14 files to download between 2 to 7 GB, I just download the first one: https://imgur.com/a/DE2byOB I try the model and it works: https://imgur.com/a/2vtPcui I should download all the 14 files to get better results…

Honest question from someone new to exploring and using these models; why do you need uncensored? What are the use-cases that would call for it? Again, not questioning your motives or anything, just straight up curious. To use your example, any of us can find bomb building info online fairly easily, and has been a point of social contention since the Anarchist's cookbook. Nobody needs an uncensored LLM for that, of c…

It’s very easy to hit absurd “moral” limits on chatgpt for the most stupid things.

Earlier I was looking for “a phrase that is used as an insult for someone who writes with too much rambling” and all I got was some bullshit about how it’s sorry but it can’t do that because it’s allegedly against its OpenAI rules.

So I asked again “a phrase negatively used to mean someone that writes too much while rambling” and it worked.

I simply cannot be bothered to deal with stupid insipid “corporate friendly language” and other dumb restrictions.

Imagine having a real conversation with someone and they freaked out any time anything negative was discussed?

TLDR: Thought police ruining LLMs

Re: LM Studio – Discover, download, and run local LLMs

#89

Post by paul_n »

cooperaustinj wrote: post #83
ekianjo wrote: post #58

Earlier quoted context omitted.

You dont need thebloke. Its trivial to make gguf files from bin models by yourself.

What a comment. Why do it the easy way when the more difficult and slower way works ok it to the same result‽ For people who just want to USE models and not back at them, TheBloke is exactly the right place to go. Like telling someone interested in 3D printing minis to build a 3D printer instead of buying one. Obviously that helps them get to their goal of printing minis faster right?

Actually, consider that the commenter may have helped un-obfuscate this world a little bit by saying that it is in fact easy. To be honest the hardest part about the local LLM scene is the absurd amount of jargon introduced - everything looks a bit more complex than it is. It’s really is easy with llama.cpp, someone even wrote a tutorial here: https://github.com/ggerganov/llama.cpp/discussions/2948 .

But yes, TheBloke tends to have conversions up very quickly as well and has made a name for himself for doing this (+more)

Re: LM Studio – Discover, download, and run local LLMs

#90

Post by paul_n »

3cats-in-a-coat wrote: post #25
andy99 wrote: post #17

I don't mean this as a criticism, I'm just curious because I work in this space too: who is this for? What is the niche of people savvy enough to use this who can't run one of the many open source local llm software? It looks in the screenshot like it's exposing much of the complexity of configuration anyway. Is the value in the interface and management of conversation and models? It would be nice to see info or even…

It's for people who want to discover LLMs and either don't have the skill to deploy it, or value their time, and prefer not to fool around for hours getting it to work before they can try it. The fact it has configuration is good, as long as it has some defaults.

Yeah, I think I fit into this category. If I see a new model announced, it’s been nice to just click and evaluate for myself if it’s useful for me. If anyone knows other tools for this kind workflow I’d love to hear about them. Right now I just keep my “test” prompts in a text file.
Post reply on HN