Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LM Studio – Discover, download, and run local LLMs

lmstudio.ai

Post reply on HN
Print view View original

131–140 of 157 posts

Re: LM Studio – Discover, download, and run local LLMs

#131

Post by cellis »

MuffinFlavored wrote: post #119
shironandonon_ wrote: post #118

Earlier quoted context omitted.

Why don’t you wrap it with a verification system (eg: web scraper) and auto-regenerate / jailbreak any things you don’t like?

Because I pay $20/mo for GPT-4 and don't understand why anybody would run a "less-good" version locally that you can trust less/that has less functionality. That's why I wanted to try to understand, what am I missing about local-toy LLMs. How are they not just noise/nonsense generators?

The first reason is because GPT-4 is censored for many types of tasks/topics.

Re: LM Studio – Discover, download, and run local LLMs

#132

Post by PeterStuer »

0xEF wrote: post #79
pentagrama wrote: post #41

Curious about this and I just download it. Want to try uncensored models. I have a question, looking for the most popular "uncensored" model I just find "TheBloke/Luna-AI-Llama2-Uncensored-GGML", but it has 14 files to download between 2 to 7 GB, I just download the first one: https://imgur.com/a/DE2byOB I try the model and it works: https://imgur.com/a/2vtPcui I should download all the 14 files to get better results…

Honest question from someone new to exploring and using these models; why do you need uncensored? What are the use-cases that would call for it? Again, not questioning your motives or anything, just straight up curious. To use your example, any of us can find bomb building info online fairly easily, and has been a point of social contention since the Anarchist's cookbook. Nobody needs an uncensored LLM for that, of c…

Most people wanting 'uncensored' are not looking for anything illegal. They just don't want to be forced into some Californian/SV idea of 'morality'.

Re: LM Studio – Discover, download, and run local LLMs

#133

Post by ekianjo »

cooperaustinj wrote: post #83
ekianjo wrote: post #58

Earlier quoted context omitted.

You dont need thebloke. Its trivial to make gguf files from bin models by yourself.

What a comment. Why do it the easy way when the more difficult and slower way works ok it to the same result‽ For people who just want to USE models and not back at them, TheBloke is exactly the right place to go. Like telling someone interested in 3D printing minis to build a 3D printer instead of buying one. Obviously that helps them get to their goal of printing minis faster right?

This is a helpful comment because the Bloke only converts a small fraction of models and hardly ever updates them timely after the first release.

So learn to cook.

Re: LM Studio – Discover, download, and run local LLMs

#134

Post by tvararu »

MuffinFlavored wrote: post #119
shironandonon_ wrote: post #118

Earlier quoted context omitted.

Why don’t you wrap it with a verification system (eg: web scraper) and auto-regenerate / jailbreak any things you don’t like?

Because I pay $20/mo for GPT-4 and don't understand why anybody would run a "less-good" version locally that you can trust less/that has less functionality. That's why I wanted to try to understand, what am I missing about local-toy LLMs. How are they not just noise/nonsense generators?

So do I, but on a flight 2 days ago, I forgot the name of a Ruby method, but knew what it does. I tried looking it up in Dash (offline rdocs) but didn’t find it.

On a whim, I asked Zephyr 7B (Mistral based) “what’s the name of that Ruby method that does ” and it gave me 3 different correct ways of doing what I wanted, including the one I couldn’t remember. That was a real “oh wow” moment.

So offline situations is the most likely use case for me.

Re: LM Studio – Discover, download, and run local LLMs

#135

Post by flexagoon »

whartung wrote: post #120
LeoPanthera wrote: post #113

Earlier quoted context omitted.

There isn't enough text in the Tolkein works to generate a functional LLM. You need to start with a base model that contains enough English (or language of your choice) to become functional. Such a model (LLaMa is a good example) is not "ignorant," but rather a generalized model capable of a wide range of language tasks. This base model does not have specialized knowledge in any one area but has a broad understanding…

So, it wouldn't necessarily "know" much about Middle Earth, but might take a stab at writing like Tolkien?

If you give Tolkien books to a newborn child, they won't become a Tolkien expert. You need to first teach them English, and then give them the books. They will be able to answer questions about Middle Earth, but they'll also be able to form any other English sentence based on what they learned previously. It's basically the same with LLMs.

Re: LM Studio – Discover, download, and run local LLMs

#136

Post by compinter »

andy99 wrote: post #17

I don't mean this as a criticism, I'm just curious because I work in this space too: who is this for? What is the niche of people savvy enough to use this who can't run one of the many open source local llm software? It looks in the screenshot like it's exposing much of the complexity of configuration anyway. Is the value in the interface and management of conversation and models? It would be nice to see info or even…

For me it's pretty simple- LM Studio supports Apple Silicon GPU acceleration out of the box, and I like the interface better than Gradio Web UI. It saves me the headache and the tinkering of the alternatives. That said, free software that's hiring developers probably won't stay free for long, so I'm keeping my eye on other options.

Tl;DR It's for Mac users

Re: LM Studio – Discover, download, and run local LLMs

#138

Post by theoldgit »

xcv123 wrote: post #101
porcoda wrote: post #91

Amusing qualifications for the senior engineering roles they’re hiring for: “Deep understanding of what is a computer, what is computer software, and how the two relate.” Right after the senior ML role that requires people understand how to write “algorithms and programs.” Kinda hard to take those kinds of requirements seriously.

> “Deep understanding of what is a computer, what is computer software, and how the two relate.” Seems like a joke, but many developers do not really understand what's going on behind the scenes. This gets straight to the point. They don't care about HR keyword matching on your CV, or how many years of experience you have of being a mediocre developer with language X or framework Y. I guess during the interview they…

I tend to agree. I have been working in IT for 48 years and it is not at all uncommon to come across developers who have a very narrow and niche view of software development. I have had the privilege to work with a wide range of architects and senior engineers over the years and I have found that the ones who tended to be the most creative (solution wise) were the ones who had deep knowledge right from the bottom of the stack all the way to the top - they did not look at a problem through the lens of a specific language (they all knew multiple languages) - when I have hired for senior positions, unless its been for a very specific skill set, experienced generalists have tended to impress more

Re: LM Studio – Discover, download, and run local LLMs

#139

Post by hugovie »

codeisawesome wrote: post #129
hugovie wrote: post #33

LMStudio is great to run local LLMs, also support OpenAI-compatible API. In the case you need more advance UI/UX, you can use LMStudio with MindMac( https://mindmac.app ), just check this video for details https://www.youtube.com/watch?v=3KcVp5QQ1Ak .

Wow, brand new product but already 'trusted by Oracle, Paypal, Amazon, Cisco' based on the logos I'm seeing on the home page. How was that achieved?

I suppose it was through word of mouth. I know this because they used their work email to purchase MindMac.

Re: LM Studio – Discover, download, and run local LLMs

#140

Post by codeisawesome »

hugovie wrote: post #139
codeisawesome wrote: post #129

Earlier quoted context omitted.

Wow, brand new product but already 'trusted by Oracle, Paypal, Amazon, Cisco' based on the logos I'm seeing on the home page. How was that achieved?

I suppose it was through word of mouth. I know this because they used their work email to purchase MindMac.

Thanks for the reply! Always been curious about this with various startups.
Post reply on HN