Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LM Studio – Discover, download, and run local LLMs

lmstudio.ai

Post reply on HN
Print view View original

111–120 of 157 posts

Re: LM Studio – Discover, download, and run local LLMs

#111

Post by OJFord »

0xEF wrote: post #79
pentagrama wrote: post #41

Curious about this and I just download it. Want to try uncensored models. I have a question, looking for the most popular "uncensored" model I just find "TheBloke/Luna-AI-Llama2-Uncensored-GGML", but it has 14 files to download between 2 to 7 GB, I just download the first one: https://imgur.com/a/DE2byOB I try the model and it works: https://imgur.com/a/2vtPcui I should download all the 14 files to get better results…

Honest question from someone new to exploring and using these models; why do you need uncensored? What are the use-cases that would call for it? Again, not questioning your motives or anything, just straight up curious. To use your example, any of us can find bomb building info online fairly easily, and has been a point of social contention since the Anarchist's cookbook. Nobody needs an uncensored LLM for that, of c…

Where do you typically have the 'safe search' setting when you web search? Personally I have it 'off' ('moderate' when I worked in an office I think) even though I'm not looking for anything that ought to be filtered by it.

I'm not using models censored or otherwise, but I imagine I'd feel the same way there - I won't be offended, so don't try to be too clever, just give me the unfiltered results and let me decide what's correct.

(Bing AI actually banned me for trying to generate images a rough likeness of myself by combining traits of minor celebrities - in combination it shouldn't have looked like those people either, so I don't think should violate ToS, certainly it didn't in intention (I wanted myself, but it doesn't know what I look like and I couldn't provide a photo at the time, idk if you can now (banned!)) so it does happen, 'false positive censoring' if you like.)

Re: LM Studio – Discover, download, and run local LLMs

#113

Post by LeoPanthera »

whartung wrote: post #110

I’m late to the game about this. So I’ll ask a stupid question. As a contrived example, what happens if you feed the LoTR books, the Hobbit, the Silmarillion, and whatever else is germane, into an LLM? Is there a base, empty, “ignorant” LLM that is used as a seed? Do you end up with a Middle Earth savant? Just how does all this work?

There isn't enough text in the Tolkein works to generate a functional LLM. You need to start with a base model that contains enough English (or language of your choice) to become functional.

Such a model (LLaMa is a good example) is not "ignorant," but rather a generalized model capable of a wide range of language tasks. This base model does not have specialized knowledge in any one area but has a broad understanding based on the diverse training data.

If you were to "feed" Tolkien's specific books into this general LLM, the model wouldn't become a Middle Earth savant. It would still provide responses based on its broad training. It might generate text that reflects the style or themes of Tolkien's work if it has learned this from the broader training data, but its responses would be based on patterns learned from the entire dataset, not just those books.

Re: LM Studio – Discover, download, and run local LLMs

#114

Post by oceanplexian »

seydor wrote: post #106
reustle wrote: post #38

This looks great! If you're looking to do the same with open source code, you could likely run Ollama and a UI. https://github.com/jmorganca/ollama + https://github.com/ollama-webui/ollama-webui

oobabooga is no longer popular?

oobabooga is the king.

As far as I know, ollama doesn’t support exllama, qlora fine tuning, multi-GPU, etc. Text-generation-webui might seem like a science project, but it’s leagues ahead (Like 2-4x faster inference with the right plugins) of everything else. Also has a nice openai mock API that works great.

Re: LM Studio – Discover, download, and run local LLMs

#115

Post by smcleod »

reustle wrote: post #38

This looks great! If you're looking to do the same with open source code, you could likely run Ollama and a UI. https://github.com/jmorganca/ollama + https://github.com/ollama-webui/ollama-webui

With a couple of other folks I'm currently working on an Ollama GUI, it's in early stages of development - https://github.com/ai-qol-things/rusty-ollama

Re: LM Studio – Discover, download, and run local LLMs

#117

Post by MuffinFlavored »

1. Mistral

2. Llama 2

3. Code Llama

4. Orca Mini

5. Vicuna

What can I do with any of these models that won't result in 50% hallucinations/it recommending code with APIs that don't exist/it recommending basically regurgitated StackOverflow historical out of date answers (that it was trained on) for libraries that have had their versions/APIs change, etc?

Can somebody share one real use case they are using any of these models for?

Re: LM Studio – Discover, download, and run local LLMs

#118

Post by shironandonon_ »

MuffinFlavored wrote: post #117

1. Mistral 2. Llama 2 3. Code Llama 4. Orca Mini 5. Vicuna What can I do with any of these models that won't result in 50% hallucinations/it recommending code with APIs that don't exist/it recommending basically regurgitated StackOverflow historical out of date answers (that it was trained on) for libraries that have had their versions/APIs change, etc? Can somebody share one real use case they are using any of these…

Why don’t you wrap it with a verification system (eg: web scraper) and auto-regenerate / jailbreak any things you don’t like?

Re: LM Studio – Discover, download, and run local LLMs

#119

Post by MuffinFlavored »

shironandonon_ wrote: post #118
MuffinFlavored wrote: post #117

1. Mistral 2. Llama 2 3. Code Llama 4. Orca Mini 5. Vicuna What can I do with any of these models that won't result in 50% hallucinations/it recommending code with APIs that don't exist/it recommending basically regurgitated StackOverflow historical out of date answers (that it was trained on) for libraries that have had their versions/APIs change, etc? Can somebody share one real use case they are using any of these…

Why don’t you wrap it with a verification system (eg: web scraper) and auto-regenerate / jailbreak any things you don’t like?

Because I pay $20/mo for GPT-4 and don't understand why anybody would run a "less-good" version locally that you can trust less/that has less functionality.

That's why I wanted to try to understand, what am I missing about local-toy LLMs. How are they not just noise/nonsense generators?

Re: LM Studio – Discover, download, and run local LLMs

#120

Post by whartung »

LeoPanthera wrote: post #113
whartung wrote: post #110

I’m late to the game about this. So I’ll ask a stupid question. As a contrived example, what happens if you feed the LoTR books, the Hobbit, the Silmarillion, and whatever else is germane, into an LLM? Is there a base, empty, “ignorant” LLM that is used as a seed? Do you end up with a Middle Earth savant? Just how does all this work?

There isn't enough text in the Tolkein works to generate a functional LLM. You need to start with a base model that contains enough English (or language of your choice) to become functional. Such a model (LLaMa is a good example) is not "ignorant," but rather a generalized model capable of a wide range of language tasks. This base model does not have specialized knowledge in any one area but has a broad understanding…

So, it wouldn't necessarily "know" much about Middle Earth, but might take a stab at writing like Tolkien?
Post reply on HN