Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LM Studio – Discover, download, and run local LLMs

lmstudio.ai

Post reply on HN
Print view View original

101–110 of 157 posts

Re: LM Studio – Discover, download, and run local LLMs

#101

Post by xcv123 »

porcoda wrote: post #91

Amusing qualifications for the senior engineering roles they’re hiring for: “Deep understanding of what is a computer, what is computer software, and how the two relate.” Right after the senior ML role that requires people understand how to write “algorithms and programs.” Kinda hard to take those kinds of requirements seriously.

> “Deep understanding of what is a computer, what is computer software, and how the two relate.”

Seems like a joke, but many developers do not really understand what's going on behind the scenes. This gets straight to the point. They don't care about HR keyword matching on your CV, or how many years of experience you have of being a mediocre developer with language X or framework Y. I guess during the interview they will investigate whether you truly understand the fundamentals.

Re: LM Studio – Discover, download, and run local LLMs

#102

Post by eshack94 »

reustle wrote: post #38

This looks great! If you're looking to do the same with open source code, you could likely run Ollama and a UI. https://github.com/jmorganca/ollama + https://github.com/ollama-webui/ollama-webui

Ollama is fantastic. I cannot recommend this project highly enough. Completely open-source, lightweight, and a great community.

Re: LM Studio – Discover, download, and run local LLMs

#103

Post by xcv123 »

wongarsu wrote: post #100
porcoda wrote: post #91

Amusing qualifications for the senior engineering roles they’re hiring for: “Deep understanding of what is a computer, what is computer software, and how the two relate.” Right after the senior ML role that requires people understand how to write “algorithms and programs.” Kinda hard to take those kinds of requirements seriously.

The second one isn't that bad in context. But the Senior Systems Software Engineer is wild, with "Deep understanding of what is a computer, what is computer software, and how the two relate" followed by "Experience writing and maintaining production code in C++14 or newer". You'd think the latter would imply the former, but maybe not... They seem to have even lowered expectations a bit. Two months ago [1] they were a…

> You'd think the latter would imply the former, but maybe not...

I wouldn't put C++ devs on too high of a pedestal. I got away with writing shitty C++ code for years before I really knew what I was doing. It still worked though.

Re: LM Studio – Discover, download, and run local LLMs

#105

Post by activatedgeek »

reustle wrote: post #38

This looks great! If you're looking to do the same with open source code, you could likely run Ollama and a UI. https://github.com/jmorganca/ollama + https://github.com/ollama-webui/ollama-webui

I looked at Ollama before, but couldn't quite figure something out from the docs [1]

It looks like a lot of the tooling is heavily engineered for a set of modern popular LLM-esque models. And looks like llama.cpp also supports LoRA models, so I'd assume there is a way to engineer a pipeline from LoRA to llama.cpp deployments, which probably covers quite a broad set of possibilities.

Beyond llama.cpp, can someone point me to what the broader community uses for general PyTorch model deployments?

I haven't quite ever self-hosted models, and am really keen to do one. Ideally, I am looking for something that stays close to the PyTorch core, and therefore allows me the flexibility to take any nn.Module to production.

[1]: https://github.com/jmorganca/ollama/blob/main/docs/import.md.

Re: LM Studio – Discover, download, and run local LLMs

#107

Post by rgbrgb »

0xEF wrote: post #79
pentagrama wrote: post #41

Curious about this and I just download it. Want to try uncensored models. I have a question, looking for the most popular "uncensored" model I just find "TheBloke/Luna-AI-Llama2-Uncensored-GGML", but it has 14 files to download between 2 to 7 GB, I just download the first one: https://imgur.com/a/DE2byOB I try the model and it works: https://imgur.com/a/2vtPcui I should download all the 14 files to get better results…

Honest question from someone new to exploring and using these models; why do you need uncensored? What are the use-cases that would call for it? Again, not questioning your motives or anything, just straight up curious. To use your example, any of us can find bomb building info online fairly easily, and has been a point of social contention since the Anarchist's cookbook. Nobody needs an uncensored LLM for that, of c…

perhaps they are in a war zone and must learn to make molotov cocktails without connecting to the internet

Re: LM Studio – Discover, download, and run local LLMs

#109

Post by ramoz »

cloudking wrote: post #60

Is anyone using open source models to actually get work done or solving problems in their software architecture? So far I haven't found anything near the quality of GPT-4.

The reality is there is no general use case for open source models, such as there is gpt-4.

The decent chat ones are based on gpt data and they’re basically shitty distilled models.

The best use case is a narrow one that you decide and can create adequate fine-tuning data around. Plenty of real production ability here.

Re: LM Studio – Discover, download, and run local LLMs

#110

Post by whartung »

I’m late to the game about this. So I’ll ask a stupid question.

As a contrived example, what happens if you feed the LoTR books, the Hobbit, the Silmarillion, and whatever else is germane, into an LLM?

Is there a base, empty, “ignorant” LLM that is used as a seed?

Do you end up with a Middle Earth savant?

Just how does all this work?

Post reply on HN