121–130 of 157 posts
Disappointing, no proper Linux support. Just "ask on discord".
Curious about this and I just download it. Want to try uncensored models. I have a question, looking for the most popular "uncensored" model I just find "TheBloke/Luna-AI-Llama2-Uncensored-GGML", but it has 14 files to download between 2 to 7 GB, I just download the first one: https://imgur.com/a/DE2byOB I try the model and it works: https://imgur.com/a/2vtPcui I should download all the 14 files to get better results…
Honest question from someone new to exploring and using these models; why do you need uncensored? What are the use-cases that would call for it? Again, not questioning your motives or anything, just straight up curious. To use your example, any of us can find bomb building info online fairly easily, and has been a point of social contention since the Anarchist's cookbook. Nobody needs an uncensored LLM for that, of c…
It doesn't make sense for me personally. It does make sense if you're offering an LLM publicly, so that you doesn't get bad PR if your LLM says some politically incorrect or questionable things.
I’m late to the game about this. So I’ll ask a stupid question. As a contrived example, what happens if you feed the LoTR books, the Hobbit, the Silmarillion, and whatever else is germane, into an LLM? Is there a base, empty, “ignorant” LLM that is used as a seed? Do you end up with a Middle Earth savant? Just how does all this work?
https://magazine.sebastianraschka.com/p/understanding-large-...
https://magazine.sebastianraschka.com/p/finetuning-large-lan...
Earlier quoted context omitted.
oobabooga is no longer popular?
oobabooga is the king. As far as I know, ollama doesn’t support exllama, qlora fine tuning, multi-GPU, etc. Text-generation-webui might seem like a science project, but it’s leagues ahead (Like 2-4x faster inference with the right plugins) of everything else. Also has a nice openai mock API that works great.
ooba's worth keeping an eye on, but koboldcpp is more stable, almost as versatile, and way less frustrating. It also still supports GGML.
I really like LM Studio and had it open when I came across this post. LM Studio is an interesting mixture of: - A local model runtime - A model catalog - A UI to chat with the models easily - An openAI compatible API And it has several plugins such as for RAG (using ChromaDB) and others. Personally I think the positioning is very interesting. They're well positioned to take advantage of new capabilities in the OS eco…
How does it compare with something like FastChat? https://github.com/lm-sys/FastChat Feature set seems like a decent amount of overlap. One limitation of FastChat, as far as I can tell, is that one is limited to the models that FastChat supports (though I think it would be minor to modify it to support arbitrary models?)
The other is some python thing that requires python knowledge of creating virtual environments, installing dependencies, etc...
Curious about this and I just download it. Want to try uncensored models. I have a question, looking for the most popular "uncensored" model I just find "TheBloke/Luna-AI-Llama2-Uncensored-GGML", but it has 14 files to download between 2 to 7 GB, I just download the first one: https://imgur.com/a/DE2byOB I try the model and it works: https://imgur.com/a/2vtPcui I should download all the 14 files to get better results…
Honest question from someone new to exploring and using these models; why do you need uncensored? What are the use-cases that would call for it? Again, not questioning your motives or anything, just straight up curious. To use your example, any of us can find bomb building info online fairly easily, and has been a point of social contention since the Anarchist's cookbook. Nobody needs an uncensored LLM for that, of c…
Earlier quoted context omitted.
Why don’t you wrap it with a verification system (eg: web scraper) and auto-regenerate / jailbreak any things you don’t like?
Because I pay $20/mo for GPT-4 and don't understand why anybody would run a "less-good" version locally that you can trust less/that has less functionality. That's why I wanted to try to understand, what am I missing about local-toy LLMs. How are they not just noise/nonsense generators?
They're bad at generative tasks. Don't have it write code or scientific papers from scratch, but you can have it review anything you've written. You can also do summaries, keyword/entity extraction, and the like safely. Any reductive task works pretty well.
LMStudio is great to run local LLMs, also support OpenAI-compatible API. In the case you need more advance UI/UX, you can use LMStudio with MindMac( https://mindmac.app ), just check this video for details https://www.youtube.com/watch?v=3KcVp5QQ1Ak .
This looks great! If you're looking to do the same with open source code, you could likely run Ollama and a UI. https://github.com/jmorganca/ollama + https://github.com/ollama-webui/ollama-webui
oobabooga is no longer popular?