Run Llama 2 uncensored locally • Hacker News Board

Re: Run Llama 2 uncensored locally

#221

Post by jokethrowaway » Thu, Aug 03, 2023, 9:26 PM UTC

joshstrange wrote: post #149

joebiden2 wrote: post #129

Ollama forks llama.cpp. The value-add is marginal. Still I see no attribution on https://ollama.ai/ . Please instead of downvoting, see if this is fine from your point of view. No affiliation at all, I just don't like this kind of marketing. See also https://news.ycombinator.com/item?id=36806448

It would be nice to add some attribution but llama.cpp is MIT licensed so what Ollama is doing is perfectly acceptable. Also, Ollama is open source (also MIT). You can bet any for-profit people using llama.cpp under the hood aren't going to mention it, and while I think we should hold open source projects to a slightly higher standard this isn't really beyond the pale for me. While you find the value-add to be "margi…

Running make vs go build? I don't see much difference

I personally settled on the text-generation-webui

Re: Run Llama 2 uncensored locally

#222

Post by sundarurfriend » Fri, Aug 04, 2023, 12:04 AM UTC

jbellis wrote: post #176

jarenmf wrote: post #10

Which graphics card would you recommend to run Llamma 2 locally? I'm about to buy a laptop and considering choosing a model with a good Nvidia GPU.

If you insist on running models locally on a laptop then a Macbook with as much unified ram as you can afford is the only way to get decent amounts of vram. But you'll save a ton of money (and time from using more capable hardware) if you treat the laptop as a terminal and either buy a desktop or use cloud hardware to run the models.

Cloud hardware like? (Is Google Colab the best option, or even one of the best? Is Paperspace Gradient any good? Others?)

Re: Run Llama 2 uncensored locally

#223

Post by Tagbert » Fri, Aug 04, 2023, 12:38 AM UTC

1123581321 wrote: post #199

Tagbert wrote: post #198

Earlier quoted context omitted.

Yes, that occurred to me just after posting and and I immediately removed my question. Sorry you saw it before my edit. Very quick response on your part. :)

Sorry. :) I should’ve thought to delete instead of edit.

normally I would annotate the edit, but I thought that it was fast enough to skip that step. alas.

Re: Run Llama 2 uncensored locally

#224

Post by stathibus » Fri, Aug 04, 2023, 5:56 PM UTC

SV_BubbleTime wrote: post #133

stathibus wrote: post #77

Earlier quoted context omitted.

The mapping from latent space to the low-dimension embarassing/correct/offensive continuum is extremely complex.

Maybe we could make it a lot easier, just by going back to the idea that if you are offended, that a you problem. Not that we had a perfect time for this ever, but it’s never been worse than it is now.

classic neckbeard take

Re: Run Llama 2 uncensored locally

#225

Post by joebiden2 » Wed, Aug 09, 2023, 8:45 PM UTC

dang wrote: post #148

joebiden2 wrote: post #135

This post seems to be upvoted for the "uncensored" keyword. But this should be attributed to https://huggingface.co/ehartford and others. See also https://news.ycombinator.com/item?id=36977146 Or better: https://erichartford.com/uncensored-models

The latter link had a major thread at the time: Uncensored Models - https://news.ycombinator.com/item?id=35946060 - May 2023 (379 comments) We definitely want HN to credit the original sources and (even more so) researchers but I'm not sure what the best move here would be, or whether we need to change anything.

Thanks, didn't notice this discussion.