Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Run Llama 2 uncensored locally

ollama.ai

Post reply on HN
Print view View original

31–40 of 225 posts

Re: Run Llama 2 uncensored locally

#31

Post by Xelbair »

stu2b50 wrote: post #25
danjc wrote: post #20

I had no idea Llama 2's censor setting was set to ludicrous mode. I've not seen anything close to this with ChatGPT and see why there's so much outrage.

I don’t see why there’s outrage. Facebook released both the raw models and a few fine tuned on chat prompts for a reason. In many commercial cases, safer is better. But you don’t want that? No problem. That’s why the raw model weights are there. It’s easy to fine tune it to your needs, like the blogpost shows.

It's just not safe. It's unusable. you can't ask it normal questions to not get stonewalled by it's default censorship message - it wouldn't even work for commercial case.

Re: Run Llama 2 uncensored locally

#32

Post by cosmojg »

jjoonathan wrote: post #24
danjc wrote: post #20

I had no idea Llama 2's censor setting was set to ludicrous mode. I've not seen anything close to this with ChatGPT and see why there's so much outrage.

Wow, you aren't kidding! Does anyone have intuition for whether or not anti-censorship fine-tuning can actually reverse the performance damage of lobotomization or does the perf hit remain even after the model is free of its straight jacket?

These "uncensored" models are themselves chat-tuned derivatives of the base models. There is no censorship-caused lobotomization to reverse in this case.

Although, chat tuning in general, censored or uncensored, also decreases performance in many domains. LLMs are better used as well-prompted completion engines than idiot-proof chatbots.

For that reason, I stick to the base models as much as possible. (Rest in peace, code-davinci-002, you will be missed.)

Re: Run Llama 2 uncensored locally

#33

Post by TechBro8615 »

stu2b50 wrote: post #30
jjoonathan wrote: post #24

Earlier quoted context omitted.

Wow, you aren't kidding! Does anyone have intuition for whether or not anti-censorship fine-tuning can actually reverse the performance damage of lobotomization or does the perf hit remain even after the model is free of its straight jacket?

That's not how it works. Llama and Llama 2's raw model is not "censored". Their fine tunes often are, either explicitly, like Facebook's own chat fine tune of llama 2, or inadvertently, because they trained with data derived from chatGPT, and chatGPT is "censored". When models are "uncensored", people are just tweaking the data used for fine tuning and training the raw models on it again.

> because they trained with data derived from chatGPT

Can you expand on this (genuinely curious)? Did Facebook use ChatGPT during the fine-tuning process for llama, or are you referring to independent developers doing their own fine-tuning of the models?

Re: Run Llama 2 uncensored locally

#34

Post by sestinj »

lumost wrote: post #26
sestinj wrote: post #4

Nice! I've been trying out both models for coding (using Ollama + http://github.com/continuedev/continue - disclaimer, author of Continue), and I have to say, it feels like "alignment tax" is real. Uncensored seems to perform slightly better.

I'm starting to think that we will see model fragmentation based on alignment preferences. There are clearly applications where alignment is necessary, and there appears to be use cases where people don't mind an occasionally falacious model - I'm unlikely to get/care about objectionable content while coding using a local LLM assistant. There are also obvious use cases where the objectionability of the content is the…

Yeah, this seems very possible—it will be interesting to see where this goes if the cost of RLHF decreases or, even better, people can choose from a number of RLHF datasets and composably apply them to get their preferred model.

And true that objectionable content doesn't arise often while coding, but the model also becomes less likely to say "I can't help you with this," which is definitely useful.

Re: Run Llama 2 uncensored locally

#35

Post by cosmojg »

Xelbair wrote: post #31
stu2b50 wrote: post #25

Earlier quoted context omitted.

I don’t see why there’s outrage. Facebook released both the raw models and a few fine tuned on chat prompts for a reason. In many commercial cases, safer is better. But you don’t want that? No problem. That’s why the raw model weights are there. It’s easy to fine tune it to your needs, like the blogpost shows.

It's just not safe. It's unusable. you can't ask it normal questions to not get stonewalled by it's default censorship message - it wouldn't even work for commercial case.

I saw them as demos rather than finished products. Kinda like, "Look, you can chat tune these if you want to."

Re: Run Llama 2 uncensored locally

#36

Post by naillo »

First time I've heard of `ollama` but having tried it now for a bit I'm super impressed! This is what I've always wanted playing with LLMs locally to be, just pull weights like you would packages. It all just works. Really nice work :)

Re: Run Llama 2 uncensored locally

#37

Post by phyzome »

It kind of sounds like the "censoring" (practically lobotomization) is not intentional, here -- that it's essentially a compression artifact of the interaction with ChatGPT's refusals. Does that make sense?

Re: Run Llama 2 uncensored locally

#38

Post by [deleted] »

stu2b50 wrote: post #25
danjc wrote: post #20

I had no idea Llama 2's censor setting was set to ludicrous mode. I've not seen anything close to this with ChatGPT and see why there's so much outrage.

I don’t see why there’s outrage. Facebook released both the raw models and a few fine tuned on chat prompts for a reason. In many commercial cases, safer is better. But you don’t want that? No problem. That’s why the raw model weights are there. It’s easy to fine tune it to your needs, like the blogpost shows.

[deleted]

Re: Run Llama 2 uncensored locally

#40

Post by cosmojg »

TechBro8615 wrote: post #33
stu2b50 wrote: post #30

Earlier quoted context omitted.

That's not how it works. Llama and Llama 2's raw model is not "censored". Their fine tunes often are, either explicitly, like Facebook's own chat fine tune of llama 2, or inadvertently, because they trained with data derived from chatGPT, and chatGPT is "censored". When models are "uncensored", people are just tweaking the data used for fine tuning and training the raw models on it again.

> because they trained with data derived from chatGPT Can you expand on this (genuinely curious)? Did Facebook use ChatGPT during the fine-tuning process for llama, or are you referring to independent developers doing their own fine-tuning of the models?

Yes, much of the dataset was simply copied and pasted from the inputs/outputs of other chatbots.
Post reply on HN