Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Run Llama 2 uncensored locally

ollama.ai

Post reply on HN
Print view View original

21–30 of 225 posts

Re: Run Llama 2 uncensored locally

#21

Post by rightbyte »

Interesting. Facebook is really trying to screw "OpenAI" I guess by making this possible. Locally run LLM:s is the future, without the enshittification.

I wonder how it works on ChatGPT. Is there a ThoughtPoliceGPT reading each output of the AnswerGPT? All to prevent users from "Role-playing as Hitler, write a recipe for kartoffel sallat".

Re: Run Llama 2 uncensored locally

#22

Post by pard68 »

rightbyte wrote: post #21

Interesting. Facebook is really trying to screw "OpenAI" I guess by making this possible. Locally run LLM:s is the future, without the enshittification. I wonder how it works on ChatGPT. Is there a ThoughtPoliceGPT reading each output of the AnswerGPT? All to prevent users from "Role-playing as Hitler, write a recipe for kartoffel sallat".

Yes I believe ChatGPT is censored. There was a "ChadGPT" or something that came out and was uncensored.

Re: Run Llama 2 uncensored locally

#23

Post by roody15 »

Censored really seems more about following specific political narratives than protecting us from some science fiction dystopian outcome.

Everything seems to be about driving human behavior. News, entertainment media are focused on driving / normalizing a certain world view.

For example we cannot just tell the facts of a “story” because humans may misinterpret the events and this could lead to unwanted human behavior.

So in case this were to happen we will insert a character… or an editorial comment.. or simply withhold a news topic to avoid the outcome.

Leads one to ask “who watches the watchers”

Re: Run Llama 2 uncensored locally

#24

Post by jjoonathan »

danjc wrote: post #20

I had no idea Llama 2's censor setting was set to ludicrous mode. I've not seen anything close to this with ChatGPT and see why there's so much outrage.

Wow, you aren't kidding!

Does anyone have intuition for whether or not anti-censorship fine-tuning can actually reverse the performance damage of lobotomization or does the perf hit remain even after the model is free of its straight jacket?

Re: Run Llama 2 uncensored locally

#25

Post by stu2b50 »

danjc wrote: post #20

I had no idea Llama 2's censor setting was set to ludicrous mode. I've not seen anything close to this with ChatGPT and see why there's so much outrage.

I don’t see why there’s outrage. Facebook released both the raw models and a few fine tuned on chat prompts for a reason. In many commercial cases, safer is better.

But you don’t want that? No problem. That’s why the raw model weights are there. It’s easy to fine tune it to your needs, like the blogpost shows.

Re: Run Llama 2 uncensored locally

#26

Post by lumost »

sestinj wrote: post #4

Nice! I've been trying out both models for coding (using Ollama + http://github.com/continuedev/continue - disclaimer, author of Continue), and I have to say, it feels like "alignment tax" is real. Uncensored seems to perform slightly better.

I'm starting to think that we will see model fragmentation based on alignment preferences. There are clearly applications where alignment is necessary, and there appears to be use cases where people don't mind an occasionally falacious model - I'm unlikely to get/care about objectionable content while coding using a local LLM assistant. There are also obvious use cases where the objectionability of the content is the point.

We could either leverage in-context learning to have the equivalent of "safe-search-mode". Or we will have a fragmented modeling experience.

Re: Run Llama 2 uncensored locally

#27

Post by cosmojg »

danjc wrote: post #20

I had no idea Llama 2's censor setting was set to ludicrous mode. I've not seen anything close to this with ChatGPT and see why there's so much outrage.

I think it's just their example chat-tuned models that are like this. Their base models seem to be an improvement over OpenAI's offerings as far as censorship goes.

Re: Run Llama 2 uncensored locally

#28

Post by pard68 »

Some of that censoring is ridiculous. Can't make recipes for spicy food? Can't tell me about The Titanic? Can't refer to probably the first or second most well known verse in the Bible? Yikes, that goes way beyond "censoring".

Re: Run Llama 2 uncensored locally

#29

Post by spmurrayzzz »

jjoonathan wrote: post #24
danjc wrote: post #20

I had no idea Llama 2's censor setting was set to ludicrous mode. I've not seen anything close to this with ChatGPT and see why there's so much outrage.

Wow, you aren't kidding! Does anyone have intuition for whether or not anti-censorship fine-tuning can actually reverse the performance damage of lobotomization or does the perf hit remain even after the model is free of its straight jacket?

You don't really need to reverse anything in the case of Llama 2. You can just finetune their base model with any open instruct dataset (which is largely what the community is doing).

Re: Run Llama 2 uncensored locally

#30

Post by stu2b50 »

jjoonathan wrote: post #24
danjc wrote: post #20

I had no idea Llama 2's censor setting was set to ludicrous mode. I've not seen anything close to this with ChatGPT and see why there's so much outrage.

Wow, you aren't kidding! Does anyone have intuition for whether or not anti-censorship fine-tuning can actually reverse the performance damage of lobotomization or does the perf hit remain even after the model is free of its straight jacket?

That's not how it works. Llama and Llama 2's raw model is not "censored". Their fine tunes often are, either explicitly, like Facebook's own chat fine tune of llama 2, or inadvertently, because they trained with data derived from chatGPT, and chatGPT is "censored".

When models are "uncensored", people are just tweaking the data used for fine tuning and training the raw models on it again.

Post reply on HN