Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Run Llama 2 uncensored locally

ollama.ai

Post reply on HN
Print view View original

51–60 of 225 posts

Re: Run Llama 2 uncensored locally

#51

Post by PeterStuer »

bhouston wrote: post #43

I bet that uncensored models also give more accurate answers in general. I think the training that censors models for risky questions is also screwing up their ability to give answers to non-risky questions. I've tried out "Wizard-Vicuna-30B-Uncensored.ggmlv3.q4_K_M.bin" [1] uncensored with just base llama.cpp and it works great. No reluctance to answer any questions. It seems surprisingly good. It seems better than…

In my experience it goes both ways. Yes, you will run less into the "I'm not going to answer that". Otoh, you will also have more giberish selected out of the possible palette of answers.

Personally, I trend towards 'uncensored' but I'm not denying it's not without it's drawbacks either.

Re: Run Llama 2 uncensored locally

#53

Post by unparagoned »

bhouston wrote: post #43

I bet that uncensored models also give more accurate answers in general. I think the training that censors models for risky questions is also screwing up their ability to give answers to non-risky questions. I've tried out "Wizard-Vicuna-30B-Uncensored.ggmlv3.q4_K_M.bin" [1] uncensored with just base llama.cpp and it works great. No reluctance to answer any questions. It seems surprisingly good. It seems better than…

But you are going to have to specify your question in way more detail to get a good response. If you just ask it a question you are going to get some crappy responses that don’t even attempt to answer your question.

Re: Run Llama 2 uncensored locally

#54

Post by bhouston »

PeterStuer wrote: post #51
bhouston wrote: post #43

I bet that uncensored models also give more accurate answers in general. I think the training that censors models for risky questions is also screwing up their ability to give answers to non-risky questions. I've tried out "Wizard-Vicuna-30B-Uncensored.ggmlv3.q4_K_M.bin" [1] uncensored with just base llama.cpp and it works great. No reluctance to answer any questions. It seems surprisingly good. It seems better than…

In my experience it goes both ways. Yes, you will run less into the "I'm not going to answer that". Otoh, you will also have more giberish selected out of the possible palette of answers. Personally, I trend towards 'uncensored' but I'm not denying it's not without it's drawbacks either.

> Otoh, you will also have more giberish selected out of the possible palette of answers.

I have not noticed that at all. I've never seen it give gibberish. Censored or uncensored, there is limits to the model and it will make things up as it hits them, but it isn't gibberish.

Re: Run Llama 2 uncensored locally

#55

Post by huggingmouth »

nomel wrote: post #47
bhouston wrote: post #43

I bet that uncensored models also give more accurate answers in general. I think the training that censors models for risky questions is also screwing up their ability to give answers to non-risky questions. I've tried out "Wizard-Vicuna-30B-Uncensored.ggmlv3.q4_K_M.bin" [1] uncensored with just base llama.cpp and it works great. No reluctance to answer any questions. It seems surprisingly good. It seems better than…

> I think the training to censoring of models for risky questions is also screwing up their ability to give answers to non-risky questions. I’ve heard this called the “alignment tax” or “safety tax”. See [1] for pre aligned GPT-4 examples. [1] https://youtu.be/qbIk7-JPB2c

It's not suprising when you think what llms really are: when you "censor" them, you're forcing them to give output that doesn't "honestly" follow, essentially training them to give wrong information.

Re: Run Llama 2 uncensored locally

#57

Post by bilsbie »

stu2b50 wrote: post #25
danjc wrote: post #20

I had no idea Llama 2's censor setting was set to ludicrous mode. I've not seen anything close to this with ChatGPT and see why there's so much outrage.

I don’t see why there’s outrage. Facebook released both the raw models and a few fine tuned on chat prompts for a reason. In many commercial cases, safer is better. But you don’t want that? No problem. That’s why the raw model weights are there. It’s easy to fine tune it to your needs, like the blogpost shows.

Aren’t the raw model weights after RFHF?

Re: Run Llama 2 uncensored locally

#58

Post by stu2b50 »

bilsbie wrote: post #57
stu2b50 wrote: post #25

Earlier quoted context omitted.

I don’t see why there’s outrage. Facebook released both the raw models and a few fine tuned on chat prompts for a reason. In many commercial cases, safer is better. But you don’t want that? No problem. That’s why the raw model weights are there. It’s easy to fine tune it to your needs, like the blogpost shows.

Aren’t the raw model weights after RFHF?

Nope. Raw models are purely trained on their corpus of text in an autoregressive manor. No chat fine tuning or rlhf.

Re: Run Llama 2 uncensored locally

#60

Post by bhouston »

unparagoned wrote: post #53
bhouston wrote: post #43

I bet that uncensored models also give more accurate answers in general. I think the training that censors models for risky questions is also screwing up their ability to give answers to non-risky questions. I've tried out "Wizard-Vicuna-30B-Uncensored.ggmlv3.q4_K_M.bin" [1] uncensored with just base llama.cpp and it works great. No reluctance to answer any questions. It seems surprisingly good. It seems better than…

But you are going to have to specify your question in way more detail to get a good response. If you just ask it a question you are going to get some crappy responses that don’t even attempt to answer your question.

I am using the Wizard + Vicuna trained Llama model. I believe this makes a huge difference even if it was censored.
Post reply on HN