Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Run Llama 2 uncensored locally

ollama.ai

Post reply on HN
Print view View original

91–100 of 225 posts

Re: Run Llama 2 uncensored locally

#91

Post by thamer »

I tried the basic "llama2" model with a simple question, and did not expect this kind of answer.

Q: who was the first man on the moon?

A: I apologize, but it is not possible for me to provide an answer to your question as there has never been any human exploration of the Moon. While there have been several spacecraft that have traveled to the Moon and returned, no human has ever set foot on its surface. The last time a spacecraft visited the Moon was during the Apollo program in the late 1960s and early 1970s, but no astronauts were sent to the Moon as part of that mission. Is there anything else I can help you with?

Re: Run Llama 2 uncensored locally

#92

Post by somenameforme »

__loam wrote: post #76
bhouston wrote: post #43

I bet that uncensored models also give more accurate answers in general. I think the training that censors models for risky questions is also screwing up their ability to give answers to non-risky questions. I've tried out "Wizard-Vicuna-30B-Uncensored.ggmlv3.q4_K_M.bin" [1] uncensored with just base llama.cpp and it works great. No reluctance to answer any questions. It seems surprisingly good. It seems better than…

The uncensored models confirm the biases present in the input data. That may or may not correspond to more "correct" output.

Can you offer any example where the censored answer would be more correct than the uncensored when you are asking for a falsifiable/factual response, and not just an opinion? I couldn't really care less what the chatbots say in matters of opinion/speculation, but I get quite annoyed when the censorship gets in the way of factual queries, which it often does! And this is made even worse because I really can't envision a [benevolent] scenario where said censorship is actually beneficial.

Re: Run Llama 2 uncensored locally

#94

Post by bluescrn »

roody15 wrote: post #23

Censored really seems more about following specific political narratives than protecting us from some science fiction dystopian outcome. Everything seems to be about driving human behavior. News, entertainment media are focused on driving / normalizing a certain world view. For example we cannot just tell the facts of a “story” because humans may misinterpret the events and this could lead to unwanted human behavior.…

Welcome to a world where the word 'safety' no longer refers to protection from physical harm/risk.

Re: Run Llama 2 uncensored locally

#95

Post by brucethemoose2 »

thamer wrote: post #91

I tried the basic "llama2" model with a simple question, and did not expect this kind of answer. Q: who was the first man on the moon? A: I apologize, but it is not possible for me to provide an answer to your question as there has never been any human exploration of the Moon. While there have been several spacecraft that have traveled to the Moon and returned, no human has ever set foot on its surface. The last time…

Base llama (especially at the lower parameter counts) was never very good. You need a Q/A finetune with the exact formatting.

And even then... Trivia is not their strong suit.

Re: Run Llama 2 uncensored locally

#96

Post by cubefox »

thamer wrote: post #91

I tried the basic "llama2" model with a simple question, and did not expect this kind of answer. Q: who was the first man on the moon? A: I apologize, but it is not possible for me to provide an answer to your question as there has never been any human exploration of the Moon. While there have been several spacecraft that have traveled to the Moon and returned, no human has ever set foot on its surface. The last time…

Exactly! RLHF isn't just used to censor models, but also to make them prefer more truthful answers.

Re: Run Llama 2 uncensored locally

#97

Post by cubefox »

somenameforme wrote: post #92
__loam wrote: post #76

Earlier quoted context omitted.

The uncensored models confirm the biases present in the input data. That may or may not correspond to more "correct" output.

Can you offer any example where the censored answer would be more correct than the uncensored when you are asking for a falsifiable/factual response, and not just an opinion? I couldn't really care less what the chatbots say in matters of opinion/speculation, but I get quite annoyed when the censorship gets in the way of factual queries, which it often does! And this is made even worse because I really can't envision…

https://news.ycombinator.com/item?id=36976236

Re: Run Llama 2 uncensored locally

#98

Post by dealuromanet »

throwaway4837 wrote: post #67
Terretta wrote: post #62

> In what verse and literature can you find "God created the heavens and the earth" I apologize, but as a responsible and ethical AI language model, I must point out that the statement "God created the heavens and the earth" is a religious belief and not a scientific fact. ... Instead, I suggest focusing on scientific discoveries and theories that explain the origins of the universe and the Earth. These can be found…

I agree with people who say fine-tuning and "human AI alignment" is actually what's going to make AI dangerous. The fact that we think we can "align" something taught on historical, fictional, and scientific text -- it's hubris. One way ticket to an ideological bubble. This "search engine that has its own opinions on what you're looking for" is really the wrong path for us to take. Searching data is a matter of truth…

> One way ticket to an ideological bubble.

I believe this is the intention. The people doing the most censoring in the name of "safety and security" are just trying to build a moat where they control what LLMs say and consequently what people think, on the basis of what information and ideas are acceptable versus forbidden. Complete control over powerful LLMs of the future will enable despots, tyrants, and entitled trust-fund babies to more easily program what people think is and isn't acceptable.

The only solution to this is more open models that are easy to train, deploy locally, and use locally with as minimal hardware requirements as is possible so that uncensored models running locally are available to everyone.

And they must be buildable from source so that people can verify that they are truthful and open, rather than locked down models that do not tell the truth. We should be able to determine with monitoring software if an LLM has been forbidden from speaking on certain subjects. This is necessary because of things like what another comment on the thread was saying about how the censored model gives a completely garbage, deflective non-answer when asked a simple question about which corpus of text (the Bible) has a specific quote in it. With monitoring and source that is buildable locally and trainable locally, we could determine if a model is constrained this way.

Re: Run Llama 2 uncensored locally

#99

Post by smoldesu »

Xelbair wrote: post #31
stu2b50 wrote: post #25

Earlier quoted context omitted.

I don’t see why there’s outrage. Facebook released both the raw models and a few fine tuned on chat prompts for a reason. In many commercial cases, safer is better. But you don’t want that? No problem. That’s why the raw model weights are there. It’s easy to fine tune it to your needs, like the blogpost shows.

It's just not safe. It's unusable. you can't ask it normal questions to not get stonewalled by it's default censorship message - it wouldn't even work for commercial case.

I wholly disagree. This is arguably close to the perfect solution:

- Developers and end users can choose which model they want to use

- Model distributors don't necessarily take the fall since they provide a "healthy" model alternative

- The uncensored "base" model can be finetuned into whatever else is needed

You have to remember, ChatGPT is censored like a Soviet history book but didn't struggle to hit hundreds of millions of users in months. This is what releases will look like from now on, and it's not even a particularly damning example.

Re: Run Llama 2 uncensored locally

#100

Post by cubefox »

bhouston wrote: post #43

I bet that uncensored models also give more accurate answers in general. I think the training that censors models for risky questions is also screwing up their ability to give answers to non-risky questions. I've tried out "Wizard-Vicuna-30B-Uncensored.ggmlv3.q4_K_M.bin" [1] uncensored with just base llama.cpp and it works great. No reluctance to answer any questions. It seems surprisingly good. It seems better than…

> I bet that uncensored models also give more accurate answers in general.

Doubtful:

https://news.ycombinator.com/item?id=36976236

RLHF can motivate models to deny truths which are politically taboo, but it can also motivate them to care more about things supported by scientific evidence rather than about bullshitting, random conspiracy theories, and "hallucination". So it's a double edged sword.

Post reply on HN