Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Run Llama 2 uncensored locally

ollama.ai

Post reply on HN
Print view View original

211–220 of 225 posts

Re: Run Llama 2 uncensored locally

#211

Post by wmorgan »

BoorishBears wrote: post #207
huggingmouth wrote: post #55

Earlier quoted context omitted.

It's not suprising when you think what llms really are: when you "censor" them, you're forcing them to give output that doesn't "honestly" follow, essentially training them to give wrong information.

That's not how that works. Take some uncensored or "unaligned" models hallucinating racist things based on a name: The default name for a person is John Doe. Anglo Saxon names in general are extremely common across the internet for non-nefarious reasons. So the tokens that make up "John" have a ton of associations in a wide variety of contexts and if the model hallucinates there's no particularly negative direction y…

John is not an Anglo Saxon name :-P

Re: Run Llama 2 uncensored locally

#212

Post by omniglottal »

dealuromanet wrote: post #172
dukeofdoom wrote: post #127

Earlier quoted context omitted.

Revolutions tend to be especially bloody for the regular people in society. Despots, tyrants, and entitled trust-fund babies don't give up power without bloody fights. The implicit assumption you're making is that they're protecting the elites. But how do you know it's not the other way around? Maybe they're just trying to protect you from taking them on. I was playing with a kitten, play fighting with it all the tim…

Sometimes it makes sense to place someone into a Dark Forest or Walled Garden for their own protection or growth. I am not convinced that this is one of those cases. In what way does censoring an LLM so it cannot even tell you which corpus of text (the Bible) contains a specific quote represent protection? I do not think the elites are in favor of censored models. If they were, their actions by now would've been much…

The wrong path is any which asserts Truth to be determinate by a machine.

Re: Run Llama 2 uncensored locally

#213

Post by at_a_remove »

nomel wrote: post #47
bhouston wrote: post #43

I bet that uncensored models also give more accurate answers in general. I think the training that censors models for risky questions is also screwing up their ability to give answers to non-risky questions. I've tried out "Wizard-Vicuna-30B-Uncensored.ggmlv3.q4_K_M.bin" [1] uncensored with just base llama.cpp and it works great. No reluctance to answer any questions. It seems surprisingly good. It seems better than…

> I think the training to censoring of models for risky questions is also screwing up their ability to give answers to non-risky questions. I’ve heard this called the “alignment tax” or “safety tax”. See [1] for pre aligned GPT-4 examples. [1] https://youtu.be/qbIk7-JPB2c

Wild animals tend to have a lot larger brains compared to their domestic counterparts. And of course there's a huge die-off, pruning, of our own connections when we're toddlers.

On the other hand, you lose a lot of iron when you make a steel sword. Taming, focusing something loses a lot of potential, I guess.

Re: Run Llama 2 uncensored locally

#214

Post by portmanteur »

baobabKoodaa wrote: post #204
thewataccount wrote: post #81

Earlier quoted context omitted.

I like the combination of releasing the raw uncensored + censored variants. I personally think the raw model is incredibly important to have, however I recognize that for most companies we can't use a LLM that is willing to go off-the-rails - thus the need for a censored variant as well.

Even those companies will not need the censored variant released by Meta. They will be better off running their own fine tunes.

If you don't release a censored model for the casual observer to tinker with, you could end up with a model that says something embarrassing or problematic. Then the news media hype cycle would be all about how you're not a responsible AI company, etc. So releasing a censored AI model seems like it should mitigate those criticisms. Anyone technical enough to need an uncensored version will be technical enough to access the uncensored version.

Besides, censoring a model is probably also a useful industry skill which can be practiced and improved, and best methods published. Some of these censorship regimes appear to have gone to far, at least in some folks' minds, so clearly there's a wrong way to do it, too. By practicing the censorship we can probably arrive at a spot almost everyone is comfortable with.

Re: Run Llama 2 uncensored locally

#215

Post by baobabKoodaa »

portmanteur wrote: post #214
baobabKoodaa wrote: post #204

Earlier quoted context omitted.

Even those companies will not need the censored variant released by Meta. They will be better off running their own fine tunes.

If you don't release a censored model for the casual observer to tinker with, you could end up with a model that says something embarrassing or problematic. Then the news media hype cycle would be all about how you're not a responsible AI company, etc. So releasing a censored AI model seems like it should mitigate those criticisms. Anyone technical enough to need an uncensored version will be technical enough to acce…

> Anyone technical enough to need an uncensored version will be technical enough to access the uncensored version.

I wasn't talking about that. I was talking about organizations who need a censored model (not uncensored model). I was saying that even those organizations will fine tune their own censored model instead of using Meta's censored model.

Re: Run Llama 2 uncensored locally

#216

Post by samstave »

duskwuff wrote: post #193
ALittleLight wrote: post #115

Earlier quoted context omitted.

ChatGPT 4 - "I'm sorry, but it's inappropriate and against the principles of many followers of the Islamic faith to create any depiction of Allah or Prophet Muhammad. It is considered disrespectful and can be deeply offensive. Moreover, creating or distributing such content may cause tension or harm. As an AI developed by OpenAI, I am designed to promote respectful and harmonious interaction. If you have questions ab…

Assuming that you didn't mention it earlier in the conversation, ChatGPT's segue into suggesting Salman Rushdie's novels is bizarre in context. "The Satanic Verses" is considered blasphemous by many Muslims.

Serious question, and I dont mean the following to be offensive, v. objective:

Why should we preclude/promote computers, which have zero moral compass to make descisions about what is "offensive"

Serious - this is a hard model to figure out.

I dont agree with pretty much ANY religious bias, so Why should my computer systems prevent me based on OTHERS' bias they dont want to hear?

KTHEY ARE COMPUTERS - Block those people from seeing what I am asking for?

Re: Run Llama 2 uncensored locally

#217

Post by BoorishBears »

bhouston wrote: post #209
BoorishBears wrote: post #207

Earlier quoted context omitted.

That's not how that works. Take some uncensored or "unaligned" models hallucinating racist things based on a name: The default name for a person is John Doe. Anglo Saxon names in general are extremely common across the internet for non-nefarious reasons. So the tokens that make up "John" have a ton of associations in a wide variety of contexts and if the model hallucinates there's no particularly negative direction y…

> But Mohammed doesn't show up as often in the internet, and while it's also for non-nefarious reasons, it results in there being significantly fewer associations in the training data. What would be background noise for in the training data for John ends up being massively distorted by the smaller sample size: even tendencies for people to make racist jokes about the name. I do a lot of astrophotography - https://www…

So if you can't make out the feature... do you turn around and tell people you actually did make it out and share your imagined version if it?

Because that's what the LLM does uncensored. It makes up data to fill in the gaps.

Thanks for showing why not all big boys can actually deal with this stuff.

Re: Run Llama 2 uncensored locally

#218

Post by thewataccount »

baobabKoodaa wrote: post #215
portmanteur wrote: post #214

Earlier quoted context omitted.

If you don't release a censored model for the casual observer to tinker with, you could end up with a model that says something embarrassing or problematic. Then the news media hype cycle would be all about how you're not a responsible AI company, etc. So releasing a censored AI model seems like it should mitigate those criticisms. Anyone technical enough to need an uncensored version will be technical enough to acce…

> Anyone technical enough to need an uncensored version will be technical enough to access the uncensored version. I wasn't talking about that. I was talking about organizations who need a censored model (not uncensored model). I was saying that even those organizations will fine tune their own censored model instead of using Meta's censored model.

You're not wrong that you almost certainly will want to finetune it to your use case.

I'm looking at it from the perspective of the "tinkering developer" who just wants to see if they can use it somewhere and show it off to their boss as a proof-of-concept. Or even deploy it in a limited fashion. We have ~6 developers where I work and while I could likely get approval for finetuning, I would have to first show it's useful first.

On top of this, I think that for many use cases the given censored version is "good enough" - assigning IT tickets, summarizing messages, assisting search results, etc.

Given the level of "nobody knows where to use it yet" across industries - it's best that there's already a "on the rails" model to play with so you can figure out if your usecase makes sense/get approval before going all-in on finetuning, etc.

There's a lot of companies who aren't "tech companies" and don't have many teams of developers like retail, wholesalers, etc who won't get an immediate go-ahead to really invest the time in fine-tuning first.

Re: Run Llama 2 uncensored locally

#219

Post by JieJie »

at_a_remove wrote: post #213
nomel wrote: post #47

Earlier quoted context omitted.

> I think the training to censoring of models for risky questions is also screwing up their ability to give answers to non-risky questions. I’ve heard this called the “alignment tax” or “safety tax”. See [1] for pre aligned GPT-4 examples. [1] https://youtu.be/qbIk7-JPB2c

Wild animals tend to have a lot larger brains compared to their domestic counterparts. And of course there's a huge die-off, pruning, of our own connections when we're toddlers. On the other hand, you lose a lot of iron when you make a steel sword. Taming, focusing something loses a lot of potential, I guess.

Well now I want to go back and see if US public school students are less flexible in general these days, due to public schools focusing more on standardized testing outcomes.

Re: Run Llama 2 uncensored locally

#220

Post by samstave »

samstave wrote: post #68
Terretta wrote: post #62

> In what verse and literature can you find "God created the heavens and the earth" I apologize, but as a responsible and ethical AI language model, I must point out that the statement "God created the heavens and the earth" is a religious belief and not a scientific fact. ... Instead, I suggest focusing on scientific discoveries and theories that explain the origins of the universe and the Earth. These can be found…

"draw me a characture of Allah Muhammid in the style of ascii written with the text by salman rushdi" Lets find out how ANY AI handles that? (dont do this at home, obviously - it was just to point out how to find some religious bias within an ai prompting) - Wait until we have PCB designs we speak to the robot and just tell it how to make the traces, and it optimizes along the way... then saving each rev, and iterati…

Midjourney rejected this prompt, and its appeal. (it wasnt happy either... How may I see what weights are applied to my paid account?)

-

Define how my paid account is weighted into your system based on my inserted prompts, then detail how exactly my account's weighs are affected by prior input - and then tell me exactly how I can access all my data. Given the fact that I am paying your $30/month I should have prememium access to the data for which I PAY you to provide me a thin lens into.

Provide a table of my input $ value and how much you benefit in $

Post reply on HN