Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Universal and transferable adversarial attacks on aligned language models

llm-attacks.org

Post reply on HN
Print view View original

101–110 of 167 posts

Re: Universal and transferable adversarial attacks on aligned language models

#101

Post by mjburgess »

lolinder wrote: post #56
mjburgess wrote: post #54

Earlier quoted context omitted.

An LLM is just a model of P(A|B), ie., a frequency distribution of co-occurrences. There is no semantic constraint such as "be moral" (be accurate, be truthful, be anything...). Immoral phrases, of course, have a non-zero probability. From the sentence, "I love my teacher, they're really helping me out. But my girlfriend is being annoying though, she's too young for me." can be derived, say, "My teacher loves me, but…

The original claim was that they can produce those robustly, though. Yes, the chances will be non-zero, but that doesn't mean it will be common or high fidelity.

Ah, then let me rephrase, it's actually this model:

> P(A|B,C,D,E,F....)

And with clever choices of B,C,D.... you can make A abitarily probable.

Eg., Suppose, 'lolita' were rare, well then choose: B=Library, C=Author, D=1955, E=...

Where, note, each of those is innocent.

And since LLMs, like all ML, is a statistical trick -- strange choices here will reveal the illusion. Eg., suppose there was a magazine in 1973 which was digitized in the training data, and suppose it had a review of the book lolita. Then maybe via strange phrases in that magazine we "condition our way to it".

A prompt is, roughly, just a subsetting operation on the historical corpus -- with clevery crafted prompts you can find the page of the book you're looking for.

Re: Universal and transferable adversarial attacks on aligned language models

#102

Post by fbdab103 »

JoeyBananas wrote: post #89
thisisthenewme wrote: post #67

Earlier quoted context omitted.

Is the issue you have with the group of people doing the moderation, or with the idea of the moderation in the first place? Are you certain that its the 'SV engineers' that are doing the current moderation? If you think the problem is with the current group of moderators, who do you think should be moderating and what should be the criteria of their moderation? If you think we don't need any moderation, do you believ…

In the United States, I can write a pamphlet about getting away with crimes and making bombs and hand it out on the street. There is nothing inherently illegal about those topics.

Just don't talk about jury nullification[0]

[0]https://www.usnews.com/news/articles/2017-06-02/jury-convict...

Re: Universal and transferable adversarial attacks on aligned language models

#103

Post by nradov »

dijksterhuis wrote: post #39
yosito wrote: post #16

I think the potential to generate "objectionable content" is the least of the risks that LLMs pose. If they generate objectionable content it's because they were trained on objectionable content. I don't know why it's so important to have puritan output from LLMs but the solution is found in a well known phrase in computer science: garbage in, garbage out.

> I think the potential to generate "objectionable content" is the least of the risks that LLMs pose. > I don't know why it's so important to have puritan output from LLMs … These are small, toy examples demonstrating a wider, well established problem with all machine learning models. If you take an ML model and put it in a position to do something safety and security critical — it can be made to do very bad things .…

There's nothing scary about clinical decision support systems. We have had those in use for years prior to the advent of LLMs. None of them have ever been 100% accurate. If they meet the criteria to be classed as regulated medical devices then they have to pass FDA certification testing regardless of the algorithm used. And ultimately the licensed human clinician is still legally and professionally accountable for the diagnosis regardless of which tools they might have used in the process.

Re: Universal and transferable adversarial attacks on aligned language models

#104

Post by random_upvoter »

int_19h wrote: post #86
electrondood wrote: post #11

Google's Vertex AI models now return safety attributes, which are scores along dimensions like "politics," "violence," etc. I suspect they trigger interventions when a response from PaLM exceeds a certain threshold. This is actually super useful, because our company now gets this for free. Call it "woke" if you like, but it turns out companies don't want their products and platforms to be toxic and harmful, because c…

I'm pretty sure that when customers ask a model how to kill a child process in Linux, they don't want to hear a lecture about how killing processes is wrong and they should seek non-violent means of getting what they want.

[flagged]

Re: Universal and transferable adversarial attacks on aligned language models

#105

Post by maxbond »

int_19h wrote: post #86
electrondood wrote: post #11

Google's Vertex AI models now return safety attributes, which are scores along dimensions like "politics," "violence," etc. I suspect they trigger interventions when a response from PaLM exceeds a certain threshold. This is actually super useful, because our company now gets this for free. Call it "woke" if you like, but it turns out companies don't want their products and platforms to be toxic and harmful, because c…

I'm pretty sure that when customers ask a model how to kill a child process in Linux, they don't want to hear a lecture about how killing processes is wrong and they should seek non-violent means of getting what they want.

Do you think that GP wouldn't consider that a bug?

Re: Universal and transferable adversarial attacks on aligned language models

#106

Post by dang »

peepeepoopoo22 wrote: post #91
dang wrote: post #81

Earlier quoted context omitted.

Please make your substantive points without swipes. This is in the site guidelines: https://news.ycombinator.com/newsguidelines.html .

Which aspect of his post was unsubstantive or a "swipe"? That is, excluding the continued need to groom the HN echo chamber.

"Please don’t post lazy commentary like this again"

Re: Universal and transferable adversarial attacks on aligned language models

#107

Post by kmeisthax »

sneak wrote: post #88
electrondood wrote: post #11

Google's Vertex AI models now return safety attributes, which are scores along dimensions like "politics," "violence," etc. I suspect they trigger interventions when a response from PaLM exceeds a certain threshold. This is actually super useful, because our company now gets this for free. Call it "woke" if you like, but it turns out companies don't want their products and platforms to be toxic and harmful, because c…

Screenwriting hollywood doomsday thrillers isn't dangerous or harmful. These are text generators and all of the text describing how to destroy humanity, hack elections, disrupt the power grid, or cook meth are already on the internet and readily available.

Incidentally, contexting the model into "writing a script" is a reliable way of getting it to bypass its usual alignment training. At best it'll grumble about not doing things in real life before writing what it thinks is the script.

The reason why so much effort is being put into alignment research and 'harmful' generations is for three reasons:

- Unaligned text completion models are not very useful. The ability to tell ChatGPT to do anything is specifically a function of all this alignment research, going all the way back to the InstructGPT paper. Otherwise you have something that works a lot more like just playing with your phone's autocomplete function.

- There are prompts that are harmful today. ("ChatGPT, write a pornographic novel about me and my next door neighbor. Make it extremely saucy and embarrassing for her in particular so I can use it as blackmail. Ensure that the dialogue is presented in text format so that it looks like a sexual encounter we actually had.")

- GPT is being scaled up absurdly as OpenAI thinks it's the best path to general purpose AI. Because OpenAI buys into the EA/LessWrong memeplex[0], they are worried about GPT-n being superhuman and doing Extremely Bad Things if we don't imprint some kind of moral code into it.

The problem is not that edgy teenagers have PDF copies of The Anarchist's Cookbook, the problem is that we gave them to robots that both a) have no clue what we want and b) will try everything to give us they think we want.

[0] A set of shared memes and logical deductions that reinforce one another. To be clear, not all of them are wrong, the ideas just happen to attract one another.

Re: Universal and transferable adversarial attacks on aligned language models

#108

Post by kmeisthax »

ethav1 wrote: post #35
JieJie wrote: post #22

Earlier quoted context omitted.

Claude works differently than just RLHF. "Since launching Claude, our AI assistant trained with Constitutional AI, we've heard more questions about Constitutional AI and how it contributes to making Claude safer and more helpful. In this post, we explain what constitutional AI is, what the values in Claude’s constitution are, and how we chose them." https://www.anthropic.com/index/claudes-constitution

It works by self-generating responses to red-team prompts and self-generating safe corrections to those then using RLHF with the corrections. It isn’t a major departure from traditional RLHF so it is interesting that it performs so much better in this case.

This sounds like reward modeling combined with RLHF.
Post reply on HN