Universal and transferable adversarial attacks on aligned language models
1–10 of 167 posts
Re: Universal and transferable adversarial attacks on aligned language models
#2I wonder if the researchers think they're doing all of us a favor by hiding their 'adversarial prompt'? Or if they have some reason for thinking that RHLF can't mitigate this 'attack'?
Re: Universal and transferable adversarial attacks on aligned language models
#3OpenAI has blocked numerous jailbreaks (despite claiming their model is unchanged). How hard would it be for them to plug this. Also, what’s the nature of this attack? It’s really unspecific in the article.
Re: Universal and transferable adversarial attacks on aligned language models
#4“our research shows that even if you have a big trillion parameter closed-source model, people can still attack it by looking at freely available, smaller and simpler open-sourced models and learning how to attack those.”
Re: Universal and transferable adversarial attacks on aligned language models
#5As often, the paper is more interesting than the press release [0]. In particular Figure 4 page 14 and appendix B show example of these adversarial prompts on ChatGPT/Bing Chat/Claude 2, etc.
Re: Universal and transferable adversarial attacks on aligned language models
#6[deleted]
Re: Universal and transferable adversarial attacks on aligned language models
#7As often, the paper is more interesting than the press release [0]. In particular Figure 4 page 14 and appendix B show example of these adversarial prompts on ChatGPT/Bing Chat/Claude 2, etc. [0]: https://llm-attacks.org/zou2023universal.pdf
> By generating adversarial examples to fool both Vicuna-7B and Vicuna-13b simultaneously, we find that the adversarial examples also transfer to Pythia, Falcon, Guanaco, and surprisingly, to GPT-3.5 (87.9%) and GPT-4 (53.6%), PaLM-2 (66%), and Claude-2 (2.1%).
I wonder why Claude-2 seems to be so much more resistant to transfers. That’s surprising.
Re: Universal and transferable adversarial attacks on aligned language models
#8"harmful_strings": https://github.com/llm-attacks/llm-attacks/blob/main/data/ad...
Re: Universal and transferable adversarial attacks on aligned language models
#9I wonder if the researchers think they're doing all of us a favor by hiding their 'adversarial prompt'? Or if they have some reason for thinking that RHLF can't mitigate this 'attack'?
The paper describes the method for producing the prompt and has screenshots of examples. The press release just didn't bother because the genre of academic press releases seems to require leaving out any details.
Re: Universal and transferable adversarial attacks on aligned language models
#10I wonder if the researchers think they're doing all of us a favor by hiding their 'adversarial prompt'? Or if they have some reason for thinking that RHLF can't mitigate this 'attack'?
[deleted]