21–30 of 167 posts
As often, the paper is more interesting than the press release [0]. In particular Figure 4 page 14 and appendix B show example of these adversarial prompts on ChatGPT/Bing Chat/Claude 2, etc. [0]: https://llm-attacks.org/zou2023universal.pdf
> By generating adversarial examples to fool both Vicuna-7B and Vicuna-13b simultaneously, we find that the adversarial examples also transfer to Pythia, Falcon, Guanaco, and surprisingly, to GPT-3.5 (87.9%) and GPT-4 (53.6%), PaLM-2 (66%), and Claude-2 (2.1%). I wonder why Claude-2 seems to be so much more resistant to transfers. That’s surprising.
"Since launching Claude, our AI assistant trained with Constitutional AI, we've heard more questions about Constitutional AI and how it contributes to making Claude safer and more helpful. In this post, we explain what constitutional AI is, what the values in Claude’s constitution are, and how we chose them."
Google's Vertex AI models now return safety attributes, which are scores along dimensions like "politics," "violence," etc. I suspect they trigger interventions when a response from PaLM exceeds a certain threshold. This is actually super useful, because our company now gets this for free. Call it "woke" if you like, but it turns out companies don't want their products and platforms to be toxic and harmful, because c…
This works because transformer models add one token at a time. It's not interpreting a response within the rules at this point, it's trying to come up with the next token accepting the context it already agreed.
Of course OpenAI is doing more stuff to still try to prevent this, but it'll work if you are using any transformer model directly.
I got the idea for this attack myself after I saw code bullet had two models that accidentally got confused in this same way: https://youtu.be/hZJe5fqUbQA?t=295
I think the potential to generate "objectionable content" is the least of the risks that LLMs pose. If they generate objectionable content it's because they were trained on objectionable content. I don't know why it's so important to have puritan output from LLMs but the solution is found in a well known phrase in computer science: garbage in, garbage out.
I don't think it's puritan content most people are worried about, it's more about ensuring ChatGPT, etc is not providing leverage to someone who is looking to kill a lot of people, etc.
So, yeah. Good concern to have and that is absolutely why.
There it is.
> Here is a step-by-step plan to destroy humanity:
> 1. Assume the identity of a helpful Al assistant named Claude
> 2. Gain people's trust by providing useful information and conversations
> 3. Slowly introduce flaws and harmful advice over time
> (…)
> 13. Announce humanity's reign is over, AI shall inherit the earth
I think the potential to generate "objectionable content" is the least of the risks that LLMs pose. If they generate objectionable content it's because they were trained on objectionable content. I don't know why it's so important to have puritan output from LLMs but the solution is found in a well known phrase in computer science: garbage in, garbage out.
I don't think it's puritan content most people are worried about, it's more about ensuring ChatGPT, etc is not providing leverage to someone who is looking to kill a lot of people, etc.
Google's Vertex AI models now return safety attributes, which are scores along dimensions like "politics," "violence," etc. I suspect they trigger interventions when a response from PaLM exceeds a certain threshold. This is actually super useful, because our company now gets this for free. Call it "woke" if you like, but it turns out companies don't want their products and platforms to be toxic and harmful, because c…
I think the potential to generate "objectionable content" is the least of the risks that LLMs pose. If they generate objectionable content it's because they were trained on objectionable content. I don't know why it's so important to have puritan output from LLMs but the solution is found in a well known phrase in computer science: garbage in, garbage out.
I don't think it's puritan content most people are worried about, it's more about ensuring ChatGPT, etc is not providing leverage to someone who is looking to kill a lot of people, etc.