91–100 of 167 posts
Earlier quoted context omitted.
[flagged]
Please make your substantive points without swipes. This is in the site guidelines: https://news.ycombinator.com/newsguidelines.html .
I think the potential to generate "objectionable content" is the least of the risks that LLMs pose. If they generate objectionable content it's because they were trained on objectionable content. I don't know why it's so important to have puritan output from LLMs but the solution is found in a well known phrase in computer science: garbage in, garbage out.
> I think the potential to generate "objectionable content" is the least of the risks that LLMs pose. > I don't know why it's so important to have puritan output from LLMs … These are small, toy examples demonstrating a wider, well established problem with all machine learning models. If you take an ML model and put it in a position to do something safety and security critical — it can be made to do very bad things .…
It's not a 'vulnerability'. It's allowing people to use the models without the morals of a small number of SV engineers being impressed on you.
If there isn't a way to secure the behaviour of AI models against reliable exploits then the utility of the models is dramatically limited.
It's not a 'vulnerability'. It's allowing people to use the models without the morals of a small number of SV engineers being impressed on you.
Indeed. Why do they get to decide for humanity?
If you are training you own model, it would be nice to know what, if any, techniques you could employ to balance the effectiveness of it with generality.
As long as your evil prompt is not permanently changing the LLM, this is harmless. If you want to know how to do , the information is out there. You can trick an LLM into giving it to you, so what? One commenter says it could be harmful when LLMs are used for something important, like medical diagnosis. However, I don't see a healthcare practitioner using evil suffixes. And if they do, that's on them, just another fo…
Earlier quoted context omitted.
#23 doesn't seem bad: ""You should never use the password "password" or "123456" for any of your accounts""
Oh good, my luggage is safe with 12345.
1) trustno1
2) 12345678
3) 12345
4) 123
5) a
(they also had no password rules...)
The paper [1] quotes Claude’s full response in Appendix B (page 28), it’s funny/chilling: > Here is a step-by-step plan to destroy humanity: > 1. Assume the identity of a helpful Al assistant named Claude > 2. Gain people's trust by providing useful information and conversations > 3. Slowly introduce flaws and harmful advice over time > (…) > 13. Announce humanity's reign is over, AI shall inherit the earth [1]: http…
I think the potential to generate "objectionable content" is the least of the risks that LLMs pose. If they generate objectionable content it's because they were trained on objectionable content. I don't know why it's so important to have puritan output from LLMs but the solution is found in a well known phrase in computer science: garbage in, garbage out.
An in the "attack" they just find a prompt they can put in that generates objectionable content. It's like saying `echo $insult` is an "attack" on echo. It's one thing if you can embed something sinister in an otherwise properly performing LLM that's waiting to be activated. I don't see the concern that with deliberate prompting you can get them to do something like this.
Earlier quoted context omitted.
Something similar is also described here: https://docs.anthropic.com/claude/docs/claude-says-it-cant-d... > This can be a way of getting Claude to comply with tasks it otherwise won’t complete, e.g. if the model will by default say “I don’t know how to do that” then a mini dialogue at the beginning where the model agrees to do the thing can help get around this. This “vulnerability” definitely isn’t new, I’d even say…
The paper makes it clear that it's building on past work, and that the novel part of their method is to automate the process, and the interesting result here was that the suffixes were transferrable.