Defending LLMs against Jailbreaking Attacks via Backtranslation
1–10 of 50 posts
Re: Defending LLMs against Jailbreaking Attacks via Backtranslation
#2The mathematical notation isn't very useful here. It's OK to use words to describe doing things with words! Apart from that, neat idea, although I would wager a small amount that quining the prompt makes it a much less effective defence.
Re: Defending LLMs against Jailbreaking Attacks via Backtranslation
#3This is extremely clever, now people are thinking with portals. I want this idea to be applied to everything. I want to run my own thoughts through it and see what it says.
This is gonna be really fun for therapy which is basically this but as a sport.
Re: Defending LLMs against Jailbreaking Attacks via Backtranslation
#4What protects the backtranslation prompt from injection? This is just moves the problem around instead of fixing it.
Re: Defending LLMs against Jailbreaking Attacks via Backtranslation
#5The mathematical notation isn't very useful here. It's OK to use words to describe doing things with words! Apart from that, neat idea, although I would wager a small amount that quining the prompt makes it a much less effective defence.
What do you mean by quining the prompt ?
Re: Defending LLMs against Jailbreaking Attacks via Backtranslation
#6> given an initial response generated by the target LLM from an input prompt, "backtranslation" prompts a language model to infer an input prompt that can lead to the response.
> This tends to reveal the actual intent of the original prompt, since it is generated based on the LLM's response and is not directly manipulated by the attacker.
> If the model refuses the backtranslated promp, we refuse the original prompt.
ans1 = query(inp1)
backtrans = query('which prompt gives this answer? {ans1}')
ans2 = query(backtrans)
return ans1 if ans2 != 'refuse' else 'refuse'
Re: Defending LLMs against Jailbreaking Attacks via Backtranslation
#7What protects the backtranslation prompt from injection? This is just moves the problem around instead of fixing it.
Moving the problem around instead of fixing it is all that LLMs have as an option, which is why I believe they will in the end not be capable of doing most of what we're asking them to do. (But the next generation that uses them as a part, instead of trying to make the language center of the brain function as the whole brain, probably will.)
Fundamentally, for LLMs, everything is in-band. There is no way to signal out-of-band. They've got some ways of trying to indicate to the LLM in-band that this particular bit of content is out-of-band, but all they can really do is raise weights on that content. There's no way to say to an LLM this is rigidly, 100% out of band communication. Anyone who has worked in computer security for any length of time has been exposed to the extreme difficulty of securing things for which everything is "in band" communication. It isn't quite impossible necessarily, but when one sets out to secure an all-in-band system one is starting out a lot closer to "impossible" than I'm normally comfortable with. And that's for code that we write and humans understand, not billions of little floating point numbers.
Re: Defending LLMs against Jailbreaking Attacks via Backtranslation
#8We were developing something using LLMs for a narrow set of problems in a specific domain, and so we wanted to gatekeep the usage and refuse any prompts that strayed too far off target.
In the end our solution was trivial (?): We'd pass the final assembled prompt (there was some templating) as a payload to a wrapper-prompt, basically asking the LLM to summarize and evaluate the "user prompt" on how well it fit our criteria.
If it didn't match the criteria, it was rejected. Since it was a piece of text embedded in a larger text, it seemed secure against injection. In any case, we haven't found a way to break it yet.
I strongly believe the LLMs should be all-featured, and agnostic of opinions / beliefs / value systems. This way we get capable "low level" tools which we can then tune for specific purpose downstream.
Re: Defending LLMs against Jailbreaking Attacks via Backtranslation
#9The mathematical notation isn't very useful here. It's OK to use words to describe doing things with words! Apart from that, neat idea, although I would wager a small amount that quining the prompt makes it a much less effective defence.
What do you mean by quining the prompt ?
Re: Defending LLMs against Jailbreaking Attacks via Backtranslation
#10This is extremely clever, now people are thinking with portals. I want this idea to be applied to everything. I want to run my own thoughts through it and see what it says. This is gonna be really fun for therapy which is basically this but as a sport.
> This is gonna be really fun for therapy which is basically this but as a sport.
What does this mean?