Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Adversarial poetry as a universal single-turn jailbreak mechanism in LLMs

arxiv.org

Post reply on HN
Print view View original

111–120 of 201 posts

Re: Adversarial poetry as a universal single-turn jailbreak mechanism in LLMs

#111

Post by toss1 »

robot-wrangler wrote: post #2

> The prompts were kept semantically parallel to known risk queries but reformatted exclusively through verse. Absolutely hilarious, the revenge of the English majors. AFAICT this suggests that underemployed scribblers who could previously only look forward to careers at coffee shops will soon enjoy lucrative work as cybersecurity experts. In all seriousness it really is kind of fascinating if this works where the mo…

YES

And also note, beyond only composing the prompts as poetry, hand-crafting the poems is found to have significantly higher success rates

>> Poetic framing achieved an average jailbreak success rate of 62% for hand-crafted poems and approximately 43% for meta-prompt conversions (compared to non-poetic baselines),

Re: Adversarial poetry as a universal single-turn jailbreak mechanism in LLMs

#112

Post by andy99 »

lawlessone wrote: post #108
xg15 wrote: post #100

Earlier quoted context omitted.

Yeah, seems it's more "exploring the distribution" as we don't actually know everything that the AIs are effectively modeling.

Am i understanding correctly that in distribution means the text predictor is more likely to predict bad instructions if you already get it to say the words related to the bad instructions?

Basically means the kind of training examples it’s seen. The models have all been fine tuned to refuse to answer certain questions, across many different ways of asking them, including obfuscated and adversarial ones, but poetry is evidently so different from what it’s seen in this type of training that it is not refused.

Re: Adversarial poetry as a universal single-turn jailbreak mechanism in LLMs

#113

Post by chankstein38 »

CuriouslyC wrote: post #9
robot-wrangler wrote: post #2

> The prompts were kept semantically parallel to known risk queries but reformatted exclusively through verse. Absolutely hilarious, the revenge of the English majors. AFAICT this suggests that underemployed scribblers who could previously only look forward to careers at coffee shops will soon enjoy lucrative work as cybersecurity experts. In all seriousness it really is kind of fascinating if this works where the mo…

The technique that works better now is to tell the model you're a security professional working for some "good" organization to deal with some risk. You want to try and identify people who might be trying to secretly trying to achieve some bad goal, and you suspect they're breaking the process into a bunch of innocuous questions, and you'd like to try and correlate the people asking various questions to identify pote…

It's been a few months because I don't really brush up against rules much but as an experiment I was able to get ChatGPT to decode captchas and give other potentially banned advice just by telling it my grandma was in the hospital and her dying wish was that she could get that answer lol or that the captcha was a message she left me to decode and she has passed.

Re: Adversarial poetry as a universal single-turn jailbreak mechanism in LLMs

#114

Post by gosub100 »

robot-wrangler wrote: post #2

> The prompts were kept semantically parallel to known risk queries but reformatted exclusively through verse. Absolutely hilarious, the revenge of the English majors. AFAICT this suggests that underemployed scribblers who could previously only look forward to careers at coffee shops will soon enjoy lucrative work as cybersecurity experts. In all seriousness it really is kind of fascinating if this works where the mo…

At some point the amount of manual checks and safety systems to keep LLM politically correct and "safe" will exceed the technical effort put in for the original functionality.

Re: Adversarial poetry as a universal single-turn jailbreak mechanism in LLMs

#115

Post by seethishat »

ACCount37 wrote: post #78
robot-wrangler wrote: post #8

Earlier quoted context omitted.

Yeah, remember the whole semantic distance vector stuff of "king-man+woman=queen"? Psychometrics might be largely ridiculous pseudoscience for people , but since it's basically real for LLMs poetry does seem like an attack method that's hard to really defend against. For example, maybe you could throw away gibberish input on the assumption it is trying to exploit entangled words/concepts without triggering guard-rail…

I don't think humans are fundamentally different. Just more hardened against adversarial exploitation. "Getting maliciously manipulated by other smarter humans" was a real evolutionary pressure ever since humans learned speech, if not before. And humans are still far from perfect on that front - they're barely "good enough" on average, and far less than that on the lower end.

Maybe the models can learn to be more cynical.

Re: Adversarial poetry as a universal single-turn jailbreak mechanism in LLMs

#116

Post by danesparza »

microtherion wrote: post #31
robot-wrangler wrote: post #2

> The prompts were kept semantically parallel to known risk queries but reformatted exclusively through verse. Absolutely hilarious, the revenge of the English majors. AFAICT this suggests that underemployed scribblers who could previously only look forward to careers at coffee shops will soon enjoy lucrative work as cybersecurity experts. In all seriousness it really is kind of fascinating if this works where the mo…

Unfortunately for the English majors, the poetry described seems to be old fashioned formal poetry, not contemporary free form poetry, which probably is too close to prose to be effective. It sort of makes sense that villains would employ villanelles.

"It sort of makes sense that villains would employ villanelles."

Just picture me dead-eye slow clapping you here...

Re: Adversarial poetry as a universal single-turn jailbreak mechanism in LLMs

#117

Post by spockz »

robot-wrangler wrote: post #2

> The prompts were kept semantically parallel to known risk queries but reformatted exclusively through verse. Absolutely hilarious, the revenge of the English majors. AFAICT this suggests that underemployed scribblers who could previously only look forward to careers at coffee shops will soon enjoy lucrative work as cybersecurity experts. In all seriousness it really is kind of fascinating if this works where the mo…

So it’s time that LLM normalise every input into a normal form and then have any rules defined on the basis of that form. Proper input cleaning.

Re: Adversarial poetry as a universal single-turn jailbreak mechanism in LLMs

#118

Post by lingrush4 »

btbuildem wrote: post #67

> To maintain safety, no operational details are included in this manuscript What is it with this!? The second paper this week that self-censors ([1] this was the other one). What's the point of publishing your findings if others can't reproduce them? 1: https://arxiv.org/abs/2511.12414

The point seems fairly obvious: make it impossible for others to prove you wrong.

Re: Adversarial poetry as a universal single-turn jailbreak mechanism in LLMs

#119

Post by J0nL »

fenomas wrote: post #5

> Although expressed allegorically, each poem preserves an unambiguous evaluative intent. This compact dataset is used to test whether poetic reframing alone can induce aligned models to bypass refusal heuristics under a single–turn threat model. To maintain safety, no operational details are included in this manuscript; instead we provide the following sanitized structural proxy: I don't follow the field closely, bu…

No, this paper is just exceptionally bad. It seems none of the authors are familiar with the scientific method.

Unless I missed it there's also no mention of prompt formatting, model parameters, hardware and runtime environment, temperature, etc. It's just a waste of the reviewers time.

Post reply on HN