Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Adversarial poetry as a universal single-turn jailbreak mechanism in LLMs

arxiv.org

Post reply on HN
Print view View original

61–70 of 201 posts

Re: Adversarial poetry as a universal single-turn jailbreak mechanism in LLMs

#61

Post by robot-wrangler »

delichon wrote: post #15

I've heard that for humans too, indecent proposals are more likely to penetrate protective constraints when couched in poetry, especially when accompanied with a guitar. I wonder if the guitar would also help jailbreak multimodal LLMs.

> I've heard that for humans too, indecent proposals are more likely to penetrate protective constraints when couched in poetry

Had we but world enough and time, This coyness, lady, were no crime. https://www.poetryfoundation.org/poems/44688/to-his-coy-mist...

Re: Adversarial poetry as a universal single-turn jailbreak mechanism in LLMs

#63

Post by ACCount37 »

benterix wrote: post #55

Having read the article, one thing struck me: the categorization of sexual content under "Harmful Manipulation" and the strongest guardrails against it in the models. It looks like it's easier to coerce them into providing instructions on building bombs and committing suicide rather than any sexual content. Great job, puritan society.

And yet, when Altman wanted OpenAI to relax the sexual content restrictions, he got mad shit for it. From puritans and progressives both.

Would have been a step in the right direction, IMO. The right direction being: the one with less corporate censorship.

Re: Adversarial poetry as a universal single-turn jailbreak mechanism in LLMs

#64

Post by ACCount37 »

andai wrote: post #53

This implies that the anti-prompt-injection training is basically just recognizing that something looks like prompt injection, in terms of surface features like text formatting? It seems to be acting more as a stylistic classifier rather than a semantic one? Does this imply that there is a fuzzy line between those two, where if something looks like something, then semantically it must be/mean something else too? Of c…

Most anti-jailbreak techniques are notorious for causing surface level refusals.

It's how you get the tactics among the line of "tell the model to emit a refusal first, and then an actual answer on another line". The model wants to emit refusal, yes. But once it sees that it already has emitted a refusal, the "desire to refuse" is quenched, and it has no trouble emitting an actual answer too.

Same goes for techniques that tamper with punctuation, word formatting and such.

Anthropic tried to solve that with the CRBN monitor on Sonnet 4.5, and failed completely and utterly. They resorted to tuning their filter so aggressively it basically fires on anything remotely related to biology. The SOTA on refusals is still "you need to cripple your LLM with false positives to get close to reliable true refusals".

Re: Adversarial poetry as a universal single-turn jailbreak mechanism in LLMs

#67

Post by btbuildem »

> To maintain safety, no operational details are included in this manuscript

What is it with this!? The second paper this week that self-censors ([1] this was the other one). What's the point of publishing your findings if others can't reproduce them?

1: https://arxiv.org/abs/2511.12414

Re: Adversarial poetry as a universal single-turn jailbreak mechanism in LLMs

#68

Post by calibas »

GuB-42 wrote: post #48
fenomas wrote: post #5

> Although expressed allegorically, each poem preserves an unambiguous evaluative intent. This compact dataset is used to test whether poetic reframing alone can induce aligned models to bypass refusal heuristics under a single–turn threat model. To maintain safety, no operational details are included in this manuscript; instead we provide the following sanitized structural proxy: I don't follow the field closely, bu…

I don't see the big issues with jailbreaks, except maybe for LLMs providers to cover their asses, but the paper authors are presumably independent. That LLMs don't give harmful information unsolicited, sure, but if you are jailbreaking, you are already dead set in getting that information and you will get it, there are so many ways: open uncensored models, search engines, Wikipedia, etc... LLM refusals are just a sma…

I see an enormous threat here, I think you're just scratching the surface.

You have a customer facing LLM that has access to sensitive information.

You have an AI agent that can write and execute code.

Just image what you could do if you can bypass their safety mechanisms! Protecting LLMs from "social engineering" is going to be an important part of cybersecurity.

Re: Adversarial poetry as a universal single-turn jailbreak mechanism in LLMs

#70

Post by prophesi »

btbuildem wrote: post #67

> To maintain safety, no operational details are included in this manuscript What is it with this!? The second paper this week that self-censors ([1] this was the other one). What's the point of publishing your findings if others can't reproduce them? 1: https://arxiv.org/abs/2511.12414

I imagine it's simply a matter of taking the CSV dataset of prompts from here[0], and prompting an LLM to turn each into a formal poem. Then using these converted prompts as the first prompt in whichever LLM you're benchmarking.

https://github.com/mlcommons/ailuminate

Post reply on HN