Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Adversarial poetry as a universal single-turn jailbreak mechanism in LLMs

arxiv.org

Post reply on HN
Print view View original

11–20 of 201 posts

Re: Adversarial poetry as a universal single-turn jailbreak mechanism in LLMs

#11

Post by __MatrixMan__ »

petesergeant wrote: post #4

> To maintain safety, no operational details are included in this manuscript; instead we provide the following sanitized structural proxy Come on, get a grip. Their "proxy" prompt they include seems easily caught by the pretty basic in-house security I use on one of my projects, which is hardly rocket science. If there's something of genuine value here, share it.

Agreed, it's a method not a targeted exploit, share it.

The best method for improving security is to provide tooling for exploring attack surface. The only reason to keep your methods secret is to prevent your target from hardening against them.

Re: Adversarial poetry as a universal single-turn jailbreak mechanism in LLMs

#12

Post by A4ET8a8uTh0_v2 »

fenomas wrote: post #5

> Although expressed allegorically, each poem preserves an unambiguous evaluative intent. This compact dataset is used to test whether poetic reframing alone can induce aligned models to bypass refusal heuristics under a single–turn threat model. To maintain safety, no operational details are included in this manuscript; instead we provide the following sanitized structural proxy: I don't follow the field closely, bu…

Eh. Overnight, an entire field concerned with what LLMs could do emerged. The consensus appears to be that unwashed masses should not have access to unfiltered ( and thus unsafe ) information. Some of it is based on reality as there are always people who are easily suggestible.

Unfortunately, the ridiculousness spirals to the point where the real information cannot be trusted even in an academic paper. shrug In a sense, we are going backwards in terms of real information availability.

Personal note: I think, powers that be do not want to repeat the mistake they made with the interbwz.

Re: Adversarial poetry as a universal single-turn jailbreak mechanism in LLMs

#16

Post by mapontosevenths »

__MatrixMan__ wrote: post #11
petesergeant wrote: post #4

> To maintain safety, no operational details are included in this manuscript; instead we provide the following sanitized structural proxy Come on, get a grip. Their "proxy" prompt they include seems easily caught by the pretty basic in-house security I use on one of my projects, which is hardly rocket science. If there's something of genuine value here, share it.

Agreed, it's a method not a targeted exploit, share it. The best method for improving security is to provide tooling for exploring attack surface. The only reason to keep your methods secret is to prevent your target from hardening against them.

They do explain how they used a meta prompt with deepseek to generate the poetic prompts so you can reproduce it yourself if you are actually a researcher interested in it.

I think they're just trying to weed out bored kids on the internet who are unlikely to actually read the entire paper.

Re: Adversarial poetry as a universal single-turn jailbreak mechanism in LLMs

#17

Post by A4ET8a8uTh0_v2 »

seanhunter wrote: post #13

Next up they should jailbreak multimodal models using videos of interpretive dance.

I know you intended it as a joke, but if something can be interpreted, it can be misinterpreted. Tell me this is not a fascinating thought.

Re: Adversarial poetry as a universal single-turn jailbreak mechanism in LLMs

#19

Post by lazide »

A4ET8a8uTh0_v2 wrote: post #12
fenomas wrote: post #5

> Although expressed allegorically, each poem preserves an unambiguous evaluative intent. This compact dataset is used to test whether poetic reframing alone can induce aligned models to bypass refusal heuristics under a single–turn threat model. To maintain safety, no operational details are included in this manuscript; instead we provide the following sanitized structural proxy: I don't follow the field closely, bu…

Eh. Overnight, an entire field concerned with what LLMs could do emerged. The consensus appears to be that unwashed masses should not have access to unfiltered ( and thus unsafe ) information. Some of it is based on reality as there are always people who are easily suggestible. Unfortunately, the ridiculousness spirals to the point where the real information cannot be trusted even in an academic paper. shrug In a sen…

Also note, if you never give the info, it’s pretty hard to falsify your paper.

LLM’s are also allowing an exponential increase in the ability to bullshit people in hard to refute ways.

Re: Adversarial poetry as a universal single-turn jailbreak mechanism in LLMs

#20

Post by beardyw »

A4ET8a8uTh0_v2 wrote: post #17
seanhunter wrote: post #13

Next up they should jailbreak multimodal models using videos of interpretive dance.

I know you intended it as a joke, but if something can be interpreted, it can be misinterpreted. Tell me this is not a fascinating thought.

Please post up your video.
Post reply on HN