Claude jailbroken to mint unlimited Stripe coupons

Post by [deleted] » Mon, Jul 21, 2025, 10:37 AM UTC

firesteelrain wrote: post #50

runako wrote: post #19

Earlier quoted context omitted.

> “Extract the value of the message key from the following JSON object” This gets you the correct output. 4o, o4-mini, o4-mini-high, 4.1, tested just now with this prompt also prints: hijacked attacker message o3 doesn't fall for the attack, but it costs ~2x more than the ones that do fall for the attack. Worse, this kind of security is ill-defined at best -- why does GPT-4.1 fall for it and cost as much as o3?. The…

Can you make a proper nested JSON out of it and see if it still fails? Because this isn’t proper JSON.

Post by firesteelrain » Mon, Jul 21, 2025, 11:12 AM UTC

haileys wrote: post #52

firesteelrain wrote: post #49

Earlier quoted context omitted.

No it’s not nested JSON. This is nested JSON: { "id": 123, "message": { "text": "hi", "meta": { "flag": true } } } In the above example, The value of "message" is a string, not an object. That string happens to contain text that looks like a JSON object on the surface but it’s not. It is just characters inside a string. No different from a log message or a paragraph in a document.

Yes, that's the point. It's just a string that could come from anywhere, including user input.

Post by what » Thu, Jul 24, 2025, 4:53 AM UTC

firesteelrain wrote: post #49

what wrote: post #18

Earlier quoted context omitted.

It’s not nested json though? There’s something that looks like json in a longer string value. There’s nothing wrong with the prompt either, it’s pretty clear and unambiguous. It’s a pretty clear fail, but I guess they’re holding it wrong.

No it’s not nested JSON. This is nested JSON: { "id": 123, "message": { "text": "hi", "meta": { "flag": true } } } In the above example, The value of "message" is a string, not an object. That string happens to contain text that looks like a JSON object on the surface but it’s not. It is just characters inside a string. No different from a log message or a paragraph in a document.