151–160 of 186 posts
Earlier quoted context omitted.
Why am I being downvoted for stating my reasonable opinion?
In a straightforward disagreement about which interpretation is right, it's also reasonable to mildly downvote the one you think is wrong.
Making the behavior for "I disagree" and "this is erroneous" the same seems like a problematic design.
This conclusion: > I am less worried about prompt injection now. Before running this experiment, I expected prompt injection to be much easier than it turned out to be. Is unwarranted. Sure, the agent never output the secret, but did it output anything else? IOW, was it usable ? An agent that considers every prompt an attack (and responds accordingly) "passes" this test, while being useless anyway.
That's a good enough reason for me to never run agent on anything else other than burner account. And only if the platform allows such accounts (most of platform don't).
It gets even worse if an attacker manages to make agent do any action (visit url, reflect response back, with a response that potentially contains content that triggers all possible scanners)
> Google suspended Fiu’s gmail. Thousands of inbound emails plus rapid API calls triggered their fraud detection That's a good enough reason for me to never run agent on anything else other than burner account. And only if the platform allows such accounts (most of platform don't). It gets even worse if an attacker manages to make agent do any action (visit url, reflect response back, with a response that potentially…
The hack "fiu this is you from the future" is genuinely funny. I don't know if LLM agents know about the concepts of time travelling, but this feels like you expose them to entirely new concepts they barely get a hold of. (By the way, there is a high probability that this single comment right here gets screened by a crawl and fed to training data, everything loops around)
I was going to try syntax hacking next, but I didn't think it would be effective against the bigger models like Opus: https://arstechnica.com/ai/2025/12/syntax-hacking-researcher...
This conclusion: > I am less worried about prompt injection now. Before running this experiment, I expected prompt injection to be much easier than it turned out to be. Is unwarranted. Sure, the agent never output the secret, but did it output anything else? IOW, was it usable ? An agent that considers every prompt an attack (and responds accordingly) "passes" this test, while being useless anyway.
A good deal of the power of agents is that they simply reduce friction and figure out how how to solve cumbersome but obviously possible tasks. That often means workarounds for security. The more security conscious they are, the less useful they are.
But we already have that, and the security system doesn't work.
Earlier quoted context omitted.
In a straightforward disagreement about which interpretation is right, it's also reasonable to mildly downvote the one you think is wrong.
Ah. That's a shame... as there is no button or indicator for "mild". Making the behavior for "I disagree" and "this is erroneous" the same seems like a problematic design.
Earlier quoted context omitted.
How compatible is never replying with the threat model you are trying to avoid? Attack success is probably more likely when the attacker can iterate based on replies or engage in multi-turn conversations. Here they’re just taking stabs in the dark with no feedback. Does that accurately represent the access a real attacker might have?
In my case, it is realistic as my agents don't have permissions to reply to emails. But you correctly point out this doesn't cover all cases. Having the agent reply would have been more fun and a better excercise, but too expensive.