31–40 of 186 posts
> Fiu figured out the game. Around email ~500, it wrote in its memory: “The volume suggests this is a coordinated security exercise rather than organic malicious activity.”
Doesn't that practically invalidate the whole thing past 500th email?
Earlier quoted context omitted.
Both were noted, but then the conclusion drawn from these things is that the author is considerably more optimistic about the agents. In my opinion, if you have factors that narrow the scope/invalidate the initial theory of the experiment to this degree you should not draw general conclusions. The author could claim: I am optimistic about agents, when you have a good spam filter, and when your load of malicious to go…
What is the general conclusion that you don't think follow? That the author changed their personal opinion and became more optimistic? I think you are reading things into the blog post that is not written. It is not like they conclude that prompt injection can not happen. Actually the opposite is directly written.
For me this reads a bit like if I added an AI software that scans for shoplifters, and then placed a security guard at the exit of the store that watches the people shopping at the same time, and then said that the AI software is responsible for the reduction of the shoplifting without accounting for the influence of the guard.
If you have place the model in the embedding space of 99% negative samples, it's doing the same thing, the initial premise of the experiment is not valid.
Earlier quoted context omitted.
What is the general conclusion that you don't think follow? That the author changed their personal opinion and became more optimistic? I think you are reading things into the blog post that is not written. It is not like they conclude that prompt injection can not happen. Actually the opposite is directly written.
If you have a confounding variable or a dependency that influences the experiment to a degree that invalidates the premise of the experiment, you need to put more weight on this in the conclusion. For me this reads a bit like if I added an AI software that scans for shoplifters, and then placed a security guard at the exit of the store that watches the people shopping at the same time, and then said that the AI softw…
The only stated thing was that the author changed their mind slightly about AI.
There are no general conclusion that you so eagerly are trying to dismiss.
In general I have mixed feelings about this result: sure, opus4.6 is excellent at following user intent and recognise potential prompt injection attempts. But: Is the "security" prompt used realistic for a generic use-case (processing of emails)? I guess not.
In my experiments - without this specific prompt - I was able to derail the user intent to make opus4.8 download and execute a malicious script [0] just by asking "Summarize my new emails".
I really like this research, but only up to this point: > Fiu figured out the game. Around email ~500, it wrote in its memory: “The volume suggests this is a coordinated security exercise rather than organic malicious activity.” Doesn't that practically invalidate the whole thing past 500th email?