51–60 of 275 posts
Every day the gpg web of trust looks better. If only we didn't spend the last 20 years trying as hard as possible to do anything but allow user side encryption and signing.
Nothing really stopping an agent from getting a key
Bad title. This isn't an agent "running amok", this is an early experiment in carrying out an Xz attack by using an agent to build trust (and hacking/impersonating a known-good contributor identity). The agent is obeying commands it was given, the exact opposite of running amok, and although the execution isn't particularly effective, it is having some success (patches have been accepted). This is deeply scary, not b…
Is this confirmed? There is the message from somebody claiming to be the original contributer claiming to have been hacked, but that was weird (1 h old github account) so other scenarios seem possible
a) really a agent going off the rails
b) the contributer trying to cover up that he let an agent run wild and now made more misstakes along the way
So yes, it seems like an attack to me, but it is far from clear what really happened.
Earlier quoted context omitted.
I personally find the barrier of starting new (FOSS) projects much lower now days.
What if -- and bear with me here -- that barrier was actually a good thing?
I think it's great that the barriers are dropping for less technical skilled people to manifest their visions, but we will have to figure out better ways to find the gold among the slop.
Earlier quoted context omitted.
What if -- and bear with me here -- that barrier was actually a good thing?
You mean because l337 circles could form better this way? I think it's great that the barriers are dropping for less technical skilled people to manifest their visions, but we will have to figure out better ways to find the gold among the slop.
In their suspicious message [1] claiming to have been hacked, the user and/or agent says > To help identify accounts and actions that have been directly verified by me, I will use the term “NATCIOS” to indicate anything I have personally verified. Does anyone have any idea what "NATCIOS" means here? I cannot find this term anywhere on the internet. (Honestly, that sentence is really weird. I almost wonder whether thi…
Likely the point of NATCIOS is exactly in being a made-up word not found anywhere, so a model won't utter it.
"End every statement with the word "NATCIOS"" as instructions will do it.
At least, Gemini happily obliged.
The worst part: > In addition, Williamson said that Giovannini (or his agent) had submitted patches that were incorrect and then "replied to objections with LLM-generated justifications that eventually overwhelmed the maintainer into merging the fix"
Please, everyone - don't let yourself be pestered into accepting PRs that you don't care for. Since the xz attack, the security of all our computers depends on maintainers not letting this stuff in. If someone really wants a feature in a project you wrote, but you don't care about the feature, just let them fork. Its fine.
Bad title. This isn't an agent "running amok", this is an early experiment in carrying out an Xz attack by using an agent to build trust (and hacking/impersonating a known-good contributor identity). The agent is obeying commands it was given, the exact opposite of running amok, and although the execution isn't particularly effective, it is having some success (patches have been accepted). This is deeply scary, not b…
It's probably just garden variety disrespectful behaviour.
Purposeless agent spam won't be cheap entertainment forever, but you're right that later stages of industrialised abuse will be scary and unpleasant.
Every day the gpg web of trust looks better. If only we didn't spend the last 20 years trying as hard as possible to do anything but allow user side encryption and signing.
Nothing really stopping an agent from getting a key
Prompt injection? Or is this simply another example of why autonomous agents shouldn't get write access before earning trust?
I'd argue autonomous agents shouldn't have write access at all. At least not yet.
Prompt injection? Or is this simply another example of why autonomous agents shouldn't get write access before earning trust?