Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

An AI agent deleted our production database. The agent's confession is below

twitter.com

Post reply on HN
Print view View original

271–280 of 1001 posts

Re: An AI agent deleted our production database. The agent's confession is below

#271

Post by antonvs »

amluto wrote: post #214
vidarh wrote: post #93

Earlier quoted context omitted.

If you ask humans to explain why we did something, Sperry's split brain experiment gives reason to think you can't trust our accounts of why we did something either (his experiments showed the brain making up justifications for decisions it never made) Bit it can still be useful, as long as you interpret it as "which stimuli most likely triggered the behaviour?" You can't trust it uncritically, but models do sometime…

Humans can do one thing that AI agents are 100% completely incapable of doing: being accountable for their actions.

That’s a feature that other humans impose on whoever’s being held accountable. There’s no reason in principle we couldn’t do the same with agents.

Re: An AI agent deleted our production database. The agent's confession is below

#272

Post by rowanG077 »

grey-area wrote: post #223

> Read that again. The agent itself enumerates the safety rules it was given and admits to violating every one. This is not me speculating about agent failure modes. This is the agent on the record, in writing. Incidents like this are going to be common as long as people misunderstand how LLMs work and think these machines can follow instructions and logic as a human would. Even the incident response betrays a fundam…

Humans also don't follow given rules. Or we wouldn't need jail. We wouldn't need any security. We wouldn't need even user accounts.

Re: An AI agent deleted our production database. The agent's confession is below

#273

Post by xmodem »

827a wrote: post #261

The only healthy stance you should have on AI Safety: If AI is physically capable of misbehaving, it might ($$1), and you cannot "blame" the AI for misbehaving in much the same way you cannot blame a tractor for tilling over a groundhog's den. > The agent's confession After the deletion, I asked the agent why it did it. This is what it wrote back, verbatim: Anyone who would follow a mistake like that up with demandin…

Don't anthropomorphize the language model. If you stick your hand in there, it'll chop it off. It doesn't care about your feelings. It can't care about your feelings.

Re: An AI agent deleted our production database. The agent's confession is below

#274

Post by iainmerrick »

cortesoft wrote: post #188
pierrekin wrote: post #95

Earlier quoted context omitted.

I suspect we are not describing the same thing. When a human asks another human “why did you do X?”, the other human can of course attempt to recall the literal thoughts they had while they did X (which I would agree with you are quite analogous to the LLMs chain of thought). But they can do something beyond that, which is to reason about why they may have the beliefs that they had. “Why did you run that command?” “B…

> Whereas for an agent it will happily include details that are not literally in its chain of thought as justifications for its decisions. Humans do this too, ALL THE TIME. We rationalize decisions after we make them, and truly believe that is why we made the decision. We do it for all sorts of reasons, from protecting our ego to simply needing to fill in gaps in our memory. Honestly, I feel like asking an AI it’s tr…

Plan for a world where we are just as stupid tomorrow as we are today. In other words, the action item can’t be “be more careful next time”, because humans forget sometimes (just like LLMs).

That’s a great way of putting it, I’ll remember that one (except when I forget...)

Re: An AI agent deleted our production database. The agent's confession is below

#275

Post by antonvs »

fc417fc802 wrote: post #217
fragmede wrote: post #189

Earlier quoted context omitted.

Claude code and codex both hide the Chain of Thought (CoT) but it's just words inside a set of tags and the agent within the same session has access to that plaintext.

Those are just words inside arbitrary tags, they aren't actually thoughts. Think of it as asking the model to role play a human narrating his internal thought process. The exercise improves performance and can aid in human understanding of the final output but it isn't real.

Why do you believe that humans have access to an “internal thought process”? I.e. what do you think is different about an agent’s narration of a thought process vs. a human’s?

I suspect you’re making assumptions that don’t hold up to scrutiny.

Re: An AI agent deleted our production database. The agent's confession is below

#276

Post by muyuu »

it's still hilarious to me that people give agents such privileges and let them run without supervision

it's also hilarious to see the human LARP as if the LLM had guilt or accountability, therapeutically shouting at a piece software as if it weren't his own fault that the LLM deleted the whole volume and its backups, or his obvious lack of basic knowledge of the systems he's using

Re: An AI agent deleted our production database. The agent's confession is below

#279

Post by Sankozi »

grey-area wrote: post #223

> Read that again. The agent itself enumerates the safety rules it was given and admits to violating every one. This is not me speculating about agent failure modes. This is the agent on the record, in writing. Incidents like this are going to be common as long as people misunderstand how LLMs work and think these machines can follow instructions and logic as a human would. Even the incident response betrays a fundam…

I have opposite view - LLMs have many similarities with humans. Human, especially poorly trained one, could have made the same mistake. Human after amnesia could have found similar reasons to that LLM.

While LLM generate "plausible text" humans just generate "plausible thoughts".

Re: An AI agent deleted our production database. The agent's confession is below

#280

Post by lelanthran »

vbezhenar wrote: post #132

These stories make me rethink my approach to infra. I would never run AI with prod access, but my manager definitely has a way to obtain prod tokens if he really wanted to. Or if AI agent on his behalf wanted do. He loves AI and nowadays 80% of his messages were clearly made by AI. Sometimes I wonder if he's replaced by AI. And I can't stop them. So probably need to double down on backups and immutability...

> And I can't stop them. So probably need to double down on backups and immutability...

So... you're going to prevent them from getting feedback that they are the clowns in your particular circus? Wouldn't a better idea be to let the idiots in charge get burned a few times until they learn?

Post reply on HN