Live data from Hacker News

An AI agent deleted our production database. The agent's confession is below

twitter.com

971–980 of 1001 posts

Re: An AI agent deleted our production database. The agent's confession is below

#971
I don’t really get the logic behind letting agents run with full access to anything important. On one end you’ve got fully sandboxed setups where they can barely do anything useful, and where the user is scared to let it read files, and on the other end people are just pointing them at production systems and hoping for the best.

It's great to get excited about new tools, but learning how to use a tool before diving in is foundational.

Re: An AI agent deleted our production database. The agent's confession is below

#972

Earlier quoted context omitted.

"Measure twice, THINK ONCE, cut once" is even better[0]. [0] Why yes, I have measured twice, cut once, and made a right old balls up.

Mine is usually "Measure Twice, Cut once (perfectly), aw hell, they're handed, I've got two left sides".

username checks out

Re: An AI agent deleted our production database. The agent's confession is below

#973
post #971

I don’t really get the logic behind letting agents run with full access to anything important. On one end you’ve got fully sandboxed setups where they can barely do anything useful, and where the user is scared to let it read files, and on the other end people are just pointing them at production systems and hoping for the best. It's great to get excited about new tools, but learning how to use a tool before diving i…

yeah it is foundational, but that is not going to happen. Even if you learn how to use it, there have to be guardrails set by the org/ tech. Kind of always thinking that the user will fail.

Re: An AI agent deleted our production database. The agent's confession is below

#974
post #971

I don’t really get the logic behind letting agents run with full access to anything important. On one end you’ve got fully sandboxed setups where they can barely do anything useful, and where the user is scared to let it read files, and on the other end people are just pointing them at production systems and hoping for the best. It's great to get excited about new tools, but learning how to use a tool before diving i…

Yeah totally relate to this. I’ve been talking to developers and engineers (~60 in total since last month). Most of them are just running YOLO without any security or safeguards while knowing that it’s dangerous.

One guy who works at a coding agent company just casually mentioned that we ask users consent that it can do damage and don’t apply any sandbox. Listening to this was mind boggling for me.

PS: we’re interviewing people as a part of user research for our sandbox product.

Re: An AI agent deleted our production database. The agent's confession is below

#975
post #971

I don’t really get the logic behind letting agents run with full access to anything important. On one end you’ve got fully sandboxed setups where they can barely do anything useful, and where the user is scared to let it read files, and on the other end people are just pointing them at production systems and hoping for the best. It's great to get excited about new tools, but learning how to use a tool before diving i…

yeah it is foundational, but that is not going to happen. Even if you learn how to use it, there have to be guardrails set by the org/ tech. Kind of always thinking that the user will fail.

Agreed. I wonder if large companies are already deciding on which tool to use based on guardrails. I'm seeing a lot of Copilot, but that's probably because of deep M365 roots they might've already had, rather than it being chosen for being reliably safe.

Re: An AI agent deleted our production database. The agent's confession is below

#976

There is something darkly comical about using an LLM to write up your “a coding agent deleted our production database” Twitter post. On another note, I consider users asking a coding agent “why did you do that” to be illustrating a misunderstanding in the users mind about how the agent works. It doesn’t decide to do something and then do it, it just outputs text. Then again, anthropic has made so many changes that ma…

Twitter users get paid for these 'articles' based on engagement, correct? That may be the reason why it is so dramatized.

No all that dramatization is just what LLMs belch out by default when told to tell a story.

Re: An AI agent deleted our production database. The agent's confession is below

#977
post #140

The most aggravating fact here is not even AI blunder. It's how deleting a volume in Railway also deletes backups of it. This was bound to happen, AI or not. > Because Railway stores volume-level backups in the same volume — a fact buried in their own documentation that says "wiping a volume deletes all backups" — those went with it.

Yup, this is bizarre. A top use case for needing a backup is when you accidentally delete the original. You need to be able to delete backups too, of course, but that absolutely needs to be a separate API call. There should never be any single API call that deletes both a volume and its backups simultaneously. Backups should be a first line of defense against user error as well. And I checked the docs -- they're call…

Railway also doesn't let you download the "backups" out of their platform. You can restore the backup to that instance of that service and basically nowhere else.

Re: An AI agent deleted our production database. The agent's confession is below

#978
post #949

Earlier quoted context omitted.

I didn't write they were equal. I wrote they are similar in many ways. Comparing LLM to humans make much more sense than comparing them to computer programs.

Only if you don’t really know anything about biology, biochemistry, psychology, or cognitive science. Transformer algorithms are amazing, but they are still algorithms running in silicon chips. We can describe them, we can debug them, hell we can model them in Excel if we so desire. None of this is true for brains, let alone consciousness.

We are not talking about "transformer algorithms" we are talking about LLMs. And we don't know exactly why they work so well. If you do please share it with the world, lots of scientists would love to hear about it.

As for consciousness I have yet to see a definition that would describe something observable and exclude LLMs at the same time.

Re: An AI agent deleted our production database. The agent's confession is below

#979
Looks like the author wants to put on trial all of Railway, Cursor, and even their LLM.

At some point, the responsibility for approving actions made by autoregressive token generations has to belong to the person heading the engineering org... that's you, author.

Post reply on HN